Wolters Kluwer
2025
Phase 3: Expert AI Summary Analysis
An AI-assisted proposal review system built to be verifiable enough for attorneys to actually act on.
Overview
Making AI Fast Enough to Help and Trustworthy Enough to Use
An AI-assisted proposal review system built to be trustworthy enough for attorneys to act on, not just accurate.
Problem
An Unverifiable Summary Is a Liability, Not a Shortcut
The business wanted AI-assisted review.
The real problem was trust. Legal professionals won't act on an algorithm's read of a multi-million-dollar engagement unless they can verify why it concluded what it did. An unverifiable summary is a liability, not a convenience. The question was how to make AI-assisted evaluation trustworthy enough that a managing attorney will actually act on it, in a domain where being wrong has legal and financial consequences.
Research
Three Trust Constraints, Not Technical Ones
Three trust constraints, not technical ones, made this hard. Auditability is non-negotiable since legal, procurement, and finance all touch these decisions, and any AI output had to trace back to source. The existing comparison model couldn't break, Expert AI had to score against the same standardized RFP criteria the MVP established, not invent its own. And human judgment had to stay in the loop, since attorneys wouldn't hand final selection to a model.
Ideation
Designing the AI to Be Legible, Not Just Accurate
Every interface decision aimed at making the AI legible, not just accurate. AI-generated summaries with a first-class human verify step, comparative scoring against fixed RFP criteria so results stayed explainable, confidence indicators with traceable sources so a skeptical attorney could always drill to the source text, predictive success forecasting framed as a signal to weigh rather than a verdict, and contextual retrieval from past matters so attorneys didn't have to research manually.
Designs
The Verify Step Isn't an Afterthought. It's the Flow.
Screens built around the verify step as a first-class part of the flow rather than an afterthought, keeping the attorney as the decider at every point.
Lessons
40% reduction in manual review time, more consistent and defensible evaluations, and real attorney trust, driven by the transparency mechanisms (confidence, traceability, human-in-the-loop) rather than the AI output alone. The 40% was real because attorneys could verify it, not just because the tool was faster.
