AI LEGAL DOCUMENT REVIEW · INDEPENDENT
Lumens AI
Designing the agent-to-lawyer handoff as an independent working build.
Killed three features that were costing trust without earning it.
TL;DR
I built the agent-to-lawyer handoff as an independent working build. The problem I set myself: trust collapses when you ask lawyers to grade AI outputs instead of designing for the cognitive shift between AI work and human judgment.
Try it
Stage three, in your hands.
Three flagged clauses from a fictional matter. Notice what the interface refuses to let you do: accept a finding whose evidence you haven't looked at.
Ashworth Health, Matter 2026-0412
Master services agreement
1 Definitions
“Confidential Information” means non-public information disclosed by either party that is marked confidential or that a reasonable person would understand to be confidential from its nature. “Services” means the contract-review software and implementation work described in Schedule A. “Data Security Incident” means unauthorized access to, or disclosure of, Client data processed under this Agreement.
2 Services
Vendor shall provide the Services in accordance with the specifications in Schedule A and the service levels in Schedule B. Vendor shall notify Client promptly of any change that would materially reduce the Services' functionality.
3 Fees
Client shall pay the fees set out in Schedule C on a monthly basis, net 30 from the invoice date. Fees are exclusive of applicable taxes, which Client shall bear in addition to the amounts stated, and unpaid balances accrue interest at 1.5% per month.
4 Term
This Agreement commences on the Effective Date and continues for 36 months, renewing automatically for successive 12-month periods unless either party gives 90 days' written notice of non-renewal.
5 Confidentiality
Each party shall protect the other's Confidential Information using at least the same degree of care it applies to its own, and in no case less than reasonable care. This obligation survives termination for five years.
6 Intellectual property
Vendor retains all rights in the Services and underlying software. Client owns its own data and grants Vendor a limited license to process that data solely to provide the Services.
7 Data Security
Each party shall maintain reasonable safeguards for data handled under this Agreement.
9 Limitation of Liability
The limits, exclusions, and allocation of risk in this Section govern the parties' claims.
9.1 Cap
Except as provided in Section 9.3, neither party's aggregate liability under this Agreement, including claims arising under Section 7 (Data Security), shall exceed the fees paid or payable in the twelve (12) six (6) months preceding the claim.
9.2 Exclusions
These limits do not apply to indemnification under Section 11, or to gross negligence.
9.3 Carve-outs
Claims arising under Section 7 (Data Security) are subject to the separate cap set out in the Term Sheet.
9.4 Survival
The limitations and exclusions in this Section 9 survive termination or expiry of this Agreement.
9.5 Aggregate application
Where more than one claim arises from a single course of conduct, those claims shall be treated as one claim for the purpose of the cap.
11 Indemnification
Each party's indemnification obligations under Section 11 remain outside the cap in Section 9.1.
12 Assignment
Neither party may assign this Agreement without the prior written consent of the other party, subject to the change-of-control language in Section 12.3.
Review minutes vs. findings, by agent confidence
Findings pile up where confidence is high. Minutes should pile up where it is low. The design's job is to make that inversion the path of least resistance.
The AI and the lawyer are not doing the same job. The AI is doing fast, comprehensive, lower-confidence retrieval and synthesis. The lawyer is doing slow, narrow, high-confidence judgment.
The reframe that scoped the build
Where the seams are.
The demo runs on written data behind real gates. The chat is where a live model call happens. Attention is the scarce resource underneath all of it: the confidence score is not decoration on a card, it is the routing signal that decides where a lawyer's minutes go.
What runs, and what was written
One box is written by hand. The rest of the demo's chain is real logic, and the site's only live model call sits in the lane below.
One box is written by hand. The rest of the demo's chain is real logic, and the site's only live model call sits in the lane below.
The decision trail
Three states of the review surface.
First pass
Accept stays disabled until every cited source has been opened.
- The verification gate went in as a working demo rather than a description of one.
- Two panes only. A flagged-clause queue beside the clause text and the agent's read.
Mid iteration
Confidence becomes a coded signal, on a color world that belongs to the product rather than the site.
- The window became a credible app. Traffic lights, centered identity, a matter toolbar.
- Confidence stopped being a ghost numeral and became a dot, a tinted meter, and a word.
Current prototype
Clause and finding each get their own titled panel, and the review opens to full size.
- Titled document and finding panels replaced the floating italic labels.
- The full view opens the way a real window would, over its own desktop.
The handoff
One job per stage.
One job per stage, so the pattern never asks the lawyer to do two things at once. The product's record of truth is the lawyer's signature, never the model's output.
Retrieval had to surface its own scoring and the draft state its own confidence, so the design and the retrieval logic were built together, not handed off.
Retrieves
User sees a soft progress state with documents being scanned in real time.
Proposes
Agent surfaces a draft answer with citations linked inline. Distinctive draft treatment so the user knows it has not yet been reviewed.
Reviews
User reads the draft, clicks any citation to verify, and either accepts, edits, or rejects with a one-line reason.
Publishes
Reviewed answer becomes canonical. Record of truth is the human signature.
Decisions made
The calls that shaped the build.
Every project has a handful of decisions that determined the outcome. These are mine, with the trade-offs I accepted.
No batch approve. Publishing an answer takes a review, one at a time.
It is the easiest feature to ask for and the easiest to abuse, because it lets a reviewer approve without reading. The Quick/Detailed toggle and the confidence slider went for the same reason: each let a lawyer skip the reading that makes an acceptance mean something.
Throughput looks slower on paper. If the friction just moves elsewhere, the design lost.
Every rejection carries a one-line reason.
A rejection with a reason is feedback instead of a dead end. It keeps what the lawyer changed, and why, on the record.
Every rejection costs a few seconds. The completion rate on that field says whether the cost was too high.
Exhibits
Two artifacts that carry the argument.
Exhibit 01A draft must look like one
Draft, not yet reviewedModerate confidence
The non-solicitation clause survives termination for 18 months under section 11.2…
Reviewed and signed by E. Okafor
The non-solicitation clause survives termination for 18 months under section 11.2, amended per the March rider.
Exhibit 02A rejection becomes structured signal
- Reject
One click, but never the last one.
- Wrong survival period…
One required line of why.
- Routed to eval set
The reason becomes structured signal.
The problem
The trust gap.
The premise Lumens starts from: a legal-AI product with a strong model and a large corpus can return accurate, cited answers and still go unused, because nothing tells the lawyer what to trust.
01Reading
Skim, pattern-match. Low-stakes attention.
02Reviewing
Verify, weigh, second-guess. Senior judgment.
03Deciding
Sign off, attach reputation.
Every AI output landed in the same visual treatment as a finished answer, so every output felt like a homework assignment that needed to be graded.
What I'd measure
There are no outcomes yet. Here is the scorecard.
If Lumens were live, this is the scorecard I would put my name to. If those numbers did not move, the pattern is wrong and I would want to know.
- 01Daily active use of the review surface against the pre-redesign baseline.
- 02The share of rejections that carry a written reason.
- 03Time from proposal to published answer.
- 04A failure-mode taxonomy for the draft state, because a draft that reads as finished is this design's most dangerous defect.
What I'd do differently
In retrospect.
- Instrument from day oneBuild the rejection-reason capture into Stage 3 from the start, not as a later add-on.
- Eval before flowDesign the eval for the draft-state failure mode before building the review flow, so that what 'reads as finished but isn't' shapes the visual language instead of following it.
More work