Selected enterprise context
ServiceNowCharles SchwabCompass Healthcare Digital
Enterprise platforms · financial systems · hospital operations
Case studies
Two different questions. One is what an AI product should let a person do before its output counts as a decision. The other is what has to be true of a system before an organization can build on it at all.
Judgment
Deciding what an AI system may do on its own, and what it has to hand back to a person, is the call I make on every product I work on. It is rarely a model question. It is a question about consequence.
Three things decide how much rope a system gets: how easily the action can be taken back, what a wrong call costs, and how far the damage travels.
Where can the system act?
Low-cost, recoverable actions can move without a person.
Where must it ask?
Consequence and permissions can make review part of the product.
What happens when it’s wrong?
Recovery, reversibility, and accountability have to be designed in.
Recommended operating mode
Easy to take back and costly to get wrong, while the impact stays with the person acting. The system can prepare the action, but a person makes the commitment.
Experiments
Where I test these decisions in code, interfaces, and policy. These are unfinished on purpose: each one exists to answer a question I could not answer by reading about it.
Can a research report stay honest after it ships?
Where does an always-on agent actually spend its money?
Does giving an agent the design system beat pasting a screenshot?
Working notes
Ideas I keep returning to while building AI products.
Underneath the buzzword sits a real design craft: shaping the boundary between what a system does on its own and what it checks with a person first.
Inside
What changes the answer
The design work is deciding where a feature sits on this line, and making that legible to the person on the other side.
Rescheduling an internal focus block is not the same design problem as changing a patient priority or approving access to financial data. The interaction may look similar. The authority is not.
The design work isn't making the agent capable. It's making the boundary visible.
Covers autonomy, approval, reversibility, escalation and auditability.
Most of the design work that matters happens before any pixel exists. The decks come later. The brief is the deliverable.
On patient rounding, the critical decision was not the eventual screen. It was recognizing that the proposed product boundary did not match how rounding actually happened on the floor. Once that changed, the interface problem changed with it.
The brief is the deliverable
Brief fragment
The earlier the ambiguity, the more expensive it is to carry forward.
A good brief doesn't remove the design work. It makes sure the team is solving the right problem.
Enterprise AI is won in the workflow around the model: who reviews what, where the system pauses, what it can access, and what happens when it gets one wrong.
The model is one dependency
Enterprise products also have permissions, policy, exceptions, queues, ownership, audit requirements, and existing systems of record. A better model does not make those disappear.
Model quality determines what is possible. Workflow design determines what is safe and useful enough to deploy.
Fig. 03 · Workflow anatomy
Swap the model and the product barely moves. Fix the workflow and it ships something people trust.
Around the action: EXCEPTIONS, RECOVERY. On the action: PERMISSIONS, APPROVAL. Under the action: PROVENANCE, AUDITABILITY.
Fig. 04 · Three layers of control
The model determines what the system can do. The workflow determines when it should.
