Ambient documentation: the questions to ask before the pilot

Ambient documentation is one of the few clinical AI applications with an obvious, immediate value proposition: clinicians spend less time typing. Pilots are easy to run, satisfaction scores come back high, and the decision to expand feels straightforward.

It frequently is straightforward. But the pilots that go badly at scale go badly for reasons that were knowable at the pilot stage and were not asked about, because the pilot was measuring the wrong things.

Baseline the burden before you start

The most common failure is having no credible before-measurement. 'Clinicians report saving two hours a day' is not a finding; it is a feeling, and it is systematically inflated by the novelty of a tool people wanted.

Pull actual EHR audit log data before the pilot begins: time in notes, time after hours, note length, time to note closure. Most modern EHRs expose this. Then measure the same fields at 30, 90 and 180 days. The 180-day number is the one that matters, because early enthusiasm decays and what remains is the real effect.

The questions the vendor demo will not raise

Who reviews the output, and what happens when it is wrong? An ambient note is a draft. If the clinician is attesting to a note they skimmed, you have moved documentation risk rather than reduced it. Define the attestation standard explicitly and make sure it is a standard a busy clinician can actually meet.

What does it do to note length and coding? Ambient tools tend to produce longer, more complete notes. That can improve severity capture, which is good. It can also produce documentation that supports a level of service the encounter did not involve, which is a compliance exposure. Have your CDI and compliance teams read fifty generated notes before you expand.

How does it perform across your actual patient population? Accented speech, multilingual encounters, interpreters, family members talking over the patient, noisy environments. Vendor performance data is generated in conditions that resemble a quiet exam room. Your emergency department is not a quiet exam room.

Where does the audio go, for how long, and who can access it? This is a business associate agreement question and a patient consent question, and it needs a real answer before the pilot, not after a patient asks.

What is the monitoring plan after go-live? Clinical AI is not a one-time validation. Model behavior changes, vendors ship updates, and your patient mix shifts. If nobody owns ongoing monitoring, nobody will notice degradation.

Pilot design that produces a decision

Pick two contrasting specialties rather than volunteers from one. Volunteers from one enthusiastic department tell you what enthusiastic people in that department think. A primary care clinic and an emergency department tell you whether the tool generalizes.

Include at least a few clinicians who did not want it. Their experience is the one that predicts scale adoption, because at scale most of your users will not have volunteered.

Define in advance what result would cause you to stop. A pilot without a stopping criterion is a procurement process wearing a pilot's clothing, and everyone involved knows it.