Conversational AI Needs Real-World Evidence, Not Just Demos
— by Vivax
Google's AMIE evidence story now has a crucial timeline. The completed BIDMC study was a prospective, single-center, single-arm feasibility study of 100…
Google's AMIE evidence story now has a crucial timeline. A completed BIDMC study was a prospective, single-center, single-arm feasibility study of 100 adults with live physician supervision; a separate nationwide randomized Included Health study was announced on February 3 as pending IRB approval, then described in the September 14 Nature Medicine commentary as IRB-approved, ongoing and pre-registered. Those are different stages of evidence.
The BIDMC study tested whether an AI-mediated visit could be run with real participants and supervising physicians. It did not establish that an autonomous chatbot improves outcomes across health systems, and it did not remove the need for clinician accountability. Feasibility is valuable precisely because it exposes the operational questions a benchmark cannot answer.
The Included Health program is the next kind of question: what happens when conversational AI is evaluated prospectively across a broader population and compared with a defined control? Until its protocol, endpoints, enrollment and results are available, it should be described as an announced or ongoing study—not as a completed randomized trial.
Real-world evidence has to measure more than diagnostic accuracy. Safety events, escalation quality, time burden, documentation, access, subgroup performance, patient understanding and clinician trust all change whether a system helps. A polished transcript can hide a dangerous handoff or a workflow that nobody can sustain.
Trust is not a marketing afterthought. Patients, clinicians, caregivers and health-system operators each experience a different risk surface, so evaluation should make those perspectives visible. Consent, privacy, auditability and the ability to recover from uncertainty belong in the protocol, not in a post-launch footnote.
For Vivax, this is the standard we want for clinical world models and AcuDx: grounded in records, explicit about uncertainty, supervised at the point of care and evaluated on trajectories rather than isolated answers. The story is promising, but the responsible conclusion is narrower: prospective evidence is being built, not finished.