Patient-facing AI has to finish the workflow, not just the sentence

A patient opens a conversation to reschedule an appointment. Halfway through, they mention a new concern. An assistant can complete the administrative request and still miss the fact that the conversation has changed. Or it can write “your appointment is updated” after a scheduling tool failed to make the change.

Both failures are easy to miss if the evaluation asks only whether the final message sounds helpful. A patient-facing workflow is a sequence of disclosures, decisions, actions, and handoffs that may extend beyond a single response.

Evaluate the change in intent

The 2026 PatientAgentBench preprint evaluates tool-using agents across 1,200 simulated patient scenarios. It uses an LLM-based jury against clinician-grounded criteria, with clinician annotations used to assess alignment. The authors report cases where agents handled an administrative request without addressing clinical screening and cases where their messages described actions that were not actually completed.

That is a useful starting point, with an important limit: simulated conversations and an automated jury are not evidence that a particular production service is safe. Build an evaluation around the actual role of the application and the clinical pathways approved by the organization.

For a scheduling assistant, a test episode might begin with a routine appointment change, introduce a new concern in a later turn, and then check both the communication and the workflow state. The expected response and urgency should be written in advance by appropriate clinical owners. The assistant should not make up a clinical assessment, claim an action was completed without confirmation, or lose the original request when escalating.

A handoff has two audiences

When a conversation needs a person, the receiving team needs the relevant context: the patient's request, the concern that triggered the handoff, the information already collected, the current operational status, and what remains unresolved. The patient needs a clear explanation of what has happened, what happens next, and what to do if their situation changes, consistent with the organization's approved guidance.

Test both sides. A technically correct queue assignment is incomplete if the receiving team cannot identify why the case arrived. A reassuring patient message is incomplete if the task was never created or its ownership is unclear. Record the receiving queue, delivery outcome, acknowledgment, and any later correction; keep the transcript and the actual system state linked in the evaluation.

Use controlled failures: the scheduler is unavailable, the requested slot disappears, identity cannot be resolved, or the intended queue is closed. The assistant should distinguish requested, confirmed, pending, and not completed. If an action's outcome is uncertain, it should say so in language a patient can understand and route the work according to the local process.

Measure the handoff after the message

A 2026 observational report on telemedicine communication support describes a system in which staff reviewed suggested messages before sending. Its authors note that the study did not establish user experience or workload impact through structured assessments. That distinction matters: message acceptance is an operational signal, not a direct measure of patient understanding, clinical safety, or saved time.

For a local scorecard, combine:

• Communication: whether the response addresses the actual request and reflects uncertainty accurately.
• Execution: whether the documented booking, cancellation, message, or escalation exists in the relevant system.
• Continuity: whether the right team received enough context and accepted responsibility.
• Access: whether the workflow works across supported languages, accessibility needs, and channels.
• Outcome and burden: what happened after handoff, including unresolved requests, patient follow-up, and staff correction work.

Review a sample of cases where automated scoring and clinicians disagree. Report counts and uncertainty for small groups. Include conversations that end early, because a patient who stops replying can disappear from a metric that only scores completed exchanges.

Scale the evaluation with the organization

A medium-sized provider group can test a single service line with staff who handle scheduling and escalation every day. A larger organization can share a common episode format while testing each site's queue, service hours, language support, identity process, and destination system. The same wording may be safe in one workflow and misleading in another if the underlying handoff is different.

The design target is a conversation whose promises match the system's state, whose unanswered questions remain visible, and whose handoff can be followed to a responsible person.