AX Field Notes

All casesNo. 071

Catholic Health Puts Voice AI on Its MyChart Line

Hold times are gone, and 64% of help desk calls no longer go to outsourced agents

Case no.
071
Primary source
Notable blog (Becker's Health IT session recap) and vendor interview
Source published
2026-04-24

1. Why this case

Catholic Health is a multi-site health system on Long Island, New York, in the United States. Its MyChart Help Desk is the phone line patients call when they cannot get into MyChart, the Epic patient portal they use to book appointments, read messages from providers, and check test results. In a 2026 interview and conference session, Catholic Health put the line at about 5,000 calls a month, at $10 a call, with patients waiting 30 to 60 seconds for a live agent and 10% of calls dropped.

Working with the vendor Notable, the health system put an AI voice agent on that line in five weeks. The goal was 30% containment, meaning calls the agent resolves without a person. At an April 2026 conference session, Allyson T. Collins, vice president of digital strategy, reported 54% on day one, 64% since, and $60,000 saved in the first two months.

The case is useful for three reasons. First, the starting line was small, routine, and priced per call, so a containment target was also a cost target. Second, the health system described how it built and checked the agent before go-live: step-by-step task instructions, a five-question grading rubric, more than a hundred test scenarios, and AI grading of calls. Third, it used the help desk as the first step of a phased expansion toward a patient access line of about 600,000 calls a year.

All figures come from Catholic Health’s own executive, speaking in a Notable-hosted interview and in a Becker’s conference session co-presented with Notable. They are customer-reported, not independently audited.


2. Problem

The problem started with the portal, not the phone. In the session recap, “While Catholic Health has an Epic-first technology stance, they received negative feedback about the MyChart Help Desk. Patient adoption of MyChart was poor, with an activation rate of only 20%.”

The help desk was meant to fix access problems but was itself a bottleneck:

  1. Volume and unit cost. “About 5,000 patients called each month, with each call costing $10.” In an interview Notable published on 2026-01-16 (the January interview below), Collins described the same line as “approximately 5,000 calls a month to our MyChart Helpdesk, and each call was costing $10.” The session abstract describes the line’s calls as ones that “would have otherwise been routed to outsourced agents.”
  2. Waiting and dropped calls. “After analyzing call data, Catholic Health discovered patients were waiting 30-60 seconds for live agents to answer, resulting in 10% of calls being abandoned.” Collins put it as “The first 30 seconds of a call can make or break the call.”
  3. Wrong answers. Patients told the health system they “weren’t always getting the correct answers.”

Collins tied the help desk to whether patients stay with the health system at all: “We were preventing patients from accessing their digital health information, which meant they wouldn’t schedule appointments online, check messages from providers or check test results,” … “What’s worse, we knew other health systems were providing a much better digital experience, so patients might never come back to Catholic Health.”

Confirmed facts: Epic-first technology stance; MyChart activation rate 20% before; about 5,000 help desk calls a month at $10 per call; calls otherwise routed to outsourced agents (session abstract); 30–60 second waits for a live agent; 10% of calls abandoned; patient feedback that answers were not always correct.


3. Solution

3.1 Leadership questions before any build

Before building, the January interview says, Catholic Health worked through three questions at the leadership level: brand (“Would AI Agents alienate patients?”), oversight (keeping patient-facing AI “accurate, empathetic, and aligned with our brand values and core mission”), and reputation (“Could a single misstep impact patient trust?”). The April recap adds that it is “essential to involve the privacy and legal teams, and the CISO.”

The help desk was one of four initiatives the January interview lists: SMS texting with live referral agents for referral scheduling, inbound voice agents for the MyChart help desk, an iterative development process with vendors, and phased go-lives of voice and text agents.

3.2 Scope set by listening to calls

The plan began narrow. Collins: “We focused our energy on an AI Voice Agent and decided to start small with password resets, one of the most common reasons patients called. But once we listened to the calls, we realized patients weren’t thinking in silos.” A caller “might think they needed a password reset but they actually forgot their username, and then they wanted help scheduling an appointment. So we shifted gears to a big bang so we could provide a cohesive experience with a bigger value.”

The team wrote step-by-step instructions for each MyChart task in scope. The examples given are username request, password reset, how to find your bill, and how to add a proxy. The April recap describes the goal as helping patients “reset passwords, recover usernames and assist with other common tasks.”

3.3 Targets and design requirements

Collins set the outcome first: “Our goal was a 20 to 25% containment rate. And then, I pushed us to aim higher and we raised it to a 30% goal because we had a substantial cost savings opportunity here.” Two more requirements came with it: “We wanted 24/7 coverage, and for the AI Agent to send text message links after verbal consent for a better user experience.”

The team also set a maximum response latency, the gap between the patient finishing a question and the agent answering, because “that silent pause is a top indicator that you’re talking to an AI Agent.” No value is given. They listened to “dozens of AI voices,” aiming for a voice that was professional but not overly formal, conversational but not too casual.

3.4 A grading rubric and a test set

“There were five criteria for grading each conversation”:

  1. Did the agent correctly identify the intent of the call?
  2. Did it follow the instructions for answering the caller’s question?
  3. Did it follow the tone and voice guidelines?
  4. “Did she properly transfer to a live agent as needed? We wanted every mention of an emergency or urgency to be directed immediately to a live agent.”
  5. “Did she hallucinate?”

Testing was, in Collins’s words, “our favorite part of the process.” “We created a spreadsheet containing over a hundred different tests that we wanted to run with the AI Agent, and then we used AI to grade every single call that came in as a perfect, pass, or fail.”

3.5 Go-live in five weeks, then hourly monitoring

The January interview and the session abstract both say the agent went live in five weeks. The session abstract: “In just 5 weeks, the organization deployed an AI voice agent capable of resolving common patient requests and providing 24/7 assistance.”

After launch, “For the first few days, my team monitored calls hourly. The tech worked predictably – exactly as we designed it. But patients were unpredictable.” The first fix was to the instructions, not the model: “we didn’t fully account for some of the minor nuances between the web and app for MyChart. We then shifted and rewrote the instructions to include a web-specific set and an app-specific set.” The team has also gone back and forth on how often the agent speaks up while a patient is in the middle of a step, such as opening an email and filling out a form, so that it does not hurry them but still sounds present.

3.6 Where people stay in the loop

Emergencies and urgent requests go straight to a live agent, per the grading rubric above. The rest of the calls the agent does not contain still go to live agents; the session abstract describes the 64% as calls “that would have otherwise been routed to outsourced agents.” The abstract also says the change allowed “staff to focus on more complex patient needs rather than routine account and access questions.”

Confirmed facts: leadership-level brand, oversight, and reputation questions settled first; involving privacy, legal, and the CISO given as advice in the session recap; scope widened from password resets to a set of MyChart tasks after listening to calls; step-by-step instructions per task; containment target 20–25% raised to 30%; 24/7 coverage; SMS links sent after verbal consent; latency ceiling set (value not given); five grading criteria including immediate transfer on any mention of emergency or urgency; 100+ test scenarios in a spreadsheet; AI grading of calls as perfect, pass, or fail; go-live in five weeks; hourly call monitoring for the first few days; instructions split into web and app sets after launch.


4. Impact

The figures come from two snapshots of the same deployment, an interview published on 2026-01-16 and an April 2026 session and recap.

MetricInterview published 2026-01-16April 2026 session and recap
Containment target20–25%, raised to 30%30% “initial objective”
Containment at launch52% (“we started with”)54% (“on day one”)
Containment since57%64%
Calls handled25,000+—
Savings“estimating $350,000 per year”“$60,000” in the first two months; “projected annual savings of $360,000”

Access. “Since agents are available 24/7, patient hold times and call abandonment have been eliminated” (April recap). In the January interview, Collins said, “now that the AI Agent is answering the phone, we’re actually answering 500 more helpdesk calls a month.”

Portal adoption. The article body gives only the 20% activation rate before the project. Notable’s summary at the top of the republished page says “MyChart activation climbed past its 20% baseline,” without a new figure.

Patients. The April recap says patient feedback has been positive. Collins quoted one caller: “You are the best virtual assistant I’ve ever spoken to in about 60 years. So thank you very much. You’re the best. God bless you.” Her broader reading: “We’ve discovered it doesn’t matter to the patient whether they’re dealing with a live human or AI, as long as they get the help they need.”

Expansion. The April recap says the help desk “has given Catholic Health confidence to expand AI voice agents to additional patient-facing processes in a phased approach”:

  • Two-way texting “to support referrals and patient communication, now live across more than 50 specialties with a 20% conversion rate and Epic integration for automated updates.” The recap does not say what counts as a conversion. The January interview places EHR AI texting for referral scheduling before the voice agent, so the two sources order the texting work differently.
  • “Inbound AI voice for patient access, targeting high call volumes — approximately 600,000 calls annually — with early use cases focused on identity authentication, triage and routing.” The session abstract names “primary care lines for tasks like refills, referrals, and follow-ups.”

The recap frames this as “a broader roadmap to scale AI across the patient access front door, sequencing new capabilities based on value and complexity.”


5. Insight

A per-call price made the target a budget line. With about 5,000 calls a month at $10 each, every point of containment had a dollar value. Collins raised the goal from 20–25% to 30% “because we had a substantial cost savings opportunity here.”

Scope came from the calls, not the org chart. The plan to start with password resets changed once the team listened to calls and saw that one call could be a forgotten username plus a scheduling request. Collins’s stated reason for covering the tasks together was a cohesive experience. It also bears on containment, since a caller whose second problem is out of scope still has to reach a person.

The test set was the specification. Five grading questions, a hundred-plus scenarios, and an AI grader defined what working meant before go-live. After launch, the first gap was in the instructions (web versus app), and the fix was to rewrite them. “The tech worked predictably”; the patients did not.

Answering at once removes the queue, not just some calls. An agent that picks up immediately at any hour takes away the 30–60 second wait and with it the 10% of callers who used to hang up. That is why the hold-time and abandonment results are reported as eliminated rather than reduced, and why the line answers about 500 more calls a month.

The human role moves to what the agent cannot or should not do. Any mention of an emergency or urgency goes straight to a live agent, and calls the agent cannot resolve still reach outsourced agents. For the next phase, Collins wants “our human agents to focus on scheduling, which is where they can have the most impact at the top of their license.”

Limits. All figures are reported by Catholic Health’s own executive through a vendor interview and a vendor co-presented session; none is independently verified. The two snapshots give different launch figures (52% and 54%). The savings are an estimate and a projection, and the sources do not say whether they are net of Notable’s fees. Portal activation after the project is described only in Notable’s page summary, without a figure. The 20% texting conversion rate is not defined.

One-line takeaway: A health system can move a routine, per-call-priced help line to a voice agent within weeks if it fixes the target, writes the tasks out step by step, and grades calls against a short rubric before letting patients in.


6. Implementation hypotheses (not in the source)

Basis: The agent was required “to send text message links after verbal consent,” its core tasks are password reset and username recovery, and the patient access expansion lists “identity authentication” as an early use case. The team also tunes how often the agent speaks while a patient is “getting an email and then needs to click to fill out a form.”

Account recovery for a patient portal cannot be done safely by reading credentials aloud. The likely flow is that the agent first matches the caller to a patient record using details already on file, then asks permission to text a link, and on a verbal yes triggers MyChart’s own reset or recovery step and sends the link to the number on record. The patient then completes the step on a phone or computer while the agent stays on the line. That would explain both requirements in the sources: the consent step before any text, and the web-specific and app-specific instruction sets, since the screens a patient sees differ between the two. Keeping the credential change inside MyChart also means the voice agent never holds a password.

6.2 The hundred-test spreadsheet as a regression suite

Basis: “five criteria for grading each conversation,” “a spreadsheet containing over a hundred different tests,” AI grading “as a perfect, pass, or fail,” hourly monitoring in the first days, and instructions rewritten into web and app sets after launch.

Each row in the spreadsheet probably describes one caller: an intent, a device, and a twist, such as a forgotten username that turns into a scheduling request, or a mention of an emergency halfway through. Test calls would run against the agent, and an AI grader would score each transcript on the five questions and assign perfect, pass, or fail. A miss on the transfer or hallucination question likely counts as a fail regardless of the other scores, since those are the two failures that harm a patient. The same rubric could then be applied to live transcripts, which would let a small team review calls every hour in the first days by reading the fails rather than every call. When the instructions were split into web and app sets, the test set would have needed web and app variants of each task, so the change could be re-run before it went live.

6.3 Containment counted against the outsourced queue

Basis: The agent “contained 64% of calls that would have otherwise been routed to outsourced agents”; the line had about 5,000 calls a month at $10 per call; “In just the first two months we saved $60,000 and projected annual savings of $360,000”; and the line now answers “500 more helpdesk calls a month.”

Containment is most likely counted per call as a call that ends without a transfer to a live agent, and the savings as contained calls times the $10 agent cost, with the annual projection extrapolated from the first two months. The extra 500 answered calls a month complicate the count. Calls that used to be abandoned cost nothing before, so whether they are counted as avoided agent calls would change the savings figure.


7. Sources

  1. Notable, republishing Becker’s Health IT — Eliminating patient hold times: Catholic Health’s rapid deployment of voice AI
    https://www.notablehealth.com/blog/eliminating-patient-hold-times-catholic-healths-rapid-deployment-of-voice-ai
    2026-04-24 (originally published by Becker’s Health IT on 2026-04-23; accessed 2026-11-17)
  2. Becker’s Hospital Review 16th Annual Meeting — session abstract, Eliminating Patient Hold Times: Catholic Health’s Rapid Deployment of Voice AI
    https://conferences.beckershospitalreview.com/april-annual-meeting-2026/session/3956554/04.-eliminating-patient-hold-times-catholic-health%E2%80%99s-rapid-deployment-of-voice-ai
    Session 2026-04-13 (accessed 2026-11-17)
  3. Notable — Catholic Health: A practical guide for your next AI deployment
    https://www.notablehealth.com/blog/catholic-health-a-practical-guide-for-your-next-ai-deployment
    2026-01-16 (accessed 2026-11-17)