Nvidia Taps Abridge to Train AI on Real Doctor Visits

Nvidia is partnering with Abridge to build an artificial intelligence (AI) model for clinical conversations, the Wall Street Journal reported Thursday (June 11).

Abridge’s platform listens to doctor-patient exchanges and automatically generates clinical notes. The model will run exclusively within Abridge’s platform. The deal gives Nvidia something it lacks: a large set of real physician-patient conversations recorded inside hospital settings. What makes that data valuable is precisely what makes it contested.

Abridge’s already operates across 100 health systems, including Kaiser Permanente, Mayo Clinic, Johns Hopkins and Yale New Haven Health. Private market research platform Sacra found that Kaiser Permanente alone has deployed the platform to 24,600 physicians across 40 hospitals and 600 clinics. Every one of those encounters produces usable conversational data.

Nvidia Vice President of Healthcare Kimberly Powell told the Journal the new model will bring clinical intelligence into an earlier stage of development than the industry has previously attempted.

Ambient AI Tools Became Healthcare’s Most Valuable Data Source

Ambient AI tools work by listening to conversations in the room and turning them into structured records without anyone typing. Hospitals bought them to cut the hours doctors spend on paperwork after patient visits. What became clear faster than most expected is that the platforms capturing those conversations were sitting on something more durable than a documentation business.

Abridge had already built its own speech recognition model because off-the-shelf tools weren’t accurate enough for medical conversations, Davis Liang, the startup’s director of applied science, told the Journal. The Nvidia partnership extends that same logic further.

Abridge CEO Shiv Rao told the Journal that even powerful general models need clinical shaping to work in practice: “Generic models are powerful, but clinical intelligence—it still has to be trained, it has to be shaped, and it has to be evaluated against real-world conditions.”

AI models trained on published medical literature miss what happens in the room. Clinical notes are compressed and shaped by billing requirements. Conversations carry what’s underneath: treatment options weighed aloud, patient history surfaced before it reaches the chart. That’s the signal the new model is built to learn from.

Patient Consent Laws Are Reshaping How Clinical AI Gets Built

Using that conversational data carries obligations that don’t apply to other AI training sets. Censinet reported in February that healthcare data breaches affected over half the U.S. population in 2024. Abridge trains on data with patient identifiers removed, but researchers found that stripped records “remain statistically tethered to identity through the very correlations that confirm their clinical utility.” Removing names doesn’t fully sever the link.

Patient consent adds another layer. The Texas Medical Liability Trust noted that several states now require explicit consent before AI systems can record and process clinical encounters. The American Bar Association reported that legal challenges around ambient scribes are reshaping what vendor contracts must cover when patient conversations become training data. Forvis Mazars noted that more than 250 AI-related bills were introduced across 46 states in the past year.

That regulatory pressure shapes how Abridge built the model’s infrastructure. Running it on Abridge’s own hardware rather than outside cloud services limits how far patient data travels, Liang told the Journal, and cuts costs.

Nvidia Builds Inside Abridge’s Walls as Microsoft Moves on Mayo Clinic

The model is expected to be ready later this year, the Journal reported. It will sit alongside other models Abridge already uses rather than replace them. Powell told the Journal the Abridge work is one example of how Nvidia’s models can be adapted for specific healthcare uses, from drug discovery to medical devices. The Journal also reported that Microsoft last week announced a similar collaboration with Mayo Clinic, drawing on Mayo’s clinical data.

Joon Lee, CEO of Emory Healthcare, which has deployed Abridge to more than 3,000 physicians, told the Journal he expects the technology to move toward real-time decision support as it matures.

Last June, Abridge announced a $300 million Series E led by Andreessen Horowitz, valuing the company at $5.3 billion. Nvidia’s venture arm has participated in multiple Abridge funding rounds.

For all PYMNTS AI and digital transformation coverage, subscribe to the daily AI and Digital Transformation Newsletters.