Corti Introduces New Benchmark to Raise the Standard for Evaluating AI-Generated Clinical Notes

COPENHAGEN – October 14, 2026 – Corti, a frontier lab for clinical-grade AI, today unveiled its Text Generation Benchmarking Whitepaper, introducing a new benchmark and multilingual dataset for evaluating AI-generated clinical notes. Designed to bring greater rigor and consistency to how clinical note quality is measured, the benchmark gives developers and healthcare organisations a clear way to compare AI scribes, understand where they perform well or fall short, and make better-informed decisions before deploying them into clinical workflows.

Despite healthcare’s rapid adoption of AI scribes and more than $2 billion in documented funding across the sector, the industry still lacks a standard for evaluating clinical note quality. Without a unified approach, healthcare providers and developers risk making deployment decisions based on incomplete evidence. Even when a note appears accurate and is supported by the recorded encounter, relevant clinical details can still be omitted, while differences in datasets, system configurations and evaluation methods make AI systems difficult to compare fairly. In healthcare, those weaknesses matter because omissions or unsupported details can enter the patient record, shape future clinical decisions and ultimately affect the care a patient receives.*

The framework addresses this gap by assessing what AI-generated notes retain, omit and add. It measures groundedness, completeness, conciseness and broader clinical quality to provide a more comprehensive view of note performance. 

Alongside the framework, Corti released MedConv, a multilingual dataset of 400 synthetic clinical encounters in English, French, Danish and German, designed to let different systems be tested against the same underlying inputs. This creates a more consistent basis for comparing clinical note quality.

To assess whether purpose-built clinical AI infrastructure could match or outperform ambient documentation tools built on general-purpose AI platforms, Corti used the new framework to evaluate 412 clinical encounters across MedConv and ACI-BENCH, comparing Corti Symphony with Heidi and Tandem Health. The benchmark showed AI-generated notes with general-purpose AI platforms can contain largely supported information yet still miss key clinical content. Groundedness (note details supported by the conversation) scored ~90–98%, while completeness (encounter details captured) ranged from ~62–77%.

Across the datasets evaluated, Corti produced the most complete notes. On MedConv’s English dataset, Corti achieved a completeness score of 73%, compared with 64.6% for Heidi and 62.9% for Tandem Health, and also scored highest on completeness across the Danish and German datasets. Although Corti performed strongest, the results show there is room for improvement across all three systems. Importantly, completeness scores should not be interpreted as the equivalent percentage of clinically important information missing from a note.

Lars Maaløe, CTO and co-founder of Corti, said:“If AI is going to write the medical record, we need to hold it to clinical standards, not just impressive demos. With this research, we’re making clinical AI quality measurable, comparable, and reproducible. Because in healthcare, what an AI leaves out can matter just as much as what it gets right.”

The benchmark also found that note quality can change materially depending on how a system is configured, reinforcing the need to evaluate performance across multiple dimensions rather than rely on a single measure of quality. 

The full whitepaper is now available here: https://arxiv.org/abs/2610.08161

_{* Funding calculation: Based on publicly reported funding totals for 15 companies operating in ambient and AI-assisted clinical documentation: Abridge, Ambience, Heidi, Suki, Tandem Health, Nabla, Eleos Health, DeepScribe, Freed, Augmedix, Robin Healthcare, voize, Lyrebird Health, TORTUS and Upheal. Combined documented funding totals approximately $2.23B–$2.55B, depending on reported funding figures for companies where sources differ. The release therefore uses the conservative formulation “more than $2 billion.” Non-dilutive financing is excluded where identified.} 

‍