Ontario auditors find doctors’ AI note takers routinely blow basic facts

The Ontario Office of the Auditor General has found that 9 out of 20 AI note-taking systems used by healthcare providers routinely produce inaccurate or fabricated patient records, raising concerns about their safety and reliability.

The audit evaluated 20 AI systems approved for use in Ontario’s healthcare sector, using simulated doctor-patient recordings. It revealed that nine systems fabricated information, such as suggesting treatments or symptoms not discussed during the recordings. Twelve systems inserted incorrect drug information into patient notes, and 17 missed key mental health details discussed during consultations.

While OntarioMD, a support group for physicians, recommends manual review of AI-generated notes, none of the approved systems include mandatory attestation features to verify accuracy. The evaluation process itself was criticized for giving disproportionate weight to criteria unrelated to accuracy, such as vendor presence in Ontario, which accounted for 30 percent of the score, while accuracy contributed only 4 percent.

Why It Matters

The findings raise serious questions about the safety and effectiveness of AI tools used in medical documentation. Inaccurate or fabricated records can lead to misdiagnosis, inappropriate treatment, and patient harm. The report suggests that current evaluation standards may inadequately prioritize accuracy and safety, potentially allowing subpar systems to be approved for clinical use.

Background

The use of AI for medical note-taking has expanded rapidly in Ontario, with more than 5,000 physicians participating in the program. The initiative aims to improve efficiency but has faced scrutiny over the quality of AI-generated documentation. Previous studies have shown that large language models often produce incorrect medical information, but this is the first formal audit highlighting systemic issues in AI scribe systems used in healthcare.

“Inaccurate weightings could result in the selection of vendors whose AI tools may produce inaccurate or biased medical records or lack adequate protection to safeguard sensitive personal health information.”

— Office of the Auditor General of Ontario

“More than 5,000 physicians in Ontario are participating in the AI Scribe program and there have been no reports of patient harms associated with the technology so far.”

— Ontario Ministry of Health spokesperson

What Remains Unclear

It remains unclear whether the issues identified are widespread across all AI scribe systems or limited to the evaluated sample. The impact on patient safety and clinical outcomes is still being assessed, and the Ministry has not yet announced specific corrective actions or changes to evaluation processes.

What’s Next

The Ontario Ministry of Health is expected to review the audit findings and may revise its evaluation criteria for AI systems. Further testing and monitoring are likely, along with possible updates to regulations or mandatory review procedures for AI-generated medical records.

Key Questions

Are AI note-taking systems currently safe to use in Ontario healthcare?

While no reports of patient harm have been confirmed, the audit raises concerns about the accuracy and safety of these AI systems, suggesting caution and the need for manual review.

What specific errors did the Ontario audit find in AI-generated medical notes?

The audit found that many systems fabricated information, inserted incorrect drug details, and missed key mental health issues discussed during patient consultations.

Will the government change how AI systems are approved for healthcare use?

The Ministry of Health is expected to review the audit results and may adjust its evaluation process, emphasizing accuracy and safety criteria more heavily.

Could these errors lead to patient harm?

Potentially, yes. Incorrect or fabricated medical records can cause misdiagnosis or inappropriate treatment, though no direct harm has been reported yet.