2025 AI Scribe Compliance Report

The first large-scale comparison of AI-scribed and human-authored psychiatry and therapy notes. Key finding: AI-scribed notes scored 0.8% lower on compliance measures, proving AI scribes are an efficiency tool—not a compliance solution.

By Zach Rosen

Executive Summary

This report provides the first comparison of AI-scribed and human-authored notes, using Brellium's compliance auditing platform to assess thousands of psychiatry and therapy charts.

What are AI medical scribes? AI medical scribes apply speech recognition and natural language processing to convert clinician-patient conversations into structured visit notes. Adoption surged in 2022-2023 as advances in conversational AI merged with mounting demands to reduce clinician burnout.

Brellium's analysis shows that these efficiency gains do not translate into more compliant notes. The analysis of thousands of psychiatry and therapy notes shows that AI-scribed documentation is not more compliant than human-authored notes. On average, AI-scribed notes scored 0.8% lower on standardized compliance and quality checks.

For healthcare organizations, the lesson is clear: AI scribes document visits, but they don't eliminate compliance obligations.

The Evidence to Date

Early studies highlight real efficiency gains:

• ~20% reductions in documentation time
• ~30% fewer after-hours time spent working in the EMR
• 38% reduction in burnout (assessed using the Stanford Professional Fulfillment Index)

Yet questions of accuracy and compliance remain unresolved. Physicians frequently note the need to edit AI-generated notes for clarity or factual correction. While outright "hallucinations" appear rare, virtually no published research has systematically tested AI-generated notes against compliance, coding, or billing standards.

Brellium's Analysis

Brellium ran the first large-scale study comparing AI-scribed and human-authored notes in psychiatry and therapy. Thousands of charts were assessed using Brellium's compliance platform, which evaluates documentation against payer and clinical requirements.

The auditing framework includes:

• Completeness: Verification that all required criteria — including medical history, therapy interventions, treatment plan, and decision-making reasoning — are present and properly documented
• Compliance with Payer Requirements: Verification of accurate CPT coding, time and duration of service, evidence of medical necessity, required signatures/attestations, and individualized narratives
• Quality Indicators: Evaluation of note clarity, organization, and relevance, including checks for individualized clinical reasoning and avoidance of copy-and-paste content

Audit Performance: Human vs. AI

Our analysis found that AI-scribed notes scored 0.8% lower on average than human-authored notes across compliance and quality measures. This means that while AI scribes may save time and reduce clinician burden, they do not produce higher-quality documentation.

The implication is straightforward: Healthcare organizations already audit clinician-authored notes. Since AI-generated notes perform no better, they too must be audited.

Limits of AI Medical Scribes

AI scribes record what was said, but they do not judge whether the right clinical questions were asked, whether the encounter followed required protocols, or whether care decisions were appropriately justified. A perfectly transcribed note may still fail to establish medical necessity.

AI scribes therefore represent an efficiency innovation, not a compliance solution. Auditing remains necessary.

Conclusion and Recommendations

AI scribes deliver efficiency, but they don't change compliance requirements. Every note — whether authored by a clinician, a human scribe, or an AI system — must meet payer, regulatory, and quality standards.

The organizations setting the bar for responsible AI use adopt AI scribes while holding them to the same compliance standards as human documentation. By combining AI scribes with auditing tools like Brellium, healthcare organizations can reduce compliance risk, protect revenue, and ensure documentation meets clinical standards.

Notable results from Brellium customers:

• Lightfully Behavioral Health reduced chart auditing time by 87%
• DoubleCare ABA lowered QA-related costs by over 80%
• Cerebral improved compliance with insurance requirements by 87%