The Essential Need for AI Accuracy in Healthcare

Striving for Accuracy in the Age of AI

AI use cases in healthcare have expanded rapidly, with electronic health records (EHRs) evolving to help clinicians offload repetitive tasks and devote more time to patient care. But as machine learning, predictive analytics, and generative AI become more prevalent in healthcare IT, human oversight remains essential, especially to ensure AI accuracy in clinical documentation and other sensitive workflows.

Read on to see how AI can enhance clinical decision-making, streamline EHR workflows, and reduce administrative burden. You’ll also learn the safeguards needed to ensure accuracy, security, and privacy when using AI-driven healthcare tools.

How AI is used in healthcare IT today

Healthcare uses of AI continue to expand, particularly within AI-powered EHRs where accuracy and reliability are paramount:

• AI is helping structure large datasets within EHRs so that patient information is more usable for clinicians. Having structured data within a single, streamlined workflow also helps payers and providers improve interoperability across the healthcare ecosystem, whether it’s a large organization, small practice, hospital, lab, or ambulatory surgery center (ASC).
• Similarly, by synthesizing data across patient populations, AI can help clinicians track trends and uncover potential care or diagnosis gaps to meet value-based care (VBC) requirements.
• Using AI to extract clinical information relevant to billing and reporting can help save time, improve accuracy, and reduce claims denials as well as the overall administrative burden on practices.
• AI chatbot capabilities allow patients to securely message with providers to answer common questions, locate resources, or even schedule appointments for care. Additionally, some AI functionalities can help identify patients overdue for care and automate appointment reminders as a means of promoting patient engagement.
• Natural Language Processing (NLP) and generative AI tools, such as Ambient Notes, help transcribe clinical notes via text or voice inputs, summarize visits, and identify clinician diagnoses and ICD-10 codes to pass through for more efficient chart closure and movement from the patient encounter to billing.

While AI embedded in healthcare workflow can dramatically improve efficiency and data quality, its accuracy ultimately depends on two things:

1. The safeguards built into AI models, such as validation, transparency, and human-in-the-loop review.
2. How clinicians engage with the information, ensuring machine-generated insights are contextualized and confirmed before decisions are made.

When those safeguards are in place, AI helps EHRs become not just faster but more reliable, reducing manual reconciliation and supporting faster clinical decisions.

Why AI accuracy in healthcare is critical

As AI models and tools advance, their role in healthcare continues to grow and diversify. But as AI capabilities grow — especially in healthcare, where models touch sensitive patient data — clinicians must be vigilant in ensuring the accuracy of their inputs for AI models and establish oversight and processes of error reporting to help guide AI-driven outputs.

While AI can effectively surface and organize patient information, the true value of that data depends on a clinician’s expertise. The complexity and variability of patient care require nuanced judgment that integrates clinical expertise, ethical considerations, and contextual understanding—elements that AI systems currently cannot fully replicate. If clinicians rely solely on AI-generated data or decisions, without verifying its accuracy, it can lead to misdiagnosis, delayed care, or incorrect treatment plans — outcomes that may unintentionally widen care gaps and create additional work. Therefore, AI should be seen as a supportive tool, augmenting but never replacing the review and informed decisions of clinicians.

While AI can effectively surface and organize patient information, the true value of that data depends on a clinician’s expertise

We mentioned how NLP tools can help transcribe clinical notes during patient encounters so clinicians can focus more on the patient. But these tools alone may not always maximize outcomes, as recent research has demonstrated.

Researchers used AI models to help measure physician “fatigue” in clinical documentation. According to the study, the rate of model-identified fatigue in notes generated by large language models (LLMs) was 74% higher than notes written by the physicians themselves¹. That indicates just how imperative human oversight is when using and interacting with AI documentation.

It’s also important to acknowledge the potential biases in AI models, due in part to limitations in training data. For example, disparities in access to care can result in certain racial biases that affect predictive analysis conducted by AI models². Without having a refined understanding of these limitations, clinicians could potentially fail to provide appropriate care plans or better identify prospective patients in need of extra care.

From a payer perspective, there is a risk of misappropriation when AI-generated data is used without proper oversight. Errors or inconsistencies in how AI surfaces information may create complications when filing claims – which could lead to claim denials and a greater strain on already over-taxed medical practices.

Finally, accuracy looms large in how clinicians understand differences in public- or private-facing health programs. The AI model can help provide information for these subsets – but clinicians need to know how to apply or act on that information and do so while maintaining compliance.

Clinicians utilizing AI tools within EHR platforms typically do so in an effort to help offload repetitive tasks and get structured data that can help optimize data sharing between payers, providers, and patients. But these AI solutions should always be contextualized in how they complement clinician workflows, rather than replacing them. The buck for decision-making always stops with clinicians.

Safeguarding against AI inaccuracy

In some cases, AI utilization in healthcare has helped improve accuracy by surfacing human error. Past research found an AI system screening mammography reduced false positives for breast cancer in the U.S. by 5.7%, and false negatives by 9.4%³. The ability for AI to access and interpret large data sets can serve as a vital tool.

Nevertheless, AI models are not infallible. That’s why payers and providers should implement safeguards for AI accuracy to handle the complexities of the healthcare ecosystem and massive amounts of personalized data constantly being processed and analyzed. Some of these safeguards include:

• Ensuring AI is always used as a tool and having qualified professionals review clinician inputs and AI outputs before making critical decisions
• Constantly evaluating data surfaced by AI and looking for ways to increase AI training data by offering more diverse and representative datasets to help reduce bias
• Providing trainings for clinicians and staff to help them understand the limitations of AI models and best practices for working with AI outputs
• Encouraging reporting on suspected AI inaccuracies for investigation in the hopes of finding ways to improve AI-driven models
• Continuously monitoring and evaluating AI models in healthcare while opening feedback loops to help improve quality and performance of AI-driven models
• Providing defined roles and responsibilities for AI oversight within different organizations and creating policies for handling AI error or inaccuracy
• Developing quality assurance teams to work in collaboration, ensure proper communication across channels, assess potential risks, and analyze potential data-related errors

Putting systems in place for human oversight helps enhance AI accuracy. It can also give insights into how AI-driven platforms can evolve with better patient outcomes in mind. After all, that’s the ultimate objective for payers, providers, and patients alike.

At athenahealth, we strive to consistently improve the accuracy of our solutions by communicating openly with our customers and having robust testing processes. We want to offer AI services that help enhance practicality in healthcare, rather than introduce complications. Most importantly, we welcome input from physicians and payers – because we want to provide a holistic EHR that is usable, helpful, and adaptable. Keeping in touch with healthcare trends and looking for ways to expand access to data helps promote better accuracy within our AI-driven solutions.