Epic Launches Open-Source AI Validation Tool for Health Systems to assist healthcare institutions in testing and keeping an eye on AI models. This free software package for AI validation was announced on Wednesday, and it is currently accessible on GitHub. The announcement was announced to Fierce Healthcare by Corey Miller, vice president of research and development at Epic, who clarified that health systems can include the technology into their electronic health record (EHR) systems.
The validation of AI models from Epic and other developers is supported by this tool. According to Epic executives, this open-source platform will allow enterprises to adopt additional standards in addition to the AI validation features as AI best practices emerge. Miller underlined that this is Epic’s first experiment with open-source tools, making it unrestrictedly available to a worldwide audience.
“Publishing on GitHub makes it fully accessible to everyone; we don’t control any locks or keys. We can’t wait to explore this world. In an interview, Miller stated, “It seems sense that a tool designed to guarantee the equity of health AI will be accessible to the public and welcome contributions from all over the world.
Epic unveiled plans for this AI validation package earlier in April with the goal of enabling healthcare institutions to assess and track AI models on an ongoing basis. Senior vice president of R&D at Epic, Seth Hain, explained that the suite relieves data scientists of the most labor-intensive aspects of validation by automating data collecting and mapping to produce metrics and analysis in almost real-time.
Although Epic intends to increase the tool’s functionality, the current version does not validate generative AI models. Epic’s software suite is intended to be used for local AI model validation by the Health AI Partnership (HAIP), which counts Kaiser Permanente, Duke Health, and the Mayo Clinic among its members.
Additionally, Epic will work with HAIP and the University of Wisconsin to investigate how to apply this validation tool with a prediction model. Miller was upbeat about the partnership’s ability to direct the tool’s development based on feedback and real-world use.
Healthcare companies have been using machine learning and predictive AI models for almost ten years. But additional difficulties have emerged with the introduction of generative AI and large language models (LLMs). These cutting-edge models are eager to be implemented by health systems for applications like clinical note automation and medical record summarization. Nevertheless, in order to guarantee accuracy, functionality, and safety, they are still creating efficient techniques for validating these AI models.
Hain stated in April that the AI software suite had easily navigable reporting dashboards that are updated automatically. These offer analysis segmented by age, sex, and race/ethnicity, among other factors. The program makes it easier to integrate additional AI models in the future by including a uniform data structure and monitoring template.
The open-source solution, according to Epic executives, enables healthcare organizations to use patient data and workflows to do AI validation within their EHR systems. It may monitor subsequent results and actions, allowing for in-depth examination of various patient groups. This feature, which looks at model performance across protected classes like age, sex, color, and ethnicity, is essential for guaranteeing AI equity.
“The technology organizes all of this data into easily understood representations, much like a funnel. You can delve farther into any one piece of data with a few clicks and single points,” Miller said. “The tool is targeted primarily at data scientists and clinicians, but we hope that it will be easy enough to understand that a clinician without a data science background can also dig in and learn where the tools are fair and equitable, and where they might need to change the way they are delivering care to improve equity.”
A possible conflict of interest has been brought up by some in relation to Epic, a significant provider of health IT, and their development of AI validation tools. In response, Miller said, “This is totally independent of any of our code.” It is intended to function with both predictive models that we developed and models that we did not contribute to. “At Epic, we like to live by the motto ‘Do good, have fun, make money,’ and we feel that this one really fits in the ‘Do good’ part,” he continued. It seems like a chance to use our knowledge in this area to help the international community assess and securely apply artificial intelligence read more about Epic Launches Open-Source AI Validation Tool for Health Systems at https://www.fiercehealthcare.com/. ( article source)
