athenahealth's Intervention Risk Management (IRM) for Predictive Decision Support Intervention (PDSI)

The Assistant Secretary for Technology Policy and Office of the National Coordinator for Health Information Technology (“ASTP/ONC”) through its Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing (HTI-1) Final Rule requires certified health information technology (“health IT”) developers to meet specific obligations regarding the use of artificial intelligence (“AI”) in their certified technology products by December 31, 2024. Specifically, ASTP/ONC regulates Predictive Decision Support Interventions (“PDSI”), which means technology that “supports decision-making based on algorithms or models that derive relationships from training data and then produces an output that results in prediction, classification, recommendation, evaluation, or analysis" that is “supplied as part of the health IT developer’s Health IT module”.

The HTI-1 Rule requires that each PDSI is subject to intervention risk management (“IRM”) practices, which include risk analysis, risk mitigation, and governance. While athenahealth has instituted internal policies and procedures governing the use of certain types of AI for years, athenahealth reviewed and updated internal AI policies and procedures to ensure alignment with HTI-1 and the National Institute of Standard and Technology (“NIST”) AI Risk Management Framework. athenahealth designed its IRM practices to align with ASTP/ONC’s FAVES principles – fair, appropriate, valid, effective, and safe. In addition to the FAVES principles, as required by HTI-1, athenahealth’s IRM practices assess a functionality’s reliability, robustness, intelligibility, security, and privacy.

PDSI at athenahealth is subject to multistage, cross-functional review and governance. To comply with company standards, athenahealth employees developing functionality that includes PDSI are required to review and comply with all applicable policies and procedures, including completing risk assessments to identify ethical, security, data privacy, equity, operational and regulatory risks in new functionality.  Once risks are identified, strategies to control and mitigate such risks are implemented to adjust or safeguard algorithms, data input, or user interfaces. If critical risks are identified, the proposed PDSI is escalated to a responsible AI council staffed with senior subject matter experts to review and assess the functionality and its necessity.

athenahealth’s IRM practices review and address AI model trustworthiness, development, testing, deployment, data handling, and decommissioning through risk-based requirements on performance measurement, AI model monitoring, incident management, and internal audits. Monitoring processes are enforced to detect new or emerging risks post-implementation. Relevant stakeholders track, evaluate and internally report adverse incidents related to PDSI and inform, as needed, PDSI users and relevant stakeholders. All AI models will be subject to review and monitoring to ensure compliance with internal policies and regulatory requirements.