Cutting Through the AI Hype in Healthcare: A Realistic Examination

By Jocelyn Keegan, Payer Provider Practice Lead

When a revolutionary technology emerges, there inevitably follows a period of calibration when its potential is weighed against the reality of its performance. This pattern, well documented by the Gartner Hype Cycle, is particularly pertinent to the use of artificial intelligence (AI) in healthcare. We want to provide a balanced exploration of AI's applications and limitations within this critical sector.

We are neither alarmists nor blind optimists. We’re just a team of pragmatists who acknowledge AI as a significant and revolutionary element of our technological toolkit, with great promise for the future of healthcare. By examining both the successes and shortcomings of currently available AI, we aim to offer a nuanced perspective that can aid healthcare professionals and policymakers in their decision making.

There are various use cases in which AI has historically shown promise, such as in predictive analytics for population health and optimized workflows to gain operational efficiencies, but there are also scenarios that have raised concerns and ethical questions. AI developments in healthcare are fast moving, so any evaluation of current capabilities needs to be revisited frequently to identify changes and new developments. Additionally, we consider the viewpoints of policymakers, whose decisions are crucial in shaping the regulatory environment that governs the use of AI in healthcare.

Let's examine some practical insights into how AI may fit into your technology stack and contribute to your organization’s interoperability roadmap. 

AI in Healthcare: Understanding Its Position on the Gartner Hype Cycle

As AI continues to evolve within the healthcare industry, its position on the Gartner Hype Cycle, as referenced in the Hype Cycle for Healthcare Data, Analytics and AI, 2023 Report, offers critical insights for stakeholders. According to Gartner's latest analysis, AI, particularly large language models and generative AI, are making significant inroads, although their most transformative applications are still several years away from mainstream adoption​​.

What's Working Well

Current applications of AI in healthcare primarily focus on reducing administrative burdens, such as clinical documentation, which has shown significant near-term tactical value. These applications align well with reducing strain on healthcare providers and addressing the pervasive issue of clinician burnout by potentially increasing workforce capacity without the need for additional human resources. This positive impact is mirrored in the rapid uptake and support at events like HIMSS24, where the practical use of AI in easing administrative loads was prominently featured​​.

Areas of Caution

Despite these promising developments, the road to broader, high-impact clinical applications of AI are fraught with complexities. Higher risk scenarios, such as those involving clinical decision making and direct patient care, present significant technical and ethical challenges. These applications require meticulous development to ensure safety, efficacy and ethical integrity, likely delaying their widespread adoption. Use of AI in such high-stakes environments necessitates a cautious approach, balancing innovation with potential risks to patient safety​​.

Ethical Considerations and Risk Management

The hype surrounding AI also brings to light numerous ethical considerations, particularly related to data privacy, bias in AI algorithms and overall reliability of AI systems in clinical settings. The industry is still navigating these issues, with a significant portion of AI capabilities being explored under controlled conditions to mitigate risks before full-scale implementation​​.

AI's trajectory in healthcare, as plotted on the Gartner Hype Cycle, illustrates a technology oscillating between the peak of inflated expectations and the trough of disillusionment. AI has experienced these hype cycles before, but the magnitude of recent AI advances makes this time unique. While administrative efficiencies driven by AI are being realized in the short term, the more profound clinical impacts are developing at a slower pace, understandably slowed by ethical, technical and operational challenges. The goals of improving outcomes and reducing costs are necessarily being balanced with the need to ensure patient safety and privacy. Stakeholders must navigate this landscape with care, promoting AI adoption where it is effective while continuously evaluating its broader implications in patient care and clinical outcomes​​.

AI in Healthcare: Perception, Education and Need for Transparency

Integration of AI into healthcare has elicited differing reactions among providers and patients, underlining the complex dynamics of implementing this unique technology in a high-stakes environment. This section explores the perceptions of AI among healthcare professionals, the crucial role of education in its adoption and the paramount importance of transparency in AI applications.

Provider and Patient Perspectives on AI

Reaction to AI in healthcare settings varies significantly among different groups of healthcare providers. A striking example is the response from nursing staff at Kaiser Permanente, where nurses have expressed strong opposition to the use of AI in clinical settings. This sentiment was manifested in a strike highlighting concerns over patient safety, job security, and the ethical implications of AI in care delivery. Such protests underscore the apprehension frontline healthcare workers may feel toward rapidly evolving technologies that are perceived as potentially intrusive or even harmful to patient care​​. We anticipate that these concerns, amplified by potential elimination of jobs, will be echoed across healthcare labor markets worldwide.

Conversely, a recent survey by Wolters Kluwer suggests a shift in perception among physicians, with over two-thirds now viewing generative AI technologies as beneficial to healthcare. This change is attributed to increased exposure to AI tools, improved understanding of their capabilities, and the advantages of AI in enhancing clinical decision making. This evolving acceptance appears to require transparency regarding the origin of the AI-generated content and data used for AI model training and performance measurement​​.

The Need for AI Education and Interaction

The differing views amongst clinicians on the utility and safety of AI technologies highlight the need for effective communication with employees and physicians, comprehensive education, and thorough testing where testing results are shared with employee and physician leadership. Education programs aimed at demystifying AI functionalities and demonstrating their clinical benefits could alleviate some concerns. Similarly, rigorous testing environments where providers can interact with AI tools and witness their development and outputs firsthand could foster a deeper understanding and trust in these systems. The threat of layoffs or staff reductions due to AI automation will be a persistent issue that needs to be managed hereon as AI becomes more capable and visible.

Interplay with Consumer Use of AI

While healthcare practitioners and organizations navigate new AI capabilities slowly and carefully, some consumers will not be so conservative. Driven by their desire for answers to their health issues and questions, consumers are already accessing AI applications such as ChatGPT, Claude, Gemini and Microsoft’s CoPilot. Patients equipped with symptoms and diagnoses provided by these AI models are both empowering the patient-physician conversation as well as confusing it. This dynamic must be considered by organizations developing their AI strategy, and anticipated in physician and other healthcare provider training. 

The Imperative for Transparency

Transparency in the use of AI software and services in healthcare is not just a preference but a necessity. For AI tools to be integrated successfully into clinical workflows, they must not only be effective but also understandable by those who rely on them. In the AI world, this is called “interpretability” and “explainability.” AI systems and their implementers should provide clear explanations for their recommendations and decisions. Such transparency is crucial for building trust among users and for ensuring that AI-enhanced decisions are well received and integrated by healthcare professionals.

Moreover, the rapid pace of development in AI technologies can sometimes be a double-edged sword. While it allows for continuous improvement and adaptation to new challenges, it also means that tools can become quickly outdated or may change significantly shortly after their implementation. This fast evolution may hinder trust building as users find it hard to keep up with or adapt to frequent updates. 

As AI technologies continue to advance and proliferate within the healthcare sector, balancing innovation with such user-centric considerations as education, testing and transparency will be essential. By addressing these key areas, healthcare organizations can enhance acceptance and effectiveness of AI applications, ultimately leading to improved patient outcomes and provider satisfaction. In the next section, we will delve into policy implications of these rapid technological changes and the challenges they pose for regulation and oversight in healthcare.

AI in Healthcare Policy: Navigating Federal and State Regulations

The integration of AI in healthcare is not only transforming clinical practices but also significantly influencing health information technology (health IT) policy at both federal and state levels. As these technologies evolve at a rapid pace, policymakers are striving to keep up with regulations that ensure AI's safe, equitable and effective integration into the healthcare ecosystem.

Federal Initiatives: HTI-1 Rule

At the federal level, the Office of National Coordinator for Health IT (ONC) Health Data, Technology and Interoperability (HTI-1) final rule is a pioneering effort to embed transparency, accountability and interoperability within healthcare AI applications. This rule mandates that developers of certified health IT products disclose comprehensive performance information of predictive decision support interventions (DSIs). These attributes cover details on the development, performance and inherent risks of predictive algorithms, aiming to equip healthcare providers with the necessary information to assess these tools' fairness, validity and safety.

Key components of the HTI-1 rule include:

• • Transparency and Disclosure: Developers must provide detailed information about the algorithms' operation, including their development process and any associated risks, ensuring users can make informed decisions.
  • Risk Management: There is an emphasis on continual risk assessment and mitigation, requiring developers to publicly share summaries of their risk management practices, thus fostering ongoing accountability.
  • Interoperability Standards: Adoption of the United States Core Data for Interoperability (USCDI), Version 3, facilitates broader and more efficient health information exchange across diverse IT systems, enhancing system compatibility and data fluidity.

State-Level Policy Dynamics

Nearly 100 pieces of legislation are currently under consideration at state levels, addressing various aspects of AI in healthcare. These proposals vary widely but generally focus on several critical areas:

• • Transparency and Accountability: Policies are being formulated to enhance the clarity and openness of AI applications in public services, necessitating detailed reporting on AI methodologies and their impacts.
  • Ethical Standards and Fairness: There is a growing call for setting ethical standards to prevent AI technologies from perpetuating biases or discrimination, especially in sensitive areas like healthcare and employment.
  • Regulation and Oversight: Some states are considering the establishment of advisory boards to oversee AI development and application, ensuring adherence to ethical standards and state laws.
  • Education and Workforce Development: Initiatives are underway to improve workforce skills in AI technologies and to educate the public on AI's implications.
  • Consumer Protection: Policies are increasingly focusing on safeguarding consumers from potential harms related to AI, particularly concerning data security and privacy.

Special Focus: Impact on Life Sciences

Life sciences companies, especially those with digital solutions arms, need to pay close attention to these evolving policies:

• • Enhanced Transparency: Companies should ensure detailed disclosure of the algorithms used in predictive DSIs, including information about development, biases, and intended uses.
  • Interoperability Compliance: Updating systems to align with new interoperability standards such as USCDI and Fast Healthcare Interoperability Resources is crucial for seamless data exchange.
  • Robust Risk Management: Implementing comprehensive risk management strategies to maintain the validity, safety and privacy of AI applications will be critical.

For manufacturers focused more on traditional roles within life sciences rather than digital or interoperability aspects, state policies could impose:

• • Greater Disclosure Requirements: There may be increased demands for transparency in how AI is used within drug development processes, demanding clearer explanations of decision-making processes to regulators.
  • Stricter Compliance and Ethical Standards: Ensuring AI systems do not perpetuate biases, particularly in drug development and patient testing, will be crucial.

The rapid development of AI technologies presents unique challenges and opportunities in healthcare policy. Policymakers are indeed making valiant efforts to keep pace with technological advancements, and it's more crucial than ever for stakeholders to engage actively during public comment periods. As AI continues to permeate various facets of the healthcare system, comprehensive and forward-thinking policies will be vital to harness its potential responsibly and equitably.

Industry Initiatives on AI in Healthcare: Collaborative Efforts for a Complex Landscape

The complexity and rapid evolution of AI in healthcare necessitates a collaborative approach to fully leverage its benefits and mitigate associated risks. No single organization can tackle the vast potential and intricacies of AI alone. Engaging with industry groups, whether through one-off events, working sessions or ongoing initiatives, is essential for unraveling AI complexities and developing best practices.

CHAI: A Pioneering Industry Consortium

The Coalition for Health AI (CHAI) exemplifies a dedicated effort to address AI challenges and opportunities in healthcare. Founded in March 2023, CHAI aims to create a federated network of assurance labs and has quickly established itself as a pivotal forum for discussions on real-world AI use cases. Its ambitious scope includes developing guidelines and educating the industry and patients on how AI is being used, emphasizing the need for transparency akin to a "nutrition label for technology."

CHAI's founding partners include leading healthcare organizations, professionals and patient advocacy groups, highlighting its comprehensive approach to stakeholder engagement. This coalition not only facilitates a deeper understanding of AI applications but also pioneers the development of industrywide best practices that address pivotal concerns such as ethics, transparency, and interoperability. We’re proud to share that Point-of-Care Partners (POCP) is the program management organization for CHAI, and we look forward to supporting this collaborative community and continuing to learn about the challenges and opportunities of AI.

NCPDP's Ideation Lab: Fostering Innovation Through Collaboration

Another significant event is the Ideation Lab organized by the National Council for Prescription Drug Programs (NCPDP). This event gathers experts and leaders across the healthcare spectrum to discuss and shape the future of healthcare technologies, with a focus on AI. The 2023 session, themed "Driving Healthcare through AI: The Opportunities and the Challenges," was facilitated by AI expert Ed Daniels, a long-time member of the POCP team. This gathering helped establish a foundational understanding of AI's current state, its potential benefits and the challenges it poses.

The Ideation Lab was a platform for brainstorming and strategizing on how to effectively integrate AI into healthcare. By involving a wide range of stakeholders, from developers to end users, the Lab ensures diverse perspectives are considered in shaping actionable plans that align with broader healthcare goals.

The complexity and scope of AI in healthcare are vast, requiring more than individual efforts to harness its full potential and ensure its safe integration. Industry initiatives like CHAI and NCPDP's Ideation Lab are crucial as they provide structured environments for collaboration, discussion and strategic planning. These forums not only facilitate the sharing of insights and best practices but also play a vital role in shaping policies and standards that govern AI's application in healthcare.

The journey of AI in healthcare is marked by a complex interplay between rapid technological evolution, policy adaptations, shifting stakeholder perceptions and dynamic industry initiatives. Positioned along various stages of the Gartner Hype Cycle, AI's integration into healthcare reflects a landscape where expectations are continually recalibrated against real-world outcomes. This fast-paced evolution demands not only agile policy frameworks that can address ethical, privacy, and efficacy concerns but also a deep engagement with all healthcare stakeholders to align perceptions with the evolving capabilities of AI. Industry initiatives play a pivotal role in this ecosystem, fostering collaboration and sharing best practices that help navigate the intricacies of AI deployment. As we move forward, the collective efforts of these stakeholders will be crucial in harnessing AI’s potential responsibly, ensuring that its integration into healthcare not only meets current needs but also anticipates future challenges and opportunities. For those seeking to deepen their understanding of this vibrant landscape or to craft strategic approaches to AI integration, POCP offers expertise and guidance tailored to navigate these complexities effectively. Reach out to our Business Strategy Lead, Brian Dwyer to set up time to discuss your challenges.