As AI gains momentum and captivates the media and the general public, physiologists and AI specialists have a crucial responsibility to collaborate, combining their complementary skillsets to ensure the development of safe, robust, and equitable AI tools, that do not perpetuate health inequalities."
Dr Richard Siow, Director of ARK and Chair of the Steering Committee that produced the report
29 June 2023
Report warns of potential AI has to perpetuate health inequalities
Dr Richard Siow, Director of Ageing Research at King’s, presented a landmark report exploring the role of Artificial Intelligence (AI) in healthcare to the Houses of Parliament on Tuesday 27 June.
‘From ‘Black Box’ to Trusted Healthcare Tools’, created by members of the Physiological Society, also flags the necessity of integrating physiological evidence into the application of AI to prevent health inequalities.
The report’s primary focus is exploring ways to establish the UK as a leading ‘science superpower’ in AI and health. A crucial aspect of this ambition is the safe and responsible deployment of AI tools in healthcare. However, The Society raises concerns about potential challenges and risks, ranging from inaccurate diagnoses to the exacerbation of health inequalities due to biased data and access, which could result in patient deaths.
AI healthcare tools are often developed, approved, and implemented without sufficient physiological input, the report states. Based on consultations with over 30 experts in the field, it proposes that limited incorporation of physiological evidence in the development of AI tools undermines trust, poses applicability challenges, and, at worst, lead to the identification of spurious correlations that could potentially harm patients.
The report argues for greater political commitment to regulating the use of AI within healthcare. The first step would involve establishing a Physiology & AI Framework which will provide improved guardrails for AI in healthcare. This would need to be created by stakeholders across healthcare as well as experts in AI and physiology.
Physiologists will be crucial for this framework to ensure AI has the required understanding of physiology to understand human health conditions, and that it can effectively interpret and contextualise medical data.
“I can see the transformative power of AI in healthcare. However, its potential can only be fully realised when supported by the scientific rigour of physiology, enabling a deep understanding of how the human body functions in health and disease. The Physiological Society’s creation of an AI & Physiology Framework is an important step to informing this.”
This report from The Physiological Society highlights an important issue in the use of current machine learning technologies – that while they are already bringing amazing improvements to healthcare, their lack of domain knowledge can potentially sometimes limit their applicability and value."
Professor Michael Luck, King's Institute of Artificial Intelligence
"The report advocates for what is a “responsible AI” approach, keeping human experts in the loop, prioritising safety and trust, and ensuring that systems are fair and explainable. Only in this way will we be able to ensure that the benefits of AI are truly realised."
In this story
Director, Ageing Research at King's
Visiting Professor of Computer Science