Although government enthusiasm for artificial intelligence remains undimmed, both the medical profession and the general public remain to be convinced.
The enthusiasm for artificial intelligence at a government level remains undimmed. Barely a day goes by without another announcement championing technology.
At the beginning of April, the Medicines and Healthcare products Regulatory Agency (MHRA) secured a funding uplift to expand its AI Airlock programme, the UK’s first regulatory sandbox for Artificial Intelligence as a Medical Device (AIaMD).
It selected the first five technologies for the scheme in December 2024.
The Department of Health and Social Care (DHSC) has allocated £1.2 million per year for the next three years until 2029 to the programme. This will enable the programme to scale beyond the constraints of yearly financial cycles and, the government hopes, support more ambitious, longer-term testing models while helping to create a more sustainable regulatory pathway for future AI medical technologies.
“This additional investment will allow us to scale up and ultimately strengthen our ability to ensure that AI-powered medical devices can reach patients safely, efficiently and with the confidence of robust regulatory oversight,” said James Pound, executive director of innovation and compliance at the MHRA.
In a blog post, Jennifer Dixon, chief executive of the Health Foundation and who is on the National Commission for the Regulation of AI in Healthcare, comments on how to seize growing opportunities now and ahead of us, in particular from technology and artificial intelligence.
She sees AI as the great leveller and wants to ensure AI is safe and accurate for all.
“The big task ahead is to get better at demand signalling what are the highest priority technologies to be developed and tested, test them better, faster and more cheaply, and work out effective ways to spread them safely,” she wrote.
Struggling to deploy
All good news, but as Healthcare Today has reported in the past, there are several problems, notably the barriers to deployment as well as broad scepticism.
First, the tech problems. Research from KTSL, an independent service management and integration specialist, highlights the transitional state of AI agent adoption in the UK. The independent survey of UK enterprises across retail, pharmaceutical, and financial services finds that while most enterprises are enthusiastically adopting AI agents, there are still gaps to bridge before deployments reach full fruition.
Almost nine in ten (88%) firms are actively deploying AI agents, and 73% say that the ability to deploy them within existing systems is a priority. More to the point, the survey also found that 71% of AI agent deployments are meeting or exceeding ROI expectations.
But, and it is a significant but, 56.5% of all respondents indicated that they were facing a number of barriers to meaningful progress, including a lack of internal knowledge or skills (29%), insufficient business case, lack of quality data and no suitable technology partner (all 28%), highlighting an issue for some enterprises in identifying and deploying the right use cases.
“Enterprises are clear about the value of AI agents, but our research highlights a market in transition,” said Aaron Perrott, chief technical officer at KTSL. “Confidence is high, yet many are struggling when it comes to deploying agents in practice and realising tangible business impact.”
Lack of reliability
But the biggest issue remains a wider scepticism both from doctors and the public. As Pierluigi Gardella, director of industrial & IoT system engineering and marketing – healthcare solutions at NXP Semiconductors, wrote earlier this month, “Edge AI enables faster and better-informed decisions, but treatment, care, and accountability all stay firmly with healthcare professionals”. There are concerns that doctors will be on the hook for AI’s decisions.
On top of this, there has been little guidance about how these tools should be used.
A Nuffield Trust and Royal College of General Practitioners survey in January found that although 28% of GPs say they use AI tools to help with their work, there is confusion about which tools to use and how, due to a lack of consistent national guidance or formal AI training.
The icing on the cake is that – at the moment – they are not as reliable as they should be. More than a third of medical references generated by some widely used artificial intelligence (AI) platforms may be fabricated, according to a new study published in the journal The Annals of the Royal College of Surgeons of England.
The research found that popular AI chatbots answering common surgical health questions sometimes produced hallucinated citations – references to sources that do not exist. As more patients turn to AI tools to understand symptoms, explore diagnoses and seek medical advice, the authors warn that fabricated references undermine users’ ability to check whether information is accurate or evidence-based.
“As AI evolves, improving transparency, accountability and the reliability of references must be a priority to ensure patient care is enhanced, not compromised,” said Tim Mitchell, president of the Royal College of Surgeons of England.
Public concerns
Perhaps most significantly of all, the concerns from the public are not shifting. A recent report from AXA Health highlighted an AI health paradox, where the same tools are both reassuring people into delaying care and alarming others into seeking unnecessary appointments.
It found that 59% of people who check symptoms with AI have delayed speaking to a healthcare professional because the tool reassured them. And the same percentage say AI made them more worried, leading them to seek help, which they later discovered they didn’t need
“Our findings show a complicated truth: AI can increase anxiety or give false reassurance, but it can also give people clarity and confidence. The challenge is that people are navigating this alone,” said Heather Smith, chief executive of AXA Health.
This type of concern has led to hesitancy and can be seen in detail in aged care. Data from AI voice companion Sentai has found that 54% of families are hesitant to introduce technology into the home, despite 79% of people believing that technology could help support an elderly relative.
The firm believes that the issue isn’t awareness, it’s perception. Families are being held back by persistent myths around AI, privacy and independence.
“Many families assume technology is something to introduce later, when there’s a crisis or clear decline,” said Peter Otto, chief product officer at Sentai. “But our research shows the opposite – the earlier support is introduced, the more naturally it fits into daily life,” he added.
It is easy to understand government enthusiasm. Any government is attracted to the latest shiny technology, especially one that promises money saved. But there is a long way still to go before that same enthusiasm is matched by doctors and the public.
