Add us as a preferred source
The debate over how useful AI in health and social care settings really is has ramped up. But despite active PR drives, not everyone is on board.
Artificial Intelligence (AI) is increasingly creeping into our daily lives – and doctors’ workloads are far from untouched.
It comes with a bold quest – to revolutionise their chaotic agendas, making it easier to manage huge numbers of patients each with their own distinct needs.
The NHS has, of course, also faced wide-reaching challenges in areas such as waiting times and workforce retention, many of which can be found within the private sector too.
Yet many doctors, across specialities, remain unconvinced. With origins from the 1950s, AI isn’t new – but its particularly energetic burst onto the scene in the past few years still hasn’t convinced everyone.
Justified caution?
Last month, Healthcare Today reported on some of these concerns, including hurdles stopping smooth deployment, and a still-lacking knowledge in the sector.
And, testing just why some scepticism is warranted, in 2024, Swedish researchers invented a fake disease dubbed “bixonimania”. Within weeks, ChatGPT, Google, Copilot and Perplexity were telling patients it was real, but even more worryingly, the fake papers got cited in peer-reviewed medical journals.
Shortly after, JAMA Network Open tested 21 AI models, including GPT-5, Claude and Gemini on real clinical scenarios, yet differential diagnosis failure rates exceeded 80% across every model.
Meanwhile, a Nature Medicine study found ChatGPT Health under-triaged 52% of emergency cases.
When a patient’s prompt included language suggesting family or friends were minimising symptoms, the model was 11.7 times more likely to recommend less urgent care, even if they were life-threatening. It’s worth remembering that most health conversations on ChatGPT happen outside conventional clinic hours, and online tools at the flick of a switch are more accessible, whatever the time of day or night.
Around 40 million people used these tools for health advice alone in April this year. In their rush for answers, they might not have considered how AI of this nature lacks crucial context that only human doctors can use to create a full picture of their health.
Matters into own hands
To counter this, tech companies that strive to put people first are developing their own approaches to AI.
Dimer Health established AiME to connect to each patient’s actual longitudinal record, including medications, allergies, diagnoses and clinician notes. When it flags a concern, it doesn’t send a message suggesting a follow-up, but escalates to a licensed provider who “owns the outcome”.
The US-based provider claims every interaction runs inside a clinician-governed model with accountability built in from the outset.
Dimer has looked into the psychology behind using tools such as ChatGPT and their outcomes and concludes that doctors who truly know a user’s specific records will always beat “smarter models”, whatever form they take.
Shadowing the doctor?
Meanwhile, Shadow AI – an umbrella term for unsanctioned and fragmented tools – is also something doctors need to remain attuned to. This can add major compliance, privacy and transparency risks. Is it lurking like a ghost in your clinic?
Global information provider Wolters Kluwer Health warns that governance and policy frameworks are struggling to keep up with the rise of AI, including these shadow kinds. This is particularly stark as healthcare organisations increasingly look to AI to aid their enormous workloads, and in their eagerness and haste, might miss crucial warnings, allowing poor examples of AI to make their way in.
Although Wolters Kluwer Health’s recent survey of more than 500 healthcare workers was in the US, it feels the results – with 40% of respondents having encountered others using unapproved AI applications in their organisation – are worth paying attention to in the UK too.
They asked participants about the risks of AI, finding nearly a quarter (23%) ranked patient safety as their primary concern. Almost a fifth (17%) admitted to having used unsanctioned tools at work themselves.
Their reasons included needing to achieve faster workflows, a lack of approved tools and approved tools lacking the desired functionality.
Offering a UK perspective, Derek Bell, group chair for University Hospitals Tees, says: “NHS England is clear that the use of AI in healthcare must promote safety and be explainable and grounded in trusted evidence. What matters is not simply what AI can generate, but what it is built on and how it is governed.”
“If we don’t document, don’t evaluate, and don’t understand the source, we’re not managing risk – we’re creating it. It’s not just about having data – it’s about having trusted, dependable sources that clinicians can rely on,” he warns.
The UK government announced in 2025 that the regulatory rulebook from a new National Commission will be published later this year. The Commission’s goal is to help accelerate safe access to AI across the NHS and wider healthcare.
The current picture
Trying to answer the pressing question of whether AI helps cut NHS backlogs in reality or just becomes another system for staff to manage, healthcare AI platform Heidi published its UK Impact Report in May. However, it’s worth noting that Heidi is used extensively in NHS consultations already.
It looked at ambient voice technology, essentially generative AI as a scribe-like service to create patient notes, in everyday services, including A&E, Same Day Emergency Care (SDEC) and Primary Care Networks. It suggests an AI scribe is starting to improve how the NHS handles paperwork.
This comes at a particularly apt time. In April this year, NHS England updated its calls to ask Integrated Care Boards and providers to adopt ambient voice technology.
Heidi’s report shares that more than four million hours of clinical time have been returned to frontline teams, documentation time was cut by up to 86% in high throughput settings such as Emergency Departments and Same Day Emergency Care (SDEC) and clinical correspondence backlogs reduced by up to 99%. One Emergency Department saw discharge letter turnaround fall from 9.03 days to 2.6 minutes. In October last year, sites using Heidi reported large reductions in paperwork and backlogs, freeing clinicians to spend more time talking to patients and delivering more personalised care.
AI here in the UK
To also put their stamp on the issue, the Professional Standards Authority for Health and Social Care (PSA) and the University of Bristol have investigated safe and ethical use of AI in health and social care.
Noting how AI in these settings can raise specific professional, ethical and legal questions for users, they explore how far professionals can or should rely on, question or entirely ignore AI-generated recommendations.
They ask: “If something were to go wrong, and the AI gives a bad recommendation, then who will be held ethically and legally responsible if that wrong AI recommendation reaches the service user and leads to harm?”
The PSA commissioned a recent workshop, facilitated by Bristol academics, and published a subsequent report in April.
The team focused on identifying areas where regulatory clarity is needed, sharing best practices for ensuring patient safety and ethical deployment of AI and studying real-world scenarios.
They identified key findings as committing to career-long learning and Continuing Professional Development, improving reporting for easier ways for doctors to learn from AI and patients to raise concerns and better sharing of responsibility and accountability. The researchers stressed the need for diverse input and flexibility in our era of fast-moving technology.
They will use their findings to inform the PSA’s contributions to the UK National Commission on the Regulation of AI in Healthcare, led by the Medicines and Healthcare products Regulatory Agency.
It remains to be seen just how far AI can change – and hopefully improve UK doctors’ working lives. But for now, tech providers, universities and healthcare organisations alike are doing their utmost to either prove or question its merits.
