AI healthcare startup Babylon Health believes it can appropriately triage patients in 85 percent of cases.
Babylon Health is best known for GP at Hand, a service which is supported by UK health secretary Matt Hancock and integrated into Samsung Health.
GP at Hand links patients with health experts 24/7 using video calls and can facilitate any prescriptions to be sent to local pharmacies. The service, however, has been criticised for an AI chatbot which repeatedly gave unsafe advice and for only taking on healthier, often younger individuals while redirecting cash away from local surgeries relied on by older and sicker patients.
Correct triaging is essential to ensure patients receive the appropriate care. As the world responds to the coronavirus pandemic, many of us will have seen the harrowing headlines from the worst-hit countries like Italy where doctors are having to essentially decide who is worth trying to save due to limited resources.
Having to make such decisions, on top of all the other pressures medical professionals are currently facing, is unimaginable. An automated system would help to reduce the mental impact from any doubt their decisions are correct.
The company used reinforcement learning, which uses rewards for completing tasks to incentivise an agent, to create their AI system.
Babylon Health’s agent learned “an optimised policy” based on 1,374 clinical vignettes crafted by experts. Each vignette was supported by 3.36 expert triage decisions on average, and each was independently reviewed by two clinicians.
The best performing model achieved an appropriateness score of 85 percent and a safety score of 93 percent, which is around the same for humans (84 percent appropriateness and 93 percent safety.)
If true, it’s an impressive result, but Babylon Health’s studies have been called into question in the past. Just three years ago, the company tried and failed to get a legal injunction to block the publication of a report from the NHS care standards watchdog.
In 2018, Babylon Health published a paper which claimed that its AI could diagnose common diseases as well as human physicians. The Royal College of General Practitioners, the British Medical Association, Fraser and Wong, and the Royal College of Physicians all issued statements disputing the paper’s claims.
As with the rest of Babylon Health’s solutions, there’s a lot of promise in what they’re aiming to do. However, the company’s history casts some doubt over whether these latest claims are as impressive as they seem.
You can find Babylon Health’s full paper on arXiv here (PDF)
