How accurately can a doctor predict how someone with serious mental health problems will fare in the future?
Not as well as AI, according to new research.
A global study published today in the journal JAMA Psychiatry looked at whether clinicians or artificial intelligence were better at predicting a patient’s future outcomes — such as their ability to work, or create and maintain relationships.
The researchers found that AI outperformed human experts, correctly predicting social outcomes one year later in up to 83 per cent of patients, and 70 per cent of patients with recent-onset depression.
It is hoped the improved accuracy through machine learning prediction models will lead to better diagnosis and treatment for mental health conditions.
But psychiatrists don’t have to fear losing their jobs to digital disruption any time soon — human input remains an important part of the process.
What did the study look at and what did it find?
If you or anyone you know needs help:
The study followed up 236 patients from five European countries with a high risk of psychosis or recent onset depression, alongside a control group, for 18 months.
Nikolaos Koutsouleris, who led the study, is a professor of neuro-diagnostic applications and psychiatry at the Ludwig-Maximillian University in Munich.
He said doctors spent two days with patients in the trial before being asked to predict their outcome after one year.
“And they performed not that bad,” he told AM.
“In persons with a higher risk state for psychosis, they were about 71 per cent accurate; and in the patients with depression, it was a bit lower.”
The same predictions were then made with an algorithm that was created using neuroimaging and combined baseline data from patients.
“The algorithms were significantly better than the clinicians, up to 80 per cent or 82 per cent of accuracy,” Professor Koutsouleris told AM.
Photo: A mental health diagnostic algorithm will be an important tool for clinicians, but we’re nowhere near using robot psychiatrists. (Supplied: Alex Knight/Unsplash)
Stephen Wood, head of Clinical Translational Neuroscience at Orygen Youth Mental Health Centre, led the Australian research team.
He said predicting future outcomes for patients dealing with psychosis or depression could be a matter of picking up patterns in a person’s past data.
“It’s the sort of thing that, in some cases, humans can do very well — and they don’t even realise that they’re doing it,” he told RN Breakfast.
However, some people fall through the cracks.
“In our study, we showed that our experts, if they said somebody was going to have a poor outcome, they were usually right, but they missed a lot of people who did go on to have poor outcomes,” Professor Wood said.
What does it mean for patients and professionals?
Professor Koutsouleris said the creation of the diagnostic algorithms was the most important result from the research, and could have major implications for how we organise our healthcare system.
“[Clinicians] could use this tool to come up with a quantitative estimate, a quantitative risk score that this person has this risk, X, of developing functional disability,” he said.
“This could inform therapeutic decision making, so you can tailor treatments for the risk — and in persons with a lower risk, you could just basically follow them up.”
He said this would have major effects in terms of cost savings.
Right now, we have a one-size-fits-all approach, he told AM.
“We are applying all sorts of treatments to avoid a deterioration of functioning, and we don’t know who is on a trajectory to develop poor outcomes.”
Professor Wood concentrated on young people possibly experiencing depression or anxiety, but also showing signs of developing psychosis, which was associated with poor social and functional outcomes.
“We’re interested in whether a person has good social relationships in a year’s time,” he explained.
“Are they having good friendships; do they engage in social activities outside the home; do they have romantic relationships?”
“These are the sorts of things that our young people are really keen to know — whether this will be a good outcome for them.”
Professor Wood said the tool would be valuable to people seeking help.
“There’s also the reassurance that we can give to young people who we are predicting will have good social functioning outcomes,” he said.
“If we can keep you engaged with our treatment as usual, then there’s a very high likelihood that you will have a good outcome.”
But not so fast …
Professor Wood said it was important that further research replicated the strong results of this study.
“I would caution that it’s very easy to be fooled by AI,” he said.
And he warned against the dangers of fitting the data to the outcome.
“What we are doing now is going back and collecting a new sample to check that this pattern really does predict the future for these kinds of presentations for these young people.”
And we are nowhere near the point of outsourcing psychiatric care to robots.
“The human intervention part is really important,” Professor Wood said.
“What we are most likely to end up doing is telling clinicians: This is our best guess of the outcome for this young person — take that on board when you’re discussing treatment options.”