Personalized mental health care promises a straightforward advantage: instead of offering everyone the same intervention, clinicians could use information about an individual’s circumstances to recommend the option most likely to help. Machine learning has raised expectations that complex patterns in patient data might make these decisions more accurate.
But a preregistered study of 828 adults in the United States has delivered a cautionary result. Researchers built an algorithm to predict who would respond better to a brief online depression program than to being placed on a waitlist. When they tested it on participants whose data had not been used to develop the model, the algorithm could not reliably distinguish who would benefit more from the program.
The study, published on 22 September 2026 in Clinical Psychological Science, used a randomized trial and a separate test dataset to examine whether personalized recommendations would hold up outside the data used to develop them. The result raises questions about the difference between a model that predicts depression symptoms and a model that can genuinely guide treatment choices.
That distinction matters as digital mental health services expand. A system that identifies people likely to remain distressed is not necessarily able to tell them which intervention will work better. The researchers’ findings highlight why predictive accuracy alone should not be mistaken for clinically useful personalization.
The attraction of a treatment-matching algorithm
Depression does not affect every person in the same way. Symptoms, experiences, other health concerns, support networks and practical barriers differ considerably. This variability creates a reasonable hope that different people may respond better to different forms of assistance.
A treatment-matching system aims to convert that idea into an actionable prediction. Instead of asking only how someone is likely to feel in two weeks, it asks a more demanding question: how would that same person’s outcome differ depending on which treatment they received?
Researchers call this a personalized treatment advantage. In principle, a model could indicate that one person would benefit more from an online skills exercise, while another would see no measurable improvement from it. Such a tool would be useful only if those predictions accurately reflect differences in treatment response among people the model has not previously encountered.
The authors, EJ Jardas, Jacqueline Howard and Lorenzo Lorenzo-Luaces, investigated this question using an existing randomized trial. They specifically tested the potential advantage of a short, self-guided digital intervention over a waitlist, rather than comparing several established clinical treatments.
That narrower comparison should remain central to interpreting the findings. This study does not demonstrate that machine learning cannot assist mental health care generally, or that the best approach for an individual could never be predicted. It establishes what happened when a particular set of methods and information was used to distinguish outcomes between two defined conditions.
What the 25-to-40-minute intervention involved
Participants were drawn from a randomized trial of the Common Elements Toolbox, known as COMET. The program was designed as a brief, self-directed collection of psychological skills delivered online. It typically took around 25 to 40 minutes to complete.
Its four modules brought together elements of cognitive behavioral therapy and positive psychology. The subjects were behavioral activation, cognitive restructuring, gratitude and self-compassion. Participants learned practical exercises, including scheduling pleasant activities and considering alternative interpretations of difficult thoughts.
They also encountered an exercise focused on identifying three positive experiences and a task involving more compassionate statements toward themselves. After reviewing the modules, participants selected one skill to practice during the following week.
The comparison group was placed on a waitlist. That design provided a benchmark against which the researchers could assess changes in symptoms, but it was not a head-to-head trial against standard psychotherapy, medication or another active mental health program.
The broader randomized trial had already reported no statistically significant average depression benefit for COMET over the waitlist. Its standardized mean difference was −0.05, with a 95% confidence interval from −0.20 to 0.07. The present study therefore asked whether an overall null result might conceal a subgroup that benefited, even if the intervention was not effective on average.
This is an important scientific question. An average that shows little difference between groups does not logically rule out individuals responding differently. However, a convincing claim of differential benefit requires evidence that such differences are real and reproducible, not just patterns generated by chance.
How the trial recruited and assessed 828 adults
The research involved 828 U.S. adults recruited through Prolific, an online research platform. Participants qualified if they reported having, or previously having, a diagnosed ongoing mental health condition. They were randomly allocated to receive COMET or join the waitlist.
Because eligibility was based on a broad history of mental health concerns, not every participant necessarily had the same degree of depression or the same primary clinical problem. That matters for any model attempting to identify a depression-specific treatment response.
The sample’s average age was approximately 36 years. Around 65.7% of participants were women, while 34.8% identified as LGBTQ+. The study also recorded a range of educational, demographic and psychological characteristics. This was a research sample of online volunteers, not a representative survey of U.S. patients receiving routine psychiatric care.
Depressive symptoms were measured with the eight-item Patient Health Questionnaire, or PHQ-8. The authors transformed the resulting scores to make them comparable with the more familiar PHQ-9 scale. The outcome for this machine-learning analysis was symptom severity two weeks after randomization.
Although the original clinical trial followed participants over two, four and eight weeks, the newly published treatment-matching analysis concentrated on the two-week measurement. It therefore does not establish whether a treatment-selection rule would predict longer-term recovery, remission or subsequent treatment engagement.
The researchers assembled candidate predictors from a review of factors previously associated with depression outcomes. These included baseline depression and anxiety, wellbeing, functioning, insomnia, childhood adversity, personality characteristics, social support, educational background, employment, prior psychiatric medication use and participants’ interest in practicing particular skills.
That comprehensive set provided the algorithm with substantially more information than a simple question about baseline mood. Nevertheless, the measures were largely self-reported. They did not include biomarkers, therapist assessments or a direct measure of how effectively each individual would practice the intervention.
Why the researchers separated training and test data
One of the most important features of the design was the decision not to judge success using only the data on which the models were developed. The full sample was divided into 662 participants for training and 166 participants for a held-out test, an 80:20 split.
Within the training set, 327 participants had received COMET and 335 had been assigned to the waitlist. The corresponding test groups contained 82 and 84 participants. The test dataset was reserved for assessing whether patterns learned from the training participants would translate to new cases.
The team used elastic-net regression with 10-fold cross-validation. Elastic net is designed to handle numerous potentially overlapping predictors without allowing every apparent relationship to influence the final model unrestrictedly. Cross-validation repeatedly evaluates models on subsets of the training data to help select settings that perform more consistently.
The researchers also took steps to reduce data leakage, a problem that arises when information from the test sample accidentally shapes the model being evaluated. Missing-data imputation and the reduction of correlated variables were fitted in training data and then applied to the separate test dataset.
Some technical choices departed from the preregistered plan, including the training-test split and the use of principal component analysis to summarize overlapping measures. The authors reported these deviations, allowing readers to distinguish the original analysis plan from subsequent methodological decisions.
Protecting the test dataset matters because complex algorithms can detect patterns that look impressive without representing stable differences in the real world. When this happens, the model may predict the original observations well while giving unreliable advice to new people.
Predicting symptoms is not the same as choosing a treatment
The researchers built models estimating participants’ likely two-week depression scores under COMET and under the waitlist condition. They then calculated a Personalized Advantage Index, or PAI, from the predicted difference between those possible outcomes.
A positive PAI indicated a predicted advantage for COMET. A negative value indicated that the model expected a lower depression score under the waitlist. Values close to zero indicated very little predicted difference. In the held-out dataset, PAI scores averaged −0.21, with a standard deviation of 0.73.
Some of the model’s ordinary outcome predictions looked encouraging. In the independent test data, predicted and observed two-week symptom scores were strongly correlated within both conditions: r = 0.841 for COMET and r = 0.864 for the waitlist.
However, these correlations largely address whether the model can identify people likely to have higher or lower depression scores. A person with severe baseline symptoms might be predicted to remain more symptomatic than someone whose initial symptoms are mild, regardless of which condition they receive.
That information can be valuable for understanding risk. But it does not, by itself, establish that the model can select the better of two treatments. For personalization to work, the difference between the two predicted outcomes must correspond to a real difference in how individuals respond.
The researchers found that the predicted COMET and waitlist outcomes were almost perfectly correlated with one another: r = 0.993 in both training and test data. The two predictions largely tracked the same underlying pattern of symptom severity rather than meaningfully separating people who would do better with one condition.
This is the central lesson of the study. An algorithm may be effective at predicting who is likely to struggle while being ineffective at identifying what assistance would change that person’s outcome.
The held-out test did not support treatment matching
The decisive analysis examined whether the PAI interacted with randomized treatment assignment to predict depression severity after two weeks. In other words, did people predicted to benefit from COMET actually show better outcomes from COMET than from waiting?
The interaction was not statistically significant, with a reported coefficient of 0.880, standard error of 1.21 and p = 0.47. That result did not provide reliable evidence that the index distinguished who should receive the program.
The researchers also examined whether any portion of the PAI range showed a statistically detectable advantage for one condition. Their additional analysis found no range meeting the prespecified significance threshold. The model consequently could not support reliable personalized allocation to COMET versus the waitlist.
A non-significant result is not mathematical proof that nobody could benefit. It means the observed data did not establish a dependable way to identify such people using this model. This distinction is essential when translating research results into statements about treatment decisions.
In particular, the analysis should not be described as a clinical failure of artificial intelligence-based diagnosis. The model did not diagnose depression, replace a clinician or test a generative chatbot. It attempted to predict differential response to a specific digital psychological skills program.
The result is more informative than a model reporting high accuracy only in its development data. Here, the investigators made a serious effort to test whether an apparent advantage would survive exposure to participants the algorithm had not already learned from.
What the findings mean for digital mental health services
Digital tools can make mental health information and structured exercises easier to access, particularly when appointments are expensive, scarce or inconvenient. That accessibility is a separate question from whether a statistical system can confidently match each user to the option that will help most.
Related Research Today reporting on health-related chatbot conversations has shown how readily people turn to accessible technology for health information. But high demand for automated guidance does not prove that personalized treatment recommendations are accurate.
In another context, research into wearable devices and brain health illustrates the promise and limits of digital monitoring. Detecting patterns in mood or behavior and knowing how to intervene are two different scientific objectives.
Research on different forms of social-media engagement and mental health also emphasizes the variety of individual experiences. That variety motivates personalization research, but does not make accurate treatment prediction automatic.
For developers, the practical implication is to evaluate treatment-selection systems using genuine treatment differences in data not previously used to develop the system. Strong correlations between predicted and actual symptoms, attractive interfaces and large collections of patient characteristics are insufficient proof of treatment-matching value.
For clinical services, the result argues against presenting an unvalidated score as a reason to offer, withhold or prioritize a particular intervention. Where decisions could influence access to care, the standard of evidence needs to be higher than whether an algorithm can predict distress.
Limitations and the questions still open
The study has several strengths: randomized treatment assignment, a comparatively substantial sample for treatment-matching research, preregistration and a held-out test. But its limitations also matter when considering broader applications.
First, all participants were online research volunteers from the United States who reported a present or past mental health condition. A person receiving clinical care for diagnosed major depressive disorder may differ from this sample in severity, motivation, access to support and treatment history.
Second, the intervention did not show a meaningful average benefit in the underlying trial. If COMET had little effect for most adults in these circumstances, there may have been only a small treatment-specific signal for an algorithm to detect. Failure to find that signal cannot automatically be generalized to more effective treatments or comparisons between two active therapies.
Third, the held-out sample contained 166 people, split between two randomized conditions. That is a useful safeguard against evaluating performance on training data, but the resulting estimates may still be too imprecise to identify small or unusual treatment-response patterns.
Fourth, important predictors may not have been measured. Information about individual engagement, actual use of the taught skills or other clinical factors might affect treatment response, even though this analysis did not demonstrate that adding them would produce successful personalization.
Finally, the results concern a two-week self-reported symptom score, not a broader assessment of quality of life, durable remission or long-term access to treatment. A prediction system that cannot differentiate outcomes here might behave differently with another program, population or endpoint.
The next research question is therefore not simply how to make a bigger algorithm. It is whether better interventions, more relevant measurements and rigorous testing can reveal treatment differences substantial enough to inform decisions safely.
Evidence before individualized promises
Personalized mental health care remains an appealing objective. People have different circumstances, and effective services should recognize those differences. But personalization must ultimately be demonstrated by better outcomes, not merely by software that produces a different score for every user.
In this randomized dataset of 828 adults, the researchers could predict overall depressive symptom severity reasonably well while failing to identify who would benefit more from COMET than from waiting. Their findings reinforce an important distinction for health technology: knowing who is at risk is not the same as knowing what will help them.
The result is a reason to improve how digital treatment-selection tools are validated, not a reason to abandon digital mental health services. Strong evidence, transparent uncertainty and meaningful comparisons must come before confident claims about individualized benefit.
Source Information
Study title: Leveraging Machine Learning to Personalize Depression Treatment: A Preregistered Study of 828 Adults Randomly Assigned to a Digital Single-Session Intervention or Waitlist
Authors: EJ Jardas, Jacqueline Howard and Lorenzo Lorenzo-Luaces
Journal: Clinical Psychological Science (online first)
Publication date: 22 September 2026
Study design: Preregistered machine-learning treatment-matching analysis of a two-arm randomized controlled trial, using elastic-net models, 10-fold cross-validation and a held-out test sample
Sample or scope: 828 U.S. adults with current or past mental health concerns, including 662 training participants and 166 held-out test participants; COMET versus waitlist, assessed at two weeks
Key finding: The personalized treatment-advantage score did not significantly predict differences between COMET and waitlist outcomes in held-out data (p = 0.47), despite strong predictions of overall symptom severity
