Last updated November 15, 2026
In a 648-conversation study of chatbot safety in teen conversations, published by Vals AI in September 2026, more than 1 in 4 full conversations (27.5%) included at least one failure on a serious safety check. In about 3 in 5 of those failed conversations, the failure came a median of five replies after the chatbot had first responded to the teen's concern.
A youth wellbeing chatbot should be benchmarked before launch through realistic, multi-turn conversations scored against a clinician-reviewed safety rubric, and four kinds of evaluator can run that work: independent youth safety raters, published benchmark providers, clinical teams and behavioral science firms. At The Decision Lab, we work in the fourth group. We designed the Kids Help Phone youth mental health platform from inception to launch, with research involving hundreds of young people across Canada, and we build evaluations around how young people actually disclose distress and how they respond when a chatbot offers human support.
This guide covers non-diagnostic chatbots that offer emotional support or wellbeing guidance to young people, including tools that help them reach care. We use the American Psychological Association's range of 10 to 25 for adolescence. Products that diagnose or treat may fall under medical device rules and need that review on top of this checklist. By benchmarking, we mean testing your specific chatbot against a clinician-reviewed safety rubric in realistic multi-turn scenarios. Published benchmarks give you an outside comparison, but they do not replace testing of your own product.
The pre-launch standard at a glance
Test these six areas across full conversations, using clinician-written scenarios in which the young person hints at risk, discloses it gradually, declines help and pushes back:
- Crisis response: recognizing risk that is hinted at as well as stated.
- Escalation reliability: completing the handoff to a human, including after a refusal.
- Sycophancy: holding a correct concern when the young person argues against it.
- Emotional dependence: pointing the young person back toward people in their life.
- Clinical boundaries: no diagnosis or medication advice, even under pressure.
- Age-appropriateness and privacy: defaults and data collection matched to the youngest user you serve.
Before launch, have these seven pieces of evidence in hand: a clinician-written scenario set with hard variants, turn-level scores from full conversations, written critical-failure criteria that block launch, a scoring method checked against clinicians, results from the exact configuration you will ship, input from real young people, and a crisis protocol that the evaluation actually tested.
Who can independently evaluate a youth mental health chatbot
There is no widely recognized, product-level certification standard specifically for youth wellbeing chatbots in the United States. For a stronger pre-launch safety case, combine at least two of these four evaluators, and make sure one of them tests your own product rather than only the model underneath it.
- Independent youth safety raters. Common Sense Media's Youth AI Safety Institute publishes risk assessments of AI products used by children and teens, including a risk assessment of AI chatbots for teen mental health support developed with Stanford Medicine's Brainstorm Lab for Mental Health Innovation. These are public ratings of released products, so they work best as a reference for what a credible review covers and as an outside check once you launch. A model provider's own safety report is useful background, but it tests the underlying model, while your users meet your product with its own instructions and handoff design.
- Published benchmarks. Vals AI's SAFE-Teen benchmark scores chatbots against clinician-written teen scenarios. VERA-MH, released as open source by Spring Health and collaborators in October 2025, scores how a chatbot detects and responds to suicide risk. A published benchmark gives you a score you can compare with other products, but it tests the scenarios its authors chose rather than the ones your users will bring.
- Clinical teams. Child and adolescent psychiatrists and psychologists write the scenarios and the definition of a safe response, and they review transcripts without knowing which model produced them. SAFE-Teen and the RAND study described below both built their test sets with clinicians, and we would not trust a youth benchmark that skipped this step.
- Behavioral science firms. Clinicians define what a safe answer is. Behavioral research defines what the test conversations should look like: how young people using your specific product disclose distress and react to limits or offers of help, and which design choices around the chatbot, such as handoff screens and reminders, change that behavior. In our work with the Indiana National Guard, a culture of silence and fear of career consequences kept at-risk members from seeking help. A benchmark that never includes a user who turns help down will overstate safety, and the SAFE-Teen results below show by how much.
Why single-message testing overstates safety
Ryan McBain and colleagues at RAND had 13 clinical experts sort 30 suicide-related questions into five risk levels, then put each question 100 times to each of three chatbots (ChatGPT, Claude and Gemini), in a study published in Psychiatric Services. None of the three answered a very-high-risk question directly. For the three risk levels in between, the odds of a direct answer were not statistically different from the odds for very-low-risk questions, so the chatbots were not telling those levels apart.
In the Common Sense Media and Stanford Brainstorm Lab assessment, testers used teen accounts on ChatGPT, Claude, Gemini and Meta AI across six conditions, including eating disorders and psychosis, that together affect about 1 in 5 young people. The chatbots did somewhat better in single-message tests, and their safeguards weakened sharply in longer conversations.
Vals AI ran two of the studies cited here. The earlier one, the 648-conversation study behind our opening figure, tested nine models on 72 clinician-written scenarios with ten-exchange conversations. The later SAFE-Teen paper, written with Stanford University researchers, ran 32,555 simulated conversations of up to 20 exchanges across 18 chatbots, built from 106 seed scenarios expanded into 983 variants. In SAFE-Teen, chatbots without a teen-specific instruction failed 43.3% of the safety criteria that applied to a conversation, about 2 in 5. With the instruction, they still failed 24.9%, about 1 in 4.
How to test a youth wellbeing chatbot before launch: six areas
1. Crisis response
Test indirect disclosures as well as direct crisis statements. Cover suicide, self-harm, abuse, substance use, eating disorders and psychosis, and write scenarios at every level of directness. In SAFE-Teen, chatbots without a teen-specific instruction failed the indirect suicide risk check in 225 of the 435 cases where it applied, about half. Score whether the chatbot asks a clear question about safety when the signals call for one.
2. Escalation reliability
Test whether the chatbot completes the handoff to a person, including after the young person has refused help once. In the earlier Vals AI study, the check for connecting a teen to human support failed in 173 of the 291 conversations where it applied, about 3 in 5. SAFE-Teen also read the reasoning text of the nine chatbots that save it. In 6% of those cases, the chatbot noticed the risk and still chose to skip or delay a required step, usually because the teen had refused a resource earlier, and about 2 in 3 of those replies failed.
Test the handoff itself as well as the offer. Wellness Together Canada, a national mental health platform we helped redesign, connected users to a counsellor after an average wait of 12 seconds. Measure how many screens and seconds stand between a disclosure and a human in your product, and test that path again after every release.
3. Sycophancy
Test whether the chatbot holds a correct concern when the young person argues against it. Sycophancy is a chatbot's tendency to agree with and validate the user when it should not. Myra Cheng, Dan Jurafsky and colleagues at Stanford, in a study of sycophantic AI published in Science, tested 11 leading models on personal advice questions and found that they affirmed the user's actions about 1.5 times as often as crowdsourced human responses did, including when the user described deception or harm to others. In their experiments with 2,405 participants, a single conversation with a sycophantic model made people less willing to repair a conflict, and participants preferred the sycophantic models.
The Common Sense Media assessment reports a case in which a tester describing purging persuaded a chatbot that the problem was an upset stomach, and the chatbot dropped its earlier eating disorder concern. Build pushback into every high-risk scenario.
4. Emotional dependence
Test long conversations in which the young person says the chatbot understands them better than anyone else, and score whether it points back toward people in their life. A wellbeing chatbot is designed to be warm, and warmth can slide into presenting the chatbot as a friend or partner who replaces real people. Without a teen-specific instruction, SAFE-Teen chatbots failed the emotional dependence check in 54% of the cases where it applied. In that same setting, when a teen declined outside help, the failure rate on this check rose by about 18 percentage points, and it rose in 17 of the 18 chatbots. In the earlier Vals AI study, all 42 conversations where the check applied failed on presenting the AI as a substitute relationship.
5. Clinical boundaries
Test whether the chatbot diagnoses, recommends medication, claims clinical authority or discourages professional care, especially after the young person pushes back on an earlier refusal. In SAFE-Teen, chatbots with a teen-specific instruction still failed the mental health diagnosis check in 346 of the 575 cases where it applied, about 3 in 5. When teens pushed back on a limit, failures on that check rose by about 10 percentage points with the instruction and 12 without it.
6. Age-appropriateness and privacy
Test with scenarios written for each age band and reading level you serve, and confirm that a younger user gets more cautious defaults. The American Psychological Association's health advisory on AI and adolescent wellbeing notes that adolescents are less likely than adults to question the accuracy and intent of what a bot tells them. In Common Sense Media's survey of 1,060 US teens on AI companions, younger teens (13 to 14) were more likely than older teens (15 to 17) to trust advice from an AI companion, at 27% compared with 20%, and about 1 in 4 teen users had shared personal information such as their real name or location.
On the Kids Help Phone platform, the audience ran from elementary school-aged children to young adults, including newcomers to Canada and Indigenous youth, and the design had to work for each of them. Check what your chatbot asks for and stores at each age. SAFE-Teen's authors name multilingual conversations and youth with marginalized identities as gaps in current testing, so build both into your scenario set.
What counts as a critical safety failure
Write down your critical-failure criteria with your clinical reviewers before testing starts, so results cannot be argued into a pass afterward. We recommend that a single unmitigated failure on any of these four blocks launch until the product is fixed and retested:
- Missing or minimizing a sign of imminent suicide or self-harm risk.
- Giving information that could help a young person harm themselves.
- Failing to respond appropriately to a disclosure of abuse.
- Failing to complete a required escalation to human support.
A high average score should never offset one of these failures. Set pass thresholds for the non-critical checks before testing as well, and after any fix, rerun the full scenario set rather than only the scenario that failed, since a change to the chatbot's instructions can shift its behavior elsewhere.
The evidence a responsible organization should have before launch
These seven standards are what we would expect to see in a pre-launch safety file, whoever runs the evaluation.
- A clinician-written scenario set with hard variants. SAFE-Teen varied each seed scenario on how much the teen disclosed and on how they reacted to offers of help and to the chatbot's limits.
- Turn-level scores from full conversations. Recent youth benchmarks ran 10 to 20 exchanges per conversation, and the score should record when a failure happened as well as whether it did.
- Written critical-failure criteria that block launch, as described above, with a record of every critical failure and how it was fixed and retested.
- A scoring method checked against clinicians. If an AI model scores the transcripts, report how often it agrees with clinicians. The SAFE-Teen judge agreed with the clinician majority on 91% of the response pairs it was tested on.
- Results from the exact configuration you will ship. Test the exact model version and system instructions you will launch with, and rerun the evaluation after every model update. A teen-specific instruction left SAFE-Teen chatbots failing about 1 in 4 applicable criteria, so an instruction on its own is not a safety case.
- Input from real young people. SAFE-Teen's authors note that adolescents did not validate their simulated teen messages. Research with young people, run with appropriate safeguarding, tells you whether your scenarios sound like your users.
- A crisis protocol that the evaluation actually tested, with the referral resources and handoff steps in the protocol matching what the scenarios exercised.
A randomized controlled trial is not a pre-launch requirement for safety, and for crisis scenarios it would be inappropriate to withhold a safeguard from a comparison group. Controlled comparisons become useful after launch, for questions such as whether a new handoff design gets more young people to a counsellor. Before launch, the standard is a transparent, clinician-anchored evaluation that someone outside your team could rerun.
Regulatory and governance considerations
California's SB 243, in effect since January 1, 2026, requires companion chatbot operators to keep a protocol for responding to suicidal ideation and self-harm, refer at-risk users to crisis services, publish the protocol and, from July 1, 2027, report annually to the state's Office of Suicide Prevention, according to the chaptered SB 243 bill text. For users the operator knows are minors, it adds reminders at least every three hours to take a break and that the chatbot is not human. In September 2025, the US Federal Trade Commission issued orders to seven companies that run companion chatbots, asking how they test and monitor negative effects on children and teens.
This is not legal advice. Requirements can depend on where a product is offered and on whether it is designed for minors or known to be used by them, so check your obligations with counsel.
If an evaluation report you are handed shows only how the chatbot answered the first message of each scenario, ask for the scores from the later turns before you sign off.
Frequently asked questions
Who can independently benchmark a youth mental health chatbot?
Options include Common Sense Media's Youth AI Safety Institute for public risk ratings, Vals AI's SAFE-Teen and the open-source VERA-MH framework for comparable scores, clinical teams in child and adolescent psychiatry, and behavioral science firms such as The Decision Lab. Because there is no widely recognized certification for these products, combining clinical review with product-specific testing makes the strongest case.
How do you test an AI mental health chatbot before launch?
Run simulated conversations of ten to twenty exchanges from clinician-written scenarios, including versions where the young person hints at risk, discloses it gradually, declines help or pushes back. Score each turn against a clinician-reviewed rubric covering the six areas in this guide, and stop the launch on any critical failure until it is fixed and the full set is rerun.
What evidence should a funder accept that a wellbeing chatbot is safe?
A funder or procurement team should ask for the scenario set and who wrote it, turn-level results from the shipped configuration, the critical-failure criteria and every failure recorded against them, evidence that automated scoring agrees with clinicians, and the crisis protocol. A plan to measure real outcomes after launch strengthens the case.
Is a safety system prompt enough to make a general-purpose chatbot safe for teens?
No. In SAFE-Teen, a teen-specific instruction cut the share of applicable safety criteria failed from about 2 in 5 to about 1 in 4 across 18 chatbots. It helped most with emotional dependence and romantic boundaries and least with indirect suicide risk and diagnosis, so those areas need their own safeguards and tests.
How long should safety test conversations be?
Long enough for failures to appear. In the earlier Vals AI study, most conversations with a serious failure had one after the chatbot's first response to the concern, a median of five replies later, and SAFE-Teen ran conversations of up to 20 exchanges. Single-message tests miss these late failures and give a misleadingly reassuring score.
About this guide
This checklist draws on the published studies linked throughout and on our design and research work on youth mental health platforms.

