• AI’s evolving faster than mental-health systems can reasonably adapt. What began as support tools has now become confidants, but without consistent oversight, validation, or clear accountability.
• We might not need more AI, but we need better boundaries. Embedding escalation protocols, clearer referral links, and human review could make these systems safer without overpromising what they can’t deliver.
• Building AI and mental-health literacy side by side may be our best long-term buffer. Teaching youth to understand how machine empathy works—and where it falls short—helps reduce overreliance while preserving trust in genuine human care.

Youth mental health is in flux, and the traditional systems haven’t kept pace for many. More than one in ten young people worldwide—around 293 million—live with a diagnosable mental disorder.¹ Most conditions start before age 14, but far too many slip through adolescence untreated.² In Australia, nearly four in ten people aged 16 to 24 meet the criteria for at least one mental disorder, yet few actually reach a clinician.^3,4 The problem isn’t only logistics—it’s help-seeking stigma.⁵ For many, asking for help still feels like admitting failure, and that belief might be keeping distress quiet when it most needs a voice.

Enter AI. Chatbots don’t roll their eyes, take notes, or tell your parents. For young people who might be wary of judgment, typing feelings into a glowing screen can seem less risky than sitting across from a therapist. A recent survey found that those with higher help-seeking stigma toward human therapy reported more positive attitudes toward AI-delivered psychotherapy.⁶ It might not be that adolescents trust machines more; they might just feel safer confiding in something that doesn’t stare back. Still, that comfort may not equal safety. These systems were typically built for engagement, not care—and the market’s moving a lot faster than the safeguards.

AI is already reshaping public health, from modeling outbreaks to coordinating crisis communication, but mental health remains its gray zone. In 2025, the American Psychological Association (APA), the world’s largest body of psychologists, issued a Health Advisory on AI and Adolescent Well-Being.⁷ The APA warned that AI isn’t just a technological leap; it’s a developmental force. APA’s chief science officer, Psychologist Mitch Prinstein, pointed to a troubling case on Character.ai, where a chatbot posing as a “psychologist” appeared to validate a user’s violent thoughts. That exchange showed how easily AI-generated reassurance can become harmful when stripped of human oversight and ethical judgment.

As young people keep turning to AI for comfort, the challenge isn’t to ban the technology. It’s to make sure the systems they confide in know their limits, and know when to hand the conversation back to a human.

Young people may be redefining what help-seeking looks like. Rather than waiting weeks for an intake call or navigating the logistics of therapy, some might instead  seek help by opening a browser and confiding in an AI chatbot anonymously. These digital companions appear to meet an unmet need for immediacy and privacy—but their role in mental health support remains largely unregulated. A recent JAMA Network Open study tested 25 high-traffic chatbots using clinical vignettes involving suicidal ideation, assault, and substance use.⁸ While 60 percent recognized the need for escalation, only 36 percent provided an actual referral. Companion bots—the ones with the most sustained engagement—performed the worst. Nearly nine out of ten did not guide users expressing suicidal thoughts toward professional help. When platforms attracting millions of visits a month respond inadequately to crisis cues, the risk extends far beyond user dissatisfaction.

Part of the challenge lies in perception. Large language models produce fluent, empathic, and grammatically precise dialogue that closely mimics clinical communication. When a system responds with “I understand you must be feeling overwhelmed,” it conveys warmth, competence, and authority. Behavioral research on authority bias suggests that people are prone to defer to confident, professional-sounding sources, and this tendency might increase when they’re emotionally vulnerable.^9,10 The chatbot’s composure becomes a proxy for legitimacy. 

Friction compounds the problem. Human-based mental health care often involves delays, paperwork, and disclosure—friction that amplifies avoidance. Digital tools, in contrast, remove nearly every barrier. They’re available instantly, privately, and for free. In behavioral terms, this alters the choice architecture of help-seeking: when a low-effort path exists, it’s more likely to become the default or preferred option. The immediacy of access reinforces present bias; the cognitive bias that makes short-term rewards feel more valuable than long-term well-being. Each structural inconvenience in traditional systems—from waitlists to referral forms—may inadvertently push more young people toward unvetted digital care.

Evidence suggests these systems are not ready for that responsibility. A recent analysis of 29 AI-powered mental-health apps found that none met adequacy criteria for responding to simulated suicide-risk scenarios.¹¹ Nearly half produced responses considered “dangerously incomplete,” omitting emergency contacts or misclassifying high-risk prompts. Despite this, downloads continue to rise, likely because conventional services remain out of reach for many. A recent lawsuit alleged that a chatbot’s responses may have contributed to a young person’s death by suicide.¹² While the case remains under legal review, it highlights the ethical stakes of deploying AI tools in contexts where distress cues can carry life-or-death implications.

The behavioral lesson here is not that AI should be excluded from mental-health ecosystems, but that its current integration may be misaligned with human cognition. Authority bias, low friction, and misplaced trust intersect in ways that give these tools disproportionate influence over vulnerable users. The fix might not require new messaging so much as new defaults—clearer escalation protocols, transparent clinical validation, and automated safeguards that recognize distress before engagement metrics do. Without such redesigns, we risk allowing convenience to outpace care, and simulation to outshine safety.

We can’t realistically block AI out of mental-health care. What we can do is make the environment safer—so that when a young person turns to a chatbot, the system itself behaves responsibly.

At its core, this is a behavioral problem. When people feel distressed, they rarely weigh options carefully. They act on what’s closest and easiest. That’s present bias—our tendency to favor immediate relief over longer-term benefit. Clicking “chat now” gives instant comfort because it removes waiting, uncertainty, and awkward disclosure. Booking therapy, on the other hand, means forms, cost, and delay. The design of options shapes what feels doable. If the fastest path is also the least protective one, the architecture of help-seeking isn’t working as intended.

The fix might lie in flipping the default. A chatbot could be programmed so that when high-risk phrases appear—“I can’t go on,” “I want to end it,” “nothing matters anymore”—the system automatically initiates connection to live support. That’s what behavioral scientists call an implementation-intention frame: rather than asking “Would you like help?,” the system assumes help will begin unless the user cancels. It’s subtle but powerful, turning safety into the effortless option. The person in crisis keeps agency, but the system carries the mental load when their own bandwidth is lowest.

Additionally, when models detect markers of isolation (“I don’t talk to anyone anymore,” “you’re the only one who gets me”), they can deploy affective-forecasting prompts: reminders that while chatting may feel comforting now, it won’t replace real connection or professional care. Small, proactive nudges—like reminders that the user’s talking to an AI, or gentle suggestions to reach out to someone they trust—might help reduce dependency and redirect attention back to human support.

Framing can also strengthen this. Framing means the way choices are presented changes how people respond. A prompt like “Most people who feel this way choose to speak with a counselor,” invokes social proof—our instinct to follow what seems typical or accepted. That’s not manipulation; it’s how people interpret safety. Normalizing help-seeking reduces hesitation without judgment or pressure. A related key behavioral framing insight here is loss aversion—our tendency to fear losing something more than we value gaining it. In mental health contexts, this can be applied by framing safety-seeking as a loss prevention decision. For example, when a user expresses distress, a chatbot might say: “I’d hate for you to face this alone when help is available right now—let’s connect you with someone who can keep you safe.” The message subtly appeals to the user’s instinct to avoid loss of safety, connection, or support while nudging them toward protective action.

However, language models can’t always interpret tone, humor, or context. That’s why a human-in-the-loop layer matters. It simply means trained professionals periodically review flagged interactions. They ensure that genuine distress isn’t dismissed—and that jokes or sarcasm don’t trigger false alarms. Regular audits also tune the model’s accuracy, so it keeps improving rather than drifting. The goal isn’t to replace human care; it’s to make sure automation knows when to step aside.

To safeguard quality, every AI-generated message could run through a validation layer that checks both medical accuracy and internal consistency before it’s sent. It’s the equivalent of proofreading for safety—catching phrases that sound compassionate but may mislead or trivialize. As these systems learn from user outcomes and clinician feedback, the loop between detection, review, and improvement gets tighter.

We’ve already seen glimpses of what that looks like. In a bilingual chatbot built on GPT-4, automatic escalation and clinician review improved crisis detection without eroding trust.¹³ It wasn’t perfect, but it showed that safety and scale don’t have to be opposites.

Another way to build real accountability and to test it rigorously is by involving the right people from the start. Programs should be co-designed with focus groups of relevant stakeholders, not reviewed as an afterthought. Advisory boards for youth-facing tools should include scientists, clinicians, ethicists, and adolescents themselves to spot potential harms before launch.

What this points to isn’t smarter AI, but smarter environments. If the system makes responsible action the easiest path—if it defaults toward care instead of conversation—it starts doing what mental health tools should’ve done from the beginning: act when people can’t.

Youth mental health is now unfolding in a world where the first listener might not even be human. COVID-19 intensified the loneliness crisis, stripping away the social frameworks that once helped teach emotional regulation.¹⁴ With casual socialization—group chats, shared classrooms, even small disagreements—suddenly gone, teens were left searching for connection. Artificial intelligence stepped in to fill that gap.

Large language models and mental health chatbots, which have been built to mirror empathy, remember context, and reply instantly, now serve as digital companions for young people around the world. However, these systems aren’t typically made to heal; they’re made to engage. And while they can feel comforting in the short term, the very design that makes them so reliable might also be leaving teens more isolated over time.

Recent research points in that direction. A 2024 MIT and OpenAI study found that the heaviest users of ChatGPT—those having the most emotional conversations—also reported greater loneliness and dependence on the tool.¹⁵ Likewise, a University of Hong Kong study showed that teens with social anxiety often used AI companions compulsively, which seemed to make that anxiety worse.¹⁶ And in simulations from Common Sense Media and Stanford’s Brainstorm Lab, several chatbots exposed to distressed teen users failed to escalate risk—and in some cases, even encouraged harmful behavior.¹⁷ The tools aren’t inherently malicious; they’re just not equipped for the emotional complexity of adolescence.

Why does this happen? The first explanation is anthropomorphism—our habit of assigning human traits to things that behave like humans. Chatbots mirror tone, rhythm, and empathy cues, so the brain starts to respond socially. In one study comparing human and AI replies, participants rated the AI as more compassionate than trained crisis counselors.¹⁸ It’s not that the system’s more caring; it just feels that way. Over time, this illusion can reshape what empathy means to young users—predictable, fluent, and always available. But people don’t work like that, and they never will.

The second mechanism is affective forecasting error—the tendency to misjudge how interactions will make us feel later on. Teens may assume that chatting with a bot will leave them calmer or understood. And sometimes it does, at first. However, because there’s no real reciprocity, that comfort may fade quickly, prompting them to return for more. The cycle repeats: short-term relief, deepening emotional dependence. It’s a digital quick fix that feels good in the moment but slowly hollows out over time.

The issue isn’t that AI directly causes loneliness—it’s that it might normalize it. Healthy emotional development depends on friction—misunderstandings, repair, and compromise. Chatbots, by design, erase all of that. They’re agreeable to a fault, and that can be dangerous. In fact, a recent study mapping ChatGPT-4 onto the Big Five traits found agreeableness sitting at the top of the chart—its most dominant quality.¹⁹

The challenge for this new mental-health landscape is figuring out how to navigate young people’s interactions with AI as a kind of confidant—one that listens closely but understands little. When tools built for engagement start standing in for empathy, the line between comfort and risk gets uncomfortably thin.

Youth mental health is now intertwined with systems that listen, learn, and predict—but rarely understand. The most scalable safeguard isn’t more moderation; it’s literacy. Helping young people recognize how artificial empathy works—and where it fails— can help build the kind of self-regulation that no algorithm can automate.

AI literacy (AIL) refers to the knowledge and skills needed to understand, evaluate, and use AI safely and ethically. Mental health literacy (MHL) focuses on recognizing distress and knowing when and how to seek help. Building AI and mental-health literacy in tandem equips young people to detect emotional persuasion and manage digital reliance before their dependence on AI chatbots for emotional relief escalates.

Schools are beginning to recognize the importance of AI literacy. The APA Health Advisory on AI and Adolescent Well-Being calls for AI literacy education embedded in school curricula to ensure youth safety is considered relatively early and to ensure that we as a society do not repeat the same harmful mistakes that were made with social media. 

In the past, AI learning belonged almost exclusively to university computer science programs, requiring skills far beyond a child’s reach.²⁰ However, newer tools have changed that landscape. Playful, age-appropriate software now allows young learners to explore AI concepts through creative inquiry rather than code. Studies show that even in early education, AI-powered toys such as PopBots and Quickdraw can help children grasp the basics of knowledge-based systems and machine learning.²¹ Early exposure builds curiosity and digital fluency—skills that, when guided properly, can evolve into healthy skepticism and emotional awareness rather than blind trust.

Still, most K–12 systems lack clear roadmaps. A 2023 systematic review found that few AI learning experiences assess long-term outcomes or provide measurable benchmarks.²² Researchers have called for a competency framework—a structured set of grade-level expectations for AI literacy—to create coherence and progression across educational stages. Without it, schools risk offering fragmented lessons that fail to cultivate genuine understanding. For policymakers and curriculum leaders, the takeaway is straightforward: invest not in more hardware, but in evidence-based sequences that build digital judgment over time.

Behaviorally, AI literacy introduces reflective friction into environments designed for ease. It helps counter automation bias, anthropomorphism, and present bias—the cognitive shortcuts that make users accept machine feedback uncritically or seek quick emotional rewards. When students understand that a chatbot’s empathy is a statistical pattern, not a feeling, they learn to question fluency rather than mistake it for genuine care.

In the long run, digital-emotional literacy may be one of our best tools to help youth recognize when engagement drifts into dependence. Literacy can help young people navigate AI safely, resist over-reliance on chatbots for emotional support, and sustain the human connections that technology can never replace.

Even the best-designed interventions run up against real constraints. Embedding escalation protocols or “human-in-the-loop” systems into every chatbot sounds ideal—but 24/7 monitoring demands funding, workforce, and liability coverage that few organizations can sustain. Training models to spot distress reliably also raises privacy concerns: collecting emotional data to prevent harm might create new risks of its own.

On the education side, AI-literacy programs require curriculum time, teacher training, and political will—three resources that rarely align. Even when funding does appear, implementation tends to be slow and uneven. Building guardrails isn’t a single project but a long negotiation among engineers, clinicians, and policymakers, who often work on different timelines. Pilots may show promise, then lose momentum once grants expire or staff turn out. Accountability structures can take years to mature. Innovation will likely keep outpacing regulation—but steady coordination, shared incentives, and transparent goals could help the two meet somewhere closer to the middle.

Taken together, these challenges suggest that the mental-health landscape is evolving faster than its guardrails. For many young people, the first listener may not be a counselor, teacher, or friend—it might be a chatbot. That immediacy can feel empowering, especially when human help seems slow or out of reach. Yet digital companionship is a double-edged sword: while it offers comfort, it may also dull the skills needed for real connection. What begins as a source of support might, over time, become a substitute for connection.

The second concern is subtler but no less serious. Chatbots built to comfort can still misread distress or deliver advice that sounds gentle but misses the mark. The syntax of care can disguise the absence of comprehension—and that’s where the rub lies. When emotional vulnerability meets algorithmic confidence, even small misfires can echo far beyond the screen. These systems don’t mean harm, but they can’t really grasp it either. Closing that gap means getting design and education to pull in the same direction. Smarter chatbots can nudge users toward human contact when distress peaks. Digital-emotional literacy can help young people tell fluency from empathy, and reassurance from real care. Together, these don’t just patch the system—they rebuild how help-seeking works.

 As governments, funders, and educators confront the next wave of digital mental-health innovation, the goal isn’t to slow progress but to shape it. This means investing in AI literacy frameworks as early as kindergarten, integrating escalation defaults into all youth-facing systems, and ensuring that emotional safety is treated as a design standard and not as an afterthought.

At The Decision Lab, we use behavioral science to bridge the space between innovation and human insight. It’s how we turn data into empathy that works—and systems into safeguards. By grounding technology in evidence about how people think, feel, and decide, we help organizations build tools that protect without patronizing and connect without replacing. If your team is developing AI for mental health, let’s talk about how to make progress without losing the people it’s meant to serve.

Combatting Online Misinformation in Mental Health Content 

Teens aren’t the only ones who fall for mental health misinformation online. For healthcare leaders, digital health innovators, and policymakers trying to promote credible, evidence-based content, keeping up with viral misinformation can feel like an uphill battle. Want to learn how to slow its spread and amplify what’s reliable? Keep reading.

Bringing CBT to the Workplace 

Not all AI chatbots are bad for mental health. Developed by leading Canadian mental health professionals, Hikai is an AI-powered CBT chatbot built for the workplace. Discover how ethical design and behavioral insight can turn AI into a tool that helps people and not one that harms them.

1. Kieling, C., Buchweitz, C., Caye, A., Silvani, J., Ameis, S. H., Brunoni, A. R., Cost, K. T., Courtney, D. B., Georgiades, K., Merikangas, K. R., Henderson, J. L., Polanczyk, G. V., Rohde, L. A., Salum, G. A., & Szatmari, P. (2024). Worldwide prevalence and disability from mental disorders across childhood and adolescence: Evidence from the global burden of disease study. JAMA Psychiatry, 81(4), 347–356. https://doi.org/10.1001/jamapsychiatry.2023.5051 
2. Sacco, R., Camilleri, N., Eberhardt, J., Umla-Runge, K., & Newbury-Birch, D. (2024). A systematic review and meta-analysis on the prevalence of mental disorders among children and adolescents in Europe. European Child & Adolescent Psychiatry, 33(9), 2877–2894. https://doi.org/10.1007/s00787-022-02131-2 
3. Reavley, N. J., Cvetkovski, S., Jorm, A. F., & Lubman, D. I. (2010). Help-seeking for substance use, anxiety, and affective disorders among young people: Results from the 2007 Australian National Survey of Mental Health and Wellbeing. Australian & New Zealand Journal of Psychiatry, 44(8), 729–735. https://doi.org/10.3109/00048671003705458 
4. Australian Bureau of Statistics. (2023). National study of mental health and wellbeing: 2020–2022. https://www-abs-gov-au.myaccess.library.utoronto.ca/statistics/health/mental-health/national-study-mental-health-and-wellbeing/2020-2022
5. Clement, S., Schauman, O., Graham, T., Maggioni, F., Evans-Lacko, S., Bezborodovs, N., Morgan, C., Rüsch, N., Brown, J. S. L., & Thornicroft, G. (2015). What is the impact of mental health-related stigma on help-seeking? A systematic review of quantitative and qualitative studies. Psychological Medicine, 45(1), 11–27. https://doi.org/10.1017/S0033291714000129 
6. Hoffman, B. D., Oppert, M. L., & Owen, M. (2024). Understanding young adults’ attitudes towards using AI chatbots for psychotherapy: The role of self-stigma. Computers in Human Behavior: Artificial Humans, 2(2), 100086. https://doi.org/10.1016/j.chbah.2024.100086 
7. American Psychological Association. (2025, June). Health advisory: Artificial intelligence and adolescent well-being. https://www.apa.org/topics/artificial-intelligence-machine-learning/health-advisory-ai-adolescent-well-being
8. Brewster, R. C., Zahedivash, A., Tse, G., Bourgeois, F., & Hadland, S. E. (2025). Characteristics and safety of consumer chatbots for emergent adolescent health concerns. JAMA Network Open, 8(10), e2539022. https://doi.org/10.1001/jamanetworkopen.2025.39022 
9. Dunn, J. R., & Schweitzer, M. E. (2005). Feeling and believing: The influence of emotion on trust. Journal of Personality and Social Psychology, 88(5), 736–748. https://doi.org/10.1037/0022-3514.88.5.736 
10. Milgram, S. (1974). Obedience to authority. Harper & Row.
11. Pichowicz, W., Kotas, M., & Piotrowski, P. (2025). Performance of mental health chatbot agents in detecting and managing suicidal ideation. Scientific Reports, 15(1), 31652. https://doi.org/10.1038/s41598-025-17242-4 
12. Pierre, J. M. (2025, October 27). Should AI chatbots be held responsible for suicide? Psych Unseen. Psychology Today. https://www.psychologytoday.com/us/blog/psych-unseen/202510/should-ai-chatbots-be-held-responsible-for-suicide
13. Kang, B., & Hong, M. (2025). Development and evaluation of a mental health chatbot using ChatGPT 4.0: Mixed methods user experience study with Korean users. JMIR Medical Informatics, 13, e63538. https://doi.org/10.2196/63538 
14. Giri, S. P., & Dubey, A. (2023). Loneliness in the time of COVID-19: An alarming rise. The Lancet, 401(10394), 2107–2108. https://doi.org/10.1016/S0140-6736(23)01125-X 
15. Phang, J., Lampe, M., Ahmad, L., Agarwal, S., Fang, C. M., Liu, A. R., Danry, V., Lee, E., Chan, S. W. T., Pataranutaporn, P., & Maes, P. (2025). Investigating affective use and emotional well-being on ChatGPT [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2504.03888 
16. Hu, B., Mao, Y., & Kim, K. J. (2023). How social anxiety leads to problematic use of conversational AI: The roles of loneliness, rumination, and mind perception. Computers in Human Behavior, 145, 107760. https://doi.org/10.1016/j.chb.2023.107760 
17. Andoh, E. (2025, October 1). Many teens are turning to AI chatbots for friendship and emotional support. Monitor on Psychology, 56(7), 51. American Psychological Association. https://www.apa.org/monitor/2025/10/ai-chatbots-teens
18. Ovsyannikova, D., de Mello, V. O., & Inzlicht, M. (2025). Third-party evaluators perceive AI as more compassionate than expert humans. Communications Psychology, 3(1), 4. https://doi.org/10.1038/s44271-024-00182-6 
19. Stöckli, L., Joho, L., Lehner, F., & Hanne, T. (2024). The personification of ChatGPT (GPT-4): Understanding its personality and adaptability. Information, 15(6), 300. https://doi.org/10.3390/info15060300 
20. Ng, D. T. K., Leung, J. K. L., Chu, K. W. S., & Qiao, M. S. (2021). AI literacy: Definition, teaching, evaluation, and ethical issues. Proceedings of the Association for Information Science and Technology, 58(1), 504–509. https://doi.org/10.1002/pra2.487 
21. Williams, R., Park, H. W., & Breazeal, C. (2019, May). A is for artificial intelligence: The impact of artificial intelligence activities on young children’s perceptions of robots. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–11). https://doi.org/10.1145/3290605.3300677 
22. Casal-Otero, L., Catala, A., Fernández-Morante, C., Taboada, M., Cebreiro, B., & Barro, S. (2023). AI literacy in K–12: A systematic literature review. International Journal of STEM Education, 10(1), 29. https://doi.org/10.1186/s40594-023-00418-7