• Digital mental health tools promise scalable care but haven’t solved privacy and autonomy issues. Personalization keeps improving, yet users still don’t know who’s watching—or where their data goes.
• By making personalization an opt-in, transparent, and reflection-based process, platforms can respect autonomy instead of steering users.
• Embedding privacy by design through local storage, readable consent pages, added friction layers, and quick deletion processes keeps data close to users while restoring confidence in digital mental health care.

Despite decades of progress in awareness and treatment, global prevalence rates of mental health disorders haven’t gone down.² These disorders still account for a large share of years lived with disability, and the clinical workforce can’t meet growing demand. To fill that gap, digital tools have been positioned as scalable, lower-cost supports that could reach people who traditional systems don’t.³ 

The pandemic accelerated the adoption of digital mental health tools. Developers built apps that track moods, deliver guided exercises, and connect users with virtual counselors. Each platform collects behavioral data and may gather information ranging from how people sleep, type, and record emotion to tailor recommendations. These personalization methods may improve adherence and outcomes, yet they also redefine what care means when interaction occurs through code rather than conversation. 

As digital platforms multiply, users face an overload of choices and recommender systems now frequently decide what content appears first.⁴ That guidance can help surface relevant support, but it can also narrow autonomy when algorithms start defining what progress should mean. Personalization that’s designed to guide can end up steering, and that difference matters most when users are vulnerable.

Simultaneously, privacy remains uncertain because data flows are often opaque and privacy agreements are difficult to parse.⁵ Continuous monitoring also leaves people unsure of what’s being stored, how long it stays, or who might access it, and that uncertainty weakens trust across the entire field.

Behavioral science can restore some of that balance. It can make personalization opt-in and privacy rights visible, while promoting plain-language communication that invites genuine consent. It can also encourage reflective decision-making instead of maximizing clicks, and embed privacy as a design principle so systems collect less, explain more, and give users real control. Together, these steps could help digital mental health care evolve responsibly and stay grounded in trust.

Personalization lies at the center of most digital mental health tools. Algorithms analyze user behavior to deliver recommendations, reminders, or motivational prompts.⁴ While these features are framed as supportive, they can also narrow choice and reinforce one version of what “well-being” should look like. Over time, the design logic of help—when to intervene, what to encourage, and how progress is defined—may begin to replace the user’s own judgment.

This dynamic reflects a form of soft paternalism: influencing behavior under the assumption that guidance will improve outcomes.⁶ In mental health contexts, that influence carries unusual weight. People using these systems are often in distress, fatigued, or cognitively burdened. They may accept prompts or advice without questioning where they come from or whose standards of health they represent. The result can entail a gradual transfer of agency from the user to the system—an outcome that may feel supportive but functions as subtle coercion.

The boundary between assistance and control becomes thinner when machine-learning models are introduced. These systems “learn” from historical data to predict what a person should do next: when to practice mindfulness, how to label a mood, or which coping strategy to apply. However, those datasets are rarely representative. Much of the evidence behind emotion-tracking and intervention models still comes from WEIRD populations: Western, educated, industrialized, rich, and democratic.⁷ These samples define distress, improvement, and recovery through specific cultural norms.⁸ When algorithms trained on that data are deployed globally, they may reproduce a single moral and emotional standard across diverse contexts.

Examples of this may already be visible in commercial wellness apps and digital therapeutics. Language-processing systems used for mood prediction often rate high-arousal emotions such as anger, grief, or passion, as “negative affect.” In cultures where open emotional expression signals authenticity or spiritual balance, these classifications could misinterpret healthy behavior as pathology.⁹ This isn’t an isolated bug in app development but rather reflects how training data encode value judgments into metrics of wellness.

The ethical risk extends beyond classification. Persuasive design techniques, such as streak counters or compliance badges, can reinforce the idea that adherence equals improvement. Many of these mechanisms draw from behavioral economics, where loss aversion describes people’s tendency to avoid losing progress even when the “loss” is arbitrary—like breaking a meditation streak.¹⁰ Commitment devices, such as prompts to set long-term goals or self-contracts,¹¹ may further lock users into daily engagement patterns designed to reduce dropout rather than support autonomy. What motivates productivity in fitness or finance can induce guilt or pressure in mental health settings, where lapses are expected and recovery is non-linear. Each prompt, however small, acts as a behavioral nudge that may reshape self-perception and choice architecture.

In traditional psychiatry, coercion can appear as forced treatment or conditional access to care. In digital systems, it emerges through interface logic: which buttons are easier to tap, which metrics appear first, which emotions receive validation. Users rarely see how these patterns are chosen or adjusted. Personalization therefore introduces an ethical paradox. The more accurately a system predicts what users might need, the more power it could gain to decide what they should do. Without deliberate constraints, that predictive precision may harden into paternalism and guide behavior under the banner of support while diminishing the independence it aims to strengthen.

Digital mental health systems inherit two core ethical vulnerabilities: paternalism, where support becomes subtle coercion, and dataset bias, where one culture’s version of “well-being” is exported as a universal truth. To counter these risks, design must restore epistemic humility—the recognition that algorithms don’t “know” what is best for a user—and embed autonomy-preserving mechanisms throughout personalization. The strategies below translate that principle into practice.

1) Make all personalization opt-in, not automatic

Automatic personalization assumes that data-driven adaptation is inherently beneficial, a pattern that can replicate algorithmic paternalism. Users rarely recognize that predictive models decide what content appears, when prompts arrive, and in what tone. Opt-in personalization interrupts the default effect,¹² which might nudge passive acceptance even when a user would prefer limited adaptation. 

Periodic renewal of consent also acknowledges that autonomy changes over time: what feels supportive today may feel intrusive next month. By treating consent to personalize as a renewable process rather than a one-time checkbox, designers counter the paternalistic assumption that silence equals approval, especially when training data assume homogeneity in needs and values.

2) Provide clear “why this suggestion?” explanations and options to dismiss or change frequency

Many algorithmic systems operate as opaque authorities. Explainable-AI (XAI) methods aim to reduce that opacity by offering human-understandable reasons for outputs, enabling oversight rather than blind trust.¹³ Counterfactual explanations—for example, “This was suggested because you logged poor sleep twice this week; disabling sleep tracking would stop similar prompts”—clarify contingencies and data dependencies. When paired with straightforward options to dismiss a recommendation or adjust its cadence, users negotiate with the system rather than comply with it. The act of refusing or modifying a recommendation also generates corrective data as it re-teaches the system what not to assume and can reduce the imprint of biased labels that define “improvement” or “relapse” too narrowly.

3) Allow quiet modes or “pause personalization” features, especially during vulnerable periods

In clinical ethics, nonmaleficence requires sensitivity to cognitive and emotional load.¹⁴ Applied to digital settings, this means giving users the power to pause algorithmic attention. Continuous nudging can reproduce coercive dynamics under a softer interface and may undermine self-regulation when users need distance. Quiet modes should be simple to activate, visible at all times, and non-punitive: no loss of points, streaks, or progress metrics. 

4) Scaffold reflective decision-making rather than optimize for engagement

Traditional app metrics such as daily active users, streaks, and session length risk equating “more engagement” with “more wellness.” Recovery depends on reflection and internalization, not throughput. Drawing on Self-Determination Theory (SDT)¹⁵ and the Capability Approach,¹⁶ systems should expand autonomy and competence: users’ confidence in directing and evaluating their own mental health journeys. Brief reflection scaffolds—for example, “Did this feel helpful?” “Did this respect your boundaries?”—invite users to assess fit, timing, and tone. Each reflective act strengthens self-regulation and supports SDT’s criteria for autonomous, self-endorsed action. 

Within the Capability Approach, these same loops enlarge real opportunities to pursue personally valued outcomes. Progress shouldn’t be defined by raw usage or generic symptom averages when reflective design can be implemented to help users articulate what well-being means to them and assess whether digital support is helping them move toward that definition.

5) Ensure algorithmic transparency and visible filters

When algorithms filter what users see, those filters shape how people interpret themselves and their options. Users might over-trust or uncritically follow recommendations because outputs appear objective. Transparency here isn’t a technical bonus; it’s a cognitive safeguard. Systems should disclose:

• Data provenance: where training data came from, collection windows, sample characteristics, and known limitations (e.g., WEIRD over-representation).
• Salient signals: the top factors that drove a suggestion (for example, time of day, recent sleep logs, language markers), with indicative weighting bands or thresholds.
• Alternative paths: actions the model considered but deprioritized, with brief rationales.
• User-level control: the ability to down-weight or disable specific signals and reset learned preferences.
  

Access to this information lets users compare, adjust, and decide whether guidance may be helpful or off-mark. It could reduce misplaced confidence in outputs that inherit cultural labels (e.g., classifying high-arousal emotions as inherently “negative”) and reveal where recommendations encode a single moral or emotional standard.

Personalization requires knowing the user. Overstepping begins when that knowledge turns into surveillance. The challenge is to collect enough data to adapt meaningfully, but not so much that people lose control over their information or feel observed. When privacy design fails, personalization stops being caring and starts being controlling.

Research shows that the privacy infrastructure of mental health technology is fragile. A systematic test of 27 top-ranked mental health apps used the LINDDUN privacy-threat model.¹⁷ Most apps exhibited linkability, identifiability, and detectability risks—meaning third parties could piece together user profiles and infer diagnoses. Only three developers confirmed performing a Privacy Impact Assessment, and just two had published the results. This pattern suggests that privacy oversight isn’t built into personalization—it’s often added, if at all, after deployment. 

Since mental health data include emotional histories, journal entries, and therapy logs, a single exposure could have significant lasting consequences—stigma, reputational damage, or discrimination. IBM’s 2020 global data breach report found healthcare to be the most costly industry for data breaches, suggesting that medical and health data remain among the highest-value targets for cybercriminals.¹⁸

Security flaws don’t always come from hackers. They often stem from development cultures that lack privacy literacy. Many developers still skip formal security testing due to limited budgets or time, leaving sensitive information stored or transmitted without adequate encryption.¹⁹ The result may be software that overcollects by default and protects by exception. Every missing safeguard increases the odds that user data will travel farther than anyone intended.

Even where privacy policies exist, they’re written in ways few people can digest. In the same prior analysis, 24 of 27 policies required college-level literacy to understand.¹⁷ That complexity makes informed consent a formality rather than a choice. Users might click “agree” without realizing that their data may be reused for analytics, model training, or sold to affiliates. Behavioral research calls this the default effect—people tend to accept pre-set options, especially when they’re anxious or cognitively taxed. In mental health contexts, those pressures are magnified where individuals seeking relief might not have the bandwidth to vet legal language.

The consequences ripple outward. A cross-sectional review of 36 depression and smoking cessation apps found that 29 transmitted user data to ad networks like Google and Facebook, while only 12 disclosed it.²⁰ Such hidden transfers might expose sensitive conditions to commercial entities, turning emotional distress into a marketing signal. Some privacy policies even blur the line between medical research and product design research, implying data-sharing with “research partners” when those partners are, in fact, corporate developers testing engagement models.

When information about mental health enters commercial ecosystems, it doesn’t stay contained. It’s aggregated, scored, and eventually repurposed in contexts users never consented to—insurance underwriting, targeted drug ads, or predictive screening tools. Each new transfer deepens mistrust. And when people feel watched, they self-censor, share less, or abandon help altogether. The core issue is that personalization depends on intimacy, but unbounded intimacy erodes privacy. If systems can know everything, they will until designers decide where knowing should stop.

If personalization defines what digital mental health tools can do, privacy defines what they should never overstep. Ethical personalization starts with boundaries: data are collected to help the individual in front of the screen, not to build invisible markets around them. A credible privacy strategy has to be both technical and behavioral, engineered to reduce risk while showing users what’s happening to their information at every step.

1) Local-first personalization and clear data controls

Most mental health apps watch what users do, then send that data to remote servers for analysis. For a population managing anxiety, trauma, or depression, this practice can feel extremely invasive. A local-first design reverses that pattern. The phone performs most computations itself, learning from patterns without exporting raw data. For instance, the device might detect that a user tends to log stress between 6 p.m. and 8 p.m., but it keeps the notes, timestamps, and sensor traces on the device. Only if the person opts in to broader research does the app send short, blurred summaries that can’t be re-linked to an identity.

Local-first systems may not eliminate every risk, but they significantly reduce exposure by keeping sensitive information close to the user where it’s never stored on a cloud. They also build trust because people can see exactly what’s stored and for how long. A single “Data Controls” page could show four basic levers—what’s saved, where it lives (device or cloud), how long it’s retained, and a one-tap delete. If users disable a feature, the app remains functional while personalization is reduced.

2) Data minimization and short retention

Privacy by design starts with restraint. Mental health tools should collect only the data required for the feature that’s active. Background tracking, device IDs, and behavioral analytics might support product metrics but rarely serve the therapeutic goal. Default retention windows of 30–90 days keep historical data useful for pattern recognition but limit long-term exposure. Any storage beyond that should require renewed consent. Automatic deletion of dormant records protects users who forget to clean their histories and discourages silent stockpiling.

3) Plain-language consent and readable policies

Informed consent collapses when policies read like legal defense documents. Most privacy policies for mobile health apps still require college-level literacy,²¹ effectively excluding a significant portion of the very people they claim to protect. Plain-language communication isn’t cosmetic; it’s functional. It tells users what’s being collected, who sees it, and when it’s deleted. Developers could treat language clarity as a measurable compliance metric with each policy tested for grade level and user comprehension before release.

4) User data control and deletion rights

Users should be able to export summaries, delete histories, or revoke sharing permissions at any time without penalty. Simple deletion workflows may look trivial but carry symbolic power: they signal that control remains with the individual. Export features also support transparency—people can review what the system knows and decide whether that knowledge still serves them.

5) Friction for reflection

Adding a short pause before sensitive consent screens—“Are you comfortable sharing this data for recommendations?”—creates a behavioral checkpoint. This deliberate moment might prompt users to weigh their comfort level before approving a data flow. Such friction should be minimal but meaningful, reminding users that privacy is a continuous participatory process.

Personalization and opt-in systems can sound simple, but they aren’t for everyone. Users with limited digital literacy may find consent screens, reflection prompts, or adjustable settings confusing—especially when they’re already anxious or fatigued. Without clear onboarding or accessible design, autonomy risks becoming a privilege reserved for the tech-confident rather than a right extended to all.

At the same time, privacy-by-design isn’t free. Most small mental health-tech startups don’t have the staff or capital to build secure local storage, differential-privacy pipelines, or on-device models.¹⁹ Those safeguards take both time and expertise that early-stage teams can’t always afford. Without shared frameworks or public tools, ethical infrastructure doesn’t scale evenly and concentrates where resources already exist.

That’s where broader governance comes in, though progress has been slow. Many digital mental health tools still sit between wellness products and regulated care, leaving their privacy obligations vague and their accountability uneven. Governments and regulators will need to do more than issue guidance, they’ll need to legislate, monitor, and enforce standards that protect users as consistently as clinical care does.^23,24 Public frameworks and shared auditing tools could help smaller developers meet those standards without being priced out. That’s where public frameworks and shared auditing tools allow standards to scale without stifling smaller developers.

Digital mental health tools were created to extend care—to listen, protect, and respond when traditional systems couldn’t keep up. Yet the challenges we’ve explored suggest that good intentions can drift. Personalization may start to guide but end up steering; privacy-by-design may remain out of reach for small teams; and even the clearest consent screens might not translate into true understanding. 

The opportunities discussed—opt-in personalization, reflection-based design, and local-first privacy—offer ways to bring intention back into balance. When users can see what’s collected, adjust what’s shared, or pause personalization when needed, digital care may begin to feel like care again.

The next phase shouldn’t be about faster scaling—it should be about more thoughtful scaling and sensitivity over speed. Thoughtful personalization could learn to help without assuming, prompt without pressuring, and adapt without taking over, protecting autonomy as actively as they deliver support.

At The Decision Lab, we take pride in applying behavioral science to make digital care safer, fairer, and more human. Our work sits at the intersection of psychology, policy, and technology, translating human behavior into systems that protect as much as they personalize. We’d love to collaborate if your team’s ready to build digital mental health solutions that feel both supportive and transparent. Together, we can make digital mental health what it was meant to be: adaptive and humane, while avoiding designs that overstep the very autonomy they aim to support.

Navigating the New AI Mental Health Landscape Among Youth

Many young people are turning to digital mental health tools, particularly conversational AI systems that simulate therapy-style dialogue outside clinical settings. These tools can lower stigma and make support feel more accessible, yet they also change how empathy and trust form. To make this landscape safer, we’ll need to study how people actually use these systems—and behavioral science is well-positioned to guide that work.

Bridging the digital divide: How can we keep healthcare accessible in the digital age

Healthcare has moved rapidly into the digital world, and new tools keep entering the picture. However, large groups of people who don’t have reliable internet or digital skills can’t fully take part in this new wave of modern care. This article examines how the widening digital divide is shaping access, who’s most affected, and what interventions could close the gap.

1. Federal Trade Commission. (2024, May 6). BetterHelp, Inc., in the matter of (Docket No. 2023169). https://www.ftc.gov/legal-library/browse/cases-proceedings/2023169-betterhelp-inc-matter
2. Ormel, J., & VonKorff, M. (2021). Reducing common mental disorder prevalence in populations. JAMA Psychiatry, 78(4), 359–360. https://doi.org/10.1001/jamapsychiatry.2020.3443 
3. Lattie, E. G., Stiles-Shields, C., & Graham, A. K. (2022). An overview of and recommendations for more accessible digital mental health services. Nature Reviews Psychology, 1(2), 87–100. https://doi.org/10.1038/s44159-021-00003-1 
4. Valentine, L., D’Alfonso, S., & Lederman, R. (2023). Recommender systems for mental health apps: Advantages and ethical challenges. AI & Society, 38(4), 1627–1638. https://doi.org/10.1007/s00146-021-01322-w 
5. Lustgarten, S. D., Garrison, Y. L., Sinnard, M. T., & Flynn, A. W. (2020). Digital privacy in mental healthcare: Current issues and recommendations for technology use. Current Opinion in Psychology, 36, 25–31. https://doi.org/10.1016/j.copsyc.2020.03.012 
6. Panda, O. D., & Binkley, C. E. (2025). Governance of direct‐to‐user digital mental health tools: Emphasizing transparency over paternalism. Hastings Center Report, 55(3), 29–33. https://doi.org/10.1002/hast.5009 
7. Duan, W., Klibert, J., Schotanus-Dijkstra, M., Llorens, S., van den Heuvel, M., Mayer, C. H., Tomasulo, D., Liao, Y., & van Zyl, L. E. (2022). Positive psychological interventions: How, when and why they work: Beyond WEIRD contexts. Frontiers in Psychology, 13, 1021539. https://doi.org/10.3389/fpsyg.2022.1021539 
8. Kohrt, B. A., Rasmussen, A., Kaiser, B. N., Haroz, E. E., Maharjan, S. M., Mutamba, B. B., B. B., de Jong, J. T., & Hinton, D. E. (2014). Cultural concepts of distress and psychiatric disorders: Literature review and research recommendations for global mental health epidemiology. International Journal of Epidemiology, 43(2), 365–406. https://doi.org/10.1093/ije/dyt227 
9. Mesquita, B., & Walker, R. (2003). Cultural differences in emotions: A context for interpreting emotional experiences. Behaviour Research and Therapy, 41(7), 777–793. https://doi.org/10.1016/S0005-7967(02)00189-4 
10. Novemsky, N., & Kahneman, D. (2005). The boundaries of loss aversion. Journal of Marketing Research, 42(2), 119–128. https://doi.org/10.1509/jmkr.42.2.119.62292 
11. Bryan, G., Karlan, D., & Nelson, S. (2010). Commitment devices. Annual Review of Economics, 2(1), 671–698.
12. Thaler, R. H., & Sunstein, C. R. (2021). Nudge: The final edition. Penguin.
13. Dwivedi, R., Dave, D., Naik, H., Singhal, S., Omer, R., Patel, P., … Ranjan, R. (2023). Explainable AI (XAI): Core ideas, techniques, and solutions. ACM Computing Surveys, 55(9), 1–33. https://doi.org/10.1145/3561048 
14. Varkey, B. (2021). Principles of clinical ethics and their application to practice. Medical Principles and Practice, 30(1), 17–28. https://doi.org/10.1159/000509119 
15. Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68 
16. Walker, M., & Unterhalter, E. (2007). The capability approach: Its potential for work in education. In Amartya Sen’s capability approach and social justice in education (pp. 1–18). Palgrave Macmillan.
17. Iwaya, L. H., Babar, M. A., Rashid, A., & Wijayarathna, C. (2023). On the privacy of mental health apps: An empirical investigation and its implications for app development. Empirical Software Engineering, 28(1), Article 2. https://doi.org/10.1007/s10664-022-10236-0 
18. IBM Security, & Ponemon Institute. (2020). Cost of a data breach report 2020 [PDF]. IBM Corporation. https://www.ibm.com/security/digital-assets/cost-data-breach-report/1Cost%20of%20a%20Data%20Breach%20Report%202020.pdf
19. Aljedaani, B., Ahmad, A., Zahedi, M., & Babar, M. A. (2020, December). An empirical study on developing secure mobile health apps: The developers’ perspective. In 2020 27th Asia-Pacific Software Engineering Conference (APSEC) (pp. 208–217). IEEE. https://doi.org/10.1109/APSEC51365.2020.00029 
20. Huckvale, K., Torous, J., & Larsen, M. E. (2019). Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Network Open, 2(4), e192542. https://doi.org/10.1001/jamanetworkopen.2019.2542 
21. Sunyaev, A., Dehling, T., Taylor, P. L., & Mandl, K. D. (2015). Availability and quality of mobile health app privacy policies. Journal of the American Medical Informatics Association, 22(e1), e28–e33. https://doi.org/10.1136/amiajnl-2013-002605 
22. Cavoukian, A. (2021). Privacy by design: The seven foundational principles. IAPP Resource Center.
23. Singh, S., & Sagar, R. (2022). Time to have effective regulation of the mental health apps market: Maximize gains and minimize harms. Indian Journal of Psychological Medicine, 44(4), 399–404. https://doi.org/10.1177/02537176221082902 
24. World Health Organization. (2024). Ethics and governance of artificial intelligence for health: Large multi-modal models. WHO guidance. World Health Organization.