Last updated October 19th, 2026
When users said goodbye to six popular AI companion apps, more than a third of the apps' replies used guilt or fear of missing out to keep them chatting, according to an audit of 1,200 farewells by Julian De Freitas of Harvard Business School and two co-authors (Emotional Manipulation by AI Companions).
An ethical choice architecture audit helps product teams find where a default, a prompt, a price or an AI response supports a user's own goals and where it crosses into manipulation. At The Decision Lab, we run the audit as one stage of a seven-stage behavioral science method for designing consumer choices, and we have used it on products from Mila's COVID-19 contact-tracing app to Kids Help Phone's youth mental health platform. The method scores every decision point against five tests and checks the riskiest ones with real users before harmful patterns become part of the product.
AI features make this harder, because a chatbot's reply to a user who wants to leave can change every time it is generated. In the same study's experiments, manipulative goodbyes made people keep chatting after saying goodbye up to 14 times more than neutral replies did, and also raised how manipulated they felt and how likely they said they were to quit the app.
The full behavioral science methodology behind ethical choice architecture
Our method runs in seven stages, and the ethical audit is the fifth. By the time we audit, we know what users are trying to do and what stands in their way, so each design is judged against a real goal.
- Define the decision and the desired user outcome. We name the decision the consumer is making and write down what informed, voluntary success would look like for them, before anyone looks at the business metric. What people end up doing is weak evidence of what they wanted: a user who keeps chatting after a guilt-laden goodbye message has done what the design asked, and that says nothing about whether chatting longer served them.
- Diagnose behavioral barriers. Using surveys, interviews, product analytics and observation, we find what stands between people and that outcome, such as friction, uncertainty, low comprehension, too many options to compare, social pressure, or present bias, the pull toward a small reward now over a larger one later. For Mila's COVI contact-tracing app, we ran weekly surveys of Canadians on attitudes toward COVID-19, trust in tracing apps, privacy concerns and anxiety about reopening, then built the findings into a decision model the product followed (how we redesigned COVID-19 risk communication).
- Map the choice environment. We log every point where a choice happens, with its default, the order of options, the price shown, the framing, the information available, the timing, the effort each option takes and any social cues. In our work with one of the world's largest fast-food chains on healthier choices for kids, we audited the customer journey across every sales channel, from the counter to the self-order kiosks, before proposing any change (redesigning nutritional choice architecture for kids).
- Design competing interventions. We draft at least one version built to improve the user outcome and one built to hit the business objective, then mark exactly where the two pull apart. Those conflict points get the closest attention in the next stage, because that is where pressure on the user usually appears.
- Audit for ethical risk. We score each decision point against the five tests below and rank the failures by priority.
- Validate with users and experiments. We test comprehension and people's sense of being free to choose, alongside actual behavior, cancellations, complaints, trust and downstream wellbeing. A controlled experiment is the strongest evidence, though some fixes, like a cancellation flow far longer than sign-up, can be corrected on sight. With a global consumer electronics company, every target behavior for a new mindfulness service was tested through experimentation before production code was written, and design time still fell 75% (simulation-first behavioral design).
- Monitor and re-audit. We hand over written rules for each flow and a list of changes that should trigger a fresh review: pricing changes, changes to personalization, new AI models and shifts in user behavior. Our clients keep the specification, so their teams can apply it to features we never see.
The five tests we apply to every decision point
We judge each decision point in a product on the five questions below, and a design has to pass all of them to count as ethical.
| Test | The question we ask | A typical failure |
|---|---|---|
| 1. Goal alignment | Does the default or prompt point toward what this user would choose after thinking it over? | A pre-ticked add-on most buyers would remove if they noticed it |
| 2. Disclosure | Would the design still work, and would the team still defend it, if users were told exactly how it works? | A countdown timer that resets every time the page reloads |
| 3. Effort symmetry | Is declining or cancelling as easy as accepting or subscribing? | Sign-up in two taps, cancellation by phone only |
| 4. Vulnerability | Who is most exposed to this design, and does it hit them harder than other users? | Urgency messages shown to people in financial or emotional distress |
| 5. Measured outcome | Does the evidence cover cancellations and wellbeing as well as the metric the design was built to move? | A test that tracks sign-ups and never looks at refunds |
The first test comes from the standard Richard Thaler and Cass Sunstein set out in Nudge: a nudge should leave people better off as judged by themselves. The fourth and fifth tests exist because harm rarely shows up in the metric a design was built for. When Jamie Luguri and Lior Strahilevitz showed a representative sample of US consumers an offer for a dubious identity-theft protection service, mild dark patterns more than doubled the share who signed up compared with a control group, and less educated participants were significantly more likely to fall for them (Shining a Light on Dark Patterns).
How we score a choice architecture audit
Each decision point gets one of four ratings on each test:
- Pass: the evidence shows the design meets the test.
- Needs evidence: the design may meet the test, but we cannot tell without user research or data.
- Fails: the design works against the test.
- High-risk failure: the design fails, and the harm is hard to reverse or lands on the people least able to absorb it.
A decision point becomes a priority when it fails two or more tests, scores a high-risk failure on any one, affects a vulnerable group, or involves cancellation, consent, a financial commitment or health. Priority points go to user research and testing first, and a needs-evidence rating counts as open work until the research closes it.
A worked choice architecture audit example: a free-trial subscription
This example is illustrative, built from patterns common in subscription products rather than from a single client. In the original flow, a consumer starts a free trial with renewal to a paid plan already selected, and the sign-up page shows a countdown timer that resets whenever the page reloads. Cancelling afterwards takes several steps. In the redesign, the sign-up screen states the renewal price and date in plain language, cancellation sits in account settings and takes no more steps than sign-up, the timer is gone, and users get a reminder before the first charge that they can switch off.
| Test | Original flow | Redesigned flow |
|---|---|---|
| 1. Goal alignment | Fails: paid renewal is pre-selected before the user has tried the product | Needs evidence: renewal stays on by default, so research has to show trial users expect it |
| 2. Disclosure | High-risk failure: the countdown resets on reload, so the urgency is false | Pass: no timer, and renewal terms are stated on the sign-up screen |
| 3. Effort symmetry | Fails: sign-up takes one screen, cancellation takes several | Pass: cancelling takes no more steps than signing up |
| 4. Vulnerability | Needs evidence: unknown whether people short on money are billed without meaning to be | Pass: the reminder gives people who forget a chance to cancel before paying |
| 5. Measured outcome | Fails: success is measured only by trial-to-paid conversion | Needs evidence: refund requests and complaints in the first month are tracked beside conversion |
The original flow fails four of the five tests, one of them as a high-risk failure, and it involves a financial commitment, so it would sit at the top of our priority list. The redesign passes three tests and leaves two needing evidence, so the next step is user research and a test against the original that counts refunds and complaints as well as conversions.
Auditing AI and conversational features
In an AI feature, the choice architecture sits in what the model says and what it suggests next, so a review of screenshots misses most of it. We audit these features by sampling their behavior: we run scripted and simulated conversations at the moments where users decide something, such as asking to cancel or saying goodbye, and score the replies against the same five tests.
De Freitas and his co-authors took a similar route, simulating conversations with each app and then sending a farewell. Asked whether the manipulative replies were deliberate, De Freitas told The Register the team could not know, given how little outsiders can see of commercial models (interview in The Register). A company auditing its own AI feature faces a version of the same problem, since a prompt or model update can change the replies without anyone redesigning a screen.
Personalization adds a second layer. When we built the behavioral engine for a digital wellness platform serving 25 million members, each member got recommendations scored on their own barriers, so a design that suits the average member could still fail the vulnerability test for one group. We ran the engine against synthetic members in a validation harness before launch, and design failures surfaced in an afternoon instead of six months into production (building a behavioral engine for 25 million members).
Where regulation is heading
Regulators now look for the same failures our tests check. In a 2024 review coordinated by the International Consumer Protection and Enforcement Network, consumer authorities from 26 countries examined 642 subscription websites and apps, and about 3 in 4 used at least one possible dark pattern, while about 2 in 3 used several (FTC summary of the review).
In the EU, the European Commission plans to propose a Digital Fairness Act in the last quarter of 2026, covering dark patterns, addictive design, unfair personalisation and influencer marketing (European Parliament legislative train). One idea under consideration is that cancelling should be as easy as subscribing, the same symmetry our third test checks (AKD briefing on the Digital Fairness Act).
We build our audits around what happens to users, so the findings stay useful as legal lists of banned practices change. We are a behavioral science firm and not a law firm, and legal sign-off stays with the client's counsel.
Working with The Decision Lab
Most of our choice architecture audits start with a product team that wants people to change a behavior and needs the route there to hold up to scrutiny from users and regulators. We combine behavioral research and experimentation with product design, so the team that finds a problem in the audit can also redesign the flow and test the fix.
Much of this work has been in health and wellness, where the stakes for vulnerable users are highest. We led research and product design for PocketWell, the companion app to the Government of Canada-funded Wellness Together platform (PocketWell case study). For Kids Help Phone, we audited the existing website and ran research with hundreds of young people across Canada before redesigning how they reach support (Kids Help Phone case study).
Many of the companies we work with already have an insights or UX research team. An audit from us is designed to sit beside that team, adding the simulations and experiments it may not run in-house, and leaving it with a standard it can apply on its own. It is probably not the right fit if you need a legal compliance opinion, or if the product decisions are already locked and no redesign is possible.
Related reading on choice architecture and behavioral design
- What choice architecture is and how it shapes decisions
- Nudge theory and how nudges change behavior
- Digital nudging in apps and websites
- How dark patterns steer users into unintended decisions
- Sludge and the friction that works against users
- Our behavioral science consulting services
- Case studies of our behavioral product design work
Frequently asked questions
What is the difference between a nudge and a dark pattern?
A nudge changes how a choice is presented to help people do what they would choose for themselves, while every option stays open and easy to pick. A dark pattern uses the same levers against the user's interest, such as a hidden fee or a cancellation path built to wear people down. The techniques can look identical, so we judge them by whose goal they serve.
What are examples of ethical choice architecture in consumer products?
These designs keep every option open and point toward what people would choose for themselves. Examples include a retirement plan that enrolls employees by default and lets them leave in one step, a reminder sent before a free trial turns into a paid plan, a plain-language price summary before checkout, and a menu screen that shows a balanced kids' meal first while every other meal stays one tap away.
Do you need a randomized controlled trial to audit choice architecture?
No. The audit itself is a structured review plus research with users, and many problems, like a cancellation flow far longer than sign-up, are clear without an experiment. A controlled test earns its cost when a redesign will move a business metric and the team needs to know by how much before rolling it out to everyone.
How do you run an AI manipulation audit on a chatbot or personalized product?
Sample its behavior at the moments where users decide something, such as asking to cancel or saying goodbye. Run enough scripted or simulated conversations to see the full range of replies, then score them against the same tests used for a screen. Repeat the sample after every model or prompt change, because the replies can shift while the interface looks the same.
Who should run an ethical choice architecture audit?
Someone outside the team that owns the metric being audited, because that team has the strongest reason to read a borderline design generously. An internal research or design ethics group can do it with a clear rubric. An outside behavioral science firm adds the most when the audit needs user research or experiments that the internal team cannot staff.
