Last updated November 8, 2026
Just over 1 in 4 US adults (27%) say they trust businesses "some" or "a lot" to use AI responsibly, down from 31% a year earlier, according to the 2026 Bentley-Gallup Business in Society study of 3,270 adults. Among adults under 30, the share fell from 3 in 10 to 1 in 5.
At The Decision Lab, we design customer-facing AI so that customers rely on it as much as its performance on each task justifies, and we test how they act on its output before launch. In practice that comes down to five design levers: capability clarity, uncertainty communication, meaningful user control, error recovery, and feedback. Our work on this started before generative AI reached consumers, when we helped Mila turn COVI, a contact tracing app driven by an AI risk score, into a product people could understand and act on.
What calibrated trust in AI means
Calibrated trust in AI means a person's reliance on an AI system matches the system's actual capabilities and limitations. Users should rely on AI when it performs well and question it when uncertainty is high, while keeping the ability to override important decisions or escalate them to a person.
This guide covers how to design for that. For a framework to measure whether users rely on AI appropriately, read our guide to calibrated trust in AI.
Three ways reliance on AI fails
- Over-reliance. Users accept incorrect, incomplete or unsuitable AI outputs without enough review.
- Under-reliance. Users reject useful AI recommendations even when the system is reliable.
- Misplaced reliance. Users trust the AI for tasks it is not designed or qualified to handle, such as asking a budgeting assistant whether a deduction is legal.
We start each project by working out which of these is happening, and for which group of customers, because the fixes pull in opposite directions. Adding a confirmation step helps with over-reliance and makes under-reliance worse. Misplaced reliance usually needs neither and calls for a clearer statement of what the system is for.
The five design levers for calibrated trust
1. Capability clarity
Tell customers what the system can and cannot do before they rely on it, at the point where they will act on its output. A support assistant that says it can explain refund policies but cannot approve exceptions gives customers a reason to check before treating its answer as a promise.
The wording of the disclosure matters more than its presence. Across 13 experiments with more than 5,000 participants, Oliver Schilke and Martin Reimann found that disclosing AI use lowered trust, and that the steepest drop came when hidden AI use was discovered by someone else. Sacha Altay and Fabrizio Gilardi's experiments with 4,976 people in the US and UK found that headlines labeled as AI-generated were rated less accurate whether or not they were true, largely because people assumed the content had been written with no human supervision. A label can also change beliefs without changing behavior. In an experiment with more than 1,500 US participants, Isabel Gallegos and colleagues found that labeling a message as AI-generated changed who people thought wrote it but did not change how persuasive or accurate they found it.
So we recommend specific disclosure over a generic AI badge: what the AI did with which information, and whether a person checked it. A note that a spending summary was generated from the customer's last 12 statements, and that no person reviewed it, gives the customer something to calibrate against.
Disclosure is also a legal requirement in some markets. Since 2 August 2026, Article 50 of the EU AI Act has required providers of AI systems that interact directly with people, such as chatbots, to make sure users know they are dealing with AI by their first interaction, unless it is obvious from the context. The Act's scope covers providers outside the EU whose systems are used there.
None of this moves responsibility away from the company. In Moffatt v. Air Canada, British Columbia's Civil Resolution Tribunal ordered the airline to compensate a customer in 2024 after its website chatbot told him he could claim a bereavement discount after travel, contrary to the airline's actual policy. The tribunal rejected the argument that the chatbot was responsible for its own statements. The chatbot sat on the airline's own site and answered with confidence, so the customer relied on it the way the design invited.
2. Uncertainty communication
Show confidence, missing information, limitations and when a result needs review, wherever they change what the customer should do. Explanations are the most common response, and the evidence on them is mixed. In experiments presented at the 2021 ACM CHI Conference, Gagan Bansal and colleagues found that AI explanations made people more likely to accept the AI's recommendation whether or not it was correct. Yunfeng Zhang, Q. Vera Liao and Rachel Bellamy found that showing a confidence score with each prediction helped people calibrate their trust, while the explanations they tested did not.
For customers, uncertainty works best as a graded answer with a next step attached. A support assistant that would otherwise say "You are eligible for a refund" can say: "Based on the details provided, you may be eligible. Review the conditions below or ask a support specialist to confirm before you book." In COVI, we replaced blunt exposed-or-not alerts with graded risk guidance, so each message carried its own level of concern.
3. Meaningful user control
Let customers edit, reject, pause, override or escalate. Berkeley Dietvorst, Joseph Simmons and Cade Massey found that people were much more willing to use an imperfect forecasting algorithm when they could adjust its forecasts, even when the adjustment allowed was small. In a customer product, control can be modest: editing a draft before it is sent, moving a recommended budget up or down, excluding a category from recommendations, or asking for a person.
The route to a person needs to be visible before the customer needs it. An escalation option that appears only after several failed attempts reads as a trap to the customer who finally finds it.
4. Error recovery
Make it easy to correct a mistake and get back to a safe path, with a clear account of what went wrong. The first error a customer notices does disproportionate damage. In Dietvorst, Simmons and Massey's study of algorithm aversion, people who watched a forecasting algorithm err became less willing to use it than a human forecaster, even after seeing the algorithm outperform the human.
We plan the first visible error before launch: how the product acknowledges it, how the customer reports and corrects it inside the flow, what changes afterwards, and how to reach a person when the stakes are high. The Air Canada ruling adds a practical requirement. The AI's answers have to agree with the company's published policies, because the company owns both.
5. Feedback and learning
Where the system genuinely adapts, show customers how their feedback changes what they see. For a digital wellness platform serving 25 million members, we built a behavioral engine that models each member individually and adapts its recommendations as the member changes. For trust, the design question is whether members can see their own input reflected in what changes.
We treat one further point as a design hypothesis to test. A control that changes nothing, such as a feedback button with no visible effect, may make a system feel less responsive than having no control at all. The test compares a real adjustment control with a passive feedback control and measures whether customers notice and use each.
Design for the moment of reliance
The decisive moment comes when a customer is about to act on an AI output. We sort those moments by what happens if the AI is wrong, and match the safeguard to the stakes:
- Routine and reversible, such as checking an order status: a fast answer and light disclosure.
- Consequential but reversible, such as comparing plans or explaining a policy: show uncertainty and link to the source the AI used.
- Hard to reverse, such as approving a refund exception or moving a large sum: ask the customer to confirm, show the AI's rationale, route the decision to human review and keep an audit trail.
- High-impact or safety-related, such as a health risk or a legally binding commitment: escalate to a qualified person whatever confidence the AI reports.
Confirmation steps work, at a cost. Zana Buçinca, Maja Barbara Malaya and Krzysztof Gajos found that designs asking people to commit to their own judgment before seeing the AI's answer reduced over-reliance more than simple explanations in an experiment with 199 participants. Participants also rated the most effective designs least favorably, and the benefits went mainly to people who enjoy effortful thinking. We reserve that friction for the hard-to-reverse tier and test it with customers who differ in expertise and digital confidence, since a check that protects an expert can shut out a less confident customer.
The interface can also push customers toward acceptance without anyone intending it. We look for four patterns in every review: an "accept" button far more prominent than "edit" or "reject", the AI's suggestion pre-selected as the default, time pressure such as an offer that expires while the customer reads, and a route to a person that takes more steps than accepting the answer.
Test before launch
Before launch, we check that customers accept the AI's correct outputs far more often than its incorrect ones. For a global consumer electronics company, every target behavior for a new mindfulness service in a health app used by 60 million people was tested through experimentation before any production code was written, and design time fell 75% even though the process added research steps. For the wellness platform, we ran the full engine against synthetic members in a validation harness, so design failures surfaced in an afternoon instead of six months into production.
For the measurement framework behind these tests, including over-reliance and under-reliance rates, see our guide to measuring calibrated trust in AI. To test how a model behaves under adversarial use, see our guide to red teaming a large language model, and for decisions made by people working with AI, our guide to evaluating human-AI collaboration before you scale.
When not to put AI in front of customers
We advise against fully automated customer-facing AI when an error would be costly and customers have no practical way to correct it or reach a person. The same applies when a team cannot yet measure how often customers accept wrong outputs, since over-reliance stays invisible until it causes harm. In those cases, we recommend keeping the AI behind a person, for example by having it draft responses that a staff member reviews and sends, until testing shows customers rely on it appropriately.
Where calibrated trust matters most
- Financial services. Budgeting tools, fraud alerts, suitability checks and investment guidance ask customers to act on a machine's judgment about their money, and over-reliance costs them directly. Our investor protection work with the Ontario Securities Commission shapes how we approach these products.
- Digital health and wellness. A clinically correct recommendation can still go nowhere if people do not act on it, and a wrong one can do harm if they act without checking.
- Mental health services. Chat and coaching tools used by people in distress need an escalation route to a person that works every time. We led the redesign of Kids Help Phone's digital support gateway from research to launch, and we study how AI is changing youth mental health journeys.
- Customer service. Chatbots that answer questions about policies and prices speak for the company, and the Air Canada ruling shows the company owns every answer they give.
If you are building a customer-facing AI product, our team can test whether customers rely on it appropriately before launch. You can talk to us about behavioral testing for AI products or see our behavioral science consulting services.
Calibrated trust checklist
- Users can tell what the AI is doing and what it is for.
- Users can see uncertainty or limitations where they affect the decision.
- Users can challenge or override an output.
- There is a clear recovery path after an error, including a route to a person.
- Higher-stakes decisions are routed to appropriate human review.
- The team has tested whether users are over-relying on the system or underusing it.
Frequently asked questions
How do you build calibrated trust in AI products?
Design so customers rely on the AI as much as its performance on each task justifies. That means stating what the system can and cannot do, showing uncertainty where it matters, giving real control, planning for the first visible error and matching safeguards to the stakes. Then test whether customers accept correct outputs more often than incorrect ones.
Does labeling advice as AI-generated change trust?
Usually it lowers trust in the advice, especially when people assume no human checked it, and hidden AI use that is later discovered lowers trust most. Labels do not reliably change behavior, though: in one large experiment they changed who people thought wrote a message but not how persuasive they found it. Specific disclosure works better than a badge.
Do explanations help customers trust AI appropriately?
On their own they often do not. In experiments by Bansal and colleagues, explanations made people more likely to accept an AI's recommendation whether it was right or wrong. Confidence scores did more to help people calibrate, and designs asking people to decide first reduced over-reliance further, though people found them less pleasant to use.
When should an AI recommendation require human review?
When acting on a wrong output would be costly or hard to reverse, such as approving a refund exception, moving a large sum, changing medical information or signing a contract. Decisions with safety or legal consequences should go to a qualified person whatever confidence the AI reports. Routine, reversible tasks rarely need it.
Why do customers stop using AI after one mistake?
Research on algorithm aversion found that people lose confidence in an algorithm faster than in a person after seeing it err, even when the algorithm is more accurate. Follow-up experiments by the same researchers found that letting people adjust an imperfect algorithm's output, even slightly, made them more willing to keep using it.
What is misplaced reliance on AI?
Misplaced reliance happens when people trust an AI system for a task it was not built or qualified to handle, such as asking a shopping assistant for medical advice. The system may do well on its intended task, so overall accuracy figures will not reveal it. Clear statements of scope, repeated where customers act, address it better than warnings.

