Psychology of Human-AI Interaction (Trust & Appropriate Reliance)

Parent: Applied Human Psychology · researched 2026-05-31T20:28:42.848Z· 9 sources · 16 concepts · skill human-ai-interaction-psychology

> Standalone skill authored via the /dr deep-research workflow. Full SKILL.md

Psychology of Human-AI Interaction: Trust & Appropriate Reliance

> Standalone skill authored via the /dr deep-research workflow. Full SKILL.md [source]
> with TRIGGER/SKIP frontmatter and three references/ files is installed at [source]
> ~/.claude/skills/human-ai-interaction-psychology/. [source]
How humans decide whether to follow, override, or ignore an AI system, and how [source]
to design and coach for the right amount of reliance. This is human-factors [source]
and decision psychology applied to trust in machines, not interpersonal [source]
trust. The central problem is not "more trust" or "less trust" but **calibrated [source]
trust**: reliance that tracks the system's actual reliability in the specific [source]

When to use this skill

A TAM, customer, or team is over-relying (rubber-stamping AI output) or [source]
under-relying (ignoring a tool that outperforms them). [source]
An AI feature hits adoption resistance rooted in distrust, or **dangerous [source]
over-adoption** where users stop checking. [source]
You are designing an AI-assisted workflow (copilot, recommender, triage [source]
assistant, autoremediation gate) and must decide what to surface (confidence, [source]
explanations, friction) to get appropriate reliance. [source]
A confidently wrong AI answer was believed and you need the vocabulary to [source]
You are coaching a customer on a human-in-the-loop override policy. [source]

The one thing to get right

**Trust is an attitude; reliance is a behavior; appropriate reliance is the [source]
goal.** They are routinely conflated and must be kept separate. Optimizing for [source]
"trust" (a survey number) is the wrong target - optimize for **reliance that [source]
matches reliability**: follow the AI when it is right, override it when it is [source]
wrong. Most failures in AI-assisted decisions are miscalibration, not a global [source]

1. Calibrated trust and the trust–reliance distinction (Lee & See, 2004)

Trust = "the attitude that an agent will help achieve an individual's goals [source]
in a situation characterized by uncertainty and vulnerability." Reliance = [source]
the observable behavior that follows. Calibration = correspondence between [source]
trust and the system's true capability. [source]
Over-trust → over-reliance / misuse. Defers when it shouldn't. [source]
Under-trust → under-reliance / disuse. Rejects help that would have worked. [source]
The trust-calibration curve plots trust against true reliability; the [source]
diagonal is perfect calibration. Resolution = fine-grained trust that [source]
discriminates which cases the system handles well from those it doesn't (good [source]
calibration on average can still have poor resolution). Calibration is a closed [source]
loop, updated by performance feedback, disposition, and organizational norms. [source]
> Operator translation: don't ask "do you trust the tool?" Ask "for which [source]
> decisions does it earn the follow?" Coach for resolution, not blanket trust. [source]

2. Automation bias & complacency (Parasuraman & Manzey, 2010)

With an imperfect aid: commission errors (following a wrong automated [source]
directive without cross-checking) and omission errors (missing what the [source]
automation failed to flag because you weren't monitoring). **Automation [source]
complacency** is the attentional root - under load, monitoring drops. Appears in [source]
experts and novices; not reliably removed by training; occurs in teams [source]
(redundancy can backfire via diffused responsibility); worsens with very high [source]
automation reliability. [source]
> Operator translation: "a human reviews it" is a control only if it forces [source]
> engagement; under load it decays to rubber-stamping. [source]

3. Algorithm aversion (Dietvorst et al., 2015)

People abandon algorithms faster than humans after seeing them err, even [source]
when the algorithm outperforms them. Error visibility is the trigger (seeing [source]
it fail, not the failure rate). The 2018 follow-up: letting people **adjust the [source]
algorithm's output even slightly** restores willingness to use it (control lever). [source]
> Operator translation: a single visible miss can sink a net-better tool — [source]
> counter with adjustability, expectation-setting before the first error, and [source]
> framing errors as bounded. [source]

4. Algorithm appreciation (Logg et al., 2019) — reconciling the two

In six experiments people **weighted advice MORE heavily when told it came from [source]
an algorithm (Weight-On-Advice). Appreciation waned with domain expertise** [source]
and when choosing algorithm-vs-their own judgment. The two literatures are [source]
reconciled by moderators: [source]

5. Why explanations & confidence displays often FAIL to calibrate reliance

Plausible-but-wrong explanations increase over-reliance (Bansal et al., [source]
2021, "Does the Whole Exceed Its Parts?"): explanations raised acceptance [source]
whether the AI was right or wrong - agreement up, accuracy not. [source]
Confidence helps only if calibrated; miscalibrated confidence degrades [source]
decision quality, and displayed AI confidence shifts the human's own [source]
self-confidence (anchoring uncertainty without improving ability). [source]
Mechanism (dual-process): explanations feed the accept-heuristic rather [source]
than interrupting it. [source]
> Operator translation: "we added explanations/confidence" is not evidence of [source]
> appropriate reliance - verify behaviorally (does override-rate track [source]
> error-rate?). Ship confidence numbers only if validated as calibrated. [source]

6. Cognitive forcing functions (Buçinca, Malaya & Gajos, 2021)

Friction that compels analytical engagement at decision time: commit-first [source]
(judge before the AI is revealed), on-demand reveal / wait, **show reasoning [source]
on request** + surface disagreement/uncertainty. These reduced over-reliance on [source]
incorrect AI more than explanation-only designs. Costs: effort, often disliked, [source]
benefit interacts with the user (Need for Cognition) - reserve for high-stakes / [source]
likely-wrong cases. Adjacent levers: onboarding on error boundaries, selective [source]
explanations, adjustable outputs. [source]

7. Human-AI complementarity (CTP)

Complementary Team Performance = human+AI beat both alone, achieved only [source]
when their errors differ and each defers where the other is better. **CTP is [source]
rare by default** - teams often do worse than the AI alone. Put the human where [source]
they have an information edge the model lacks (context, unobservables), not as a [source]

8. Anthropomorphism, persona & the uncanny valley

Anthropomorphic cues (persona, warmth, avatar) can raise initial trust but are [source]
mediated by perceived empathy/interaction quality. Uncanny valley (Mori, [source]
1970): near-human-but-not affinity drops sharply; an "uncanny valley of trust" [source]
raises competence expectations the bot can't meet. A warm, fluent, confident [source]
persona manufactures over-trust regardless of correctness (fluency reads as [source]
competence) - match persona confidence to validated capability. [source]

Design & coaching checklist

Target appropriate reliance, measured behaviorally (override tracks error) — [source]
not a trust survey number or raw agreement. [source]
Set honest expectations before the first error. [source]
Show confidence only if calibrated; communicate uncertainty honestly. [source]
Don't expect explanations to create skepticism (they raise acceptance); pair [source]
with friction; prefer selective explanations on likely-error cases. [source]
Engineer friction where stakes are high (commit-first, on-demand reveal) — [source]
and reserve it; it has a cost. [source]
Give users control/adjustability over outputs (restores reliance after errors). [source]
Place the human where they have an information edge, not as a generic reviewer. [source]
Match persona confidence to validated capability. [source]
Treat "a human reviews it" as a design problem, not a safeguard. [source]

Anti-patterns

Optimizing for "trust" as a survey number instead of calibrated reliance. [source]
Shipping explanations/confidence and declaring over-reliance solved (they [source]
Treating a human-in-the-loop step as a guaranteed control. [source]
Letting one visible AI error kill adoption of a net-better tool. [source]
Maxing out a confident anthropomorphic persona on a high-stakes tool. [source]
Assuming "human + AI" beats either alone (complementarity is rare). [source]
Conflating trust and reliance in instrumentation. [source]

Operator scenarios (TAM / AI-native workflow)

"Team rubber-stamps the AI triage." → automation bias/complacency + [source]
over-reliance. Fix: commit-first workflow, surface disagreement, instrument [source]
agreement-on-wrong, reserve the human for context the model lacks. [source]
"Analysts refuse the new recommender." → likely algorithm aversion [source]
(experts, post-error, model-vs-own-judgment). Fix: adjustability, [source]
expectation-setting, advisor framing, show win-rate vs. baseline. [source]
"We added explanations and people trust it more - ship it?" → more [source]
agreement is not more appropriate reliance; verify override tracks error. [source]
"Friendly human persona for the assistant?" → lifts likability but risks [source]
over-trust and the uncanny valley; keep high-stakes tools capability-honest. [source]
"Human-in-the-loop / override policy?" → define by resolution (specific [source]
case classes needing independent judgment), not a blanket "review everything." [source]

Key sources

Lee & See (2004), Trust in Automation: Designing for Appropriate Reliance, [source]
Parasuraman & Manzey (2010), *Complacency and Bias in Human Use of [source]
Automation*, Human Factors 52(3). [source]
Dietvorst, Simmons & Massey (2015), Algorithm Aversion, JEP:General 144(1); [source]
and Dietvorst et al. (2018), Overcoming Algorithm Aversion, Management Science. [source]
Logg, Minson & Moore (2019), Algorithm Appreciation, OBHDP 151. [source]
Bansal et al. (2021), Does the Whole Exceed Its Parts?, CHI 2021. [source]
Buçinca, Malaya & Gajos (2021), To Trust or to Think, Proc. ACM HCI (CSCW1). [source]
Mori (1970/2012), The Uncanny Valley, IEEE Robotics & Automation Magazine. [source]
Microsoft Research (2024), Appropriate Reliance on Generative AI; plus CHI [source]
2024-2025 work on miscalibrated AI confidence and confidence/self-confidence [source]

Children

Calibrated Trust and the Trust-Calibration Curve (Lee & See) (frontier)
Trust as Attitude vs. Reliance as Behavior (frontier)
Over-trust/Over-reliance vs. Under-trust/Disuse (frontier)
Trust Resolution and Specificity (frontier)
Automation Bias (Commission vs. Omission Errors) (frontier)
Automation Complacency (Parasuraman & Manzey) (frontier)
Algorithm Aversion (Dietvorst) (frontier)
Algorithm Appreciation (Logg) (frontier)
Reconciling Aversion vs. Appreciation (Moderators) (frontier)
Explanation/Transparency Effects on Reliance (Bansal) (frontier)
Confidence Display and Miscalibrated Confidence (frontier)
Cognitive Forcing Functions (Bucinca) (frontier)
Human-AI Complementarity / Complementary Team Performance (frontier)
Appropriate-Reliance Interventions (frontier)
Anthropomorphism, AI Persona and Uncanny Valley (frontier)
Design Principles for Calibrated Reliance (frontier)

Frontier under this node: Algorithm Appreciation (Logg), Algorithm Aversion (Dietvorst), Anthropomorphism, AI Persona and Uncanny Valley, Appropriate-Reliance Interventions, Automation Bias (Commission vs. Omission Errors), Automation Complacency (Parasuraman & Manzey), Calibrated Trust and the Trust-Calibration Curve (Lee & See), Cognitive Forcing Functions (Bucinca), Confidence Display and Miscalibrated Confidence, Design Principles for Calibrated Reliance, Explanation/Transparency Effects on Reliance (Bansal), Human-AI Complementarity / Complementary Team Performance, Over-trust/Over-reliance vs. Under-trust/Disuse, Reconciling Aversion vs. Appreciation (Moderators), Trust Resolution and Specificity, Trust as Attitude vs. Reliance as Behavior

← the whole tree · 3D view· how to read this page

Psychology of Human-AI Interaction (Trust &amp; Appropriate Reliance)