Algorithm Aversion and Algorithm Appreciation

Two robust findings in the literature on machine advice appear to contradict each other. People prefer algorithmic judgement to human judgement (Logg, Minson and Moore, 2019). People abandon algorithms after observing errors that they would forgive in a human (Dietvorst, Simmons and Massey, 2015). Both replicate. Reconciling them is more useful than choosing between them.

Task type is the first moderator

Appreciation is strongest for tasks perceived as objective and quantitative — estimating a number, forecasting a measurable outcome. Aversion is strongest for tasks perceived as requiring judgement about people, values or uniqueness: hiring, medical diagnosis, creative evaluation.

Longoni, Bonezzi and Morewedge (2019) documented reluctance to use medical AI and traced it to uniqueness neglect — the belief that an algorithm cannot account for the person's particular circumstances. The belief is often mistaken but it is coherent, and it predicts resistance better than general technophobia does.

Error visibility is the second

The aversion result depends on the participant having observed a mistake. Before that, reliance is high. The asymmetry with human advisers arises because people hold different models of the two error sources: a human's error is attributed to circumstance, a machine's to a defect that will recur.

An operational consequence follows. Systems that fail visibly and rarely may be trusted less than systems that fail invisibly and often, which is exactly backwards from a safety perspective. Designing for appropriate reliance sometimes means making errors more visible, not less, and setting expectations before deployment.

Control is the third and most actionable

Dietvorst and colleagues (2018) found that allowing participants to adjust an algorithm's output — even within tight bounds — substantially increased adoption and satisfaction. The adjustment did not need to be large enough to materially change accuracy. What mattered was that the person retained agency.

This is the most useful finding in the area for practitioners, because it is cheap to implement and it does not require convincing anyone of anything. A recommendation the person can modify is accepted where an identical recommendation issued as a verdict is not.

Expertise cuts both ways

Domain experts show more resistance to algorithmic advice in their own field than novices do, which is well documented and unsurprising: accepting the algorithm implies something about the value of their expertise. Yet experts are also the group best placed to catch algorithmic error.

The design implication is that decision-support systems for experts should be framed as instruments rather than as replacements — providing inputs the expert integrates, rather than conclusions the expert accepts or rejects. The framing changes the identity threat without changing the information.

What this means in practice

Before deploying an advisory system, establish which of the three moderators dominates. If the task is judgement-heavy and personal, expect uniqueness neglect and address it directly by showing what individual factors the model uses. If reliance is high but calibration is poor, expect a collapse at first visible error and pre-empt it with honest performance framing. And build in adjustment, because the acceptance return on modest control is larger than on almost any other intervention. The applied version of this is covered in the psychology behind AI recommendations.

References

1. Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114–126.
2. Logg, J. M., Minson, J. A., & Moore, D. A. (2019). Algorithm appreciation: People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151, 90–103.
3. Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019). Resistance to medical artificial intelligence. Journal of Consumer Research, 46(4), 629–650.
4. Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018). Overcoming algorithm aversion. Management Science, 64(3), 1155–1170.
5. Burton, J. W., Stein, M.-K., & Jensen, T. B. (2020). A systematic review of algorithm aversion in augmented decision making. Journal of Behavioral Decision Making, 33(2), 220–239.