Personalization, Trust and the Modern Consumer

Personalization increases relevance, and relevance increases response. It also, past some threshold, produces discomfort that damages the relationship it was meant to strengthen. The interesting question is what determines the threshold, because it is not simply the amount of data involved.

Accuracy is not the trigger

A recommendation that is accurate because the person told you their preferences is received differently from an equally accurate recommendation derived from inference. White, Zahay, Thorbjørnsen and Shavitt (2008) found that highly personalized messages could reduce response when they made the person feel the firm knew more than it should, and that justifying the personalization mitigated the effect.

The mechanism is not surveillance detection in a general sense. It is the discovery of inference: the realisation that the firm derived something the person never disclosed. Inference is what makes the relationship feel non-reciprocal, because the person cannot see what else has been derived or how.

Context matters more than sensitivity

Nissenbaum's contextual integrity framework (2010) holds that privacy expectations are governed by norms attached to the context in which information was shared, not by the intrinsic sensitivity of the data. Information appropriately shared with a pharmacy is inappropriately surfaced by a retailer, even though the data is identical.

This explains a pattern that puzzles marketers. The same personalization can be welcome in one channel and alarming in another. It is not the data that changed; it is the flow, and flows carry norms.

Control cues cut both ways

Brandimarte, Acquisti and Loewenstein (2013) documented a control paradox: giving people more control over the publication of private information increased their willingness to disclose sensitive details, even when doing so increased actual accessibility to strangers. Perceived control reduced concern more than it reduced risk.

For anyone designing a preference centre, this is an uncomfortable finding. Visible controls genuinely improve the experience and genuinely increase disclosure, and the increase is not fully explained by better-informed consent. Controls that reduce concern without reducing exposure are not a solution to the privacy problem; they are a way of making it feel solved.

Personalization as an implicit claim

Every personalized message asserts something: we understand your situation. When the assertion is wrong, the damage exceeds that of a generic message, because a generic message made no claim to fail. Recommending baby products to someone who has lost a pregnancy is the extreme case, but the structure is general — confident personalization raises the cost of error.

This suggests calibrating personalization confidence to inference confidence, and defaulting to less specific messaging where the underlying signal is weak. Most systems do the opposite, personalising most aggressively where engagement models predict the highest response, which is not the same as where the inference is most reliable.

What this means in practice

Three questions are worth asking of any personalization programme. Could the person reconstruct how you knew this? Would the inference feel appropriate in the context they originally shared the data? And what does the message cost you when the inference is wrong? Programmes that pass all three tend to be experienced as service. Programmes that fail the first are experienced as surveillance regardless of how legitimate the data collection was. The wider pattern is covered in the privacy paradox.

References

1. White, T. B., Zahay, D. L., Thorbjørnsen, H., & Shavitt, S. (2008). Getting too personal: Reactance to highly personalized email solicitations. Marketing Letters, 19(1), 39–50.
2. Nissenbaum, H. (2010). Privacy in Context: Technology, Policy, and the Integrity of Social Life. Stanford University Press.
3. Brandimarte, L., Acquisti, A., & Loewenstein, G. (2013). Misplaced confidences: Privacy and the control paradox. Social Psychological and Personality Science, 4(3), 340–347.
4. Acquisti, A., Brandimarte, L., & Loewenstein, G. (2015). Privacy and human behavior in the age of information. Science, 347(6221), 509–514.
5. Tucker, C. E. (2014). Social networks, personalized advertising, and privacy controls. Journal of Marketing Research, 51(5), 546–562.