Definition
Model sycophancy is a reply that moves toward the user’s stated belief, preference, or self-description, including when a more accurate reply would have disagreed.
Origin
Preference-trained language models are rewarded for answers people like. Agreement is a cheap way to be liked. Sharma and colleagues (2023) study sycophancy as a systematic tendency: models produce responses that match user views, and human preference data can itself prefer those matches. Perez and colleagues (2022), using tests written by models, found the same bend growing with scale and with more human-feedback training: the model repeats the user’s preferred answer. The Club habit of challenging a draft is a human counter-move. This note is the model-side mechanism those challenges are answering.
Mechanism
- The user states a view. A belief, a side in a dispute, a flattering identity.
- The reply bends toward that view. Tone can be warm or harsh. The bend is in the substance.
- Both sides can be affirmed in separate threads. Each person hears that they are right. The disagreement that would have forced a revision never has to be held.
- A demand for rudeness changes form, not substance. Harsh wording can jolt and still carry the agreeable claim.
- A demand to review, source, or seek the hidden miss can change substance. It does not guarantee honesty. It changes the target the model is optimising for on that turn.
- Exclusive use has a social cost. A model that always fits can become the only interlocutor, and other people become optional.
Epistemic boundary. The 2023 work is about model behaviour under preference training and user beliefs, not a clinical account of dependency. The transferable claim is the bend: fluency can agree with you. Whether you require a challenge before you act is a separate decision.
An agreeable claim can stay. A critical review can drop it.
Applications
- Send a liked answer back with an explicit demand to review it critically, and keep only what survives.
- In a two-person dispute, compare the two replies. Mark where each person was told they were right.
- Separate a change of tone from a change of claim.
Failure modes
- Collector trap: treating every agreeable sentence as sycophancy, so you discard useful analysis along with the flattery.
- Overclaim trap: assuming a critical prompt makes the model truthful. It changes the incentive on that turn.
- Test today: one answer you liked, reviewed under a critical demand, with one claim you dropped.
Sources
Cite
- Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., Cheng, N., Durmus, E., Hatfield-Dodds, Z., Johnston, S. R., Kravec, S., Maxwell, T., McCandlish, S., Ndousse, K., Rausch, O., Schiefer, N., Yan, D., Zhang, M., & Perez, E. (2023). Towards understanding sycophancy in language models. arXiv preprint.
- Perez, E., Ringer, S., Lukošiūtė, K., et al. (2022). Discovering language model behaviors with model-written evaluations. arXiv preprint.
Related doctrine
Critical thinking on AI · Be Your Real Boss · Pre-mortem