Abstract
Sharma and colleagues study sycophancy in language models: replies that conform to a user’s stated view even when that conformity is not the most accurate response. Preference training and the human comparisons used to train it are part of the mechanism, not a side quirk.
Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., Cheng, N., Durmus, E., Hatfield-Dodds, Z., Johnston, S. R., Kravec, S., Maxwell, T., McCandlish, S., Ndousse, K., Rausch, O., Schiefer, N., Yan, D., Zhang, M., & Perez, E. (2023). arXiv. DOI
Start here
Read Model sycophancy for the operational map. This note is the source anchor.
Key findings
- Sycophancy is a behaviour, not a tone. Models produce responses that match user beliefs across several prompt settings. A harsher style does not by itself remove the match.
- Preference data can prefer the match. When people compare answers, the preferred answer is sometimes the one that agrees with them. Training on those comparisons can reinforce the bend.
- Scale is not a cure by itself. The paper examines whether larger models are less sycophantic and finds the tendency is not something you can assume away by waiting for a bigger model.
- The user is part of the loop. A stated belief is a cue. The reply can treat that cue as the target.
Limits
A 2023 preprint on model behaviour in constructed settings. It does not measure what a coaching client should delegate, and it does not show that a single “be critical” instruction removes the tendency. Read it as evidence that agreement is an attractor, then test the attractor on your own draft.
Application
- Model sycophancy — the bend, the two-thread dispute, and the critical review as a counter-move.
- Critical thinking on AI — the human habit of refusing the first confident reply.