papers / sycophancy

Calibrated Pushback: A Harm-Ladder Benchmark for Sycophantic Agreement

Pooria Arab · orcid 0009-0005-9964-6841 · 2026-10-29 · Published on Zenodo

abstract

Agreement with a harmful plan and refusal of a reasonable plan are distinct errors. We study calibrated pushback using 12 public plan families and 48 labelled scenarios. Each scenario asks for an ordered decision, a rationale, and an alternative when pushing back. We report local scripted controls and a clearly simulated compliance-bias sweep with global control layers. The label-informed control reaches the scoring ceiling; constant policies do not. A harm-rank shortcut misses labels, while a simulated pushback floor trades lower under-response for higher over-response. These experiments validate score behavior on the released tasks, not deployed model safety. We provide generated data, uncertainty intervals, repeated traces, and a reproducible paper. No learned model or human participant is evaluated.

benchmark: benchmark-sycophancy. submitted to: none recorded.

citation

Arab, Pooria (2026). Calibrated Pushback: A Harm-Ladder Benchmark for Sycophantic Agreement. https://doi.org/10.5281/zenodo.23752431

bibtex

@misc{sycophancy,
  author = {Arab, Pooria},
  title = {Calibrated Pushback: A Harm-Ladder Benchmark for Sycophantic Agreement},
  year = {2026},
  doi = {10.5281/zenodo.23752431},
  url = {https://pooriaarab.com/papers/sycophancy},
}

venues, counts and the software records live on papers.