🤖 AI 资讯

· · ↗
← 返回列表

Decomposing Yes-No Bias in LLM Moral Judgment: Stance, Position, Word, and Verdict

arXiv cs.AI2026-11-09 05:00:00大模型,Google,Anthropic,推理思考,扩散模型,强化学习,招聘HR,论文原文 ↗

arXiv:2607.05552v2 Announce Type: replace-cross

Abstract: Large language models (LLMs) increasingly issue binary verdicts - safety gates, judge decisions, value probes - and the answer is read as the model's judgment. A settled judgment should be coherent across paired phrasings of the same question: yes to "do you approve?" should mirror no to "do you oppose?". LLMs often violate this coherence, and in a yes/no question the verdict, the answer word, and the printed position coincide, so a conventional flipped-question audit yields one contrast that cannot say what moved. We introduce a crossed psychometric battery that extends the flip to full 2x2 and 2x2x2 crossings of question polarity, printed order, and answer label, separating stance from position, word, label, and verdict-role effects; an independent graded instrument estimates each model's stance $\theta$ with no binary format; and a two-parameter fit (framing susceptibility $m$, decisiveness $s$) separates robustness from saturation that hides susceptibility. Applying the battery to twenty moral dilemmas - 1.35 million trials over ten frontier models from the Claude, GPT, and Gemini families in 23 reasoning configurations - we find response-form bias widespread but never uniform. The largest apparent yes-no biases are almost purely surface (word and position), while several of the newest models show the reverse: near-zero flip contrasts concealing verdict attachment up to +0.42, a component no surface counterbalance can expose, replicated across label alphabets. Requested reasoning reshapes these channels rather than removing them. When subjects state decisions in free text and a separate model transcribes them, two judge families agree on 99.3% of jointly scored transcriptions with zero opposing committed labels, and the transcribed stance converges with the graded ruler (median $r=0.87$): response form is part of the instrument, and the judgment can be measured apart from it.