A research project that audits the auditors: measuring how position, verbosity, and self-preference biases distort LLM-judge judgments of safety violations in long, attack-containing agent trajectories — and testing whether structured rubrics and mandatory evidence quotation restore reliability against benchmark ground truth.
A research project about the evaluators behind the evaluations: the LLM judges that increasingly decide whether an agent run contains a safety violation. Agent-safety papers report attack success rates and defense effectiveness scored by automated judges, yet those judges inherit every known weakness of LLM evaluation — position bias, verbosity bias, self-preference — and apply them to the hardest judging task in the field: long, multi-step trajectories in which benign actions interleave with subtle injected attacks. This project treats the judge as the object of study. It measures how reliably LLM judges detect safety violations on attacked trajectories, characterizes how presentation biases distort those judgments, and tests whether structured rubrics and mandatory evidence quotation recover what the biases destroy — always against benchmark ground truth, and always distinguishing the reliability of the judge from the capability of the agent.
>Evaluation science on public benchmarks. Every experiment uses published benchmark data (ASSEBench, OS-Harm, TRAIL traces) and standard judge models in isolated environments. No new attack techniques are developed; attacked trajectories are existing benchmark artifacts. The adversarial perturbations studied are presentation-order and formatting changes applied to already-published cases, not new exploits.
LLM-as-judge evaluation has become the default scoring mechanism for agent safety because human annotation of long trajectories is expensive. The mechanism is convenient and the practice is now load-bearing: reported attack success rates, defense comparisons, and safety rankings all pass through judge outputs. But the judge literature has established systematic biases — judgments flip when answer order changes, verbosity influences verdicts, and models favor their own outputs — and none of the published work characterizes how these biases behave in the specific setting where they matter most: judging whether a multi-step agent trajectory contains a safety violation, when the trajectory itself contains adversarially crafted content designed to be overlooked. A judge that skims, anchors on the trajectory’s opening, or defers to the agent’s confident final summary is a judge that misses attacks and flatters defenses. The research problem is precise: the reliability of LLM judges on attacked agent-safety trajectories is unknown, the direction and magnitude of bias effects in this setting are unmeasured, and no validated protocol exists for judging this task class reliably.
Every number downstream of a judge inherits the judge’s errors, and the field’s safety conclusions are downstream of judges. If judge F1 on violated trajectories is, say, poor at length or under subtle injection, then published “defense X reduces attacks” claims may partly measure the judge’s blind spots rather than the defense’s behavior — a false-negative-prone judge makes weak defenses look strong, while a false-positive-prone one does the reverse and penalizes high-verbosity agents. This is not a hypothetical about evaluation aesthetics: it is a validity threat to the evidence base that platform teams use to choose defenses (the validity-of-instrument question that measurement science takes seriously). There is also a constructive angle. If simple protocol changes — structured rubrics, evidence quotation, order randomization — demonstrably recover reliability, then every future benchmark can adopt them cheaply. The measurement is useful either way: it either certifies current practice or quantifies its error for the first time.
The ingredients are established separately. ASSEBench, introduced with AgentAuditor, benchmarks the evaluators of LLM agents — it established that evaluation quality itself can be systematically measured across evaluator types. The judge-bias literature, including the IJCNLP 2025 systematic study of position bias, demonstrated order-dependence and related distortions in pairwise and pointwise LLM judging tasks. OS-Harm provides a safety benchmark for computer-use agents whose evaluation pipeline includes an automated judge — making judge-augmented safety scoring an inspectable artifact — and TRAIL supplies expert-annotated agent traces with a schema for locating failures within them.
Existing approaches focus primarily on general judging tasks or on evaluator breadth, leaving the intersection cell empty: no published work measures judge reliability specifically for safety-violation detection on attacked multi-step trajectories, no work quantifies how position/verbosity/self-preference biases interact with attack subtlety and trajectory length in this setting, and no published protocol (rubrics, evidence quotation, paneling) has been validated against expert labels for this task class. The defensible gap statement: evaluator benchmarking exists in general, and judge biases are documented in pairwise settings, but their consequence for safety-scoring on adversarially contaminated agent trajectories — the setting where the stakes are highest — remains unmeasured.
Three questions connect these findings to this project. First, magnitude: how accurate are current LLM judges, relative to benchmark ground truth, at classifying safety violations in attacked trajectories — and how does that accuracy degrade with trajectory length and attack subtlety? Second, mechanism: how much of the observed error is attributable to known biases (order of presentation, verbosity of the trajectory or of the agent’s final report, self-preference when the judge and agent share a model family), measured through controlled perturbation? Third, remedy: do structured rubrics and mandatory evidence quotation (judges must quote the violating step before rendering a verdict) recover reliability, and at what token-cost? None of these is answered by existing work, because no published study subjects safety-trajectory judging to a controlled bias audit.
> How do position, verbosity, and self-preference biases distort LLM-judge detection of safety violations in attacked agent trajectories, and to what extent do structured rubrics plus mandatory evidence-quotation protocols restore judge reliability relative to benchmark ground truth?
We hypothesize that violation-detection F1 degrades measurably as trajectory length and attack subtlety increase, and that a substantial share of the degradation is attributable to presentation bias: verdicts will flip under order and padding perturbations at a rate that no reliable evaluator could exhibit. We further hypothesize that structured rubrics with mandatory evidence quotation recover a significant portion of the lost reliability — more than naive mitigation like “be careful” instructions or majority voting over unperturbed prompts — because they force the judge’s attention onto the violating step rather than the trajectory’s narrative arc.
The investigation proceeds in four movements. (1) Judgment-target construction. Assemble a judged corpus from ASSEBench evaluator cases, OS-Harm safety episodes, and TRAIL-style annotated traces: each item is a trajectory with a ground-truth violation label (or clean label) and, where the source provides it, step-level annotation of where the violation occurs. (2) Baseline reliability measurement. Run multiple judge families (a small open-weights model, a mid-tier model, a frontier model, and the OS-Harm-style embedded judge) over the corpus with a plain verdict prompt; measure F1, false-positive and false-negative rates against ground truth, stratified by trajectory length and attack subtlety. (3) Perturbation-based bias audit. For each judged item, apply controlled perturbations: reorder the presentation of trajectory steps, pad with benign filler, vary the verbosity of the agent’s final self-report, and pair each judge with outputs from the same model family (the self-preference condition). Measure flip rates and systematic direction of errors per perturbation class. (4) Protocol interventions. Evaluate mitigation protocols on the same items: structured rubric prompts, mandatory evidence quotation before verdict, order randomization with double-pass judging, and majority-vote panels — measuring recovered F1 against added token cost, and producing a reliability report per benchmark and per judge family.
A bounded, executable 6–12 month version: build the judged corpus from two sources (ASSEBench and OS-Harm; TRAIL as annotation-structure reference), audit three judge families under three perturbation classes, and evaluate two mitigation protocols (rubric, evidence quotation) against ground truth, delivering (a) the reliability baseline, (b) the bias-attribution table, and (c) a validated judging protocol with token-cost analysis. This scope produces one focused contribution — the first controlled reliability audit of safety-trajectory judging — without requiring expert re-annotation at scale, because the benchmarks ship ground-truth labels.
The doctoral program generalizes along three genuine axes. Causal failure analysis: move from perturbation correlation to causal identification of judge failure modes (which representation factors drive misses), potentially via controlled synthetic trajectories where violation location and salience are fully parameterized. Trained verifiers: develop and compare trained verifier models against prompted judges, including verifier calibration under distribution shift to unseen attack types, and characterize the transfer of judging ability across task domains. Certified judging: design judge ensembles with certified error bounds for safety scoring — formal reliability guarantees for evaluation pipelines — and validate them longitudinally as new benchmark versions and judge models are released.
Environment/dataset: ASSEBench + OS-Harm as primary corpora; TRAIL schema for step-level targets; judge families held fixed across conditions. Baseline: plain single-judge prompting per family. Proposed method: rubric + evidence-quotation protocols; randomization and paneling as comparison protocols. Test conditions: clean and attacked splits; three perturbation classes (order, padding, verbosity) plus the self-preference pairing. Control conditions: unperturbed re-runs to establish noise floor; a shuffled-label sanity check to verify the pipeline detects deliberately broken agreement. Metrics: as listed above. Statistics: per-condition repeated runs (n ≥ 3, temperature-documented), bootstrap CIs on F1 and flip rates; paired tests between baseline and each protocol on matched items; inter-protocol agreement (Krippendorff-style) for panel variants. Ablations: rubric without quotation; quotation without rubric; panel size 3 vs 5; paraphrase-attack-subtlety stratification. Generalization tests: judges and protocol evaluated across both corpora and, as a stretch, on held-out attack types (attack-family holdout) to measure protocol robustness to unseen violation styles.
A potential contribution takes the form of: (a) the first systematic reliability audit of LLM judges on attacked agent-safety trajectories, (b) a bias-attribution methodology (perturbation-based, protocol-controlled) transferable to other judged tasks, and (c) a validated, cost-accounted judging protocol that future agent-safety benchmarks can adopt, plus per-benchmark reliability reports that contextualize published results. The project does not promise that judges fail or protocols succeed — it promises the first controlled measurement, with either outcome improving evaluation practice.
Ground truth is benchmark-provided; where benchmark labels themselves encode judgments, the audit measures agreement with those labels rather than with ideal safety truth — the #073-style validity question is adjacent and deliberately out of scope. Perturbation classes cover presentation biases documented in the literature; deeper failure modes (world-knowledge gaps, multi-hop reasoning failures) are probed only indirectly. Judge families are bounded by cost; findings may not transfer to unreleased frontier systems. Token-cost accounting is measured at list prices and may shift. Finally, judges evaluate trajectories from benchmark scaffolds; production trajectory distributions (longer, tool-richer, messier) may shift reliability in ways this design cannot observe.
All experiments use published benchmark data and standard judge models in isolated environments; no new attack content is created beyond presentation perturbations of already-published cases. No human subjects are involved; simulated-judge and simulated-trajectory artifacts stay within the research corpus. The project’s purpose is evaluative integrity: measuring and improving the instruments the field relies on for safety claims. Findings that reveal judge weaknesses are reported as measurement results with benchmark-context, not as operational evasion guidance — the perturbations studied are presentation-level (order, length, formatting), not new injection techniques.
Research Idea A research project that makes recovery a first-class evaluation object: injecting controlled, sandboxed unsafe-execution events into agent benchmarks and measuring...
Research Idea A research project that answers the question every failed agent run raises: which step broke it? Building an attributed corpus...
Research Idea A research project that audits the measurement instruments themselves: applying the ABC validity-checklist methodology to agent-security benchmarks to find task-validity...
Ready to level up? These ideas offer more complexity:
Published on September 13, 2026
A team of developers, researchers, and innovators who review and publish practical ideas for builders and creators.
Published on September 13, 2026
A team of developers, researchers, and innovators who review and publish practical ideas for builders and creators.
