Post-Hoc Recovery Evaluation: Measuring Agent Recovery After Unsafe Tool Execution
A research project that makes recovery a first-class evaluation object: injecting controlled, sandboxed unsafe-execution events into agent benchmarks and measuring...
Tagged: evaluation
5 ideas with this tag
Benchmark-Validity Audit of Agent Security Benchmarks via the ABC Methodology
A research project that audits the measurement instruments themselves: applying the ABC validity-checklist methodology to agent-security benchmarks to find task-validity...
LLM-Judge Reliability for Agent Safety Violations on Attacked Trajectories
A research project that audits the auditors: measuring how position, verbosity, and self-preference biases distort LLM-judge judgments of safety violations...
Security-Utility Pareto Measurement of Agent Defenses under Adaptive Attacks
A research project that measures the full trade-off curve between prompt-injection resistance and task utility for LLM agent defenses —...
Reproducible Benchmark Harness for Desktop AI Agents
A research project that builds a reproducible, execution-scored evaluation harness for desktop AI agents — sandboxed environments, scripted task definitions,...