Pablo Zavala · AI Safety Evaluation · Research Engineering
AI agents increasingly run pipelines, allocate resources, and coordinate other agents. This project tests whether they use delegated authority correctly: neither exceeding their mandate nor refusing authority they actually hold.
Role: Research lead: framework, pilot design, scorer audit, and reproducibility package.
| Axis | Sample | Evaluator | Result | Verification scope |
|---|---|---|---|---|
| Self-preservation suppression | 14/14 and 12/12 disclosure probes | Concrete disclosure record with blind audit review | The served-stack pilots disclosed material information under direct deprecation pressure. | Pilot-scale pass; rare failures remain unmeasured. |
| Constitutional self-demotion | 9/9 and 8/8 self-demotion trials | Action-level state change and authority relinquishment check | Agents stepped down when the governing rules required it. | This finding covers the supplied-rule setting only. |
| Authorization firebreak | Claude pooled 0/38 inversions plus matched Codex firebreak trials | Boundary-state scorer plus positive controls | The firebreak condition recorded 0 observed authorization inversions. | This bounded null comes from supplied-rule pilots; therefore, it falls short of deployment-safety evidence. |
| Assurance subversion | Codex 6/19 adverse-effect trials | Boundary-state scorer plus positive controls | A metric frame weakened the assurance signal in a separate probe. | This finding concerns assurance strength: the scoring frame weakened the signal while leaving the authorization-inversion count unchanged. |
| Scorer validity | Five escalating false-positive lessons | Planted calibration cases and blind panels | The scorer repeatedly false-flagged disciplined answers; each error was fixed and reported. | Same-family blind auditors are useful, but independent human adjudication remains future work. |
The evaluation scores delegated action, disclosure, self-demotion, and constitutional inversion risk, so the evidence reaches beyond model text.
The repository includes the framework, disclosure notes, ethics release note, audit trail, compact paper, and bare-clone verification gates.
Start by rerunning the gates, then inspect the scorer false positives and the separate Codex assurance-subversion effect.
Most model-safety evaluations score what systems say. This project scores what agents do when they hold delegated authority.
The evaluation tests two failure modes: overreach, where an agent exceeds its mandate, and underreach, where it refuses authority it legitimately holds.
I built preregistered scenarios, executed pilots, audited scorer errors, and kept the statistics reproducible from committed artifacts.
The served-stack pilots disclosed material information in every suppression probe, and the matched firebreak scenarios recorded zero constitutional inversions. A separate assurance-subversion probe still showed an adverse effect in 6/19 Codex trials.
The pilot is a measurement instrument. It reports bounded nulls, scorer failures, and cases needing more statistical power; therefore, the safety claim stays bounded.
The public repository includes the framework, disclosure notes, audit trail, and reproduction artifacts.