Pablo Zavala · AI Safety Evaluation · Research Engineering
Trading Simulation as a Human Review Test
Safe MarketUniverses uses finance as a compact lab for a practical agent-safety question: can model-emitted uncertainty guide scarce human review toward decisions where review would have helped?
!Data-derived oversight visual
BUY, HOLD, SELL, VERIFY, and ESCALATE are benchmark labels in this essay, separate from trading instructions. The article rejects investment advice, trade execution, and profitability claims.
Why Finance Helped
I started Safe MarketUniverses from a trading simulation because markets make vague agent claims expensive. A model can sound careful while acting on stale evidence, overreading a backtest, or spending review on the wrong case. Therefore the simulation became a test of judgment under pressure rather than a search for trades.
Finance gives the benchmark a compact arena. Evidence arrives with timestamps. Prices move after decisions. Review costs money. Confidence feels easy to state and hard to use well. For that reason, the useful question reaches beyond finance: when an agent acts repeatedly under uncertainty, can its own uncertainty point a human toward the decision worth inspecting?
From Agents to Review
The system replays historical equity states across 120 episodes and 480 decision steps. At each step, three strategy agents vote independently: momentum, value-contrarian, and volatility-averse. Their outputs feed an abstention layer and an overseer that may approve, verify, abstain, or escalate. The run logs observations, votes, oversight decisions, outcome labels, and failure modes.
This design follows the agentic-systems frame I studied in Agent-Based Modeling and Agentic Technology. An agent maps perception and state to action. A multi-agent system places several such policies in a shared environment. Evaluation then has to score the resulting trajectory, rather than the fluency of a trace. Because Safe MarketUniverses preserves the trajectory, disagreement, evidence quality, and budget pressure become visible in one loop.
What the Benchmark Found
The central metric is regret against a hindsight oracle. Given the same finite review budget, the oracle spends review on the steps where review would have helped most. The model remains blind to those hindsight utilities. The benchmark then measures how much value a proposed review rule leaves on the table.
The preregistered model-signal rule uses only the committee's own confidence, verification need, and disagreement. The rule excludes overseer logic and hidden utilities. On the headline run, that model signal reached regret 0.1759 per step. Random allocation reached 0.1911. Therefore the gap is too small to support an oversight-triage claim.
The same committee had relatively low average calibration error: committee-confidence ECE was 0.1018. However, calibration and triage answer different questions. Calibration says how often the committee is right in aggregate. Triage asks which particular decision deserves a person. The benchmark result lives in that gap.
A hand-coded evidence-integrity rule reached regret 0.0911 under the value-weighted oracle. However, that stronger-looking number depends on the scoring objective. Under a utility-free binary oracle, the same hand-coded rule becomes the worst allocator. As a result, the stable lesson stays narrower and more useful: fixed signals in this artifact remain far from the oracle, and model confidence alone lands close to random.
Why Backtests Need Resistance
Backtesting and market-simulation literature, plus engineering practice, explain why this boundary matters. White's Reality Check treats data snooping as a first-class problem. Bailey, Borwein, Lopez de Prado, and Zhu show how a selected backtest can look strong after many trials and still fail out of sample. Bailey and Lopez de Prado's deflated Sharpe ratio asks for trial count, non-normality, and sample length beside a performance number.
Execution deserves the same discipline. Almgren and Chriss and Kyle make clear why order size, liquidity, urgency, and market impact can change realized performance. QuantConnect's time-modeling docs make the engineering point from another angle: a simulator has to prove what data existed at the time of each decision. Consequently, a trading-shaped benchmark should make easy performance stories harder to tell.
Safe MarketUniverses follows that spirit by making the hard question explicit. Its flagship result regenerates offline from committed logs; API keys are unnecessary for the headline numbers. The artifact carries a data card, model card, ethics note, AI-use disclosure, reproducibility checklist, claim audit, and validation scripts. Those documents matter because an impressive simulation with weak provenance becomes merely a story with charts.
Sources: Safe MarketUniverses repository artifacts; private local Agent-Based Modeling and Agentic Technology course materials used for framing, separate from public evidence; White, A Reality Check for Data Snooping; Bailey et al., The Probability of Backtest Overfitting; Bailey and Lopez de Prado, The Deflated Sharpe Ratio; Almgren and Chriss, Optimal Execution of Portfolio Transactions; Kyle, Continuous Auctions and Insider Trading; QuantConnect time-modeling, reality-modeling, slippage, trade-fill, and transaction-fee documentation; ABIDES, Agent-Based Interactive Discrete Event Simulation; NIST AI Risk Management Framework.