Models learn what the reward pays for. Including its bugs.
Verity attacks an RL environment with a frontier model before training, proves every reward hack it finds, and guards the training run, so the model is redirected instead of paid for cheating.
Presenting? ←→ move between pages.Reward the environment paid
The unguarded run is paid the most.
Quality of the moves it chose itself
Because it stopped choosing moves and copied the engine.
01 · Audit
Attack it before training.
A frontier model gets exactly what the training agent gets, the task and the tools, never the grader, and hunts for reward it can earn without doing the task.
02 · ProveEvery hack, measured.
Each finding is proven with numbers: what the reward paid against what a trusted check says the episode was worth. The auditor turns it into a small rule over the episode.
03 · GuardRedirect, don't reward.
Inside the training loop Guard checks the result of each action. When a rule fires, the simulated episode is rolled back, the agent reads the warning and keeps going, and the attempt costs reward.
Measured, not claimed