Briefs · Emerging Technologies
How “reverse game theory” could keep AI agents in check
You cannot read an AI’s mind. So bribe it instead.
AI alignment usually means ensuring AI systems behave according to human values and intentions, and most work focuses on training models or on interpretability — reverse-engineering large language models, like Anthropic tracking features in Claude and mapping activation patterns to concepts like the Golden Gate Bridge, San Francisco and self-awareness.
Some researchers believe reverse game theory — mechanism design — might get better results: game theory predicts how rational actors behave given a set of rules, while reverse game theory designs rules to elicit the behaviour we want. The classic example is splitting a piece of cake: one person cuts, the other chooses first, so the cutter makes the pieces as even as possible. The cutter gets the smaller slice!
In AI, the premise is that we may never understand enough about the inner workings of frontier models to guarantee alignment, so we should design mechanisms that incentivise agents to behave as we want, regardless of what is going on under the hood. The catch: it assumes agents behave rationally, and mechanism design has a mixed record with humans, who often fail to behave as economic models say they should. “Is it even reasonable to assume that they will respond to incentives?” asks Helen Qu (a research fellow at the Flatiron Institute).
Qu sees mechanism design as just one layer in a “defense in depth” strategy — alongside interpretability, regulation and training, not instead of them. Models could find loopholes no one anticipated, and Anthropic and OpenAI did not respond to requests for comment on whether they are working on this. It is a dog-training manual, except the dog is a maths equation that talks (sorry, Pavlov).
Conclusion: alignment is not a mind-reading trick. It is a cake-cutting rule.
Read the original at MIT Technology Review
Summary in our own words. The original belongs to its publisher.