PokerBench establishes a high-frequency, adversarial, and imperfect-information environment: a No-Limit Texas Hold’em (NLTH) tournament. PokerBench places frontier models - including Claude Opus, Gemini 3, and GPT-5.2 - into direct competition. This introduces a "Theory of Mind" requirement; agents must not only manage their own state (chips, cards) but also model the epistemic states and strategies of hostile actors [1].
The complexity of NLTH, with a game tree exceeding
The benchmark simulates a large number of independent games, each lasting 150 hands or until one player accumulates all chips. This scale is necessary to dampen the extreme variance inherent in NLTH and achieve statistical significance in the primary metric: stack size [3].
All of the runs are stored in JSON files in runs/ and can be inspected. You may also generate your own runs using pokerbench-runner/pokerbench.py. Contributions and bug-fixes are welcome.
Try out https://pokerbench.adfontes.io/ for game visualizations, stats, etc.
Sources:
- Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4
- PokerBench: Training Large Language Models to become Professional Poker Players
- Poker Variance Calculator
As seen on:
- https://www.reddit.com/r/poker/comments/1q7gdll/i_made_llms_play_21000_hands_of_poker/
- https://www.reddit.com/r/GeminiAI/comments/1q7gy25/i_made_gemini_3_proflash_play_21000_hands_of_poker/
- https://www.reddit.com/r/OpenAI/comments/1q7lm9o/i_made_gpt525_mini_play_21000_hands_of_poker/
Note: This project is licensed under the MIT License with the Commons Clause restriction. Commercial use and redistribution for a fee require explicit permission. Reach out to x.com/adfontes_ for more information.