Chessenv is an ORS environment for evaluating agents on playing chess against an LLM opponent. This environment wraps the Chess implementation from TextArena, a framework for text-based game environments.
- Strategic planning and tactical execution in chess
- Position evaluation and move generation
- Opening theory, middlegame strategy, and endgame technique
- Testing long-horizon planning and competitive gameplay
Chess does not require a sandbox. It has minimal compute requirements.
MIT.
There are two splits: train (150 tasks) and test (150 tasks). Each split contains 50 tasks across each of 3 variants:
- Chess-v0
- Chess-v0-blind
- Chess-v0-long
Each task is seeded for reproducibility.
This is a sparse reward environment. Rewards are mapped from TextArena's native range of {-1, 0, 1} to {0.0, 0.5, 1.0} via (raw + 1) / 2.
We do not use LLM graders for this environment; reward is determined programmatically.
Game state is generated procedurally by the TextArena engine using seeded randomness. No external data files are required.
Agents are given a single tool:
make_move(move): Make a chess move in UCI format (e.g. 'e2e4', 'g1f3', 'e7e8q' for promotion).
Chess is a multi-turn environment.
Hard to Very Hard. Chess requires deep strategic thinking, position evaluation, tactical calculation, and long-term planning. The blind variant increases difficulty by limiting board visibility.
This environment requires an OpenAI API key (passed via secrets) to power the LLM opponent.
Agents in Chess interact only with a board game and have no access to external systems, the internet, or sensitive data. The environment does not present safety risks.
@software{textarena2024,
author = {Guertler, Leon and Banting, Wilfried and Pignatelli, Eduardo},
title = {TextArena},
year = {2024},
publisher = {GitHub},
url = {https://github.com/LeonGuertler/TextArena}
}