WordChains

Description

WordChains is an environment for evaluating agents on word association and vocabulary knowledge through a chaining game where each word must start with the last letter of the previous word. This environment wraps the WordChains implementation from TextArena, a framework for text-based game environments.

Capabilities

• Testing vocabulary breadth and word recall
• Evaluating linguistic pattern recognition and word formation rules
• Assessing strategic word choice to limit opponent options
• Testing memory of previously used words to avoid repetition

Compute Requirements

WordChains does not require a sandbox. It has minimal compute requirements.

License

MIT.

Tasks

There are two splits: train (150 tasks) and test (150 tasks). Each split contains 50 tasks across each of 3 variants:

• WordChains-v0
• WordChains-v0-train
• WordChains-v0-raw

Each task is seeded for reproducibility.

Reward Structure

This is a sparse reward environment. Rewards are mapped from TextArena's native range of {-1, 0, 1} to {0.0, 0.5, 1.0} via (raw + 1) / 2.

We do not use LLM graders for this environment; reward is determined programmatically.

Data

Game state is generated procedurally by the TextArena engine using seeded randomness. No external data files are required.

Tools

Agents are given a single tool:

• submit_word(word): Submit a word for the chain. Must start with the last letter of the previous word.

Time Horizon

WordChains is a multi-turn environment.

Environment Difficulty

Easy to Medium. The task requires basic vocabulary knowledge and pattern matching. Success depends on knowing valid English words and strategic planning to avoid difficult letters. Variants may adjust word length requirements or difficulty.

Other Environment Requirements

This environment requires an OpenAI API key (passed via secrets) to power the LLM opponent.

Safety

Agents in WordChains interact only with a word game and have no access to external systems, the internet, or sensitive data. The environment does not present safety risks.

Citations

@software{textarena2024,
  author    = {Guertler, Leon and Banting, Wilfried and Pignatelli, Eduardo},
  title     = {TextArena},
  year      = {2024},
  publisher = {GitHub},
  url       = {https://github.com/LeonGuertler/TextArena}
}