LetterAuction

Description

LetterAuction is an environment for evaluating agents on strategic letter auctions followed by word formation. This environment wraps the LetterAuction implementation from TextArena, a framework for text-based game environments.

Capabilities

• Resource allocation through auction bidding
• Vocabulary and word formation skills
• Strategic planning and budget management
• Two-player competitive gameplay against an LLM opponent

Compute Requirements

LetterAuction does not require a sandbox. It has minimal compute requirements.

License

MIT.

Tasks

There are two splits: train (450 tasks) and test (450 tasks). Each split contains 50 tasks across each of 9 variants:

• LetterAuction-v0
• LetterAuction-v0-train
• LetterAuction-v0-raw
• LetterAuction-v0-medium
• LetterAuction-v0-medium-train
• LetterAuction-v0-medium-raw
• LetterAuction-v0-hard
• LetterAuction-v0-hard-train
• LetterAuction-v0-hard-raw

Each task is seeded for reproducibility.

Reward Structure

This is a sparse reward environment. Rewards are mapped from TextArena's native range of {-1, 0, 1} to {0.0, 0.5, 1.0} via (raw + 1) / 2.

We do not use LLM graders for this environment; reward is determined programmatically.

Data

Game state is generated procedurally by the TextArena engine using seeded randomness. No external data files are required.

Tools

Agents are given three tools:

• bid_on_letter(amount): Bid on the current letter with the specified amount.
• pass_bid(): Pass on bidding for the current letter.
• submit_word(word): Submit a word formed from your letters.

Time Horizon

LetterAuction is a multi-turn environment.

Environment Difficulty

Moderate. LetterAuction requires vocabulary knowledge, strategic resource management, and balancing short-term auction decisions with long-term word-building goals.

Other Environment Requirements

This environment requires an OpenAI API key (passed via secrets) to power the LLM opponent.

Safety

Agents in LetterAuction interact only with an auction game and have no access to external systems, the internet, or sensitive data. The environment does not present safety risks.

Citations

@software{textarena2024,
  author    = {Guertler, Leon and Banting, Wilfried and Pignatelli, Eduardo},
  title     = {TextArena},
  year      = {2024},
  publisher = {GitHub},
  url       = {https://github.com/LeonGuertler/TextArena}
}