Adaptation of the TextWorld text-based reinforcement learning (RL) environment for materials synthesis procedures analysis. This is intended to become a proof-of-concept for "Text-To-Quest", a new approach to procedural text understanding, by which free natural language instructions or narratives are interpreted as a game, whose solution by a text-based agent is equivalent to the successful extraction of the procedure in the required structured form.
More formally, the objective is the extraction of an action graph
An example from our materials science domain:
We approach this from a game solving perspective, where an agent will be trained on increasingly harder generated texts simulating materials synthesis procedures (hopefully eventually approaching something like
For a detailed system overview, refer to our paper, Playing by the Book: An Interactive Game Approach for Action Graph Extraction from Text.
This repository is essentially a fork of TextWorld (version 1.0.0) with the following main modifications:
- Incorporation of domain specific "lab" entities and logic in the TextWorld knowledge base.
- A
QuestGeneratorfor generating materials synthesis "quests" consistent with the domain-specific logic. This is similar in function (and utilizes) the existingChainerhelper class. - Surface Generator for generating the quest instructions for the generated quests. This replaces the TextWorld
Grammar. - A new "lab challenge".
Note that the implementations are still preliminary and are under active development!
We call our extension package tw_textlabs to avoid confusion with the original textworld package.
We follow the Textworld installation and usage instructions, with minor modifications.
TextLabs requires Python 3 and only supports Linux and macOS systems at the moment. We have only tested TextLabs in Linux environments.
TextLabs requires some system libraries for its native components. On a Debian/Ubuntu-based system, these can be installed with
sudo apt install build-essential uuid-dev libffi-dev python3-dev curl git
And on macOS, with
brew install libffi curl git
After cloning the repo, go inside the root folder of the project (i.e. alongside setup.py) and run
pip install .
TextWorld's visualization options aren't yet supported for TextLabs.
TextLabs text-based games can be generated via the tw-make-lab script.
tw-make-lab challenge --seed 1234 --output gen_games/game_1234.ulx --challenge tw-lab_game-level12
challenge indicates we want to create a random game, where --challenge specifies the type of challenge and difficulty level. Under this mode, text generation flags are currently set automatically.
In both modes, --max_search_steps sets the number of steps for the search for a valid quest satisfying the constraints. Note that a valid quest may not be found within max_search_steps, especially for higher levels.
Alternatively, games can be hand-crafted in a Jupyter Notebook environment. For details, see example.
To play a game, one can use the tw-play script. For instance, the command to play the game generated in the previous section would be
tw-play gen_games/game_1234.ulx
Note: Only Z-machine's games (*.z1 through .z8) and Glulx's games (.ulx) are supported.
Here's how you can interact with a text-based game from within Python.
import tw_textlabs
env = tw_textlabs.start("gen_games/game_1234.ulx") # Start an existing game.
agent = tw_textlabs.agents.NaiveAgent() # Or your own `tw_textlabs.Agent` subclass.
# Collect some statistics: nb_steps, final reward.
avg_moves, avg_scores = [], []
N = 10
for no_episode in range(N):
agent.reset(env) # Tell the agent a new episode is starting.
game_state = env.reset() # Start new episode.
reward = 0
done = False
for no_step in range(100):
command = agent.act(game_state, reward, done)
game_state, reward, done = env.step(command)
if done:
break
# See https://textworld-docs.maluuba.com/textworld.html#textworld.core.GameState
avg_moves.append(game_state.nb_moves)
avg_scores.append(game_state.score)
env.close()
print("avg. steps: {:5.1f}; avg. score: {:4.1f} / 1.".format(sum(avg_moves)/N, sum(avg_scores)/N))The code for training the baseline LSTM-DQN reinforcement learning agent described in the paper can be found at TextLabs-Agent-Gym.
If you use TextLabs, please cite the following BibTex:
@inproceedings{tamari-etal-2019-playing,
title = "Playing by the Book: An Interactive Game Approach for Action Graph Extraction from Text",
author = "Tamari, Ronen and
Shindo, Hiroyuki and
Shahaf, Dafna and
Matsumoto, Yuji",
booktitle = "Proceedings of the Workshop on Extracting Structured Knowledge from Scientific Publications",
month = jun,
year = "2019",
address = "Minneapolis, Minnesota",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/W19-2609",
pages = "62--71",
abstract = "Understanding procedural text requires tracking entities, actions and effects as the narrative unfolds. We focus on the challenging real-world problem of action-graph extraction from materials science papers, where language is highly specialized and data annotation is expensive and scarce. We propose a novel approach, Text2Quest, where procedural text is interpreted as instructions for an interactive game. A learning agent completes the game by executing the procedure correctly in a text-based simulated lab environment. The framework can complement existing approaches and enables richer forms of learning compared to static texts. We discuss potential limitations and advantages of the approach, and release a prototype proof-of-concept, hoping to encourage research in this direction.",
}
This project welcomes contributions and suggestions.
- Add SynthesisApparatus entities to generated quests
- More complex surface texts.
- Add component to convert annotated action graphs to TextLabs format.