Run z-machine games in parallel

[YunqiuXu]

Hi, is there any method to run multiple z-machine games (e.g. zork1.z5) in parallel.
For games generated by tw-make this can be done via textworld.gym.make_batch, but how can I achieve this on z-machine games?

[MarcCote]

Hi @YunqiuXu, you can still use textworld.gym.make_batch to play several Z-Machine games.

Here's how.

import textworld.gym
env_id = textworld.gym.register_games(['zork1.z5', 'zork2.z5', 'zork3.z5'])
batch_env_id = textworld.gym.make_batch(env_id, batch_size=5)

env = gym.make(batch_env_id)
obs, infos = env.reset()
for i, ob in enumerate(obs):
    print(ob)
    print("----")

NB: of course, for Z-Machine games, no additional infos can be requested, i.e., the second argument of register_game should be None (the default).
NB2: the list of supported Z-Machine games is the same as with Jericho.

[YunqiuXu]

Hi @MarcCote , thanks for your reply. I tried your code but there's still some error:

FileNotFoundError                         Traceback (most recent call last)
<ipython-input-8-d888bed448f1> in <module>
----> 1 env = gym.make(batch_env_id)

~/anaconda3/lib/python3.6/site-packages/gym/envs/registration.py in make(id)
    165 
    166 def make(id):
--> 167     return registry.make(id)
    168 
    169 def spec(id):

~/anaconda3/lib/python3.6/site-packages/gym/envs/registration.py in make(self, id)
    117         logger.info('Making new env: %s', id)
    118         spec = self.spec(id)
--> 119         env = spec.make()
    120         # We used to have people override _reset/_step rather than
    121         # reset/step. Set _gym_disable_underscore_compat = True on

~/anaconda3/lib/python3.6/site-packages/gym/envs/registration.py in make(self)
     84         else:
     85             cls = load(self._entry_point)
---> 86             env = cls(**self._kwargs)
     87 
     88         # Make the enviroment aware of which spec it came from.

~/anaconda3/lib/python3.6/site-packages/textworld/gym/envs/batch_env.py in __init__(self, env_id, batch_size)
    218         assert len(self.env_ids) == self.batch_size
    219 
--> 220         self.envs = [gym.make(self.env_ids[i]) for i in range(self.batch_size)]
    221         self.observation_space = self.envs[0].observation_space
    222         self.action_space = self.envs[0].action_space

~/anaconda3/lib/python3.6/site-packages/textworld/gym/envs/batch_env.py in <listcomp>(.0)
    218         assert len(self.env_ids) == self.batch_size
    219 
--> 220         self.envs = [gym.make(self.env_ids[i]) for i in range(self.batch_size)]
    221         self.observation_space = self.envs[0].observation_space
    222         self.action_space = self.envs[0].action_space

~/anaconda3/lib/python3.6/site-packages/gym/envs/registration.py in make(id)
    165 
    166 def make(id):
--> 167     return registry.make(id)
    168 
    169 def spec(id):

~/anaconda3/lib/python3.6/site-packages/gym/envs/registration.py in make(self, id)
    117         logger.info('Making new env: %s', id)
    118         spec = self.spec(id)
--> 119         env = spec.make()
    120         # We used to have people override _reset/_step rather than
    121         # reset/step. Set _gym_disable_underscore_compat = True on

~/anaconda3/lib/python3.6/site-packages/gym/envs/registration.py in make(self)
     84         else:
     85             cls = load(self._entry_point)
---> 86             env = cls(**self._kwargs)
     87 
     88         # Make the enviroment aware of which spec it came from.

~/anaconda3/lib/python3.6/site-packages/textworld/gym/envs/textworld_games_env.py in __init__(self, game_files, request_infos, action_space, observation_space)
     64             # Extract vocabulary from games.
     65             games_iter = (textworld.Game.load(os.path.splitext(gamefile)[0] + ".json") for gamefile in self.gamefiles)
---> 66             vocab = textworld.text_utils.extract_vocab(games_iter)
     67 
     68         self.action_space = action_space or text_spaces.Word(max_length=8, vocab=vocab)

~/anaconda3/lib/python3.6/site-packages/textworld/text_utils.py in extract_vocab(games)
    121     text = ""
    122     seen = set()
--> 123     for game in games:
    124         if game.kb not in seen:
    125             seen.add(game.kb)

~/anaconda3/lib/python3.6/site-packages/textworld/gym/envs/textworld_games_env.py in <genexpr>(.0)
     63         if action_space is None or observation_space is None:
     64             # Extract vocabulary from games.
---> 65             games_iter = (textworld.Game.load(os.path.splitext(gamefile)[0] + ".json") for gamefile in self.gamefiles)
     66             vocab = textworld.text_utils.extract_vocab(games_iter)
     67 

~/anaconda3/lib/python3.6/site-packages/textworld/generator/game.py in load(cls, filename)
    419     def load(cls, filename: str) -> "Game":
    420         """ Creates `Game` from serialized data saved in a file. """
--> 421         with open(filename, 'r') as f:
    422             return cls.deserialize(json.load(f))
    423 

FileNotFoundError: [Errno 2] No such file or directory: 'detective.json'

Maybe the error is due to the load() function at https://github.com/microsoft/TextWorld/blob/master/textworld/generator/game.py#L419
that the .json file of z5 game is not found.

Is there any method to tackle this?

BTW I could run single .z5 game successfully via benchmark.py and I found some z5-related information in https://github.com/microsoft/TextWorld/blob/master/benchmark/games.json
Can it be used to run .z5 games in parallel?

[MarcCote]

Okay, I see why. We fixed that in the master branch. We are planning on making a new TextWorld release next week (which will include that fix). In the meantime, you can install the master branch using
pip install https://github.com/microsoft/TextWorld/archive/master.zip.

Regarding the .z5 games generated via benchmark, those are running fine because they were generated by TextWorld along with their .json file. Unfortunately, non-TextWorld games do not some with a .json containing the needed metadata.

[vzhong]

Hi @MarcCote , if batch_size is bigger than the number of games, how does this batched environment behave? What do reset and step return in this case? Also if batch_size is exactly equal to the number of games, then are we guaranteed that there is 1 instance of each game?

[MarcCote]

Hi @vzhong, batch_size actually corresponds to the number of "environments" to will be run concurrently. So, if you have an environment that was created from a list of 10 games, a batch_size=3 would mean you are going to play 3 games at the same time, i.e. one per environment. Every time reset is called, the next 3 games are loaded, i.e. the next game from each environment. Calling step (expecting 3 commands) will dispatch each command to the respective game. If a game is finished, calling step won't have any effect on it, i.e. the return state/information is copy from the last step.

I think it is easier to understand with an example. Let's assume we have 5 games and batch_size of 2:
env_id = textworld.gym.register_games([g1, g2, g3, g4, g5]) # Registering the environment with 5 games.

# E = [g1, g2, g3, g4, g5]

benv_id = textworld.gym.make_batch(env_id, batch_size=2)  # Duplicating the environment.
env = gym.make(benv_id)
# E1 = [g1, g2, g3, g4, g5]
# E2 = [g1, g2, g3, g4, g5]

env.seed(123)  # Shuffle the order in which the games are going to be played.
# E1 = [g3, g1, g5, g4, g2]
# E2 = [g5, g2, g3, g1, g4]

env.reset() #Playing [g3, g5]
env.reset() #Playing [g1, g1]
...
env.reset() #Playing [g2, g4]

As you can see you, in the end, you are playing 2*5 games per "epoch", where each game is seen batch_size times per epoch.

I understand this behavior differs from what traditionally the term "batch" means. If you actually wanted to split the games in batch, you will have to register batch_size environment, each one containing a different split of the games.

[vzhong]

ah I see - thank you for the detailed response!