Tetris wrapper's reset_game() doesn't seem to work.

[ogabrielluiz]

quiet = False 
filename  = './roms/Tetris.gb' 
pyboy = PyBoy(filename, window_type="headless" if quiet else "SDL2", window_scale=3, debug=True, game_wrapper=True) 
pyboy.set_emulation_speed(0) 
tetris = pyboy.game_wrapper() 
tetris.start_game() 

After this, calling reset_game() does nothing. Even after sending a few WindowEvents and tick().

[Baekalfen]

What happens if you run python examples/gamewrapper_tetris.py [tetris rom file]?

[ogabrielluiz]

This:

First brick touched the bottom!
Tetris:
Score: 0
Level: 0
Lines: 0
Fitness: 0
Sprites on screen:
Sprite [4]: Position: (72, 128), Shape: (8, 8), Tiles: (Tile: 133), On screen: True
Sprite [5]: Position: (80, 128), Shape: (8, 8), Tiles: (Tile: 133), On screen: True
Sprite [6]: Position: (88, 128), Shape: (8, 8), Tiles: (Tile: 133), On screen: True
Sprite [7]: Position: (80, 136), Shape: (8, 8), Tiles: (Tile: 133), On screen: True
Sprite [8]: Position: (120, 112), Shape: (8, 8), Tiles: (Tile: 129), On screen: True
Sprite [9]: Position: (128, 112), Shape: (8, 8), Tiles: (Tile: 129), On screen: True
Sprite [10]: Position: (136, 112), Shape: (8, 8), Tiles: (Tile: 129), On screen: True
Sprite [11]: Position: (136, 120), Shape: (8, 8), Tiles: (Tile: 129), On screen: True
Tiles on screen:
     0   1   2   3   4   5   6   7   8   9   
____________________________________________
0  | 47  47  47  47  47  47  47  47  47  47  
1  | 47  47  47  47  47  47  47  47  47  47  
2  | 47  47  47  47  47  47  47  47  47  47  
3  | 47  47  47  47  47  47  47  47  47  47  
4  | 47  47  47  47  47  47  47  47  47  47  
5  | 47  47  47  47  47  47  47  47  47  47  
6  | 47  47  47  47  47  47  47  47  47  47  
7  | 47  47  47  47  47  47  47  47  47  47  
8  | 47  47  47  47  47  47  47  47  47  47  
9  | 47  47  47  47  47  47  47  47  47  47  
10 | 47  47  47  47  47  47  47  47  47  47  
11 | 47  47  47  47  47  47  47  47  47  47  
12 | 47  47  47  47  47  47  47  47  47  47  
13 | 47  47  47  47  47  47  47  47  47  47  
14 | 47  47  47  47  47  47  47  47  47  47  
15 | 47  47  47  47  47  47  47  47  47  47  
16 | 47  47  47  47  47  47  47  133 133 133 
17 | 47  47  47  47  47  47  47  47  133 47  
Final game board mask:
Tetris:
Score: 0
Level: 0
Lines: 0
Fitness: 0
Sprites on screen:
Sprite [4]: Position: (72, 24), Shape: (8, 8), Tiles: (Tile: 129), On screen: True
Sprite [5]: Position: (80, 24), Shape: (8, 8), Tiles: (Tile: 129), On screen: True
Sprite [6]: Position: (88, 24), Shape: (8, 8), Tiles: (Tile: 129), On screen: True
Sprite [7]: Position: (88, 32), Shape: (8, 8), Tiles: (Tile: 129), On screen: True
Sprite [8]: Position: (128, 112), Shape: (8, 8), Tiles: (Tile: 134), On screen: True
Sprite [9]: Position: (136, 112), Shape: (8, 8), Tiles: (Tile: 134), On screen: True
Sprite [10]: Position: (120, 120), Shape: (8, 8), Tiles: (Tile: 134), On screen: True
Sprite [11]: Position: (128, 120), Shape: (8, 8), Tiles: (Tile: 134), On screen: True
Tiles on screen:
     0   1   2   3   4   5   6   7   8   9   
____________________________________________
0  | 47  47  47  47  47  47  47  47  47  47  
1  | 47  47  47  47  47  47  47  47  47  47  
2  | 47  47  47  47  47  47  47  47  47  47  
3  | 47  47  47  47  47  47  47  129 129 129 
4  | 47  47  47  47  47  47  47  47  47  129 
5  | 47  47  47  47  47  47  47  47  47  47  
6  | 47  47  47  47  47  47  47  47  47  47  
7  | 47  47  47  47  47  47  47  47  47  47  
8  | 47  47  47  47  47  47  47  47  47  47  
9  | 47  47  47  47  47  47  47  47  47  47  
10 | 47  47  47  47  47  47  47  47  47  47  
11 | 47  47  47  47  47  47  47  47  47  47  
12 | 47  47  47  47  47  47  47  47  47  47  
13 | 47  47  47  47  47  47  47  47  47  47  
14 | 47  47  47  47  47  47  47  47  47  47  
15 | 47  47  47  47  47  47  47  47  47  47  
16 | 47  47  47  47  47  47  47  133 133 133 
17 | 47  47  47  47  47  47  47  47  133 47  

[ogabrielluiz]

I built a py_environment for this wrapper to run with tf-agents and for some reason it seems it sometimes works and sometimes doesn't.

MarioWrapper works every time.

Maybe it is the speed? You said it can lose accuracy on higher speeds. I'm using 0.

Just one more thing: when it the reset_game() doesn't work, it gets stuck in the score/level screen.

[ogabrielluiz]

I think I understand what is happening.

Since the emulation is too fast, it probably is going through the game over state too quickly and my algorithm doesn't catch that so set the episode_ended to True.

No bug, then. Thanks for project. I'm having a lot of fun with it.

[Baekalfen]

If that example run through, then you have something with your code. Because that tests the reset function (look for the asserts). You should try to find the difference between the example and your code.

If you check what’s on the screen at every tick, then it cannot run too fast. Are letting it run while measuring time? If you let it run 60 ticks, it’s equivalent to one second in “real life”.

[ogabrielluiz]

I'm not. I'm just measuring the number of steps the agent takes and penalizing it if it takes too many. I'll try to lower the speed because it seems to bug out after a few thousand episodes.

The example ran and it was due to the way I was checking the agent lost the game.
Now I'm using game_wrapper._game_area_np()[4,5:9] == [21, 14, 31, 14, 21] to check if the game is in the score/level screen.

[Baekalfen]

Is it working for you now?

Unless there is a major, scary bug somewhere, the speed of execution cannot change the outcome.

[MathisFederico]

I think he means that there is no function to detect automaticaly if the game is over (like game_over()) and if so reset the environment.

This is why the openai_gym wrapper needed that function too !
It will make usage realy easier for ML users ahah ^^

[Baekalfen]

I'll close this, as I think we have solved this by including the OpenAI Gym wrapper.