Pointer event exploration causes crash

[unixpickle]

Expected behavior

Play flashgames.EasterEggsChallenge-v0 repeatedly with the following code:

import time
import random

import gym
import universe
from universe import wrappers

env = wrappers.BlockingReset(gym.make('flashgames.EasterEggsChallenge-v0'))
env.configure(remotes=1, fps=5)

env.reset()
while True:
        # Random pointer event in playable region.
        x = random.randint(284, 744)
        y = random.randint(148, 594)
        clicked = random.randint(0, 1)
        obs, rew, done, inf = env.step([[('PointerEvent', x, y, clicked)]])
        if done[0]:
                print('Finished an episode')
                # imagine doing some training here or something.
                time.sleep(3)
                env.reset()

The above agent sends random pointer events, which is reasonable for an RL agent that is starting out from scratch. All pointer events are constrained to the meaningful rectangle of the screen, as shown in red here:

Actual behavior

The program crashes with this error:

universe.error.Error: 1/1 environments have crashed! Most recent error: {'0': 'Rewarder session failed: Lost connection: connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake) (clean=False code=1006)'}

The agent usually makes it through a few successful runs before the crash. However, the crash usually happens within 5-10 episodes.

When I run the latest version of the flashgames Docker container manually

docker run --rm -p 5900:5900 -p 15900:15900 --cap-add SYS_ADMIN --ipc host --privileged quay.io/openai/universe.flashgames:0.20.29

and connect to it via

env.configure(remotes='vnc://localhost:5900+15900', fps=5)

the crash still occurs. The crash occurs with both versions 0.20.28 and 0.20.29 of the flashgames container.

When I prevent the agent from clicking by setting clicked = 0 in the above script, the crash does not seem to occur. I suspect that the crash is due to the agent clicking a button right as the game is finishing. If this is the case, I see no easy way to prevent this bug; further, I see no safe way to implement mouse-based exploration for RL agents.

Versions

Linux saturn 4.8.0-52-generic #55~16.04.1-Ubuntu SMP Fri Apr 28 14:36:29 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Python 3.5.2
Name: universe
Version: 0.21.5
Summary: Universe: a software platform for measuring and training an AI's general intelligence across the world's supply of games, websites and other applications.
Home-page: https://github.com/openai/universe
Author: OpenAI
Author-email: universe@openai.com
License: UNKNOWN
Location: /home/alex/go/src/github.com/openai/universe
Requires: autobahn, docker-py, docker-pycreds, fastzbarlight, go-vncdriver, gym, Pillow, PyYAML, six, twisted, ujson
---
Name: gym
Version: 0.9.1
Summary: The OpenAI Gym: A toolkit for developing and comparing your reinforcement learning agents.
Home-page: https://github.com/openai/gym
Author: OpenAI
Author-email: gym@openai.com
License: UNKNOWN
Location: /usr/local/lib/python3.5/dist-packages
Requires: requests, six, numpy, pyglet
---
Name: numpy
Version: 1.12.1
Summary: NumPy: array processing for numbers, strings, records, and objects.
Home-page: http://www.numpy.org
Author: NumPy Developers
Author-email: numpy-discussion@scipy.org
License: BSD
Location: /usr/local/lib/python3.5/dist-packages
Requires:
---
Name: go-vncdriver
Version: 0.4.19
Summary: UNKNOWN
Home-page: UNKNOWN
Author: UNKNOWN
Author-email: UNKNOWN
License: UNKNOWN
Location: /usr/local/lib/python3.5/dist-packages
Requires: numpy
---
Name: Pillow
Version: 4.1.1
Summary: Python Imaging Library (Fork)
Home-page: https://python-pillow.org
Author: Alex Clark (Fork Author)
Author-email: aclark@aclark.net
License: Standard PIL License
Location: /usr/local/lib/python3.5/dist-packages
Requires: olefile

Logs

python-stderr.txt
universe.txt
docker.txt