Skip to content
This repository has been archived by the owner. It is now read-only.

Pointer event exploration causes crash #187

unixpickle opened this issue May 23, 2017 · 0 comments

Pointer event exploration causes crash #187

unixpickle opened this issue May 23, 2017 · 0 comments


Copy link

@unixpickle unixpickle commented May 23, 2017

Expected behavior

Play flashgames.EasterEggsChallenge-v0 repeatedly with the following code:

import time
import random

import gym
import universe
from universe import wrappers

env = wrappers.BlockingReset(gym.make('flashgames.EasterEggsChallenge-v0'))
env.configure(remotes=1, fps=5)

while True:
        # Random pointer event in playable region.
        x = random.randint(284, 744)
        y = random.randint(148, 594)
        clicked = random.randint(0, 1)
        obs, rew, done, inf = env.step([[('PointerEvent', x, y, clicked)]])
        if done[0]:
                print('Finished an episode')
                # imagine doing some training here or something.

The above agent sends random pointer events, which is reasonable for an RL agent that is starting out from scratch. All pointer events are constrained to the meaningful rectangle of the screen, as shown in red here:


Actual behavior

The program crashes with this error:

universe.error.Error: 1/1 environments have crashed! Most recent error: {'0': 'Rewarder session failed: Lost connection: connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake) (clean=False code=1006)'}

The agent usually makes it through a few successful runs before the crash. However, the crash usually happens within 5-10 episodes.

When I run the latest version of the flashgames Docker container manually

docker run --rm -p 5900:5900 -p 15900:15900 --cap-add SYS_ADMIN --ipc host --privileged

and connect to it via

env.configure(remotes='vnc://localhost:5900+15900', fps=5)

the crash still occurs. The crash occurs with both versions 0.20.28 and 0.20.29 of the flashgames container.

When I prevent the agent from clicking by setting clicked = 0 in the above script, the crash does not seem to occur. I suspect that the crash is due to the agent clicking a button right as the game is finishing. If this is the case, I see no easy way to prevent this bug; further, I see no safe way to implement mouse-based exploration for RL agents.


Linux saturn 4.8.0-52-generic #55~16.04.1-Ubuntu SMP Fri Apr 28 14:36:29 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Python 3.5.2
Name: universe
Version: 0.21.5
Summary: Universe: a software platform for measuring and training an AI's general intelligence across the world's supply of games, websites and other applications.
Author: OpenAI
License: UNKNOWN
Location: /home/alex/go/src/
Requires: autobahn, docker-py, docker-pycreds, fastzbarlight, go-vncdriver, gym, Pillow, PyYAML, six, twisted, ujson
Name: gym
Version: 0.9.1
Summary: The OpenAI Gym: A toolkit for developing and comparing your reinforcement learning agents.
Author: OpenAI
License: UNKNOWN
Location: /usr/local/lib/python3.5/dist-packages
Requires: requests, six, numpy, pyglet
Name: numpy
Version: 1.12.1
Summary: NumPy: array processing for numbers, strings, records, and objects.
Author: NumPy Developers
License: BSD
Location: /usr/local/lib/python3.5/dist-packages
Name: go-vncdriver
Version: 0.4.19
Summary: UNKNOWN
Home-page: UNKNOWN
Author-email: UNKNOWN
License: UNKNOWN
Location: /usr/local/lib/python3.5/dist-packages
Requires: numpy
Name: Pillow
Version: 4.1.1
Summary: Python Imaging Library (Fork)
Author: Alex Clark (Fork Author)
License: Standard PIL License
Location: /usr/local/lib/python3.5/dist-packages
Requires: olefile



@gdb gdb closed this Apr 5, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.