(change request) Fix atari flickering

[justheuristic]

Hello!

Could you please add a version of atari environments that does not "flicker".
Flickering = only displaying part of all sprites each frame for performance reasons (back then it was an issue).

The code that allows one to observe flickering in AirRaid (only half of "enemies" are displayed each tick, some screen frames are just black):

import gym
import time
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make("AirRaid-v3")
for i in range(100):
    env.render()
    env.step(5)
    time.sleep(0.1)

if you replace AirRaid with e.g. Alien, only half of dots are displayed on each tick.

The original deepmind paper mentioned taking max over each pixel to get rid of flickering, but in gym this cannot be done without modifying the AtariEnv class since the game is also using random time delays between frames.

One option is to make each gym step worth N timesteps in ALE (N = length of the flickering loop) and take max/min over pixels depending on the game.
There may or may not be a better solution on ALE level.

Otherwise all atari games with flickering are much farther from "fully observable" than the ones in papers, making those irreproducible.

[justheuristic]

Anyone?

[tlbtlbtlb]

There are versions of the environments that do what I think you want, formed by adding Deterministic or NoFrameSkip to the name. For example, env.make("AirRaidDeterministic-v3").

[ppwwyyxx]

The same as #280.

[hholst80]

The best solution to this is to use the frame interpolation option available in ALE.

[christopherhesse]

Closing in favor of #280

[0xcaff]

There are in fact multiple versions of the envs without filters applied to the atari output.

gym/gym/envs/__init__.py

Lines 626 to 659 in c33cfd8

	# Use a deterministic frame skip.
	register(
	id='{}Deterministic-v0'.format(name),
	entry_point='gym.envs.atari:AtariEnv',
	kwargs={'game': game, 'obs_type': obs_type, 'frameskip': frameskip, 'repeat_action_probability': 0.25},
	max_episode_steps=100000,
	nondeterministic=nondeterministic,
	)
	register(
	id='{}Deterministic-v4'.format(name),
	entry_point='gym.envs.atari:AtariEnv',
	kwargs={'game': game, 'obs_type': obs_type, 'frameskip': frameskip},
	max_episode_steps=100000,
	nondeterministic=nondeterministic,
	)
	register(
	id='{}NoFrameskip-v0'.format(name),
	entry_point='gym.envs.atari:AtariEnv',
	kwargs={'game': game, 'obs_type': obs_type, 'frameskip': 1, 'repeat_action_probability': 0.25}, # A frameskip of 1 means we get every frame
	max_episode_steps=frameskip * 100000,
	nondeterministic=nondeterministic,
	)
	# No frameskip. (Atari has no entropy source, so these are
	# deterministic environments.)
	register(
	id='{}NoFrameskip-v4'.format(name),
	entry_point='gym.envs.atari:AtariEnv',
	kwargs={'game': game, 'obs_type': obs_type, 'frameskip': 1}, # A frameskip of 1 means we get every frame
	max_episode_steps=frameskip * 100000,
	nondeterministic=nondeterministic,
	)