Breakout doesn't end after losing the game

[familyld]

As mentioned in the title and demonstrated by the gif, after the paddle misses the ball’s rebound, the game still goes on. I check done returned by env.step(action) and it remains false. Is this normal? I am using BreakoutNoFrameskip-v0 and the code I use is as follows,

def eval_policy(policy, env, seed, eval_episodes=1):
    
    avg_reward = 0.
    for _ in range(eval_episodes):
        state, done = env.reset(), False
        while not done:
            env.env.render()
            action = policy.select_action(np.array(state), eval=True)
            state, reward, done, _ = env.step(action)
            avg_reward += reward
            # sleep(0.02)
            
        print(f"Evaluation over {steps} steps: {episode_reward:.3f}")

    avg_reward /= eval_episodes

    print("---------------------------------------")
    print(f"Evaluation over {eval_episodes} episodes: {avg_reward:.3f}")
    print("---------------------------------------")
    return avg_reward

where env is produced by,

# Create environment, add wrapper if necessary and create env_properties
def make_env(env_name, atari_preprocessing):
	env = gym.make(env_name)
	
	is_atari = gym.envs.registry.spec(env_name).entry_point == 'gym.envs.atari:AtariEnv'
	env = AtariPreprocessing(env, **atari_preprocessing) if is_atari else env

	state_dim = (
		atari_preprocessing["state_history"], 
		atari_preprocessing["frame_size"], 
		atari_preprocessing["frame_size"]
	) if is_atari else env.observation_space.shape[0]

	return (
		env,
		is_atari,
		state_dim,
		env.action_space.n
	)

I am using atari-py==0.2.6 and gym==0.17.2.

[familyld]

This is related to Breakout doesn't end after losing the game #76 and the cause is that the agent didn't choose the "fire" action to start a new game. This somewhat reveals that your agent is acting too greedily.