Dockerized runs of poke-env fail to access port 8000 during imports

[nacharya114]

Hello! This is my first time submitting an issue on Github so I'm a bit scared 😓

I'm basically trying to dockerize the training process of a DQN bot to keep my local environment clean. I've made a dockerized version of the pokemon showdown server that runs with the '--no-security' flag (docker.io/nacharya114/pokemonshowdown), and I've created a jupyter notebook docker stack that mounts to my working directory that runs pokebot. I'm getting errors at import that read:

2021-05-05 17:14:24,768 - RandomPlayer 1 - ERROR - Multiple exceptions: [Errno 111] Connect call failed ('127.0.0.1', 8000), [Errno 99] Cannot assign requested address
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/asyncio/base_events.py", line 1010, in create_connection
    sock = await self._connect_sock(
  File "/opt/conda/lib/python3.8/asyncio/base_events.py", line 924, in _connect_sock
    await self.sock_connect(sock, address)
  File "/opt/conda/lib/python3.8/asyncio/selector_events.py", line 496, in sock_connect
    return await fut
  File "/opt/conda/lib/python3.8/asyncio/futures.py", line 260, in __await__
    yield self  # This tells Task to wait for completion.
  File "/opt/conda/lib/python3.8/asyncio/tasks.py", line 349, in __wakeup
    future.result()
  File "/opt/conda/lib/python3.8/asyncio/futures.py", line 178, in result
    raise self._exception
  File "/opt/conda/lib/python3.8/asyncio/selector_events.py", line 528, in _sock_connect_cb
    raise OSError(err, f'Connect call failed {address}')
ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 8000)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/poke_env/player/player_network_interface.py", line 242, in listen
    async with websockets.connect(
  File "/opt/conda/lib/python3.8/site-packages/websockets/legacy/client.py", line 604, in __aenter__
    return await self
  File "/opt/conda/lib/python3.8/site-packages/websockets/legacy/client.py", line 622, in __await_impl__
    transport, protocol = await self._create_connection()
  File "/opt/conda/lib/python3.8/asyncio/base_events.py", line 1033, in create_connection
    raise OSError('Multiple exceptions: {}'.format(
OSError: Multiple exceptions: [Errno 111] Connect call failed ('127.0.0.1', 8000), [Errno 99] Cannot assign requested address

All the cell is doing is:

# Imports
import dotmap
import json
import asyncio
import os
import importlib

from poke_env.server_configuration import ServerConfiguration
from poke_env.player.random_player import RandomPlayer
from poke_env.player_configuration import PlayerConfiguration
from poke_env.player.baselines import MaxBasePowerPlayer, SimpleHeuristicsPlayer

from pokebot import BotPlayer



PIPELINE_PATH = os.path.join(os.curdir, "hparams.json")
SAVE_PATH = os.path.join(os.curdir, "fakemodel.h5")

my_server_config = ServerConfiguration(
    "ps:8000",
    "https://play.pokemonshowdown.com/action.php?"
)

I've run these two docker containers via compose and have used ping to verify that they can talk to eachother via tcp. I'm a little bit stuck on why the process of importing is causing a block due to localhost 8000 being unavailable. Thanks to anyone whom this concerns!

[hsahovic]

Hey @nacharya114,

Thanks for opening this issue. This sounds like an exciting project! I think I need more information to help you out efficiently, in particular regarding the way you are using docker compose.
On my local setup, I was able to run a poke-env agent on your PS docker image with:

$ docker pull nacharya114/pokemonshowdown 
$ docker run -p 8000:8000 nacharya114/pokemonshowdown 
RESTORE CHATROOM: lobby
RESTORE CHATROOM: staff
Worker 1 now listening on 0.0.0.0:8000
Test your server at http://localhost:8000

And, in my poke-env folder:

$ PYTHONPATH=src python examples/cross_evaluate_random_players.py
--------------  --------------  ---------------               
RandomPlayer 1  RandomPlayer 2
RandomPlayer 1                  1.0
RandomPlayer 2  0.0
--------------  --------------  --------------

Are you sure that port 8000 is being forwarded as intended?

[nacharya114]

Thank you so much for the fast reply! Yes it should be! The compose is pretty simple, I'm just running jupyter lab ephemerally and having it all standing up with open ports. I can access both the PS server and labs server from each container with ping, but any time I try to make a user in the PS UI, I get this error:

The docker compose is just:

version: "3.9"
services:
  notebook:
    image: nacharya114/pokebot-notebooks
    user: root
    environment:
      - GRANT_SUDO=yes
    volumes: 
        - "/home/nacharya/pokebot:/home/jovyan/work"
    ports:
      - "8888:8888"
  ps:
    image: nacharya114/pokemonshowdown:latest
    ports:
      - "8000:8000"

[hsahovic]

I've been able to run poke-env using a docker-compose.yml file that's very similar to the one you used above.

Docker compose file:

version: "3.9"
services:
  notebook:
    image: jupyter/datascience-notebook
    user: root
    environment:
      - GRANT_SUDO=yes
    volumes: 
        - <path>
    ports:
      - "8888:8888"
  ps:
    image: nacharya114/pokemonshowdown
    ports:
      - "8000:8000"

poke-env code executed in jupyter:

from poke_env.player.random_player import RandomPlayer
from poke_env.player.utils import cross_evaluate
from tabulate import tabulate
from poke_env.server_configuration import ServerConfiguration

server_config = ServerConfiguration(
    "ps:8000",
    "authentication-endpoint.com/action.php?"
)

async def main():
    # We create three random players
    players = [RandomPlayer(max_concurrent_battles=10, server_configuration=server_config) for _ in range(3)]

    # Now, we can cross evaluate them: every player will player 20 games against every
    # other player.
    cross_evaluation = await cross_evaluate(players, n_challenges=20)

    # Defines a header for displaying results
    table = [["-"] + [p.username for p in players]]

    # Adds one line per player with corresponding results
    for p_1, results in cross_evaluation.items():
        table.append([p_1] + [cross_evaluation[p_1][p_2] for p_2 in results])

    # Displays results in a nicely formatted table.
    print(tabulate(table))


await main()

Let me know if that helps, or if you still have this issue :)

[nacharya114]

Okay, weird. I can run your cell, but when I run my cell (the only difference is my own class import) it fails with the same socket error.
Specifically, the line in the cell causing the issue is:
from pokebot import BotPlayer

Which shouldn't be running anything since nothing is being instantiated and the only import statements in that class are:

import numpy as np
from typing import Any, Callable, List, Optional, Tuple, Union

from poke_env.environment.battle import Battle
from poke_env.player.env_player import EnvPlayer
from poke_env.player_configuration import PlayerConfiguration
from poke_env.server_configuration import ServerConfiguration
from poke_env.teambuilder.teambuilder import Teambuilder

from .state_engine              import StateEngine
from .utils                     import *

class BotPlayer(EnvPlayer):
...

[hsahovic]

If you can share the code causing the issue and a full traceback, I could take a deeper look at what is going on :)

[nacharya114]

For sure, thanks so much for the amazing help :)
Here's my class that extends EnvPlayer:
https://github.com/nacharya114/pokebot/blob/master/pokebot/bots/bot.py

Here's the notebook:
https://github.com/nacharya114/pokebot/blob/master/notebooks/Training%20Notebook.ipynb

And here's the stack trace ( I wish it showed more):

2021-05-07 01:43:31,649 - RandomPlayer 4 - ERROR - Multiple exceptions: [Errno 111] Connect call failed ('127.0.0.1', 8000), [Errno 99] Cannot assign requested address
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/poke_env/player/player_network_interface.py", line 242, in listen
    async with websockets.connect(
  File "/opt/conda/lib/python3.8/site-packages/websockets/legacy/client.py", line 604, in __aenter__
    return await self
  File "/opt/conda/lib/python3.8/site-packages/websockets/legacy/client.py", line 622, in __await_impl__
    transport, protocol = await self._create_connection()
  File "/opt/conda/lib/python3.8/asyncio/base_events.py", line 1033, in create_connection
    raise OSError('Multiple exceptions: {}'.format(
OSError: Multiple exceptions: [Errno 111] Connect call failed ('127.0.0.1', 8000), [Errno 99] Cannot assign requested address

[hsahovic]

Are you sure this is the code you are using? I was able to run the code below successfully:

import asyncio

from poke_env.player.random_player import RandomPlayer
from poke_env.player.utils import cross_evaluate
from tabulate import tabulate
from poke_env.server_configuration import ServerConfiguration

from pokebot import BotPlayer

class BotPlayer(BotPlayer):
    def choose_move(self, battle):
        return self.choose_random_move(battle)
    
    def _battle_finished_callback(*args, **kwargs):
        pass
    
server_config = ServerConfiguration(
    "ps:8000",
    "authentication-endpoint.com/action.php?"
)

async def main():
    # We create three random players
    players = [
        RandomPlayer(max_concurrent_battles=10, server_configuration=server_config),
        BotPlayer(max_concurrent_battles=10, server_configuration=server_config),
    ]

    # Now, we can cross evaluate them: every player will player 20 games against every
    # other player.
    cross_evaluation = await cross_evaluate(players, n_challenges=20)

    # Defines a header for displaying results
    table = [["-"] + [p.username for p in players]]

    # Adds one line per player with corresponding results
    for p_1, results in cross_evaluation.items():
        table.append([p_1] + [cross_evaluation[p_1][p_2] for p_2 in results])

    # Displays results in a nicely formatted table.
    print(tabulate(table))


await main()

I needed to redefine some of the BotPlayer code as it looks like some of its implementation was not fully ready.

[nacharya114]

Okay, thank you SO much for your help. Seriously. I just copied the code you had up there and everything works. I still get those weird OS errors but everything does run now. I appreciate your dedication to keeping issues cleared on this repo and all the ground work you've done for this!