Important
If you have questions or find bugs or anything, you can contact us in our organisation's discussion.
We use code in this repository to generate new missions to facilitate self-play, to run the agent in the environment, and to evaluate the agent for the eval.ai leaderboard.
Important
We only ever ran this on a Ubuntu 20 Linux machine. This has not been tested or verified on MacOS or Windows. Your mileage may vary.
You can run the convenience script at scripts/prepare-user-area.sh.
Caution
I HIGHLY RECOMMEND reading the script to know what it does, because you might not need all of it! The convenience script does do some sudo-based file changes so if you don't want to permanently dirty your computer, look at it!
Tip
If you need sudo to run Docker, prefix the poetry run command with: sudo -E env PATH=$PATH.
T1 evaluation (the one for evai.ai)
When running T1, progress is sent to wandb:emma-simbot/alexa-arena-evaluation. All session IDs are prefixed with T1-.
-
Create a new tmux session.
-
In one tmux pane, run
sudo /usr/bin/X :1 & -
In the other pane, run:
poetry run python -m simbot_offline_inference run-background-services
-
Finally, in a third pane, run:
poetry run python -m simbot_offline_inference run-their-evaluation
-
Let it run.
When running trajectories, each one is a new "run", and all the runs are tracked at wandb:emma-simbot/arena-high-level-trajectories.
-
Create a new tmux session.
-
In one tmux pane, run
sudo /usr/bin/X :1 & -
In the other pane, run:
poetry run python -m simbot_offline_inference run-background-services
-
Finally, in a third pane, run:
poetry run python -m simbot_offline_inference generate-trajectories
-
Let it run.