Skip to content
A virtual assistant bot in Minecraft
Python C++ HTML Lua Shell Makefile
Branch: master
Clone or download
drothermel Merge pull request #10 from facebookresearch/add_conda_env_instr
Update readme to include conda env instructions
Latest commit 73d375c Sep 6, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci Initial commit Jul 18, 2019
bin Initial commit Jul 18, 2019
client Initial commit Jul 18, 2019
codetools Initial commit Jul 18, 2019
cuberite @ 999faa9 Initial commit Jul 18, 2019
cuberite_config Initial commit Jul 18, 2019
cuberite_plugins Initial commit Jul 18, 2019
docker Initial commit Jul 18, 2019
logging Initial commit Jul 18, 2019
minecraft_specs Initial commit Jul 18, 2019
python Initial commit Jul 18, 2019
schematic_convert/src Initial commit Jul 18, 2019
.clang-format Initial commit Jul 18, 2019
.dockerignore Initial commit Jul 18, 2019
.flake8 Initial commit Jul 18, 2019
.gitattributes Initial commit Jul 18, 2019
.gitignore Initial commit Jul 18, 2019
.gitmodules Initial commit Jul 18, 2019
CODE_OF_CONDUCT.md Initial commit Jul 18, 2019
CONTRIBUTING.md Initial commit Jul 18, 2019
LICENSE Initial commit Jul 18, 2019
Makefile Initial commit Jul 18, 2019
README.md Update readme to include conda env instructions Sep 5, 2019
pyproject.toml Initial commit Jul 18, 2019
requirements.txt Initial commit Jul 18, 2019

README.md

The goal of this project is to build an intelligent, collaborative assistant bot in the game of Minecraft1 that can perform a wide variety of tasks specified by human players. Its primary purpose is to be a tool for artifical intelligence researchers interested in grounded dialogue and interactive learning. This project is in active development.

A detailed outline and documentation is available in this paper

This release is motivated by a long-term research agenda described here.

GIF of Gameplay With Bot

Installation & Getting Started

Do this section before cloning the repo.

Dependencies

Make sure the following packages have already been installed before moving on:

Python Requirements: Using A Conda Environment

To build a conda environment that supports this release:

# Create a new env and install pip
conda create -n minecraft_env
conda activate minecraft_env
conda install pip

# The requirements.txt contains some reqs that need to be installed with conda 
# and others that need to be installed with pip.  This switches between these 
# two installation methods
while read requirement;
    do conda install --yes $requirement || pip install $requirement;
done < requirements.txt
pip install ipdb
conda install pytorch torchvision -c pytorch

Then activate this environment whenever you want to run the agent.

Install git-lfs

# OSX
brew install git-lfs
git lfs install

# On Ubuntu
sudo add-apt-repository ppa:git-core/ppa
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt-get install git-lfs
git lfs install

Checking out the repo

Use this command, or your submodules will not be pulled, and your clone will take a very long time:

git lfs clone --recursive git@github.com:facebookresearch/craftassist.git

Now cd craftassist before proceeding to the following sections.

Building client and server

To build Cuberite and the C++ Minecraft client:

make

Run the Cuberite instance

Run the following command

python ./python/cuberite_process.py

to start an instance of cuberite instance listening on localhost:25565

Connecting your Minecraft game client (so you can see what's happening)

Buy and download the official Minecraft client.

You can inspect the world and view the Minecraft agent's actions by logging into the running Cuberite instance from the game client.

To connect the client to the running Cuberite instance, click in the Minecraft client:

Multiplayer > Direct Connect > localhost:25565

Error: Unsupported Protocol Version

Minecraft has recently release v1.13, and our Cuberite system supports at most v1.12

Please follow these instructions to add a 1.12.x profile and use it to connect.

Running the interactive V0 agent

Assuming you have set up the Cuberite server and the client, in a separate tab, run:

python ./python/craftassist/craftassist_agent.py

You should see a new bot player join the game.

Chat with the bot by pressing t to open the dialogue box, and Enter to submit.

Use the w, a, s, and d keys to navigate, left and right mouse clicks to destroy and place blocks, and e to open your inventory and select blocks to place.

Running tests

./python/craftassist/test.sh

Datasets

Download links to the datasets described in section 6 of Technical Whitepaper are provided here:

In the root of each tarball is a README that details the file structure contained within.

Citation

If you would like to cite this repository in your research, please cite the CraftAssist paper.

@misc{gray2019craftassist,
    title={CraftAssist: A Framework for Dialogue-enabled Interactive Agents},
    author={Jonathan Gray and Kavya Srinet and Yacine Jernite and Haonan Yu and Zhuoyuan Chen and Demi Guo and Siddharth Goyal and C. Lawrence Zitnick and Arthur Szlam},
    year={2019},
    eprint={1907.08584},
    archivePrefix={arXiv},
    primaryClass={cs.AI}
}

License

CraftAssist is MIT licensed.

1 Minecraft features: © Mojang Synergies AB included courtesy of Mojang AB

You can’t perform that action at this time.