Complete installation script for Ubuntu 20.04

[danijar]

Not an issue, just thought I'd share a full installation script for the DMLab Python package because it took a while to put all the pieces together and might be helpful for others. Tested on Ubuntu 20.04, might also work on newer versions:

#!/bin/sh
set -eu

# Dependencies
apt-get update && apt-get install -y \
    build-essential curl freeglut3 gettext git libffi-dev libglu1-mesa \
    libglu1-mesa-dev libjpeg-dev liblua5.1-0-dev libosmesa6-dev \
    libsdl2-dev lua5.1 pkg-config python-setuptools python3-dev \
    software-properties-common unzip zip zlib1g-dev g++
pip3 install numpy

# Bazel
apt-get install -y apt-transport-https curl gnupg
curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg
mv bazel.gpg /etc/apt/trusted.gpg.d/
echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list
apt-get update && apt-get install -y bazel

# Build
git clone https://github.com/deepmind/lab.git
cd lab
echo 'build --cxxopt=-std=c++17' > .bazelrc
bazel build -c opt //python/pip_package:build_pip_package
./bazel-bin/python/pip_package/build_pip_package /tmp/dmlab_pkg
pip3 install --force-reinstall /tmp/dmlab_pkg/deepmind_lab-*.whl
cd ..
rm -rf lab

# Stimuli
mkdir dmlab_data
cd dmlab_data
pip3 install Pillow
curl https://bradylab.ucsd.edu/stimuli/ObjectsAll.zip -o ObjectsAll.zip
unzip ObjectsAll.zip
cd OBJECTSALL
python3 << EOM
import os
from PIL import Image
files = [f for f in os.listdir('.') if f.lower().endswith('jpg')]
for i, file in enumerate(sorted(files)):
  print(file)
  im = Image.open(file)
  im.save('../%04d.png' % (i+1))
EOM
cd ..
rm -rf __MACOSX OBJECTSALL ObjectsAll.zip

apt-get clean

[tkoeppe]

You can reuse the .precommit.bazelrc file instead of making your own (--bazelrc=...).

[danijar]

Good to know about --bazelrc=.... The .precommit.bazelrc specifies --cxxopt=-std=c++11 though, wouldn't that be an issue? Without any .bazelrc I got an error that the build requires at least C++ 14.

[tkoeppe]

Oh, I think that's a pending issue: Abseil has dropped support for C++11, so I think we need to do the same here.

[cirosantilli]

We need to add a Dockerfile to this project.

[elliottower]

We need to add a Dockerfile to this project.

Would love an official dockerfile but you can pretty easily adapt this installation script to work with docker:

# Install DM lab requirements
RUN apt-get -y update \
    && apt-get install --no-install-recommends -y \
    build-essential curl freeglut3 gettext git libffi-dev libglu1-mesa \
    libglu1-mesa-dev libjpeg-dev liblua5.1-0-dev libosmesa6-dev \
    libsdl2-dev lua5.1 pkg-config python-setuptools python3-dev \
    software-properties-common unzip zip zlib1g-dev g++

# Install Bazel
RUN apt-get install -y apt-transport-https curl gnupg  \
    && curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg  \
    && mv bazel.gpg /etc/apt/trusted.gpg.d/  \
    && echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list \
    && apt-get update && apt-get install -y bazel

# Build DM lab
RUN git clone https://github.com/deepmind/lab.git \
    && cd lab \
    && echo 'build --cxxopt=-std=c++17' > .bazelrc \
    && bazel build -c opt //python/pip_package:build_pip_package  \
    && ./bazel-bin/python/pip_package/build_pip_package /tmp/dmlab_pkg \
    && pip3 install --force-reinstall /tmp/dmlab_pkg/deepmind_lab-*.whl \
    && cd .. \
    && rm -rf lab

[danijar]

Updated instal script for Ubuntu 22 (it assumes to be inside a Python 3 venv):

https://gist.github.com/danijar/ca6ab917188d2e081a8253b3ca5c36d3

Usage in Dockerfile:

RUN wget -O - https://gist.github.com/danijar/ca6ab917188d2e081a8253b3ca5c36d3/raw/install-dmlab.sh | sh

Unfortunately, I think DMLab is not compatible with Numpy 2, so you'll also need numpy<2 in your requirements.txt.

[tkoeppe]

(What does it take to make DMLab compatible with Numpy 2?)

[danijar]

My understanding is that Numpy's C API has changed a bit, so if that's used inside DMLab it would have to be update to the new API. I'm guessing Numpy is only used for the interface to Python so it shouldn't be that much work?