[CircleCI] Fix dm_control rendering #339

vmoens · 2022-08-05T08:38:24Z

Fixes the rendering for dm_control.
OsX tests now run on glfw, linux cpu tests on osmesa and linux gpu on egl (which is faster than osmesa).
Rendering is only tested on gpu machines.

For context, the EGL bug was due to running the bash command via docker with a set of env variables that were inappropriate. As regular bash execution runs smoothly without this, we simply discard the docker prefix of the run_test.sh command.

vmoens · 2022-08-05T14:02:35Z

env variable:

CIRCLECI=true
CONDA_DEFAULT_ENV=/home/circleci/project/env
CONDA_EXE=/home/circleci/project/conda/bin/conda
CONDA_PREFIX=/home/circleci/project/env
CONDA_PREFIX_1=/home/circleci/project/conda
CONDA_PROMPT_MODIFIER=(/home/circleci/project/env)
CONDA_PYTHON_EXE=/home/circleci/project/conda/bin/python
CONDA_SHLVL=2
CUDA_VERSION=10.2
DISPLAY=unix:0.0
HOME=/root
HOSTNAME=fc658fb51d7c
LANG=en_US.UTF-8
LANGUAGE=en_US.UTF-8
LC_ALL=en_US.UTF-8
LD_LIBRARY_PATH=/home/circleci/project/.mujoco/mujoco210/bin:/home/circleci/project/env/lib
LIBRARY_PATH=/usr/local/cuda/lib64/stubs
MJLIB_PATH=/home/circleci/project/.mujoco/mujoco-2.1.1/lib/libmujoco.so.2.1.1
MKL_THREADING_LAYER=GNU
MUJOCO_GL=egl
MUJOCO_PY_MUJOCO_PATH=/home/circleci/project/.mujoco/mujoco210
NCCL_VERSION=2.13.4
NVARCH=x86_64
NVIDIA_DRIVER_CAPABILITIES=compute,utility
NVIDIA_REQUIRE_CUDA=cuda>=10.2
NVIDIA_VISIBLE_DEVICES=all
NV_CUDA_CUDART_DEV_VERSION=10.2.89-1
NV_CUDA_CUDART_VERSION=10.2.89-1
NV_CUDA_LIB_VERSION=10.2.89-1
NV_LIBCUBLAS_DEV_PACKAGE=libcublas-devel-10.2.2.89-1
NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-devel
NV_LIBCUBLAS_DEV_VERSION=10.2.2.89-1
NV_LIBCUBLAS_PACKAGE=libcublas10-10.2.2.89-1
NV_LIBCUBLAS_PACKAGE_NAME=libcublas10
NV_LIBCUBLAS_VERSION=10.2.2.89-1
NV_LIBNCCL_DEV_PACKAGE=libnccl-devel-2.13.4-1+cuda10.2
NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-devel
NV_LIBNCCL_DEV_PACKAGE_VERSION=2.13.4-1
NV_LIBNCCL_PACKAGE=libnccl-2.13.4-1+cuda10.2
NV_LIBNCCL_PACKAGE_NAME=libnccl
NV_LIBNCCL_PACKAGE_VERSION=2.13.4-1
NV_LIBNPP_DEV_VERSION=10.2.89-1
NV_LIBNPP_VERSION=10.2.89-1
NV_NVML_DEV_VERSION=10.2.89-1
NV_NVPROF_VERSION=10.2.89-1
NV_NVTX_VERSION=10.2.89-1
PATH=/home/circleci/project/env/bin:/home/circleci/project/conda/condabin:/opt/rh/devtoolset-9/root/usr/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/home/circleci/project
PYOPENGL_PLATFORM=egl
PYTORCH_TEST_WITH_SLOW=1
SDL_VIDEODRIVER=dummy
SHLVL=1
SSL_CERT_FILE=/opt/_internal/certs.pem
TERM=xterm
_=/usr/bin/printenv
_CE_CONDA=
_CE_M=
__CONDA_SHLVL_1_LD_LIBRARY_PATH=/opt/rh/devtoolset-9/root/usr/lib64:/opt/rh/devtoolset-9/root/usr/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
brand=tesla,driver>=418,driver<419

env variable in SSH session

"34.136.16.61",\
"GCE")
"ubuntu-2004-cuda-11.4:202110-01",\
(image:\
(master)
--allocation=ZB32TDXW
--isMachineProvisioner=true
--launchInfo=machine-agent\
--maxRunTime=5h0m0s
--runnerAPIBaseURL=https://runner.circleci.com
--sshAdvertiseAddr=34.136.16.61:54782
-c
/tmp/circleci-machine-agent1023746743/circleci-agent
1.0.29116-1432b03\
ADB_INSTALL_TIMEOUT=120
ANDROID_HOME=/opt/android/sdk
ANDROID_NDK_HOME=/opt/android/sdk/ndk/23.0.7599858
ANDROID_NDK_ROOT=/opt/android/sdk/ndk/23.0.7599858
ANDROID_SDK_ROOT=/opt/android/sdk
AWS_PAGER=
BASH_ENV=/tmp/.bash_env-62ed1fc724ad5f2e63d3cbe6-0-build
CI=true
CIRCLECI=true
CIRCLECI_PKG_DIR=/opt/circleci
CIRCLE_BRANCH=pull/339
CIRCLE_BUILD_NUM=28716
CIRCLE_BUILD_URL=https://circleci.com/gh/facebookresearch/rl/28716
CIRCLE_INTERNAL_SCRATCH=/tmp/circleci-3492665234
CIRCLE_INTERNAL_TASK_DATA=/tmp/.circleci-task-data-62ed1fc724ad5f2e63d3cbe6-0-build
CIRCLE_JOB=unittest_linux_stable_gpu_py3.8
CIRCLE_NODE_INDEX=0
CIRCLE_NODE_TOTAL=1
CIRCLE_PROJECT_REPONAME=rl
CIRCLE_PROJECT_USERNAME=facebookresearch
CIRCLE_PR_NUMBER=339
CIRCLE_PR_REPONAME=rl
CIRCLE_PR_USERNAME=vmoens
CIRCLE_PULL_REQUEST=https://github.com/facebookresearch/rl/pull/339
CIRCLE_PULL_REQUESTS=https://github.com/facebookresearch/rl/pull/339
CIRCLE_REPOSITORY_URL=git@github.com:facebookresearch/rl.git
CIRCLE_SHA1=4cddd6e32cf032ae538a68009c3937ab84422dc8
CIRCLE_SHELL_ENV=/tmp/.bash_env-62ed1fc724ad5f2e63d3cbe6-0-build
CIRCLE_USERNAME=vmoens
CIRCLE_WORKFLOW_ID=181f97df-3dca-42e6-9019-a14e0f85eec1
CIRCLE_WORKFLOW_JOB_ID=094220f2-4b0a-4d3b-aa25-030470f084bd
CIRCLE_WORKFLOW_WORKSPACE_ID=98198707-0b73-4e57-84c5-2e93edbf1bd9
CIRCLE_WORKING_DIRECTORY=/home/circleci/project
CI_PULL_REQUEST=https://github.com/facebookresearch/rl/pull/339
CONDA_DEFAULT_ENV=/home/circleci/project/env
CONDA_EXE=/home/circleci/project/conda/bin/conda
CONDA_PREFIX=/home/circleci/project/env
CONDA_PREFIX_1=/home/circleci/project/conda
CONDA_PROMPT_MODIFIER=(/home/circleci/project/env)
CONDA_PYTHON_EXE=/home/circleci/project/conda/bin/python
CONDA_SHLVL=2
CU_VERSION=cu113
DBUS_SESSION_BUS_ADDRESS=/dev/null
DISPLAY=unix:0.0
GEM_HOME=/opt/circleci/.rvm/gems/ruby-3.0.2
GEM_PATH=/opt/circleci/.rvm/gems/ruby-3.0.2:/opt/circleci/.rvm/gems/ruby-3.0.2@global
GIT_ASKPASS=echo
GOPATH=/home/circleci/.go_workspace:/usr/local/go_workspace
HOME=/home/circleci
IRBRC=/opt/circleci/.rvm/rubies/ruby-3.0.2/.irbrc
JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
JDK_HOME=/usr/lib/jvm/java-11-openjdk-amd64
JRE_HOME=/usr/lib/jvm/java-11-openjdk-amd64
LANG=en_US.UTF-8
LD_LIBRARY_PATH=/home/circleci/project/.mujoco/mujoco210/bin:/home/circleci/project/env/lib
LOGNAME=circleci
M2_HOME=/usr/local/apache-maven
MAIL=/var/mail/circleci
MAVEN_OPTS=-Xmx2048m
MJLIB_PATH=/home/circleci/project/.mujoco/mujoco-2.1.1/lib/libmujoco.so.2.1.1
MKL_THREADING_LAYER=GNU
MUJOCO_GL=egl
MUJOCO_PY_MUJOCO_PATH=/home/circleci/project/.mujoco/mujoco210
MY_RUBY_HOME=/opt/circleci/.rvm/rubies/ruby-3.0.2
NVM_BIN=/opt/circleci/.nvm/versions/node/v14.17.3/bin
NVM_CD_FLAGS=
NVM_DIR=/opt/circleci/.nvm
NVM_INC=/opt/circleci/.nvm/versions/node/v14.17.3/include/node
OLDPWD=/home/circleci
PATH=/home/circleci/project/env/bin:/home/circleci/project/conda/condabin:/home/circleci/.yarn/bin:/home/circleci/.config/yarn/global/node_modules/.bin:/opt/android/sdk/ndk/23.0.7599858:/opt/android/sdk/emulator:/opt/android/sdk/cmdline-tools/latest/bin:/opt/android/sdk/tools:/opt/android/sdk/tools/bin:/opt/android/sdk/platform-tools:/opt/android/sdk/platform-tools/bin:/home/circleci/.go_workspace/bin:/usr/local/go/bin:/opt/circleci/.pyenv/shims:/opt/circleci/.pyenv/bin:/opt/google/google-cloud-sdk/bin:/usr/local/apache-maven/bin:/home/circleci/bin:/home/circleci/.yarn/bin:/home/circleci/.config/yarn/global/node_modules/.bin:/opt/android/sdk/ndk/23.0.7599858:/opt/android/sdk/emulator:/opt/android/sdk/cmdline-tools/latest/bin:/opt/android/sdk/tools:/opt/android/sdk/tools/bin:/opt/android/sdk/platform-tools:/opt/android/sdk/platform-tools/bin:/opt/circleci/.rvm/gems/ruby-3.0.2/bin:/opt/circleci/.rvm/gems/ruby-3.0.2@global/bin:/opt/circleci/.rvm/rubies/ruby-3.0.2/bin:/home/circleci/.go_workspace/bin:/usr/local/go/bin:/opt/circleci/.nvm/versions/node/v14.17.3/bin:/opt/circleci/.pyenv/shims:/opt/circleci/.pyenv/bin:/opt/google/google-cloud-sdk/bin:/usr/local/apache-maven/bin:/home/circleci/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/usr/local/gradle-7.1.1/bin:/opt/circleci/.rvm/bin:/snap/bin:/opt/circleci/.rvm/bin:/usr/local/gradle-7.1.1/bin:/snap/bin:/opt/circleci/.rvm/bin
PWD=/home/circleci/project
PYENV_ROOT=/opt/circleci/.pyenv
PYENV_SHELL=bash
PYOPENGL_PLATFORM=egl
PYTHON_VERSION=3.8
PYTORCH_TEST_WITH_SLOW=1
RUBY_VERSION=ruby-3.0.2
SDL_VIDEODRIVER=dummy
SHELL=/bin/bash
SHLVL=1
SSH_ASKPASS=false
SSH_AUTH_SOCK=/tmp/circleci-3492665234/ssh_auth_sock
SUDO_COMMAND=/bin/bash
SUDO_GID=0
SUDO_UID=0
SUDO_USER=root
TAR_OPTIONS=--no-same-owner
TERM=xterm-256color
UPLOAD_CHANNEL=nightly
USER=circleci
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
_=/usr/bin/printenv
_CE_CONDA=
_CE_M=
__CONDA_SHLVL_1_DISPLAY=:99
_internal
agent-runner
image_name=pytorch/manylinux-cuda113
ip:\
provider:\
public\
rvm_bin_path=/opt/circleci/.rvm/bin
rvm_gemset_create_on_use_flag=1
rvm_path=/opt/circleci/.rvm
rvm_prefix=/opt/circleci
rvm_trust_rvmrcs_flag=1
rvm_version=1.29.12-next
version\

Tony Tong and others added 28 commits August 3, 2022 00:32

update linux setup_env to include libosmesa6 and libgl1-mesa-glx

a816d0c

try just apt

b714c34

try yum

a77c0b5

try yum again

3424edd

well it is functorch now

7aabc85

remove install from git

f7692ee

Merge branch 'main' into TT_circleci

7ca29dc

remove

567c00f

revert format

2f131c9

remove conda install

69f436c

should be good

8d99c5d

remove xml

0aaba3f

save setup

8e2ae9a

git merge

c98d40d

try fix openGL version

d0dd7bb

for GPU tests?

2901caa

revert

d5dbde9

try again

b6a26df

try again

76253a8

try again

cdbeb14

try again

06a9e1c

try again

d6a8f4e

try again

a672c38

update pyopenGL platform

f436505

check env vars

35ccfaf

check mesa

cbaa2c9

ld path

93fb291

init

7f27626

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 5, 2022

amend

9e777b8

vmoens added 3 commits August 5, 2022 12:31

amend

1024a02

amend

09bb294

amend

4cddd6e

vmoens added 15 commits August 5, 2022 18:47

amend

650afc1

amend

0fb5793

amend

2c2d5e4

LIBRARY_PATH

f10e164

__CONDA_SHLVL_1_DISPLAY and __CONDA_SHLVL_1_LD_LIBRARY_PATH

47d3a32

amend

2dc443c

amend

7c692ce

amend

74af44c

amend

ea07ec9

amend

5eda0d0

amend

4d551ae

amend

018e9d3

amend

86a0b1e

amend

b678d01

amend

219f87a

vmoens mentioned this pull request Aug 6, 2022

[BugFix]: update linux setup_env to include libosmesa6 and libgl1-mesa-glx #335

Closed

6 tasks

vmoens changed the title ~~[NOMRG] fix egl~~ [CircleCI] Fix dm_control rendering Aug 6, 2022

vmoens mentioned this pull request Aug 6, 2022

Using mujoco rendering on CircleCI google-deepmind/mujoco#398

Closed

vmoens added 2 commits August 6, 2022 10:43

amend

a72d88b

amend

d4da7c4

vmoens marked this pull request as ready for review August 6, 2022 10:44

vmoens added bug Something isn't working CI Has to do with CI setup (e.g. wheels & builds, tests...) labels Aug 6, 2022

vmoens merged commit 30ef218 into pytorch:main Aug 6, 2022

vmoens deleted the fix_egl branch August 6, 2022 10:46

vmoens mentioned this pull request Aug 6, 2022

[BUG] rendering of dm_control fails on circleci #329

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[CircleCI] Fix dm_control rendering #339

[CircleCI] Fix dm_control rendering #339

Uh oh!

vmoens commented Aug 5, 2022 •

edited

Loading

Uh oh!

vmoens commented Aug 5, 2022 •

edited

Loading

Uh oh!

Uh oh!

[CircleCI] Fix dm_control rendering #339

[CircleCI] Fix dm_control rendering #339

Uh oh!

Conversation

vmoens commented Aug 5, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vmoens commented Aug 5, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

vmoens commented Aug 5, 2022 •

edited

Loading

vmoens commented Aug 5, 2022 •

edited

Loading