Fix rllab dockerfile before migration to garage #18
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reference: https://gist.github.com/lukechilds/a83e1d7127b78fef38c2914c4ececc3c#gistcomment-2649739 Full error message: => ERROR [sql 4/19] RUN apt-get install -y curl grep sed dpkg && TINI_VERSION=`curl https://github.com/krallin/tini/releases/latest 8.7s ------ > [sql 4/19] RUN apt-get install -y curl grep sed dpkg && TINI_VERSION=`curl https://github.com/krallin/tini/releases/latest | grep -o "/v.*"" | sed 's:^..\(.*\).$:\1:'` && curl -L "https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini_${TINI_VERSION}.deb" > tini.deb && dpkg -i tini.deb && rm tini.deb && apt-get clean: 0.997 Reading package lists... 1.567 Building dependency tree... 1.658 Reading state information... 1.855 sed is already the newest version (4.2.2-7). 1.855 dpkg is already the newest version (1.18.4ubuntu1.7). 1.855 grep is already the newest version (2.25-1~16.04.1). 1.855 The following NEW packages will be installed: 1.855 curl 2.269 0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded. 2.269 Need to get 139 kB of archives. 2.269 After this operation, 340 kB of additional disk space will be used. 2.269 Get:1 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 curl amd64 7.47.0-1ubuntu2.19 [139 kB] 4.510 debconf: delaying package configuration, since apt-utils is not installed 4.662 Fetched 139 kB in 1s (126 kB/s) 4.834 Selecting previously unselected package curl. (Reading database ... 10533 files and directories currently installed.) 4.840 Preparing to unpack .../curl_7.47.0-1ubuntu2.19_amd64.deb ... 4.964 Unpacking curl (7.47.0-1ubuntu2.19) ... 5.524 Setting up curl (7.47.0-1ubuntu2.19) ... 6.048 % Total % Received % Xferd Average Speed Time Time Time Current 6.048 Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 6.701 % Total % Received % Xferd Average Speed Time Time Time Current 6.701 Dload Upload Total Spent Left Speed 100 9 100 9 0 0 17 0 --:--:-- --:--:-- --:--:-- 17 7.255 dpkg-deb: error: 'tini.deb' is not a debian format archive 7.255 dpkg: error processing archive tini.deb (--install): 7.255 subprocess dpkg-deb --control returned error exit status 2 8.350 Errors were encountered while processing: 8.350 tini.deb ------ failed to solve: process "/bin/sh -c apt-get install -y curl grep sed dpkg && TINI_VERSION=`curl https://github.com/krallin/tini/releases/latest | grep -o \"/v.*\\\"\" | sed 's:^..\\(.*\\).$:\\1:'` && curl -L \"https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini_${TINI_VERSION}.deb\" > tini.deb && dpkg -i tini.deb && rm tini.deb && apt-get clean" did not complete successfully: exit code: 1
Reference: https://askubuntu.com/a/857433 Full error message: => ERROR [sql 8/19] RUN apt-get -y build-dep glfw 1.3s ------ > [sql 8/19] RUN apt-get -y build-dep glfw: 0.492 Reading package lists... 1.056 E: You must put some 'source' URIs in your sources.list ------ failed to solve: process "/bin/bash -c apt-get -y build-dep glfw" did not complete successfully: exit code: 100
Reference: https://stackoverflow.com/a/32988728 Full error message: => ERROR [sql 17/20] RUN conda env create -f /root/sql/environment.yml && conda env update 1.0s ------ > [sql 17/20] RUN conda env create -f /root/sql/environment.yml && conda env update: 0.644 /opt/conda/lib/python2.7/site-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.26.18) or chardet (3.0.4) doesn't match a supported version! 0.644 RequestsDependencyWarning) 0.719 /opt/conda/lib/python2.7/site-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.26.18) or chardet (3.0.4) doesn't match a supported version! 0.719 RequestsDependencyWarning) 0.719 Traceback (most recent call last): 0.719 File "/opt/conda/bin/conda", line 11, in <module> 0.719 sys.exit(main()) 0.719 File "/opt/conda/lib/python2.7/site-packages/conda/cli/main.py", line 182, in main 0.719 return conda_exception_handler(_main, *args) 0.719 File "/opt/conda/lib/python2.7/site-packages/conda/exceptions.py", line 644, in conda_exception_handler 0.719 return handle_exception(e) 0.719 File "/opt/conda/lib/python2.7/site-packages/conda/exceptions.py", line 634, in handle_exception 0.719 print_unexpected_error_message(e) 0.719 File "/opt/conda/lib/python2.7/site-packages/conda/exceptions.py", line 596, in print_unexpected_error_message 0.720 stderrlogger.info(get_main_info_str(get_info_dict())) 0.720 File "/opt/conda/lib/python2.7/site-packages/conda/cli/main_info.py", line 162, in get_info_dict 0.720 from ..connection import user_agent 0.720 File "/opt/conda/lib/python2.7/site-packages/conda/connection.py", line 12, in <module> 0.720 from requests import Session, __version__ as REQUESTS_VERSION 0.720 File "/opt/conda/lib/python2.7/site-packages/requests/__init__.py", line 97, in <module> 0.720 from . import utils 0.720 File "/opt/conda/lib/python2.7/site-packages/requests/utils.py", line 24, in <module> 0.720 from . import certs 0.720 ImportError: cannot import name certs ------ failed to solve: process "/bin/bash -c conda env create -f /root/sql/environment.yml && conda env update" did not complete successfully: exit code: 1
Reference: https://stackoverflow.com/a/75080916 Full error message: => ERROR [sql 18/21] RUN conda env create -f /root/sql/environment.yml && conda env update 721.4s ------ > [sql 18/21] RUN conda env create -f /root/sql/environment.yml && conda env update: 1.838 Using Anaconda API: https://api.anaconda.org 1.898 Fetching package metadata ........... 33.34 Solving package specifications: . _libgcc_mutex- 100% |###############################| Time: 0:00:00 1.30 MB/s0 B/s blas-1.0-mkl.t 100% |###############################| Time: 0:00:00 3.77 MB/s0 B/s ca-certificate 100% |###############################| Time: 0:00:00 11.99 MB/s0 B/s freeglut-2.8.1 100% |###############################| Time: 0:00:00 11.86 MB/s0 B/s icu-54.1-0.tar 100% |###############################| Time: 0:00:01 11.47 MB/s0 B/s intel-openmp-2 100% |###############################| Time: 0:00:00 11.58 MB/s0 B/s libgfortran-ng 100% |###############################| Time: 0:00:00 10.63 MB/s0 B/s libiconv-1.14- 100% |###############################| Time: 0:00:00 11.79 MB/s0 B/s libstdcxx-ng-9 100% |###############################| Time: 0:00:00 11.41 MB/s0 B/s readline-6.2-2 100% |###############################| Time: 0:00:00 11.87 MB/s0 B/s sqlite-3.13.0- 100% |###############################| Time: 0:00:00 11.83 MB/s0 B/s tk-8.5.18-0.ta 100% |###############################| Time: 0:00:00 11.01 MB/s0 B/s libgcc-ng-9.1. 100% |###############################| Time: 0:00:00 11.55 MB/s0 B/s mkl-2018.0.3-1 100% |###############################| Time: 0:00:18 11.26 MB/s0 B/s expat-2.4.4-h2 100% |###############################| Time: 0:00:00 12.99 MB/s0 B/s jpeg-9e-h7f872 100% |###############################| Time: 0:00:00 14.29 MB/s0 B/s libffi-3.3-he6 100% |###############################| Time: 0:00:00 17.72 MB/s0 B/s libgcc-7.2.0-h 100% |###############################| Time: 0:00:00 12.41 MB/s0 B/s libxcb-1.15-h7 100% |###############################| Time: 0:00:00 9.56 MB/s0 B/s openssl-1.0.2u 100% |###############################| Time: 0:00:00 11.81 MB/s0 B/s pcre-8.45-h295 100% |###############################| Time: 0:00:00 12.27 MB/s0 B/s xz-5.2.5-h7f87 100% |###############################| Time: 0:00:00 12.35 MB/s0 B/s zlib-1.2.12-h7 100% |###############################| Time: 0:00:00 9.79 MB/s0 B/s glib-2.69.1-h4 100% |###############################| Time: 0:00:00 11.59 MB/s0 B/s libpng-1.6.37- 100% |###############################| Time: 0:00:00 12.64 MB/s0 B/s libxml2-2.9.9- 100% |###############################| Time: 0:00:00 11.88 MB/s0 B/s python-3.5.2-0 100% |###############################| Time: 0:00:01 11.21 MB/s0 B/s cached-propert 100% |###############################| Time: 0:00:00 7.26 MB/s0 B/s certifi-2020.6 100% |###############################| Time: 0:00:00 13.63 MB/s0 B/s click-7.1.2-py 100% |###############################| Time: 0:00:00 16.52 MB/s0 B/s cloudpickle-1. 100% |###############################| Time: 0:00:00 7.26 MB/s0 B/s dbus-1.13.18-h 100% |###############################| Time: 0:00:00 12.13 MB/s0 B/s freetype-2.5.5 100% |###############################| Time: 0:00:00 11.81 MB/s0 B/s gstreamer-1.14 100% |###############################| Time: 0:00:00 11.44 MB/s0 B/s itsdangerous-1 100% |###############################| Time: 0:00:00 7.96 MB/s0 B/s markupsafe-1.0 100% |###############################| Time: 0:00:00 11.17 MB/s0 B/s mkl-service-1. 100% |###############################| Time: 0:00:00 7.56 MB/s0 B/s more-itertools 100% |###############################| Time: 0:00:00 14.74 MB/s0 B/s numpy-1.13.1-p 100% |###############################| Time: 0:00:00 11.44 MB/s0 B/s pyparsing-2.4. 100% |###############################| Time: 0:00:00 16.70 MB/s0 B/s pytz-2021.3-py 100% |###############################| Time: 0:00:00 12.96 MB/s0 B/s sip-4.18.1-py3 100% |###############################| Time: 0:00:00 12.37 MB/s0 B/s six-1.16.0-pyh 100% |###############################| Time: 0:00:00 8.79 MB/s0 B/s werkzeug-1.0.1 100% |###############################| Time: 0:00:00 14.21 MB/s0 B/s wheel-0.37.1-p 100% |###############################| Time: 0:00:00 28.44 MB/s0 B/s cycler-0.10.0- 100% |###############################| Time: 0:00:00 24.97 MB/s0 B/s fontconfig-2.1 100% |###############################| Time: 0:00:00 12.33 MB/s0 B/s gst-plugins-ba 100% |###############################| Time: 0:00:00 11.50 MB/s0 B/s mako-1.1.4-pyh 100% |###############################| Time: 0:00:00 17.02 MB/s0 B/s pyopengl-3.1.1 100% |###############################| Time: 0:00:00 11.95 MB/s0 B/s python-dateuti 100% |###############################| Time: 0:00:00 8.94 MB/s0 B/s retrying-1.3.3 100% |###############################| Time: 0:00:00 39.57 MB/s0 B/s scipy-1.1.0-py 100% |###############################| Time: 0:00:01 11.55 MB/s0 B/s setuptools-40. 100% |###############################| Time: 0:00:00 11.22 MB/s0 B/s zipp-0.6.0-py_ 100% |###############################| Time: 0:00:00 15.62 MB/s0 B/s importlib-meta 100% |###############################| Time: 0:00:00 12.43 MB/s0 B/s jinja2-2.11.3- 100% |###############################| Time: 0:00:00 15.66 MB/s0 B/s joblib-0.14.1- 100% |###############################| Time: 0:00:00 14.10 MB/s0 B/s pip-10.0.1-py3 100% |###############################| Time: 0:00:00 5.63 MB/s0 B/s plotly-4.14.3- 100% |###############################| Time: 0:00:00 11.75 MB/s0 B/s qt-5.6.2-5.tar 100% |###############################| Time: 0:00:03 11.55 MB/s0 B/s theano-0.8.2-p 100% |###############################| Time: 0:00:00 11.80 MB/s0 B/s flask-1.1.4-py 100% |###############################| Time: 0:00:00 16.95 MB/s0 B/s importlib_meta 100% |###############################| Time: 0:00:00 22.03 MB/s0 B/s lasagne-0.1-py 100% |###############################| Time: 0:00:00 21.60 MB/s0 B/s pyqt-5.6.0-py3 100% |###############################| Time: 0:00:00 11.46 MB/s0 B/s matplotlib-2.0Collecting gtimer################## | Time: 0:00:00 11.56 MB/s0 B/s 689.4 Downloading https://files.pythonhosted.org/packages/03/fb/dcaaf7b85aed9adb3b01f63cd5a5c779cc6216073e57395ec231fd45e083/gtimer-1.0.0b5.tar.gz 689.7 Collecting gym==0.8 689.8 Downloading https://files.pythonhosted.org/packages/cb/bb/53b742d7d64fefaca668b09b74ed875956403d88095b3e169b8e73b0bd25/gym-0.8.0.tar.gz (150kB) 690.2 Collecting mujoco_py==0.5.7 690.3 Downloading https://files.pythonhosted.org/packages/a7/86/a5a6469d553545a2742b2434931f45679e94871da1a75c803ba75957ea0f/mujoco-py-0.5.7.tar.gz 690.5 Collecting pyprind 690.6 Downloading https://files.pythonhosted.org/packages/ab/b3/1f12ebc5009c65b607509393ad98240728b4401bc3593868fb161fdd3760/PyPrind-2.11.3-py2.py3-none-any.whl 690.7 Collecting ray 691.1 Downloading https://files.pythonhosted.org/packages/68/14/2d65d5468593ae5a6e210507777042367cba9a8ba5e6d5dcab5313653423/ray-0.8.6-cp35-cp35m-manylinux1_x86_64.whl (21.9MB) 693.5 Collecting tensorflow 693.8 Downloading https://files.pythonhosted.org/packages/7c/fb/7b2c5b3e85ad335b53ca67deb2ef4af574dc0a8759f43b7f45e15005e449/tensorflow-1.14.0-cp35-cp35m-manylinux1_x86_64.whl (109.2MB) 705.7 Requirement already satisfied: numpy>=1.10.4 in /opt/conda/envs/sql/lib/python3.5/site-packages (from gym==0.8) (1.13.1) 705.8 Collecting requests>=2.0 (from gym==0.8) 705.9 Downloading https://files.pythonhosted.org/packages/29/c1/24814557f1d22c56d50280771a17307e6bf87b70727d975fd6b2ce6b014a/requests-2.25.1-py2.py3-none-any.whl (61kB) 706.0 Requirement already satisfied: six in /opt/conda/envs/sql/lib/python3.5/site-packages (from gym==0.8) (1.16.0) 706.0 Collecting pyglet>=1.2.0 (from gym==0.8) 706.1 Downloading https://files.pythonhosted.org/packages/e9/33/cbff7525a357c950e76717ea9741127a662a7ed49a92874897b8a4036db9/pyglet-2.0.10-py3-none-any.whl (858kB) 706.2 Requirement already satisfied: PyOpenGL>=3.1.0 in /opt/conda/envs/sql/lib/python3.5/site-packages (from mujoco_py==0.5.7) (3.1.1a1) 706.2 Collecting grpcio (from ray) 707.7 Downloading https://files.pythonhosted.org/packages/bd/81/6c704c002a992b9d6466c739e3e7687e0bb2365d8cd63d7fc8e95d502cb6/grpcio-1.41.1.tar.gz (21.2MB) 714.1 Collecting protobuf>=3.8.0 (from ray) 714.7 Downloading https://files.pythonhosted.org/packages/6c/be/4e32d02bf08b8f76bf6e59f2a531690c1e4264530404501f3489ca975d9a/protobuf-4.21.0-py2.py3-none-any.whl (164kB) 715.1 protobuf requires Python '>=3.7' but the running Python is 3.5.2 715.2 715.2 CondaValueError: pip returned an error. 715.2 matplotlib-2.0 100% |###############################| Time: 0:00:00 11.59 MB/s path.py-12.0.2 100% |###############################| Time: 0:00:00 10.75 MB/s0 B/s ------ failed to solve: process "/bin/bash -c conda env create -f /root/sql/environment.yml && conda env update" did not complete successfully: exit code: 1
Note that ray 0.8.6 has requirement numpy>=1.16
Command and error message:
(sql) root@a4a7e4e7014f:~/sql# python ./examples/mujoco_all_sql.py --env=swimmer --log_dir="/root/sql/data/swimmer-experiment"
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
ImportError: numpy.core.multiarray failed to import
ImportError: numpy.core.umath failed to import
ImportError: numpy.core.umath failed to import
2023-11-28 16:17:19.714721: F tensorflow/python/lib/core/bfloat16.cc:675] Check failed: PyBfloat16_Type.tp_base != nullptr
Aborted (core dumped)
Error message:
using seed 3
Traceback (most recent call last):
File "/root/rllab/scripts/run_experiment_lite.py", line 137, in <module>
run_experiment(sys.argv)
File "/root/rllab/scripts/run_experiment_lite.py", line 74, in run_experiment
from rllab.sampler import parallel_sampler
File "/root/rllab/rllab/sampler/parallel_sampler.py", line 2, in <module>
from rllab.sampler.stateful_pool import singleton_pool, SharedGlobal
File "/root/rllab/rllab/sampler/stateful_pool.py", line 3, in <module>
from joblib.pool import MemmapingPool
ImportError: cannot import name 'MemmapingPool'
Reference: rll/rllab#240
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR resolves issues about running the provided code in Docker with rllab.
The migration from rllab to garage appears to overlook Docker usages, thus it may be harder to fix.
I have fixed some of the docker issues regarding the migration to garage in another branch (the
fix-garagebranch). If I somehow managed to fix all issues, I'll open another PR for it.I'll leave this as a draft for now, making it a potential reference for other docker users.