-
Notifications
You must be signed in to change notification settings - Fork 49
feat: updated docker setup for manipulation demo #641
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
34e22c8
6aa277a
b77b5dd
768a7b6
7d0d2ae
a6c3277
b8aec43
6e4b18c
702d195
4e72628
e2f6f9f
14d5f4b
7c32345
208d4be
eb57419
a47ebc0
7b88d37
07814d9
8afa035
302aad2
329020c
d6c81ff
7a4d5bd
5fef90e
2e54d09
c7f703d
4c63430
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,182 @@ | ||
# Byte-compiled / optimized / DLL files | ||
__pycache__/ | ||
*.py[cod] | ||
*$py.class | ||
|
||
# ROS | ||
build/ | ||
devel/ | ||
install/ | ||
log/ | ||
|
||
# C extensions | ||
*.so | ||
|
||
# Distribution / packaging | ||
.Python | ||
build/ | ||
develop-eggs/ dist/ | ||
downloads/ | ||
eggs/ | ||
.eggs/ | ||
lib/ | ||
lib64/ | ||
parts/ | ||
sdist/ | ||
var/ | ||
wheels/ | ||
share/python-wheels/ | ||
*.egg-info/ | ||
.installed.cfg | ||
*.egg | ||
MANIFEST | ||
dist/ | ||
|
||
# PyInstaller | ||
# Usually these files are written by a python script from a template | ||
# before PyInstaller builds the exe, so as to inject date/other infos into it. | ||
*.manifest | ||
*.spec | ||
|
||
# Installer logs | ||
pip-log.txt | ||
pip-delete-this-directory.txt | ||
|
||
# Unit test / coverage reports | ||
htmlcov/ | ||
.tox/ | ||
.nox/ | ||
.coverage | ||
.coverage.* | ||
.cache | ||
nosetests.xml | ||
coverage.xml | ||
*.cover | ||
*.py,cover | ||
.hypothesis/ | ||
.pytest_cache/ | ||
cover/ | ||
|
||
# Translations | ||
*.mo | ||
*.pot | ||
|
||
# Django stuff: | ||
*.log | ||
local_settings.py | ||
db.sqlite3 | ||
db.sqlite3-journal | ||
|
||
# Flask stuff: | ||
instance/ | ||
.webassets-cache | ||
|
||
# Scrapy stuff: | ||
.scrapy | ||
|
||
# Sphinx documentation | ||
docs/_build/ | ||
|
||
# PyBuilder | ||
.pybuilder/ | ||
target/ | ||
|
||
# Jupyter Notebook | ||
.ipynb_checkpoints | ||
|
||
# IPython | ||
profile_default/ | ||
ipython_config.py | ||
|
||
# pyenv | ||
# For a library or package, you might want to ignore these files since the code is | ||
# intended to run in multiple environments; otherwise, check them in: | ||
# .python-version | ||
|
||
# pipenv | ||
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. | ||
# However, in case of collaboration, if having platform-specific dependencies or dependencies | ||
# having no cross-platform support, pipenv may install dependencies that don't work, or not | ||
# install all needed dependencies. | ||
#Pipfile.lock | ||
|
||
# poetry | ||
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control. | ||
# This is especially recommended for binary packages to ensure reproducibility, and is more | ||
# commonly ignored for libraries. | ||
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control | ||
#poetry.lock | ||
|
||
# pdm | ||
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control. | ||
#pdm.lock | ||
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it | ||
# in version control. | ||
# https://pdm.fming.dev/#use-with-ide | ||
.pdm.toml | ||
|
||
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm | ||
__pypackages__/ | ||
|
||
# Celery stuff | ||
celerybeat-schedule | ||
celerybeat.pid | ||
|
||
# SageMath parsed files | ||
*.sage.py | ||
|
||
# Environments | ||
.env | ||
.venv | ||
env/ | ||
venv/ | ||
ENV/ | ||
env.bak/ | ||
venv.bak/ | ||
|
||
# Spyder project settings | ||
.spyderproject | ||
.spyproject | ||
|
||
# Rope project settings | ||
.ropeproject | ||
|
||
# mkdocs documentation | ||
/site | ||
|
||
# mypy | ||
.mypy_cache/ | ||
.dmypy.json | ||
dmypy.json | ||
|
||
# Pyre type checker | ||
.pyre/ | ||
|
||
# pytype static type analyzer | ||
.pytype/ | ||
|
||
# Cython debug symbols | ||
cython_debug/ | ||
|
||
# PyCharm | ||
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can | ||
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore | ||
# and can be added to the global gitignore or merged into this file. For a more nuclear | ||
# option (not recommended) you can uncomment the following to ignore the entire idea folder. | ||
#.idea/ | ||
|
||
logs/ | ||
.vscode/ | ||
|
||
!examples/imgs/*.md | ||
|
||
src/examples/*-demo | ||
artifact_database.pkl | ||
|
||
imgui.ini | ||
|
||
src/rai_bench/rai_bench/experiments | ||
src/rai_interfaces | ||
|
||
# Demo assets | ||
demo_assets/ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
<?xml version="1.0" encoding="UTF-8" ?> | ||
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles" > | ||
<transport_descriptors> | ||
<transport_descriptor> | ||
<transport_id>CustomUdpTransport</transport_id> | ||
<type>UDPv4</type> | ||
</transport_descriptor> | ||
</transport_descriptors> | ||
<participant profile_name="participant_profile" is_default_profile="true"> | ||
<rtps> | ||
<userTransports> | ||
<transport_id>CustomUdpTransport</transport_id> | ||
</userTransports> | ||
<useBuiltinTransports>false</useBuiltinTransports> | ||
</rtps> | ||
</participant> | ||
</profiles> |
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
|
@@ -18,6 +18,10 @@ manipulation techniques. | |||||||
|
||||||||
Make sure ROS 2 is sourced. (e.g. `source /opt/ros/humble/setup.bash`) | ||||||||
|
||||||||
### Local Setup | ||||||||
|
||||||||
#### Setting up the demo | ||||||||
|
||||||||
1. Follow the RAI setup instructions in the [quick setup guide](../setup/install.md#setting-up-developer-environment). | ||||||||
2. Download additional dependencies: | ||||||||
|
||||||||
|
@@ -39,7 +43,7 @@ manipulation techniques. | |||||||
colcon build --symlink-install | ||||||||
``` | ||||||||
|
||||||||
## Running the Demo | ||||||||
#### Running the demo | ||||||||
|
||||||||
!!! note "Remain in sourced shell" | ||||||||
|
||||||||
|
@@ -75,6 +79,110 @@ manipulation techniques. | |||||||
|
||||||||
To change camera in the simulation use 1-7 keys on your keyboard once it's window is focused. | ||||||||
|
||||||||
### Docker Setup | ||||||||
|
||||||||
!!! note "ROS 2 required" | ||||||||
|
||||||||
The docker setup requires a working Humble or Jazzy ROS 2 installation on the host machine. Make sure that ROS 2 is sourced on the host machine and the `ROS_DOMAIN_ID` environment variable is set to the same value as in the [Docker setup](../setup/setup_docker.md#2-set-up-communications-between-docker-and-host-optional) | ||||||||
|
||||||||
!!! warning "ROS 2 distributions" | ||||||||
|
||||||||
It is highly recommended that ROS 2 distribution on the host machine matches the ROS 2 distribution of the docker container. A distribution version mismatch may result in the demo not working correctly. | ||||||||
|
||||||||
#### 1. Setting up the demo | ||||||||
|
||||||||
1. Set up docker as outlined in the [docker setup guide](../setup/setup_docker.md). During the setup, build the docker image with all dependencies (i.e., use the `--build-arg DEPENDENCIES=all_groups` argument) | ||||||||
and configure communication between the container and the host ([link](../setup/setup_docker.md#2-set-up-communications-between-docker-and-host-optional)). | ||||||||
|
||||||||
2. On the host machine, download the latest binary release for the Robotic Arm Demo: | ||||||||
|
||||||||
```shell | ||||||||
./scripts/download_demo.sh manipulation | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. downloaded binary is specific to ros2 version. Is it possible to run this example in such setup: I think communication-wise it might not work, so I would resolve it by mentioningin the docs that we advise against such setup with 2 different ros2 versions. Then download script will work fine There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Addressed in 6ff5bd1 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the warning should also be added to docker docs There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added in ad28165 |
||||||||
``` | ||||||||
|
||||||||
3. Run the docker container (if not already running): | ||||||||
|
||||||||
```shell | ||||||||
docker run --net=host --ipc=host --pid=host -e ROS_DOMAIN_ID=$ROS_DOMAIN_ID -it rai:jazzy # or rai:humble | ||||||||
``` | ||||||||
|
||||||||
!!! tip "NVIDIA GPU acceleration" | ||||||||
|
||||||||
If the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) is set up on your host machine, you can use the GPU within the RAI docker container for faster inference by adding the `--gpus all` option: | ||||||||
|
||||||||
```shell | ||||||||
docker run --net=host --ipc=host --pid=host -e ROS_DOMAIN_ID=$ROS_DOMAIN_ID --gpus all -it rai:jazzy # or rai:humble | ||||||||
``` | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It may be worth mentioning that if this error is reported:
the docker may need to be restarted.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added in 2e54d09 |
||||||||
|
||||||||
Sometimes, passing GPUs to the docker container may result in an error: | ||||||||
|
||||||||
```shell | ||||||||
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]. | ||||||||
``` | ||||||||
|
||||||||
Restarting the docker service should resolve this error: | ||||||||
|
||||||||
```shell | ||||||||
sudo systemctl restart docker | ||||||||
``` | ||||||||
|
||||||||
4. (Inside the container shell) Download additional ROS 2 dependencies: | ||||||||
|
||||||||
```shell | ||||||||
vcs import < demos.repos | ||||||||
rosdep install --from-paths src/examples/rai-manipulation-demo/ros2_ws/src --ignore-src -r -y | ||||||||
``` | ||||||||
|
||||||||
5. (Inside the container shell) Build the ROS 2 workspace: | ||||||||
|
||||||||
```shell | ||||||||
source /opt/ros/${ROS_DISTRO}/setup.bash | ||||||||
colcon build --symlink-install | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added and modified to be ROS 2 distribution agnostic in c7f703d |
||||||||
``` | ||||||||
|
||||||||
6. (Inside the docker container) By default, RAI uses OpenAI as the vendor. Thus, it is necessary | ||||||||
to set the `$OPENAI_API_KEY` environmental variable. The command below may be utilized to set | ||||||||
the variable and add it to the container's `.bashrc` file: | ||||||||
|
||||||||
```shell | ||||||||
export OPENAI_API_KEY=YOUR_OPEN_AI_API_KEY | ||||||||
echo "export OPENAI_API_KEY=$OPENAI_API_KEY" >> ~/.bashrc | ||||||||
``` | ||||||||
|
||||||||
!!! note AI vendor change | ||||||||
|
||||||||
The default vendor can be changed to a different provider via the [RAI configuration tool](../setup/install.md#15-configure-rai) | ||||||||
|
||||||||
#### 2. Running the demo | ||||||||
|
||||||||
!!! note Source the setup shell | ||||||||
|
||||||||
Ensure ROS 2 is sourced on the host machine and the `ROS_DOMAIN_ID` environment variable is set to the same value as in the [Docker setup](../setup/setup_docker.md#2-set-up-communications-between-docker-and-host-optional). Ensure that every command inside the docker container is run in a sourced shell using `source setup_shell.sh`. | ||||||||
|
||||||||
1. Launch the Robotic Arm Visualization on the host machine: | ||||||||
|
||||||||
```shell | ||||||||
./demo_assets/manipulation/RAIManipulationDemo/RAIManipulationDemo.GameLauncher | ||||||||
``` | ||||||||
|
||||||||
2. (Inside the container shell) Launch the Robotic Arm Demo script inside of the docker container: | ||||||||
boczekbartek marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
|
||||||||
```shell | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A step needs to be added to adjust
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Clarified that the vendor may be changed and added a step to set the OpneAI API key in 4c63430 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Regarding the error in the script, I had trouble reproducing it. However, I've noticed that similar exceptions can be thrown if one does not build the ROS 2 workspace with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You're right. I probably had missed a step. Re-ran the demo again and I couldn't repro the issue so we are good here. |
||||||||
ros2 launch examples/manipulation-demo.launch.py | ||||||||
``` | ||||||||
|
||||||||
3. (Inside the container shell) Open a new terminal for the docker container (e.g., `docker exec -it CONTAINER_ID /bin/bash`) and launch the streamlit interface: | ||||||||
|
||||||||
```shell | ||||||||
streamlit run examples/manipulation-demo-streamlit.py | ||||||||
``` | ||||||||
|
||||||||
Alternatively, run the simpler command-line version: | ||||||||
|
||||||||
```shell | ||||||||
python examples/manipulation-demo.py | ||||||||
``` | ||||||||
|
||||||||
## How it works | ||||||||
|
||||||||
The manipulation demo utilizes several components: | ||||||||
|
Uh oh!
There was an error while loading. Please reload this page.