Improvements to docker

[StevePotter]

Hello friends, thank you for building such a wonderful project. I noticed a few non-standard uses of Docker and this PR provides some suggested improvements. I don't recommend merging it until there has been discussion and testing by the team.

The current dockerfile does the work of preparing the container for developing the code. None of the commands in the dockerfile touch any of the FoundationPose code. Then, run_container.sh mounts a bunch of volumes, which in turn gives the container access to the code. Since some other things still need to happen, build_all.sh needs to be run.

Normally, a docker image is rather complete and all someone needs to run the code is pull the image. That's not the case here. These extra steps require more work to set up, and make it very difficult to simply deploy the FP image to a server.

Luckily, docker has some other approaches that are widely accepted. For development time, when you want to share files with the container, there is Docker Compose. Docker compose makes it easy to set up networks, mount volumes, and just about everything else that run_container.sh was doing. Plus since the volumes are available to every RUN command, everything in build_all.sh can be moved to the dockerfile.

So now instead of run_container.sh, you can simply use the native docker compose up command!

For use in servers, docker provides a COPY command that can place files from the project into the container. So, I added a second dockerfile for that, dockerfile.prod. If you build that image, it'll include everything from the git repo in the container and will run all the commands from build_all.sh. Then this image can quickly run anywhere, without requiring the git repo code. You could even create a Github Action that will automatically publish this image whenever an PR is merged.

I'd love to hear people's thoughts on this. I'm not a FoundationPose developer and I realize you have a workflow, but I've been using docker for 10 years, and I thought I could share some of my experience. If you switch to Docker Compose, things will become easier and experiences will be more consistent across developers. Having a 2nd dockerfile may seem weird, but actually a lot of people do it.

If you think this is too much to do all at once, I can keep the old dockerfile, run_container.sh, build_all.sh and present this new way as an alternative. If and once it's determined Compose is easier, then you can remove those files later.

Also, it could be possible to make a much lightweight container, which is safer and easier to deploy. There are quite a bit of build tools (g++, gcc, build-essential, cmake, etc) that could be removed from the final container if we took advantage of Docker's Multi-Stage Builds. This allows you to have multiple FROM statements in your dockerfile. You can use some of them to build code, then you simply copy the built cover over to the final container. I could certainly help with this if you're interested.

Thanks, I hope you like this. Have a great day!

[wenbowen123]

Hi @StevePotter thanks for your great suggestion, this seems very useful! I still need to find a time to test this myself as I'm swamped with other projects now. To be backward compatible, would you mind creating a separate docker/ folder (e.g. docker_compose) and put the new stuff there? Right now I'd prefer to keep the old one as it is, but later if many folks have verified this, I'd be happy to replace that.

[StevePotter]

Okay great, I will do that

[StevePotter]

I got a little sidetracked, but will devote some time to this next week. I also plan the following improvements:

• Use requirements.txt or conda environment.yml to declare packages
• Include weights in the docker image
• Use multi-stage build so the runtime image uses a base image like nvidia/cudagl:11.3.0-runtime-ubuntu20.04. The current image is about 20gb and when I tried it out, it cut it down to about 10gb
• Supply an argument to toggle cuda version. I tested on 11.8 and 12.1, and those work. Would be nice for users to have a choice

[EquilibriaW]

Hi! I was trying this method, and I ran into an issue where the line cd /foundationpose/mycpp/ failed because it couldn't find the directory. I was wondering if this could because I placed the docker-compose file in a different location than intended; I currently place it as a subdirectory inside of main.