feature: Added ability to run OCIM in a container.

[kewne]

Description

This adds some basic infrastructure for developing OCIM inside docker containers.
It already makes it possible to run some of the dev workflow (e.g. tests and OCIM) in Docker and could be used as a starting point for running a dockerized OCIM in production.

Kudos to @janimo for the prior work on #740

Testing

Follow the instructions in docker.md

[Kelketek]

@kewne Thank you for doing this. I made a quick pass over it and added some comments.

[bradenmacdonald]

Thanks for this! I haven't tested it, but I'm happy to see it :-)

Let me know if you need an additional reviewer/tester, and I can do that.

[Kelketek]

@kewne I'm getting this error when trying to run migrate:

❯ # --rm destroys the container at the end
docker-compose run --rm ocim bash
Creating network "opencraft_default" with the default driver
Creating opencraft_postgresql_1 ... done
Creating opencraft_ocim_run     ... done
root@ab8c60815b51:/usr/src/ocim# make migrate
coverage erase
find -name '*.pyc' -delete
find -name '*~' -delete
find -name '__pycache__' -type d -delete
rm -rf .coverage build
find static/external -type f -not -name 'Makefile' -not -name '.gitignore' -delete
find static/external -type d -empty -delete
honcho run python3 manage.py migrate
Traceback (most recent call last):
  File "manage.py", line 37, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "/usr/local/lib/python3.6/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python3.6/site-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/usr/local/lib/python3.6/site-packages/django/apps/config.py", line 90, in create
    module = import_module(entry)
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'djng'
make: *** [Makefile:102: migrate] Error 1
root@ab8c60815b51:/usr/src/ocim# exit

Also, looks like this needs a rebase, as master has changed.

[kewne]

@Kelketek argh, the django-angular package gets installed into the src directory under the project root, so it gets overwritten by the mount...
This is caused by package being installed as editable though, is that really necessary? Removing the -e gets it working.

[Kelketek]

@kewne Probably not. Chances are it was needed when first implementing the angular interface, but once it was no longer needed, no one bothered to remove the -e. I didn't even realize the package exists until it failed for me. So go ahead and try removing the -e.

We can look at re-adding the -e if we find a need for it and it can be on the person who needs it to make it work with Docker once we have that as a standard feature for OCIM. :)

[bradenmacdonald]

@kewne I can review this, but probably not until Tuesday.

[MartinKaburu]

I followed the docs and managed to get myself set up on an m1 mac. Thanks.

[kewne]

@Kelketek @bradenmacdonald I've fixed the ModuleNotFoundError but forgot to ping, can you take a look when you have the time?

@MartinKaburu in case you need to develop on top of this before it gets merged in, you can use this workflow:

git checkout my_work_branch
# rebase changes on top of docker
git rebase joaocabrita/ocim_in_docker
# hack, hack hack...
# when you're done:
git fetch
# rebase all commits in my_work_branch but not in joaocabrita/ocim_in_docker on top of origin/master
git rebase --onto origin/master joaocabrita/ocim_in_docker my_work_branch

[Kelketek]

@kewne This is working for me now. I notice that the management backend is the UI on 5000-- I think 3000 is where the customer-facing UI is supposed to be-- are there special directions for how to start this up in the docker environment?

[kewne]

@Kelketek The frontend should now be working as well; I added the instructions to the documentation but the workflow is a bit clunky because node_modules ends up being shared between the container and the host.

[Kelketek]

@kewne Much closer now!

Failed to compile.
ocim-frontend_1  | 
ocim-frontend_1  | ./src/console/components/DomainItem/DomainItem.tsx
ocim-frontend_1  | Module not found: Can't resolve 'ocim-client' in '/usr/src/ocim-frontend/src/console/components/DomainItem'

I've forced a rebuild of the items and removed node_modules before restarting, but I keep getting this error when beginning from scratch.

Also it would be good to make this container set rootless. https://pythonspeed.com/articles/root-capabilities-docker-security/ since I found I had to use sudo to remove all of the node_modules files (they were owned by root).

There are some other potential solutions to the awkwardness of syncing node_modules-- many available if you Google around, since lots of people have this problem, but I don't think it needs to be in this PR. What you're doing is probably sufficient there.

[kewne]

@Kelketek can you try docker-compose run --rm ocim-frontend bash scripts/build-api-client.sh before the npm install ?

Also it would be good to make this container set rootless.

I'd like to tackle that as part of (at least) one separate PR/story making containerized OCIM production ready;
my focus here was really to get something that works minimally for development or I'm afraid this'll never end 😓

[Kelketek]

@kewne Please create that follow-up ticket for the rootless container since I'm afraid it may get overlooked otherwise, and we do intend to containerize OCIM fully, so this work might end up the basis of something much larger.

There should be an OCIM docker epic somewhere you can attach the ticket to. Ping the current owner on it to make them aware of the creation.

I left one comment. Once addressed, this is 👍 from me. No need for me to fiddle with it again.

[bradenmacdonald]

👍 if you fix the frontend build issue I encountered.

• I tested this: started it up locally
• I read through the code
• I checked for accessibility issues: n/a
• Includes documentation: yes, thx

[kewne]

@Kelketek @bradenmacdonald I've created https://tasks.opencraft.com/browse/BB-4246 for the follow up work that came up during the review.

Since I've addressed all of your comments, I'll merge this once it passes all checks; thanks for your patience!

[Kelketek]

@kewne Looks like you'll need to do a commit --amend and force push to retrigger the tests. Also please check in the OCIM Mattermost channel to see if this failure is a known one that we're already planning on handling. I think this might be something we already have a ticket to fix and is unrelated to your changes, but I'm not sure.

[kewne]

@Kelketek it's failing on master as well: https://app.circleci.com/pipelines/github/open-craft/opencraft/3742/workflows/75ea8813-4744-4c7c-b198-260d5bbc0f16/jobs/33686
I'm asking on Mattermost if this is a know issue that'll be solved soon; I don't mind waiting until then and will rebase when possible.

[kewne]

@Kelketek @bradenmacdonald Seems like master is (finally) green again, so I'll be merging this in as soon as I get the flakes to pass.

[bradenmacdonald]

@kewne Oh man, I thought this was already merged. Better late than never though. Thanks!!!