Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

T195077/docker compose for tests #253

Merged

Conversation

ureesoriano
Copy link
Contributor

@ureesoriano ureesoriano commented May 20, 2018

Add ORES docker-compose file for testing purposes

This starts up 3 services (containers):

  • ores-api
  • ores-worker
  • redis

The tests seem to pass: make test
ores-api runs: ./utility applications.wsgi --config ./config/celery-test/
ores-worker runs: ./utility applications.celery --config ./config/celery-test/
And visiting the url http://localhost:8080/v2/scores/testwiki/revid/641962088?features=true does return results.

Considerations:

  • For simplicity, both the task queue and the score cache are pointed at the same redis. It can be split into 2 when needed.
  • I was skeptical about adding changes to test config files without breaking anything, so I added a new set of config files 'celery-test-docker'. It adds a 'docker_celery' to scoring_systems, a 'docker_redis' to score_caches and sets ores to use 'docker_celery' as its scoring system.
  • Celery seems to complain very hard when the service is run as root (which makes sense). I set the env var C_FORCE_ROOT to true to get it to start. Probably the best would be adding the user to application is supposed to be run by to the docker image.

This - plus all the stuff that I might have missed - should suffice to setup a dockerized dev environment.
Additionally, it should make the transition to kubernetes easier :) (transitioning from docker-compose to kube is easy)

Services:
* ores-api
* ores-worker
* redis

Considerations:
* For simplicity, both the task queue and the score cache are pointed at
the same redis. It can be split into 2 when needed.
* I was skeptical about adding changes to test config files without
breaking anything, so I added a new set of config files
'celery-test-docker'. It adds a 'docker_celery' to scoring_systems, a
'docker_redis' to score_caches and sets ores to use 'docker_celery' as
its scoring system.
* Celery seems to complain very hard when the service is run as root
(which makes sense). I set the env var C_FORCE_ROOT to true to get it to
start. Probably the best would be adding the user to application is
supposed to be run by to the docker image.
@@ -0,0 +1,29 @@
version: '2.1'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docker-compose in stretch is version 1.8, and seems to only support up to version 2.0 of the config file. Downgrading this version string fixes the problem for me, so I'll add a follow-up patch.

@adamwight
Copy link
Contributor

Bravo! Verified working on my box, I'll merge since this definitely adds value.

I'm learning a lot, following all your progress :-)

The Travis CI docs don't make it clear how to run tests in a docker-compose environment, but I guess that's the next step, if you find more time for this project. Thanks for all you've done here!

@adamwight adamwight merged commit acd0805 into wikimedia:master May 20, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants