Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker BugFix #3346

Merged
merged 24 commits into from Jan 6, 2017
Merged

Docker BugFix #3346

merged 24 commits into from Jan 6, 2017

Conversation

deinok
Copy link
Member

@deinok deinok commented Dec 4, 2016

No description provided.

@deinok
Copy link
Member Author

deinok commented Dec 4, 2016

Requires #3346

@deinok
Copy link
Member Author

deinok commented Dec 4, 2016

Closes #3319

@deinok
Copy link
Member Author

deinok commented Dec 4, 2016

Closes #3075

@deinok
Copy link
Member Author

deinok commented Dec 4, 2016

Closes #3347

@deinok deinok changed the title [WIP] - Docker BugFix Docker BugFix Dec 5, 2016
@wardi wardi self-assigned this Dec 6, 2016
@deinok
Copy link
Member Author

deinok commented Dec 11, 2016

@wardi any update on this

- docker run -d --name ckan -p 5001:5000 --link db:db --link redis:redis --link solr:solr ckan
- sleep 10
- curl --retry 10 --retry-delay 5 -v http://localhost:5001

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You've added about 2 minutes to the all CI containers. Would you add your docker build and test code to https://github.com/ckan/ckan/blob/master/.circleci-matrix.yml so that it only runs in one container instead of all 4?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The circlce-matrix docs are more than a little lacking, let me know if you'd like more direction here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, i will try to make it ;)
But im not sure if i can cache the images in the matrix :(

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe @amercader's suggestion of running the docker tests under travis is the best, then

@amercader
Copy link
Member

I think we definitely should test the Docker images, building and basic functional tests (as in "site is up and running"), but probably separating it from the normal tests. We could use one of the Circle CI containers for that or use Travis for Docker testing exclusively and have two checks.

@@ -13,7 +13,7 @@ RUN mkdir -p /opt/solr/server/solr/$SOLR_CORE/data

# Adding Files
ADD ./solrconfig.xml \
https://raw.githubusercontent.com/ckan/ckan/dev-v2.6/ckan/config/solr/schema.xml \
https://raw.githubusercontent.com/ckan/ckan/master/ckan/config/solr/schema.xml \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this will be need to be changed to the relevant tag as part of the release process? Also more generically I guess the release process will need to include a Docker step to build and tag the images and upload them to Docker Hub.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amercader For now i don't know how to make the diference between the schemas.
The best aproach is to set a managed schema. I will work on it ;)

@amercader
Copy link
Member

@deinok great work, it seems like we are almost there. This will be a really good base for people to work on Docker based setups

@deinok
Copy link
Member Author

deinok commented Dec 13, 2016

@amercader For now i think we should use Travis for docker, or wait until CircleCI 2.0 is released.

For now, i will try to set the Docker test in a container.
(For now, docker only tests if it can be build and executed, and the frontpage returns a HTTP-200)
(In a future, we should be able to run the tests inside the docker)

"ckan.storage_path = /var/lib/ckan" \
"email_to = disabled@example.com" \
"error_email_from = ckan@$(hostname -f)" \
"ckan.site_url = http://localhost:5000"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be easier to use the environment variables for these config options instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, i miss the ckan.storage_path. Thanks @wardi

Should I add also the other params as environment variables?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please just add the ones you need, making arbitrary config options override-able from environment vars makes me nervous.

@deinok
Copy link
Member Author

deinok commented Dec 20, 2016

@wardi or @amercader Can anybody help me with the travis.yml?

@deinok deinok mentioned this pull request Dec 21, 2016
@deinok
Copy link
Member Author

deinok commented Dec 21, 2016

Closes #3375

@wardi
Copy link
Contributor

wardi commented Dec 22, 2016

If you remove the changes to the circle tests I would merge this, then I can re-enable travis on this repo

"solr_url = ${SOLR_URL}" \
"ckan.redis.url = ${REDIS_URL}" \
"ckan.storage_path = ${CKAN_DATA}" \
"ckan.site_url = ${SITE_URL}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to call config-tool at all? We've already got environment variables for these settings.

  • CKAN_SQLALCHEMY_URL will override sqlalchemy.url
  • CKAN_SOLR_URL will override solr_url
  • CKAN_REDIS_URL will override ckan.redis.url
  • CKAN_STORAGE_PATH will override ckan.storage_path
  • CKAN_SITE_URL will override ckan.site_url

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain this a little bit more?
Where are this environment variables?
How are they used?
Where are generated?
Etc

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are environment variables recognised by CKAN and used to override INI settings. See https://github.com/ckan/ckan/blob/master/ckan/config/environment.py#L117

If you use these environment variable names you don't need to generate an ini file with paster config-tool at all.

@deinok
Copy link
Member Author

deinok commented Dec 22, 2016

Done

@deinok
Copy link
Member Author

deinok commented Jan 4, 2017

Any issue?

@wardi
Copy link
Contributor

wardi commented Jan 4, 2017

@deinok just that small improvement to skip generating an ini file, then I'll merge

@deinok
Copy link
Member Author

deinok commented Jan 6, 2017

@wardi Done, take a look ;)

@wardi wardi merged commit 1d0cee3 into ckan:master Jan 6, 2017
@wardi
Copy link
Contributor

wardi commented Jan 6, 2017

@deinok great work. Thanks!

@deinok
Copy link
Member Author

deinok commented Jan 6, 2017 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants