Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CR][ENG-481] Update WB base image for Oathpit python36 #387

Conversation

felliott
Copy link
Member

@felliott felliott commented Nov 7, 2019

Ticket

https://openscience.atlassian.net/browse/ENG-481

Purpose

Since we're doing comprehensive regression testing for OATHPIT, take advantage to apply some widespread changes to the base image and codebase.

Changes

  • Upgrade base Dockerfile image to use python3.6 and be based on buster
  • Bump tornado dep to v6 and add newrelic support.
  • Start pulling gosu from apt-get
  • Remove some code constructs that will be invalid on python3.7
  • Bump test suite dependencies to latest and fix incompatibilities
  • Replace raven-python with sentry-sdk
  • Mark some docstrings as raw to support sphinx formatting
  • Other minor misc changes.

Side effects

There shouldn't be, but it's possible.

QA Notes

This will be tested as part of the other OATHPIT regression testing.

Deployment Notes

Since this affects the Dockerfile, we may need to pay special attention to deploy step. Also needs a patch to osf.io's docker-compose.yml to work with local devel.

@coveralls
Copy link

coveralls commented Nov 7, 2019

Coverage Status

Coverage decreased (-0.02%) to 61.503% when pulling dbf1d09 on felliott:feature/oathpit-python36 into 3706373 on CenterForOpenScience:feature/oathpit.

Copy link
Contributor

@cslzchen cslzchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass done. A great PR! 🎆 Please refer to the comments for my notes and questions.

For now, I am still working on building the new image for testing locally.

Spent about two hours testing lots of move / copy / upload / delete / rename with both files and folders using 4 providers (S3, owncloud, dropbox, osfstorage), everything works as expected. New Relic is configured and looks great.

new-relic

  • Here are three expected error messages during requirements installation. The aiohttpretty one will be fixed separately. Please double check the flake8 and pyflakes version.
ERROR: flake8 3.0.4 has requirement pyflakes!=1.2.0,!=1.2.1,!=1.2.2,<1.3.0,>=0.8.1, but you'll have pyflakes 2.1.1 which is incompatible.
ERROR: aiohttpretty 19.0.0 has requirement aiohttp==3.5.4, but you'll have aiohttp 3.6.2 which is incompatible.
ERROR: aiohttpretty 19.0.0 has requirement furl==2.0.0, but you'll have furl 0.4.2 which is incompatible.
  • In addition, here is one unexpected permission warning with the new image in the WB Worker container. I may not have paid attention before: this warning shows up for normal OATHPIT and develop WB as well.
wb_worker_1         | /usr/local/lib/python3.6/site-packages/celery/platforms.py:766: RuntimeWarning: You are running the worker with superuser privileges, which is
wb_worker_1         | absolutely not recommended!

waterbutler/core/exceptions.py Outdated Show resolved Hide resolved
requirements.txt Outdated Show resolved Hide resolved
requirements.txt Outdated Show resolved Hide resolved
waterbutler/tasks/app.py Outdated Show resolved Hide resolved
waterbutler/tasks/app.py Outdated Show resolved Hide resolved
tasks.py Outdated Show resolved Hide resolved
tests/server/api/v1/test_create_mixin.py Outdated Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
tests/tasks/conftest.py Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants