Skip to content
This repository has been archived by the owner on Feb 22, 2023. It is now read-only.

Update packages and add updated Pipfile.lock files #72

Merged
merged 2 commits into from
May 6, 2021
Merged

Update packages and add updated Pipfile.lock files #72

merged 2 commits into from
May 6, 2021

Conversation

obulat
Copy link
Contributor

@obulat obulat commented May 6, 2021

The Python 3.9 update has updated the Dockerfiles, but didn't update the Pipfile.lock files. With the old lock files, Docker builds fail with the following error:

ERROR [8/8] RUN pipenv install --deploy --system --dev                 0.4s
------
 > [8/8] RUN pipenv install --deploy --system --dev:
#13 0.386 Your Pipfile.lock (fc5562) is out of date. Expected: (8b8260).

In the Dockerfile, we install the python packages as root using
RUN pipenv install --deploy --system --dev
This uses the existing Pipfile.lock, and doesn't update it.

To update the Pipfile.lock I added RUN pipenv install --dev to the Dockerfile. It installed dependencies into a virtual environment, and updated the Pipfile.lock accordingly. Then I copied the updated Pipfile.lock from docker to my computer using docker cp openverse-api_ingestion-server_1:/ingestion_server:Pipfile.lock ~/<path_to_repo>/ingestion_server/Pipfile.lock command. This process worked for ingestion server and analytics.

In the cccatalog-api, the copied Pipfile.lock file was the old one. So, I had to run pipenv install inside the docker container shell (I used the Docker Desktop's 'CLI' button for the container), and then copy the updated Pipfile.lock using the docker cp command described in the previous paragraph.

While updating the Pipfiles, pipenv also updated other dependencies listed in #71, so I am closing it in favor of this PR.
Several issues created from dependabot PRs when we transferred the repository are fixed:
Fixes #29 jinja2 2.11.2 -> 2.11.3
Fixes #33 pycodestyle 2.6.0 ->2.7.0
Fixes #34 deepdiff 5.0.2 [->5.2.1] -> 5.5.0
Fixes #39 pipdeptree 1.0.0 -> 2.0.0
Fixes #41 django-storages 1.10.1 -> 1.11.1
Fixes #43 pyjwt 1.7.1 [-> 2.0.1] -> 2.1.0
Fixes #44 djangorestframework 3.11.1 -> 3.12.4
Fixes #48 hvac 0.10.5 [-> 0.10.8] -> 0.10.10
Fixes #49 boto3 1.15.11 [-> 1.17.39] -> 1.17.67
Fixes #50 gevent 20.9.0 [-> 21.1.0] -> 21.1.2
Fixes #51 py 1.9.0 -> 1.10.0
Fixes #52 ipython 7.18.1 [-> 7.21.0] -> 7.23.1
Fixes #54 pygments 2.7.1 [-> 2.7.4] -> 2.9.0

The dependency graph after these changes
# pipenv graph
aws-requests-auth==0.4.3
  - requests [required: >=0.14.0, installed: 2.25.1]
    - certifi [required: >=2017.4.17, installed: 2020.12.5]
    - chardet [required: >=3.0.2,<5, installed: 4.0.0]
    - idna [required: >=2.5,<3, installed: 2.10]
    - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.4]
boto3==1.17.67
  - botocore [required: >=1.20.67,<1.21.0, installed: 1.20.67]
    - jmespath [required: >=0.7.1,<1.0.0, installed: 0.10.0]
    - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.1]
      - six [required: >=1.5, installed: 1.16.0]
    - urllib3 [required: >=1.25.4,<1.27, installed: 1.26.4]
  - jmespath [required: >=0.7.1,<1.0.0, installed: 0.10.0]
  - s3transfer [required: >=0.4.0,<0.5.0, installed: 0.4.2]
    - botocore [required: >=1.12.36,<2.0a.0, installed: 1.20.67]
      - jmespath [required: >=0.7.1,<1.0.0, installed: 0.10.0]
      - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.1]
        - six [required: >=1.5, installed: 1.16.0]
      - urllib3 [required: >=1.25.4,<1.27, installed: 1.26.4]
deepdiff==5.5.0
  - ordered-set [required: ==4.0.2, installed: 4.0.2]
django-braces==1.14.0
  - Django [required: >=1.11.0, installed: 2.2.16]
    - pytz [required: Any, installed: 2021.1]
    - sqlparse [required: >=0.2.2, installed: 0.4.1]
  - six [required: Any, installed: 1.16.0]
django-cors-headers==3.7.0
  - Django [required: >=2.2, installed: 2.2.16]
    - pytz [required: Any, installed: 2021.1]
    - sqlparse [required: >=0.2.2, installed: 0.4.1]
django-oauth-toolkit==1.3.3
  - django [required: >=2.1, installed: 2.2.16]
    - pytz [required: Any, installed: 2021.1]
    - sqlparse [required: >=0.2.2, installed: 0.4.1]
  - oauthlib [required: >=3.1.0, installed: 3.1.0]
  - requests [required: >=2.13.0, installed: 2.25.1]
    - certifi [required: >=2017.4.17, installed: 2020.12.5]
    - chardet [required: >=3.0.2,<5, installed: 4.0.0]
    - idna [required: >=2.5,<3, installed: 2.10]
    - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.4]
django-redis==4.12.1
  - Django [required: >=2.2, installed: 2.2.16]
    - pytz [required: Any, installed: 2021.1]
    - sqlparse [required: >=0.2.2, installed: 0.4.1]
  - redis [required: >=3.0.0, installed: 3.5.3]
django-sslserver==0.22
  - Django [required: >=1.8, installed: 2.2.16]
    - pytz [required: Any, installed: 2021.1]
    - sqlparse [required: >=0.2.2, installed: 0.4.1]
django-storages==1.11.1
  - Django [required: >=2.2, installed: 2.2.16]
    - pytz [required: Any, installed: 2021.1]
    - sqlparse [required: >=0.2.2, installed: 0.4.1]
django-uuslug==1.2.0
  - python-slugify [required: >=1.2.0, installed: 5.0.2]
    - text-unidecode [required: >=1.3, installed: 1.3]
djangorestframework-xml==2.0.0
  - defusedxml [required: >=0.6.0, installed: 0.7.1]
drf-yasg==1.20.0
  - coreapi [required: >=2.3.3, installed: 2.3.3]
    - coreschema [required: Any, installed: 0.0.4]
      - jinja2 [required: Any, installed: 2.11.3]
        - MarkupSafe [required: >=0.23, installed: 1.1.1]
    - itypes [required: Any, installed: 1.2.0]
    - requests [required: Any, installed: 2.25.1]
      - certifi [required: >=2017.4.17, installed: 2020.12.5]
      - chardet [required: >=3.0.2,<5, installed: 4.0.0]
      - idna [required: >=2.5,<3, installed: 2.10]
      - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.4]
    - uritemplate [required: Any, installed: 3.0.1]
  - coreschema [required: >=0.0.4, installed: 0.0.4]
    - jinja2 [required: Any, installed: 2.11.3]
      - MarkupSafe [required: >=0.23, installed: 1.1.1]
  - Django [required: >=2.2.16, installed: 2.2.16]
    - pytz [required: Any, installed: 2021.1]
    - sqlparse [required: >=0.2.2, installed: 0.4.1]
  - djangorestframework [required: >=3.10.3, installed: 3.12.4]
    - django [required: >=2.2, installed: 2.2.16]
      - pytz [required: Any, installed: 2021.1]
      - sqlparse [required: >=0.2.2, installed: 0.4.1]
  - inflection [required: >=0.3.1, installed: 0.5.1]
  - packaging [required: Any, installed: 20.9]
    - pyparsing [required: >=2.0.2, installed: 2.4.7]
  - ruamel.yaml [required: >=0.15.34, installed: 0.17.4]
    - ruamel.yaml.clib [required: >=0.1.2, installed: 0.2.2]
  - uritemplate [required: >=3.0.0, installed: 3.0.1]
elasticsearch-dsl==7.2.1
  - elasticsearch [required: >=7.0.0,<8.0.0, installed: 7.12.1]
    - certifi [required: Any, installed: 2020.12.5]
    - urllib3 [required: >=1.21.1,<2, installed: 1.26.4]
  - python-dateutil [required: Any, installed: 2.8.1]
    - six [required: >=1.5, installed: 1.16.0]
  - six [required: Any, installed: 1.16.0]
future==0.18.2
grequests==0.6.0
  - gevent [required: Any, installed: 21.1.2]
    - greenlet [required: >=0.4.17,<2.0, installed: 1.0.0]
    - setuptools [required: Any, installed: 56.0.0]
    - zope.event [required: Any, installed: 4.5.0]
      - setuptools [required: Any, installed: 56.0.0]
    - zope.interface [required: Any, installed: 5.4.0]
      - setuptools [required: Any, installed: 56.0.0]
  - requests [required: Any, installed: 2.25.1]
    - certifi [required: >=2017.4.17, installed: 2020.12.5]
    - chardet [required: >=3.0.2,<5, installed: 4.0.0]
    - idna [required: >=2.5,<3, installed: 2.10]
    - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.4]
hvac==0.10.10
  - requests [required: >=2.21.0, installed: 2.25.1]
    - certifi [required: >=2017.4.17, installed: 2020.12.5]
    - chardet [required: >=3.0.2,<5, installed: 4.0.0]
    - idna [required: >=2.5,<3, installed: 2.10]
    - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.4]
  - six [required: >=1.5.0, installed: 1.16.0]
ipaddress==1.0.23
ipython==7.23.1
  - backcall [required: Any, installed: 0.2.0]
  - decorator [required: Any, installed: 5.0.7]
  - jedi [required: >=0.16, installed: 0.18.0]
    - parso [required: >=0.8.0,<0.9.0, installed: 0.8.2]
  - matplotlib-inline [required: Any, installed: 0.1.2]
    - traitlets [required: Any, installed: 5.0.5]
      - ipython-genutils [required: Any, installed: 0.2.0]
  - pexpect [required: >4.3, installed: 4.8.0]
    - ptyprocess [required: >=0.5, installed: 0.7.0]
  - pickleshare [required: Any, installed: 0.7.5]
  - prompt-toolkit [required: >=2.0.0,<3.1.0,!=3.0.1,!=3.0.0, installed: 3.0.18]
    - wcwidth [required: Any, installed: 0.2.5]
  - pygments [required: Any, installed: 2.9.0]
  - setuptools [required: >=18.5, installed: 56.0.0]
  - traitlets [required: >=4.2, installed: 5.0.5]
    - ipython-genutils [required: Any, installed: 0.2.0]
piexif==1.1.3
Pillow==8.2.0
pipdeptree==2.0.0
  - pip [required: >=6.0.0, installed: 21.1.1]
psycopg2-binary==2.8.6
pycodestyle==2.7.0
PyJWT==2.1.0
pytest-django==4.2.0
  - pytest [required: >=5.4.0, installed: 6.2.4]
    - attrs [required: >=19.2.0, installed: 20.3.0]
    - iniconfig [required: Any, installed: 1.1.1]
    - packaging [required: Any, installed: 20.9]
      - pyparsing [required: >=2.0.2, installed: 2.4.7]
    - pluggy [required: >=0.12,<1.0.0a1, installed: 0.13.1]
    - py [required: >=1.8.2, installed: 1.10.0]
    - toml [required: Any, installed: 0.10.2]
python-xmp-toolkit==2.0.1
  - pytz [required: Any, installed: 2021.1]
python3-openid==3.2.0
  - defusedxml [required: Any, installed: 0.7.1]
redlock-py==1.0.8
  - redis [required: Any, installed: 3.5.3]
remote-pdb==2.1.0
requests-oauthlib==1.3.0
  - oauthlib [required: >=3.0.0, installed: 3.1.0]
  - requests [required: >=2.0.0, installed: 2.25.1]
    - certifi [required: >=2017.4.17, installed: 2020.12.5]
    - chardet [required: >=3.0.2,<5, installed: 4.0.0]
    - idna [required: >=2.5,<3, installed: 2.10]
    - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.4]
wsgi-basic-auth==1.1.0
  - webob [required: >=1.0.0, installed: 1.8.7]

@dhruvkb dhruvkb added this to Needs review in Openverse May 6, 2021
@obulat obulat mentioned this pull request May 6, 2021
@obulat obulat changed the title Add updated Pipfile.lock files to fix builds Update packages and add updated Pipfile.lock files May 6, 2021
Openverse automation moved this from Needs review to Reviewer approved May 6, 2021
@zackkrida
Copy link
Member

@obulat go ahead and merge; thanks!

@obulat obulat merged commit af3a281 into WordPress:master May 6, 2021
Openverse automation moved this from Reviewer approved to Done! May 6, 2021
@zackkrida zackkrida deleted the fix_3.9_pipfiles branch May 6, 2021 14:51
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.