Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docker: try on fixing docker issues #286

Merged
merged 1 commit into from
Oct 13, 2020
Merged

docker: try on fixing docker issues #286

merged 1 commit into from
Oct 13, 2020

Conversation

pazembrz
Copy link
Contributor

@pazembrz pazembrz commented Oct 5, 2020

No description provided.

@pazembrz pazembrz force-pushed the fix_docker branch 2 times, most recently from aecbf44 to 3f68540 Compare October 7, 2020 11:35
michamos
michamos approved these changes Oct 12, 2020
View reviewed changes
Copy link
Contributor

@michamos michamos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard to review, but besides minor comments looks ok given that the tests are passing :)

tests/Dockerfile.hepcrawl_base
WORKDIR /code
RUN pip install --upgrade wheel setuptools idutils rfc3987 bleach jsonschema inspire-utils
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need to install some of the deps explicitly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because somehow they are not installed. I'm not sure why actually, I can investigate it more if you prefer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I look into it a little bit more and looks like inspire-schemas are missing this packages in setup_requires directive. Without installing them manually build for inspire-schemas is failing:

  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-zHeIhV/inspire-schemas/setup.py'"'"'; __file__='"'"'/tmp/pip-install-zHeIhV/inspire-schemas/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-0XIc4V
       cwd: /tmp/pip-install-zHeIhV/inspire-schemas/
  Complete output (103 lines):
  /usr/local/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'build_requires'
    warnings.warn(msg)
  /home/test/.local/lib/python2.7/site-packages/autosemver/__init__.py:156: UserWarning: Using explicit parametes is deprecated and will be removed, pass a dictionary as the value of the autosemver parameter instead.
    'Using explicit parametes is deprecated and will be removed, '
  /home/test/.local/lib/python2.7/site-packages/setuptools/dist.py:476: UserWarning: Normalizing '61.3.9
  ' to '61.3.9'
    normalized_version,
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib
  creating build/lib/inspire_schemas
  copying inspire_schemas/utils.py -> build/lib/inspire_schemas
  copying inspire_schemas/errors.py -> build/lib/inspire_schemas
  copying inspire_schemas/__init__.py -> build/lib/inspire_schemas
  copying inspire_schemas/api.py -> build/lib/inspire_schemas
  creating build/lib/inspire_schemas/records
  copying inspire_schemas/records/__init__.py -> build/lib/inspire_schemas/records
  creating build/lib/inspire_schemas/countries
  copying inspire_schemas/countries/__init__.py -> build/lib/inspire_schemas/countries
  creating build/lib/inspire_schemas/builders
  copying inspire_schemas/builders/references.py -> build/lib/inspire_schemas/builders
  copying inspire_schemas/builders/seminars.py -> build/lib/inspire_schemas/builders
  copying inspire_schemas/builders/conferences.py -> build/lib/inspire_schemas/builders
  copying inspire_schemas/builders/__init__.py -> build/lib/inspire_schemas/builders
  copying inspire_schemas/builders/authors.py -> build/lib/inspire_schemas/builders
  copying inspire_schemas/builders/jobs.py -> build/lib/inspire_schemas/builders
  copying inspire_schemas/builders/signatures.py -> build/lib/inspire_schemas/builders
  copying inspire_schemas/builders/literature.py -> build/lib/inspire_schemas/builders
  copying inspire_schemas/builders/builder.py -> build/lib/inspire_schemas/builders
  creating build/lib/inspire_schemas/readers
  copying inspire_schemas/readers/conference.py -> build/lib/inspire_schemas/readers
  copying inspire_schemas/readers/__init__.py -> build/lib/inspire_schemas/readers
  copying inspire_schemas/readers/literature.py -> build/lib/inspire_schemas/readers
  creating build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/__init__.py -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/institutions.json -> build/lib/inspire_schemas/records
  copying inspire_schemas/records/jobs.json -> build/lib/inspire_schemas/records
  copying inspire_schemas/records/seminars.json -> build/lib/inspire_schemas/records
  copying inspire_schemas/records/experiments.json -> build/lib/inspire_schemas/records
  copying inspire_schemas/records/hep.json -> build/lib/inspire_schemas/records
  copying inspire_schemas/records/journals.json -> build/lib/inspire_schemas/records
  copying inspire_schemas/records/data.json -> build/lib/inspire_schemas/records
  copying inspire_schemas/records/authors.json -> build/lib/inspire_schemas/records
  copying inspire_schemas/records/conferences.json -> build/lib/inspire_schemas/records
  copying inspire_schemas/countries/iso_3166-3.json -> build/lib/inspire_schemas/countries
  copying inspire_schemas/countries/iso_3166-1.json -> build/lib/inspire_schemas/countries
  copying inspire_schemas/records/elements/address.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/reference.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/acquisition_source.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/cnum.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/source.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/contact.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/inspire_field.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/id.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/records-files.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/url.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/arxiv_categories.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/inspire_experiment_classification.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/degree_type.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/sourced_value.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/related_record.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/country_code.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/title.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/document_type.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/json_reference.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/material.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/language_code.json -> build/lib/inspire_schemas/records/elements
  copying inspire_schemas/records/elements/rank.json -> build/lib/inspire_schemas/records/elements
  warning: build_py: byte-compiling is disabled, skipping.
  
  installing to build/bdist.linux-x86_64/wheel
  running install
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-zHeIhV/inspire-schemas/setup.py", line 240, in <module>
      do_setup()
    File "/tmp/pip-install-zHeIhV/inspire-schemas/setup.py", line 235, in do_setup
      autosemver=True,
    File "/home/test/.local/lib/python2.7/site-packages/setuptools/__init__.py", line 162, in setup
      return distutils.core.setup(**attrs)
    File "/usr/local/lib/python2.7/distutils/core.py", line 151, in setup
      dist.run_commands()
    File "/usr/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
      self.run_command(cmd)
    File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
      cmd_obj.run()
    File "/home/test/.local/lib/python2.7/site-packages/wheel/bdist_wheel.py", line 326, in run
      self.run_command('install')
    File "/usr/local/lib/python2.7/distutils/cmd.py", line 326, in run_command
      self.distribution.run_command(command)
    File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-zHeIhV/inspire-schemas/setup.py", line 50, in run
      _generate_json_schemas()
    File "/tmp/pip-install-zHeIhV/inspire-schemas/setup.py", line 160, in _generate_json_schemas
      _generate_country_code(schemas_dir)
    File "/tmp/pip-install-zHeIhV/inspire-schemas/setup.py", line 139, in _generate_country_code
      from inspire_schemas.utils import COUNTRY_CODE_TO_NAME
    File "inspire_schemas/utils.py", line 34, in <module>
      import rfc3987
  ImportError: No module named rfc3987
  ----------------------------------------

It's not preventing inspire-schemas to install at the end, as this is retried after other packages are installed, but it slows down image build a little. I can remove them if you want.

docker-compose.deps.py2.yml Outdated
tty: true
environment:
- BASE_USER_UID=${BASE_USER_UID:-1000}
- BASE_USER_GIT=${BASE_USER_GIT:-1000}
command: bash -c "pip install -e .[all] && pip freeze"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got confused by this, I think you can remove it (the py3 version is simply "true")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, right. I missed it.

@pazembrz pazembrz mentioned this pull request Oct 13, 2020
Closed
@pazembrz pazembrz merged commit a2a4e1b into inspirehep:master Oct 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@michamos michamos michamos approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@pazembrz @michamos