Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch out conditional requirement parser #16636

Merged
merged 2 commits into from Sep 6, 2023

Conversation

mvdbeek
Copy link
Member

@mvdbeek mvdbeek commented Sep 1, 2023

Fixes

Traceback (most recent call last):
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 35, in __init__
    parsed = _parse_requirement(requirement_string)
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 64, in parse_requirement
    return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 73, in _parse_requirement
    name_token = tokenizer.expect(
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/_vendor/packaging/_tokenizer.py", line 140, in expect
    raise self.raise_syntax_error(f"Expected {expected}")
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/_vendor/packaging/_tokenizer.py", line 165, in raise_syntax_error
    raise ParserSyntaxError(
pkg_resources.extern.packaging._tokenizer.ParserSyntaxError: Expected package name at the start of dependency specifier
    "git+https://github.com/jakobmerljak/pyarcrest.git@dev"
    ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/storage/srv/galaxy/server/lib/galaxy/dependencies/__init__.py", line 312, in optional
    conditional = ConditionalDependencies(config_file)
  File "/storage/srv/galaxy/server/lib/galaxy/dependencies/__init__.py", line 45, in __init__
    self.get_conditional_requirements()
  File "/storage/srv/galaxy/server/lib/galaxy/dependencies/__init__.py", line 176, in get_conditional_requirements
    for req in pkg_resources.parse_requirements(open(crfile).readlines()):
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/__init__.py", line 3174, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 37, in __init__
    raise InvalidRequirement(str(e)) from e
pkg_resources.extern.packaging.requirements.InvalidRequirement: Expected package name at the start of dependency specifier
    "git+https://github.com/jakobmerljak/pyarcrest.git@dev"
    ^

for advanced package specifiers that might be used during development.

(Please replace this header with a description of your pull request. Please include BOTH what you did and why you made the changes. The "why" may simply be citing a relevant Galaxy issue.)
(If fixing a bug, please add any relevant error or traceback)
(For UI components, it is recommended to include screenshots or screencasts)

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

Fixes
```
Traceback (most recent call last):
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 35, in __init__
    parsed = _parse_requirement(requirement_string)
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 64, in parse_requirement
    return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 73, in _parse_requirement
    name_token = tokenizer.expect(
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/_vendor/packaging/_tokenizer.py", line 140, in expect
    raise self.raise_syntax_error(f"Expected {expected}")
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/_vendor/packaging/_tokenizer.py", line 165, in raise_syntax_error
    raise ParserSyntaxError(
pkg_resources.extern.packaging._tokenizer.ParserSyntaxError: Expected package name at the start of dependency specifier
    "git+https://github.com/jakobmerljak/pyarcrest.git@dev"
    ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/storage/srv/galaxy/server/lib/galaxy/dependencies/__init__.py", line 312, in optional
    conditional = ConditionalDependencies(config_file)
  File "/storage/srv/galaxy/server/lib/galaxy/dependencies/__init__.py", line 45, in __init__
    self.get_conditional_requirements()
  File "/storage/srv/galaxy/server/lib/galaxy/dependencies/__init__.py", line 176, in get_conditional_requirements
    for req in pkg_resources.parse_requirements(open(crfile).readlines()):
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/__init__.py", line 3174, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/storage/srv/galaxy/server/.venv/lib64/python3.9/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 37, in __init__
    raise InvalidRequirement(str(e)) from e
pkg_resources.extern.packaging.requirements.InvalidRequirement: Expected package name at the start of dependency specifier
    "git+https://github.com/jakobmerljak/pyarcrest.git@dev"
    ^

```

for advanced package specifiers that might be used during development.
@mvdbeek mvdbeek added kind/refactoring cleanup or refactoring of existing code, no functional changes area/dependencies labels Sep 1, 2023
@github-actions github-actions bot added this to the 23.2 milestone Sep 1, 2023
@nsoranzo
Copy link
Member

nsoranzo commented Sep 1, 2023

https://setuptools.pypa.io/en/latest/pkg_resources.html recommends packaging for requirements parsing, would https://packaging.pypa.io/en/stable/requirements.html work?
One less project dependency, which would otherwise need to be added to packages/app/setup.cfg as well.

@mvdbeek
Copy link
Member Author

mvdbeek commented Sep 1, 2023

That's not exactly a replacement for parsing requirement files, we'd still have to handle comments and empty lines and other funky stuff that is allowed ... or live with the limitations. FWIW dparse is 900 lines in total and used by pyup.io

@dannon dannon merged commit 9d7bc54 into galaxyproject:dev Sep 6, 2023
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/dependencies kind/refactoring cleanup or refactoring of existing code, no functional changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants