Skip to content

Conversation

@tieneupin
Copy link
Contributor

@tieneupin tieneupin commented May 20, 2024

Fixes issue #268.

As of pip v22.3 (in accordance with PEP 658), pip now expects an additional whl.metadata file to be served from the Simple Index it is using for the package installation. This file is not visible on the Simple Index web page, but is expected to be accessible by replacing .whl with .whl.metadata in the URL.

The bootstrap script previously only copied across what was explicitly visible on the PyPI Simple Index, so these implied URLs were not ported across as well. This commit fixes that by explicitly creating and displaying links to the whl.metadata on the proxy server as well.

Python versions tested:

  • 3.9
  • 3.10
  • 3.11
  • 3.12

@tieneupin tieneupin added bug Something isn't working ci-cd Improvements to project development environment labels May 20, 2024
@tieneupin tieneupin requested a review from d-j-hatton May 20, 2024 10:54
@tieneupin tieneupin self-assigned this May 20, 2024
@tieneupin
Copy link
Contributor Author

tieneupin commented May 20, 2024

Additionally, locked pydantic at <2 for now; depending on which combination of dependencies are installed alongside murfey, its version jumps between 2.7.* and 1.10.*.

@tieneupin
Copy link
Contributor Author

Resolved CodeSQL warning about polynomial regular expression. Unable to resolve warnings related to server-side request forgery.

@tieneupin
Copy link
Contributor Author

tieneupin commented May 22, 2024

Successfully made contact with GitHub's CodeQL team. Will dismiss alert if unable to resolve issue by end of week.

Current state of changes made:

  • Standardised sanitise functions for each folder in murfey by moving definition to __init__.py; context-specific versions left untouched
  • Used urllib.parse.quote() in tandem with re.match() (take only alphanumerics, "_", "-", and "." in string) to validate and normalise incoming package names before inserting it into the PyPI url.

@tieneupin tieneupin linked an issue May 23, 2024 that may be closed by this pull request
11 tasks

# Get HTTP response
url = f"https://pypi.org/simple/{package_clean}"
response = requests.get(url)

Check failure

Code scanning / CodeQL

Partial server-side request forgery

Part of the URL of this request depends on a [user-provided value](1). Part of the URL of this request depends on a [user-provided value](2).
@tieneupin tieneupin merged commit 9de0917 into main May 24, 2024
@tieneupin tieneupin deleted the pip-fix branch May 24, 2024 09:13
@tieneupin tieneupin removed the bug Something isn't working label Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cd Improvements to project development environment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Newer versions of pip fail to install from PyPI proxy

3 participants