PDM fails during package download when running pdm build

[m3nowak]

• I have searched the issue tracker and believe that this is not a duplicate.

Possibly related to #1879 , however the error is different.

Since yesterday, I am unable to build my library in any new environment, including my GitHub actions build. Every attempt results in the following error:

pdm build -v

Building sdist...
pdm.termui: Preparing isolated env for PEP 517 build...
pdm.termui: ======== Start resolving requirements ========
pdm.termui: pdm-backend
pdm.termui: python>=3.9.14,<3.9.15
pdm.termui: Adding requirement pdm-backend
Traceback (most recent call last):
File "/root/.local/bin/pdm", line 8, in
sys.exit(main())
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/core.py", line 255, in main
return Core().main(args)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/core.py", line 193, in main
raise cast(Exception, err).with_traceback(traceback) from None
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/core.py", line 188, in main
self.handle(project, options)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/core.py", line 154, in handle
command.handle(project, options)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/cli/commands/build.py", line 64, in handle
actions.do_build(
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/cli/actions.py", line 493, in do_build
loc = SdistBuilder(project.root, project.environment).build(dest, config_settings)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/builders/sdist.py", line 18, in build
self.install(self._requires, shared=True)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/builders/base.py", line 294, in install
install_requirements(missing, env)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/installers/core.py", line 28, in install_requirements
resolved, _ = resolve(
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/resolver/core.py", line 35, in resolve
result = resolver.resolve(requirements, max_rounds)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/resolvelib/resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/resolvelib/resolvers.py", line 397, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/resolvelib/resolvers.py", line 173, in _add_to_criteria
if not criterion.candidates:
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/resolvelib/structs.py", line 127, in bool
next(iter(self))
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/resolvelib/structs.py", line 136, in iter
self._factory() if self._iterable is None else self._iterable
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/resolver/providers.py", line 139, in matches_gen
candidates = self._find_candidates(reqs[0])
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/resolver/providers.py", line 123, in _find_candidates
return self.repository.find_candidates(requirement, requirement.prerelease or self.allow_prereleases)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/models/repositories.py", line 148, in find_candidates
cans = LazySequence(self._find_candidates(requirement))
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/pdm/models/repositories.py", line 323, in _find_candidates
for c in finder.find_all_packages(requirement.project_name, allow_yanked=requirement.is_pinned)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/unearth/finder.py", line 281, in find_all_packages
return LazySequence(self._find_packages(package_name, allow_yanked, hashes))
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/unearth/finder.py", line 263, in _find_packages
return sorted(all_packages, key=self._sort_key, reverse=True)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/unearth/collector.py", line 133, in collect_links_from_location
yield from _collect_links_from_index(session, location)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/unearth/collector.py", line 153, in _collect_links_from_index
page = fetch_page(session, location)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/unearth/collector.py", line 140, in fetch_page
resp = _get_html_response(session, location)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/unearth/collector.py", line 175, in _get_html_response
resp = session.get(
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/requests/sessions.py", line 600, in get
return self.request("GET", url, **kwargs)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/requests/sessions.py", line 745, in send
r.content
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/requests/models.py", line 899, in content
self._content = b"".join(self.iter_content(CONTENT_CHUNK_SIZE)) or b""
File "/root/.local/share/pdm/venv/lib64/python3.9/site-packages/requests/models.py", line 818, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(5254 bytes read, -2627 more expected)', IncompleteRead(5254 bytes read, -2627 more expected))

My failed GH Actions run https://github.com/m3nowak/nats-nsc/actions/runs/4894670086/jobs/8747174607

Steps to reproduce

I was able to reproduce this issue on a container, it seems it only appears on new pdm installs.

1. Create new pdm lib project with pdm init
2. Try to build this project using pdm build

Actual behavior

PDM fails to download pdm-backend, which results in crash.

[root@c6b6e6aeb2f4 ~]# mkdir test2
[root@c6b6e6aeb2f4 ~]# cd test2
[root@c6b6e6aeb2f4 test2]# pdm init
Creating a pyproject.toml for PDM...
Please enter the Python interpreter to use
0. /usr/bin/python3.9 (3.9)
1. /root/.local/share/pdm/venv/bin/python (3.9)
Please select (0): 0
Using Python interpreter: /usr/bin/python3.9 (3.9)
Using __pypackages__ because non-venv Python is used.
Would you like to create a virtualenv with /usr/bin/python3.9? [y/n] (y): 
Virtualenv is created successfully at /root/test2/.venv
Is the project a library that is installable?
If yes, we will need to ask a few more questions to include the project name and build backend [y/n] (n): y
Project name (test2): 
Project version (0.1.0): 
Project description (): 
Which build backend to use?
0. pdm-backend
1. setuptools
2. flit-core
3. hatchling
4. pdm-pep517
Please select (0): 
License(SPDX name) (MIT): 
Author name (): AB
Author email (): a@example.com
Python requires('*' to allow any) (>=3.9): 
Changes are written to pyproject.toml.
[root@c6b6e6aeb2f4 test2]# pdm build
Building sdist...
See /tmp/pdm-build-9iu2574z.log for detailed debug log.
[ChunkedEncodingError]: ('Connection broken: IncompleteRead(5254 bytes read, -2627 more expected)', IncompleteRead(5254 bytes read, -2627 more expected))
Add '-v' to see the detailed traceback

Expected behavior

PDM successfully builds package or at least does not fail during pdm build

Environment Information

# Paste the output of `pdm info && pdm info --env` below:
pdm info && pdm info --env
PDM version:
  2.5.5
Python Interpreter:
  /root/test2/.venv/bin/python (3.9)
Project Root:
  /root/test2
Local Packages:
  
{
  "implementation_name": "cpython",
  "implementation_version": "3.9.14",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "6.2.12-200.fc37.x86_64",
  "platform_system": "Linux",
  "platform_version": "#1 SMP PREEMPT_DYNAMIC Thu Apr 20 23:38:29 UTC 2023",
  "python_full_version": "3.9.14",
  "platform_python_implementation": "CPython",
  "python_version": "3.9",
  "sys_platform": "linux"
}

[frostming]

It seems the cached content generated by cachecontrol isn't compatible with urllib3 2.0, downgrade urllib3 can fix this.

[rafalkrupinski]

Same error on pdm add

[rafalkrupinski]

If you've installed pdm with pipx:
pipx inject pdm "urllib3<2.0"

[m3nowak]

I was able to use pdm self add 'urllib3<2' as a workaround.

[frostming]

Reported upstream: psf/cachecontrol#295

[rafalkrupinski]

Reported upstream: ionrock/cachecontrol#295

Doesn't look like the project is still alive