urllib3 2.0 incompatibility: 'HTTPResponse' object has no attribute 'strict'

[jorge-wonolo]

• I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

1. Run python docker image --> docker run -it --entrypoint="" python3.9 /bin/bash
2. Install PDM --> pip install pdm (requests-2.30.0 is installed)
3. Setup pdm --> pdm init

Actual behavior

Warning is displayed:

/usr/local/lib/python3.9/site-packages/pdm/cli/actions.py:823: RuntimeWarning: Failed to get latest version: 'HTTPResponse' object has no attribute 'strict'
  warnings.warn(f"Failed to get latest version: {e}", RuntimeWarning, stacklevel=1)

If we try to add a dependency we get an exception:

root@548563773aab:/# pdm add -v pytest
Adding group default to lockfile
Adding packages to default dependencies: pytest
STATUS: Resolving dependencies
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   pytest
pdm.termui:   python>=3.9
pdm.termui:   Adding requirement pytest
Traceback (most recent call last):
  File "/usr/local/bin/pdm", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/pdm/core.py", line 255, in main
    return Core().main(args)
  File "/usr/local/lib/python3.9/site-packages/pdm/core.py", line 193, in main
    raise cast(Exception, err).with_traceback(traceback) from None
  File "/usr/local/lib/python3.9/site-packages/pdm/core.py", line 188, in main
    self.handle(project, options)
  File "/usr/local/lib/python3.9/site-packages/pdm/core.py", line 154, in handle
    command.handle(project, options)
  File "/usr/local/lib/python3.9/site-packages/pdm/cli/commands/add.py", line 60, in handle
    actions.do_add(
  File "/usr/local/lib/python3.9/site-packages/pdm/cli/actions.py", line 280, in do_add
    resolved = do_lock(
  File "/usr/local/lib/python3.9/site-packages/pdm/cli/actions.py", line 98, in do_lock
    mapping, dependencies = resolve(
  File "/usr/local/lib/python3.9/site-packages/pdm/resolver/core.py", line 35, in resolve
    result = resolver.resolve(requirements, max_rounds)
  File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
  File "/usr/local/lib/python3.9/site-packages/resolvelib/structs.py", line 127, in __bool__
    next(iter(self))
  File "/usr/local/lib/python3.9/site-packages/pdm/resolver/providers.py", line 241, in matches_gen
    yield from super_find()
  File "/usr/local/lib/python3.9/site-packages/pdm/resolver/providers.py", line 139, in matches_gen
    candidates = self._find_candidates(reqs[0])
  File "/usr/local/lib/python3.9/site-packages/pdm/resolver/providers.py", line 123, in _find_candidates
    return self.repository.find_candidates(requirement, requirement.prerelease or self.allow_prereleases)
  File "/usr/local/lib/python3.9/site-packages/pdm/models/repositories.py", line 148, in find_candidates
    cans = LazySequence(self._find_candidates(requirement))
  File "/usr/local/lib/python3.9/site-packages/pdm/models/repositories.py", line 323, in _find_candidates
    for c in finder.find_all_packages(requirement.project_name, allow_yanked=requirement.is_pinned)
  File "/usr/local/lib/python3.9/site-packages/unearth/finder.py", line 281, in find_all_packages
    return LazySequence(self._find_packages(package_name, allow_yanked, hashes))
  File "/usr/local/lib/python3.9/site-packages/unearth/finder.py", line 263, in _find_packages
    return sorted(all_packages, key=self._sort_key, reverse=True)
  File "/usr/local/lib/python3.9/site-packages/unearth/collector.py", line 133, in collect_links_from_location
    yield from _collect_links_from_index(session, location)
  File "/usr/local/lib/python3.9/site-packages/unearth/collector.py", line 153, in _collect_links_from_index
    page = fetch_page(session, location)
  File "/usr/local/lib/python3.9/site-packages/unearth/collector.py", line 140, in fetch_page
    resp = _get_html_response(session, location)
  File "/usr/local/lib/python3.9/site-packages/unearth/collector.py", line 175, in _get_html_response
    resp = session.get(
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 600, in get
    return self.request("GET", url, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 745, in send
    r.content
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 899, in content
    self._content = b"".join(self.iter_content(CONTENT_CHUNK_SIZE)) or b""
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 816, in generate
    yield from self.raw.stream(chunk_size, decode_content=True)
  File "/usr/local/lib/python3.9/site-packages/urllib3/response.py", line 935, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/local/lib/python3.9/site-packages/urllib3/response.py", line 874, in read
    data = self._raw_read(amt)
  File "/usr/local/lib/python3.9/site-packages/urllib3/response.py", line 809, in _raw_read
    data = self._fp_read(amt) if not fp_closed else b""
  File "/usr/local/lib/python3.9/site-packages/urllib3/response.py", line 794, in _fp_read
    return self._fp.read(amt) if amt is not None else self._fp.read()
  File "/usr/local/lib/python3.9/site-packages/cachecontrol/filewrapper.py", line 96, in read
    self._close()
  File "/usr/local/lib/python3.9/site-packages/cachecontrol/filewrapper.py", line 76, in _close
    self.__callback(result)
  File "/usr/local/lib/python3.9/site-packages/cachecontrol/controller.py", line 353, in cache_response
    self._cache_set(cache_url, request, response, body, expires_time)
  File "/usr/local/lib/python3.9/site-packages/cachecontrol/controller.py", line 274, in _cache_set
    self.serializer.dumps(request, response, body),
  File "/usr/local/lib/python3.9/site-packages/cachecontrol/serialize.py", line 54, in dumps
    u"strict": response.strict,
AttributeError: 'HTTPResponse' object has no attribute 'strict'

Expected behavior

PDM can be initialized and packages can be installed

Environment Information

root@548563773aab:/# pdm info && pdm info --env
PDM version:
  2.5.3
Python Interpreter:
  /.venv/bin/python (3.9)
Project Root:
  /
Local Packages:

/usr/local/lib/python3.9/site-packages/pdm/cli/actions.py:823: RuntimeWarning: Failed to get latest version: 'HTTPResponse' object has no attribute 'strict'
  warnings.warn(f"Failed to get latest version: {e}", RuntimeWarning, stacklevel=1)
{
  "implementation_name": "cpython",
  "implementation_version": "3.9.16",
  "os_name": "posix",
  "platform_machine": "aarch64",
  "platform_release": "5.15.82-0-virt",
  "platform_system": "Linux",
  "platform_version": "#1-Alpine SMP Mon, 12 Dec 2022 09:15:17 +0000",
  "python_full_version": "3.9.16",
  "platform_python_implementation": "CPython",
  "python_version": "3.9",
  "sys_platform": "linux"
}
/usr/local/lib/python3.9/site-packages/pdm/cli/actions.py:823: RuntimeWarning: Failed to get latest version: 'HTTPResponse' object has no attribute 'strict'
  warnings.warn(f"Failed to get latest version: {e}", RuntimeWarning, stacklevel=1)

[chapmanjacobd]

I also get this error when trying to install anything starting yesterday

https://github.com/chapmanjacobd/library/actions/runs/4885246275/jobs/8719096764

add appdirs failed:
...
"/Users/runner/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/cachecontrol/serialize.py", line 54, in self.serializer.dumps
    u"strict": response.strict,
AttributeError: 'HTTPResponse' object has no attribute 'strict'

psf/cachecontrol#292

I think the short term solution here will be to pin Requests to pre-2.30.0:

pip install pdm 'requests<2.30.0'

edit: yep. works for me: https://github.com/chapmanjacobd/library/actions/runs/4885486057

[cp2boston]

I am also seeing a similar problem, starting yesterday afternoon:

• PDM 2.5.3
• Python 3.10.11
• install pdm via curl -sSL https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py | python - -p /usr/local
• create and cd to a new directory
• run pdm init, no installable, no package

Changes are written to pyproject.toml.
/usr/local/venv/lib/python3.10/site-packages/pdm/cli/actions.py:823: RuntimeWarning: Failed to get latest version: 'HTTPResponse' object has no attribute 'strict'
  warnings.warn(f"Failed to get latest version: {e}", RuntimeWarning, stacklevel=1)

Of note:

• urllib3 removed the strict attribute with version 2.0.0 last week
• the above short term solution worked for me as well

[nateprewitt]

pip install pdm 'requests<2.30.0'

This is likely going to create more work for you long term than following the instructions for this provided with the 2.30.0 release notes.

Cachecontrol has an unspecified required dependency on urllib3 which assumes you've installed urllib3 1.x. Because this isn't specified, either PDM or the end user can help enforce that by defining urllib3<2 until this is addressed in cachecontrol.

[radusuciu]

Same issue, pip install pdm 'requests<2.30.0' also worked. urllib3<2 did not work but I did not investigate deeply.

[frostming]

I contacted @ionrock about the maintenance issue, let's see how it's going. Before that we have to pin urllib3<2.

[illpack]

Same here with python 3.11 and pdm==2.4.9 and pdm==2.5.4

[frostming]

Same here with python 3.11 and pdm==2.4.9 and pdm==2.5.4

Can you confirm? How do you test? pdm 2.5.4 has pinned urllib3<2.

[grmnz]

Same here with python 3.11 and pdm==2.4.9 and pdm==2.5.4

Can you confirm? How do you test? pdm 2.5.4 has pinned urllib3<2.

python3.11 pdm==2.5.4 is OK (urllib3 1.26.15)

[frostming]

This is resolved by PDM side, close it now.