Skip to content

drop dependency on 'urllib3'#9273

Merged
crusaderky merged 1 commit into
dask:mainfrom
jameslamb:remove-urllib
May 26, 2026
Merged

drop dependency on 'urllib3'#9273
crusaderky merged 1 commit into
dask:mainfrom
jameslamb:remove-urllib

Conversation

@jameslamb
Copy link
Copy Markdown
Member

Closes #9267

distributed has a hard runtime dependency on urllib3 only for --preload (see #3755). That's only used for a single GET request with retries. This proposes replacing it with urllib from the standard library (h/t @jrbourbeau for the suggestion) and removing the urllib3 dependency.

Notes for Reviewers

  • Tests added / passed
  • Passes pre-commit run --all-files

I left one optional use in test-only code

Intentionally left this:

from urllib3.util.retry import Retry

And the dependency in the conda env for running that:

How I tested this

python -m venv .venv
source .venv/bin/activate

pip install \
  . \
  pytest \
  pytest-cov \
  pytest-repeat \
  pytest-rerunfailures \
  pytest-timeout

pytest ./distributed/tests/test_preload.py

@github-actions
Copy link
Copy Markdown
Contributor

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    31 files  ±0      31 suites  ±0   10h 41m 51s ⏱️ + 9m 9s
 4 074 tests ±0   3 967 ✅  -  3    104 💤 ± 0  2 ❌ +2  1 🔥 +1 
59 131 runs  ±0  56 648 ✅  - 50  2 480 💤 +47  2 ❌ +2  1 🔥 +1 

For more details on these failures and errors, see this check.

Results for commit 4876d19. ± Comparison against base commit d8fabe2.

@jameslamb jameslamb changed the title drop dependncy on 'urllib3' drop dependency on 'urllib3' May 25, 2026
@crusaderky
Copy link
Copy Markdown
Collaborator

Intentionally left this:
distributed/continuous_integration/scripts/test_report.py

That's fine, test_report.py is currently broken and needs a heavy maintenance pass anyway.

Copy link
Copy Markdown
Collaborator

@crusaderky crusaderky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've also tested this manually without mocks

@crusaderky crusaderky merged commit 6f721b8 into dask:main May 26, 2026
34 of 39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

make urllib3 an optional dependency?

2 participants