Skip to content

shutil: avoid importing bz2, lzma and compression.zstd just to probe availability #154904

Description

@maxday

shutil imports bz2/lzma/zstd at startup just to check they exist

shutil imports bz2, lzma and compression.zstd at module scope purely to discover whether those formats are available, then throws the bindings away:

try:
    import bz2
    del bz2
    _BZ2_SUPPORTED = True
except ImportError:
    _BZ2_SUPPORTED = False

Every program importing shutil pays for that, even if it never creates or extracts an archive and shutil is imported transitively by much of the stdlib: on main, each of urllib.request, tempfile, zipfile, tarfile, http.server and venv ends up with shutil in sys.modules.

Net import cost (median of 40 fresh interpreters):

net cost
import shutil 12.75 ms
import urllib.request 36.64 ms

This matters most for short-lived processes on constrained hardware: CLI tools and serverless cold starts, where a few ms per process is worth removing.

I have a patch with tests that cuts this by ~2.5 ms (−19.6%) on import shutil, and will open a PR.

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance or resource usagestdlibStandard Library Python modules in the Lib/ directory

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions