Implement PEP 706 – Filter for tarfile.extractall

[encukou]

This issue tracks implementation of PEP 706 – Filter for tarfile.extractall

• Fix WASI tests (or at least understand and document the symlink behavior)

Linked PRs

• gh-102950: Implement PEP 706 – Filter for tarfile.extractall #102953
• [3.11] gh-102950: Implement PEP 706 – Filter for tarfile.extractall (GH-102953) #103832
• [3.10] gh-102950: Implement PEP 706 – Filter for tarfile.extractall (GH-102953) #104128
• [3.10] gh-102950: Implement PEP 706 – Filter for tarfile.extractall (GH-102953) #104327
• [3.9] gh-102950: Implement PEP 706 – Filter for tarfile.extractall (GH-102953) #104382
• [3.8] gh-102950: Implement PEP 706 – Filter for tarfile.extractall (GH-102953) #104548
• [3.7] gh-102950: Implement PEP 706 – Filter for tarfile.extractall (GH-102953)  #104583

Unofficial backport

• 3.6 backport for tarfile filters encukou/cpython#26

[mcepl]

And I really think #73974 (CVE-2007-4559) should be mentioned somewhere in this PR.

[encukou]

Well, the PR was open for a month, but now it's closed. Feel free to suggest an update to the docs.
Note that Python 3.12 doesn't “fix” CVE-2007-4559 (depending on how you define “fix”).

[ned-deily]

Per the discussion in #104583, we have decided that it is not feasible to safely merge the proposed 3.7 version of this fix prior to 3.7's imminent end-of-life. Third-party distributors of cPython who plan to provide support for 3.7 past its official end-of-life are free, of course, to choose to merge or adapt the PR for their users.

[mcepl]

Well, the PR was open for a month, but now it's closed. Feel free to suggest an update to the docs. Note that Python 3.12 doesn't “fix” CVE-2007-4559 (depending on how you define “fix”).

I would say “deals with”, which should cover everything. ;)

[encukou]

I hope 3.4 is too old for me, so I won't be porting there any time soon.

(To fix the CVE you need to update callers -- and it's often enough to audit or improve whatever ad-hoc mechanisms they already have. The filter API isn't strictly necessary for security.)

[mcepl]

(To fix the CVE you need to update callers -- and it's often enough to audit or improve whatever ad-hoc mechanisms they already have. The filter API isn't strictly necessary for security.)

Did I miss it? What exactly you are talking about?

[encukou]

Did I miss it? What exactly you are talking about?

The backports add API that allows callers to avoid CVE-2007-4559, since 3.12 they get a warning if they're not using it, and the plan for 3.14 is to switch to safer behaviour by default.

Until 3.14, the default behaviour stays as it was.

[mcepl]

Oh, you mean that the change is not 100% API-stable? We have learned the hard way, unfortunately.

[mcepl]

And yes, I will be working on 3.4 as well. Happy times!

[ben-spiller]

Hi, I just upgraded from 3.11 to Python 3.12 and this PR is producing DeprecationWarnings (ok) that can't be easily avoided (not ok!) while calling shutil.unpack_archive(): Python 3.14 will, by default, filter extracted tar archives and reject files or modify their metadata. Use the filter argument to control this behavior.

While the idea of a deprecation warning for tarfile with filter=None is clearly reasonable, the impact of this on shutil.unpack_archive() is really unfortunate and could use more consideration. The whole point of unpack_archive is to provide a way to unpack an archive without the caller having to special-case different archive formats. It's now impossible to use unpack_archive for different formats in Python 3.12 without either

a) hitting the new v3.12 deprecation warning for not specifying filter='...' (imho it's not ok to ignore warnings!), or

b) special-casing how caller invokes unpack_archive() for different archive types i.e. passing filter='data' if archive contains .tar/tgz/txz/etc but filter=None if archive is a zipfile. Determining which case is not trivial for the caller of unpack_archive(), and should be taken care of by the library.

Both of those options are pretty gross and make this change very hard to adapt to without gross hacks or disabling deprecation warnings (not a great practice).

Some possible solutions:

• provide some value of filter= that can be passed to both tarfile and other archive formats that don't yet/don't need to support tar-style safe extraction (as PEP-0706 states, "ZipFile.extract’s defaults are already similar to what a 'data' filter would do")
• remove the deprecation warning asap until a future release when there's time to do the above or plan another solution. (nb: this deprecation is not documented in the "Important deprecations, removals or restrictions" section of release notes so removing it in a patch would seem reasonable)
• make unpack_archive pass the filter= flag down to tarfile but not to other archive classes

Right now it's a breaking change and I can't see any nice way to work around it.