Skip to content

gh-155286: Write a valid empty zstd archive #155342

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
RealBhupesh wants to merge 1 commit into
base: main
Choose a base branch
from
+12 −4
Open

gh-155286: Write a valid empty zstd archive #155342

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion Lib/compression/zstd/_zstdfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ def __init__(self, file, /, mode='r', *,
self._close_fp = False
self._mode = _MODE_CLOSED
self._buffer = None
self._write_started = False

if not isinstance(mode, str):
raise ValueError('mode must be a str')
Expand All @@ -68,6 +69,9 @@ def __init__(self, file, /, mode='r', *,
if level is not None and not isinstance(level, int):
raise TypeError('level must be int or None')
self._mode = _MODE_WRITE
# Do not add an empty frame when closing an existing archive in
# append mode without writing anything.
self._write_started = mode == 'a'

@chatgpt-codex-connector chatgpt-codex-connector Bot Aug 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Emit a frame for newly created append targets

When ZstdFile(path, "a") creates a nonexistent file, or opens an existing zero-byte file, setting _write_started solely from the mode makes close() short-circuit because a new compressor's last_mode is already FLUSH_FRAME. With no writes, the target therefore remains a zero-byte, invalid Zstandard archive—the same failure this change fixes for w and x. Preserve no-op behavior only when the append target already contains archive data.

Useful? React with 👍 / 👎.

self._compressor = ZstdCompressor(level=level, options=options,
zstd_dict=zstd_dict)
self._pos = 0
Expand Down Expand Up @@ -131,6 +135,7 @@ def write(self, data, /):
length = _nbytes(data)

compressed = self._compressor.compress(data)
self._write_started = True
self._fp.write(compressed)
self._pos += length
return length
Expand All @@ -153,10 +158,11 @@ def flush(self, mode=FLUSH_BLOCK):
raise ValueError('Invalid mode argument, expected either '
'ZstdFile.FLUSH_FRAME or '
'ZstdFile.FLUSH_BLOCK')
if self._compressor.last_mode == mode:
if self._compressor.last_mode == mode and self._write_started:
return
# Flush zstd block/frame, and write.
data = self._compressor.flush(mode)
self._write_started = True
self._fp.write(data)
if hasattr(self._fp, 'flush'):
self._fp.flush()
Expand Down
6 changes: 3 additions & 3 deletions Lib/test/test_zstd.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2120,16 +2120,16 @@ def test_write_empty_frame(self):
self.assertNotEqual(c.flush(c.FLUSH_FRAME), b'')
self.assertNotEqual(c.flush(c.FLUSH_FRAME), b'')

# don't generate empty content frame
# generate an empty content frame when the file is closed
bo = io.BytesIO()
with ZstdFile(bo, 'w') as f:
pass
self.assertEqual(bo.getvalue(), b'')
self.assertEqual(decompress(bo.getvalue()), b'')

bo = io.BytesIO()
with ZstdFile(bo, 'w') as f:
f.flush(f.FLUSH_FRAME)
self.assertEqual(bo.getvalue(), b'')
self.assertEqual(decompress(bo.getvalue()), b'')

# if .write(b''), generate empty content frame
bo = io.BytesIO()
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Library/2026-08-07-00-00-00.gh-issue-155286.empty-zstd-file.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix :class:`~compression.zstd.ZstdFile` creating an invalid zero-byte archive
when an output file is closed without any writes.
Loading