v0.12.3: io D port Modules/_io/bufferedio.c in full#31
Merged
Conversation
Adds buffered_repr, buffered_iternext, __enter__/__exit__, __iter__, and the closefd attribute forwarding to the raw stream on the buffered classes. Repr/Str slots wired on BufferedReaderType / BufferedWriterType / BufferedRandomType. bufferedWrite now seeks the raw stream back by the unread amount and drops the read buffer before issuing a write, so BufferedRandom no longer loses position on a read->write transition. Spec 1702 row D updated honestly: this is incremental hardening, not a top-to-bottom port. The unified-slab buffer model is still divergent and will land in its own PR.
The Buffered struct now mirrors CPython's `buffered` field-for-field: one shared `buffer` slab plus pos / raw_pos / read_end / write_pos / write_end / abs_pos / buffer_mask offsets. The read and write views overlap into the same byte array, so BufferedRandom can interleave reads and writes without the read cache going stale or the raw stream drifting. ADJUST_POSITION, RAW_OFFSET, READAHEAD, MINUS_LAST_BLOCK are ported as Buffered methods. _bufferedreader_raw_read, _bufferedreader_fill_buffer, _bufferedwriter_raw_write, _bufferedwriter_flush_unlocked, buffered_flush_and_rewind_unlocked, _bufferedreader_read_all, _bufferedreader_read_fast, _bufferedreader_read_generic, and _bufferedreader_peek_unlocked all land with CPython line citations. Other fixes that fell out of the rewrite: - closed getter now reads self.raw.closed instead of a local bool, so a raw stream closed by something else flips the wrapper's view too. - BufferedRandom __init__ validates that the raw stream is seekable. - __sizeof__ returns basicsize + buffer_size, not buffer_size alone. - seek now has the intra-buffer fast path (SEEK_SET / SEEK_CUR returns immediately when the target lies inside the current view). - _dealloc_warn forwards to the raw stream. - bufferedTell subtracts RAW_OFFSET (which collapses to the same answer but is reachable from a pending write buffer too). Not ported on purpose: per-instance thread lock + reentrancy owner (Go's concurrency model is different), _PyIO_trap_eintr (objects.Call never surfaces a raw EINTR), and the fast_closed_checks shortcut (redundant in the Go wrapper). Spec 1702 row D flipped to done. Overall io / _io row note refreshed to reflect that bufferedio joined the full-port list.
tamnd
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Full port of
Modules/_io/bufferedio.con CPython's unified-slab buffer model.The Buffered struct now mirrors the C
bufferedstruct field-for-field: one sharedbufferslab pluspos/raw_pos/read_end/write_pos/write_end/abs_pos/buffer_mask. Read and write views overlap into the same byte array, so a BufferedRandom can interleave reads and writes without the read cache going stale or the raw stream drifting away from the logical position.What landed
ADJUST_POSITION/RAW_OFFSET/READAHEAD/MINUS_LAST_BLOCKas Buffered methods._bufferedreader_raw_read,_bufferedreader_fill_buffer,_bufferedreader_read_all(with the raw.readall fast path),_bufferedreader_read_fast,_bufferedreader_read_generic,_bufferedreader_peek_unlocked._bufferedwriter_raw_write,_bufferedwriter_flush_unlocked(with the rewind-before-write step).buffered_flush_and_rewind_unlocked.closedgetter readsself.raw.closedinstead of a local bool, so a raw stream closed by something else flips the wrapper's view too.__init__validates the raw stream is seekable.__sizeof__returnsbasicsize + buffer_size.seekincludes the intra-buffer fast path (SEEK_SET / SEEK_CUR returns immediately when the target lies inside the current view)._dealloc_warnforwards to the raw stream.tellsubtractsRAW_OFFSETso a pending write buffer doesn't skew the result.buffered_iternext,buffered_repr, and the context-manager / iter dunders are wired through type slots.Not ported, on purpose
ENTER_BUFFERED/LEAVE_BUFFERED). Go's concurrency model is different from CPython's GIL+per-buffer lock; a Go-flavoredlockOSThreadguard would not match user expectations._PyIO_trap_eintrretries on EINTR. The gopybufXxxhelpers go throughobjects.Call, which never surfaces a raw EINTR to the buffered layer.fast_closed_checks(thePy_IS_TYPEshortcut that skipsgetattr(raw, "closed")). Gopy already calls the Go method directly so the optimization is moot.Spec 1702 row D flipped to done. The overall io / _io row now lists bufferedio in the full-port group.
Test plan
go test ./module/io/...greengo test ./...greengolangci-lint run ./module/io/...clean