-
-
Notifications
You must be signed in to change notification settings - Fork 33.4k
gh-141311: Avoid assertion in BytesIO readinto #141333
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Account for when self->pos is equal to PY_SSIZE_T_MAX. The length of read was correctly set to zero but the asserts assumed self->pos couldn't reach PY_SSIZE_T_MAX. Return early to avoid edge cases.
| } | ||
| } | ||
|
|
||
| assert(self->pos + len < PY_SSIZE_T_MAX); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't it enough to fix this assertion?
| assert(self->pos + len < PY_SSIZE_T_MAX); | |
| assert(self->pos + len <= PY_SSIZE_T_MAX); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will but I worry about the later PyBytes_AS_STRING(self->buf) + self->pos as self->pos is PY_SSIZE_T_MAX and adding a large integer to a pointer then doing a memcpy from that computed address (even 0 bytes) feels like a path to undefined behavior to me. Happy to implement whichever you'd prefer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. If adding PY_SSIZE_T_MAX causes troubles, adding slightly smaller value will also cause troubles. Actually, if pos larger than the size of the array, ot is an undefined behaviour. So, a specoal case for empty read is needed too.
Misc/NEWS.d/next/Library/2025-11-09-18-55-13.gh-issue-141311.qZ3swc.rst
Outdated
Show resolved
Hide resolved
For `read_bytes_lock_held` the `size` parameter is always set to 0 by calling code currently if self->pos > self->string_size. I added a range check for self->pos as an extra safety as codepaths chagne but can remove if it's too much. `write_bytes_lock_held`: `endpos` is a `size_t` so can hold `2 * PY_SSIZE_T_MAX`. Value is bounds checked in `resize_buffer_lock_held`. A number of cases use `scan_eol_lock_held` to move forward. That has code which checks `self->pos >= self->string_size` and returns `0`. Callsites all seem to handle that correctly.
|
I audited the codepaths which did For
A number of cases use |
Account for when self->pos is equal to PY_SSIZE_T_MAX. The length of read was correctly set to zero but the asserts assumed self->pos couldn't reach PY_SSIZE_T_MAX. Return early to avoid edge cases.
_io_BytesIO_readinto_impl: Assertion 'self->pos + len < PY_SSIZE_T_MAX' failed.#141311