Skip to content

Conversation

cmaloney
Copy link
Contributor

@cmaloney cmaloney commented Sep 10, 2025

In _io__Buffered_flush_impl the macro CHECK_CLOSED is used to check the buffered* is in a good state to be flushed. That differs slightly from buffered_closed.

In some cases, that difference would result in close() (_io__Buffered_close_impl) thinking the file needed to be flushed and closed while flush() thought the file was already closed.

This could happen during GC and would result in an unraisable exception.

In `_io__Buffered_flush_impl` the macro `CHECK_CLOSED` is used to check
the `buffered*` is in a good state to be flushed. That differs slightly
from `buffered_closed`.

In some cases, that difference would result in `close()` thinking the
file needed to be flushed and closed while `flush()` thought the file
was already closed.

This could happen during GC and would result in an unraisable exception.
@cmaloney
Copy link
Contributor Author

cmaloney commented Sep 16, 2025

For this, I also looked at resolving two other ways:

  1. Updating .closed member: This makes it so .close() isn't called (and therefore .flush()). A lot more tests break with that change, this to me is more minimal fix; BufferedRWPair is not often used (and was thought of being removed once).
  2. Figure out why the GC isn't clearing out in a clean order and resolve: Again fairly intricate to do with a lot more potential unintended consequences.

@encukou
Copy link
Member

encukou commented Sep 17, 2025

!buildbot PPC64LE.Fedora.Stable.LTO

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @encukou for commit 60fe309 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F138724%2Fmerge

The command will test the builders whose names match following regular expression: PPC64LE.Fedora.Stable.LTO

The builders matched are:

  • PPC64LE Fedora Stable LTO PR
  • PPC64LE Fedora Stable LTO + PGO PR

Copy link
Member

@encukou encukou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! This looks like a correct change.

Comment on lines +2290 to +2293
# gh-138720: C BufferedRWPair would destruct in a bad order resulting in
# an unraisable exception.
support.gc_collect()

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still necessary?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test is way more likely to triggered the issue (fixed behavior) with this change.

Copy link
Contributor Author

@cmaloney cmaloney Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On my dev box (64 bit archlinux) adding the explicit GC takes this from "one in a hundred" -> every time

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK! I couldn't reproduce it myself, but I'll trust the buildbots :)

@encukou encukou merged commit db68bfc into python:main Sep 18, 2025
51 checks passed
@vstinner
Copy link
Member

Should we backport this bugfix?

@encukou
Copy link
Member

encukou commented Sep 18, 2025

Can you reproduce it on previous versions? If so, please backport :)

@vstinner
Copy link
Member

Can you reproduce it on previous versions? If so, please backport :)

Oh. I'm unable to reproduce the issue on 3.14.

@cmaloney cmaloney deleted the bufferedrwpair_gc branch September 18, 2025 19:26
@cmaloney
Copy link
Contributor Author

The code has been the same since the C I/O implementation was added; unless/until someone has an issue on older versions I wouldn't backport.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants