Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup rwobject.c file object methods #2717

Merged
merged 4 commits into from
Mar 4, 2024

Conversation

Starbuck5
Copy link
Member

@Starbuck5 Starbuck5 commented Feb 13, 2024

  • Simplifies the code by removing WITH_THREAD / not WITH_THREAD conditional compilation
  • Removes path for file-like objects that are actually files
  • Removes unused data ("file") inside pgRwHelper

I've tried before to remove WITH_THREAD blocks and got a lot of skepticism, so I've done my homework this time. See CPython python/cpython#75551 "Remove support for threads-less builds," where WITH_THREAD becomes always defined (Python 3.7). See also pyport.h where it is still hardcoded as defined, and the region below references emscripten's lack of threading support, so it's not like this code hasn't taken that into account yet. Lastly, lets say we did build on a platform without threads... what's the harm in calling PyGILState_Ensure and PyGILState_Release? They're both in the stable ABI, it's not like they will be missing-- they might become no-ops but that's fine.

Secondly, my removal of the fileno path. The io protocol can only provide a fileno if it actually comes from a file, see https://docs.python.org/3/library/io.html#io.IOBase.fileno. I believe it would be very uncommon for a user to pass an io object into a pygame loader after loading from a file. What I've seen is str/pathlib path input, which creates a basic SDL_RWop, and io.BytesIO-style (memory) input which creates one of our custom RWops. io.BytesIO objects don't come from files, so they don't have a fileno. Before this PR every time they had to do anything they'd have to check if they had a fileno. If a user happens to pass in an io object that actually does come from a file, the code using the standard io APIs will work fine on it without a specialized path for that case.

Also, per the docs for PyObject_AsFileDescriptor (at https://docs.python.org/3/c-api/file.html), it is an emulation of Python 2 APIs and third party code should use io instead.

Some example code showing how uses of io can intersect with pygame-ce:

import pygame
import io

pygame.init()

# Load into BytesIO somehow (through a file this time but could come from anywhere)
with open(pygame.font.match_font("Arial"), "rb") as f:
    obj = io.BytesIO(f.read())

# print(obj.fileno()) # obj doesn't have a fileno, raises exception
font = pygame.Font(obj, 20)
print(font)

# f is an io.BufferedReader
with open(pygame.font.match_font("Arial"), "rb") as f:
    print(f.fileno()) # f has a fileno
    font = pygame.Font(f, 20)
    print(font)

BytesIO never has a fileno, and we practically only use this path for BytesIO. So it is redundant code, and slows things down slightly at runtime.
@Starbuck5 Starbuck5 added Code quality/robustness Code quality and resilience to changes rwops SDL's IO loading/streaming code labels Feb 13, 2024
@Starbuck5 Starbuck5 requested a review from a team as a code owner February 13, 2024 08:13
Copy link
Member

@ankith26 ankith26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm in for removing WITH_THREAD. But I don't think we need to remove fileno logic, simply because it is intended as an optimisation (as that codepath does not need GIL) and it doesn't add much code complexity.

@Starbuck5
Copy link
Member Author

Optimizations are all well and good, but an optimization for who? I don't think I've ever seen anyone open a file and pass it into pygame-ce in the way required to get this to work, in years of examining people's codes on discord and on reddit.

Keep in mind removing the branching and ref counting required for this questionable optimization is a micro-optimization for the far more common case of using a BytesIO. And it makes the code simpler.

@ankith26
Copy link
Member

I looked at the blame for when this code was added, and traced it back to pygame/pygame#551

Back when it was added it was a bugfix and not an optimization. Though now, I just tested your PR and it works with the testcase that used to deadlock the music playback before this fix.

Copy link
Member

@ankith26 ankith26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, these changes LGTM, thanks!

I guess it would still be nice to add a few unitests that explicitly test the python-with-fileno kind of objects while simultaneously running multiple instances of those to make sure the deadlocking issue doesn't return.

Also there are more usages of WITH_THREAD around the codebase that could be removed, this could also happen in a later PR

@Starbuck5
Copy link
Member Author

Thanks for looking into the history of this, I had assumed it was primordial and wouldn't have a PR attached.

I really was not thinking about the GIL at all while making this PR.

The old PR that added this seems like a sidestep of the issue. It avoids the deadlock only file objects from files, but what about all other file objects? Nobody has reported any deadlocks with BytesIO, and this brings (very uncommon) file-objects-from-files into the same level as BytesIO, so it should be fine.

@Starbuck5
Copy link
Member Author

FYI I was looking into these methods because in SDL3 the rwops signatures changes, so I wanted to simplify them before needing to write in SDL3-compat stuff.

Copy link
Member

@MyreMylar MyreMylar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, LGTM 👍

@MyreMylar MyreMylar merged commit ec62643 into pygame-community:main Mar 4, 2024
30 checks passed
@ankith26 ankith26 added this to the 2.5.0 milestone Mar 5, 2024
@Starbuck5 Starbuck5 deleted the rwobject-code branch March 22, 2024 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Code quality/robustness Code quality and resilience to changes rwops SDL's IO loading/streaming code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants