New Cython files for Python 3.10, 3.11 & v0.57?

[toomanybrians]

I recently updated MPF and MPF-Monitor dev branches to v0.57.0.devXX. These now support Python 3.10 and 3.11. I'm trying to get MPF-MC updated now as well. I created a python-311 branch where I'm doing this work.

It looks like there aren't many changes needed for MPF-MC. (Most of that work was in MPF.) The main thing is just updating the versions and build scripts, and config version and show version to 6.

Recall that I rewrote the MPF-MC installer in Aug 2022, and everything worked fine for awhile. But at some point, the GitHub builds of the audio libraries stopped working for Windows. (They built fine on GitHub, but they they didn't actually work, and since the audio tests don't run on GitHub we didn't know it.)

So now I want to revisit this and see if I can get things building reliably, and if I can get things building for Python 3.10 and 3.11. I have not had luck with this and need help, as I spent as much time as I could experimenting, googling, and asking ChatGPT, but I can't get any farther.

The MPF-MC repo contains .c files generated with Cython 0.29.21. It seems to me that these work for Python 3.8 and 3.9, but not 3.10 or 3.11?

I also tried specifying a newer version of Cython (0.29.36 is the latest 0.29.x), but does that generate new c files? I think no? (Here's that run: https://github.com/missionpinball/mpf-mc/actions/runs/6254185770)

I removed the .c files also to force Cython to recreate them. Here's that run with Cython 0.29.36: https://github.com/missionpinball/mpf-mc/actions/runs/6254426490

I also noticed that the Kivy project is set to allow Cython 3 which has been released recently. I tried with 3.0.2 but that also failed.

All of the errors are pretty different. Sometimes it's linker paths and includes. Other times it's GIL lock errors. And I really have no idea.

@qcapen if you're around, can you take a look at things now and see if you can get this to build? I don't know if Python 3.11 support will be possible, hopefully we can at least get 3.10. I'd love to support 3.11 too (it's been out for a year so hopefully it's possible) since that's the default that brew and a lot of other things install.

Anyway, let me know how I can help? As of now I can't really get this to build. (I see some of the builds for Python 3.9 seem to work, so I'll need to test those.) But I wonder if a cython expert can help out here? Thanks!!

[qcapen]

I will see if I can get the Windows MC builds to work properly, although it may take me a couple of weeks

[toomanybrians]

I added the .c files back and hardcoded Cython to 0.29.21 (which should not matter since the .c files are there now, so I think Cython doesn't even run).

The build is working for Python 3.8 and 3.9 across all three platforms. (Here's the latest run showing this.) As I said before, I think this makes sense because Cython 0.29.21 only has wheels up through 3.9 so I assume it doesn't support 3.10 or newer. (Even though Cython isn't running here, I assume something in the .c files it generates isn't compatible with Python 3.10+.)

So, we have MPF MC 0.57 technically running now. Yay! But just Python 3.8 and 3.9, and on my local machine (x86 mac) I do not actually have audio. (Based on the mc_demo.) The MC runs fine, no crashing or errors, just no sound.

So it seems the game plan should be something like:

• Get a newer version of Cython to generate newer .c files
• Get the wheel building process to actually compile those
• Run a test and make sure sound is really happening

Again I'm probably done with this unless someone has a suggestion of what to try next.

[toomanybrians]

I got Cython building for Python 3.10 and 3.11. The way I did it was on my x86 Mac. I manually installed the build-system requires modules and then manually ran python3.9 setup.py build_ext --inplace to generate the .c files. This worked fine for 0.29.21, so I updated Cython to 0.29.36 (which also supports Python 3.10 and 3.11), deleted the .c files, ran it again, and it worked fine. I committed those .c files and all the builds worked too. (that was dev 4)

(I had forgotten that even though the setup is pyproject.toml now, Cython still requires a setup.py for its stuff.)

The audio still does not seem to work. I'm getting a bunch of failing tests so I'll go through those, maybe that will fix it. I'm going to keep this issue open for now.

[toomanybrians]

I'm going to close this since it seems to be working. Audio is maybe not working across all platforms? I'll start a new issue to track that. It seems to work fine for some, and not others, so it might be related to whether the prereqs are successfully installing and/or other versions of products installed.