🚨 Looking for co-maintainers 🚨

[jlaine]

I am progressively returning from a much needed break regarding the opensource projects I maintain, and am looking for a more sustainable way to maintain them.

Some years ago, I stepped up to help @mikeboers maintain PyAV, and over time have become the de-facto sole maintainer for the project. Over time, the workload associated with PyAV has become too much for one person to maintain:

• The workload of triaging of issues and pull requests has increased with the projects' popularity.
• There is an alarming rise of issues along the lines of "please help me do X" (often without the "please") or "my use case doesn't work, figure out what's wrong" or "PyAV doesn't build on FOO"..
• Polishing pull requests is exhausting. More often than not the submitter has done the "satisfying" part of the work (i.e. fixed their problem), expecting the maintainer to finish the work: check upstream documentation, tidy up the code, write unit tests, write documentation, etc.
• Infrastructure needs a lot of maintenance to ensure CI keeps running and binary packages are built properly.
• Periodically forks pop up, diluting whatever good will there is to maintain the actual PyAV project, and creating additional work to merge back changes.

This had lead to a draining cycle of heavy investment on my behalf, burning out, feeling guilty about neglecting the project and repeating. A one-person maintainership is just not sustainable, so I am looking for volunteers to help co-maintain PyAV. To be clear I am not looking for funding, that won't give me more time to spent with my family, run my company or honour my volunteer commitments.

The main traits I value for co-maintainers are:

• A willingness to give. This is about helping out the community, not expediting your pet PRs :)
• A realistic view of what long-term maintainership of opensource means. I would prefer someone who can commit only moderate but regular amounts of time to someone who will go all-in and burn out in 6 months.
• High standards of quality. This means researching standards to find the "right" solution rather than the expedient one, continuing the push towards full test coverage, an attention to keeping the code easy to read and a willingness to write documentation and examples.

If you care about the future of PyAV and are willing to put in the work, please get in touch by responding to this issue!

[WyattBlue]

I think I am a lead candidate for being a co-maintainer. I've been essentially already been running PyAV for the past three weeks in my fork, and I'm happy to port all the good changes back to PyAV in a way that's up to PyAV standards.

As for this point, let me give some advice:

There is an alarming rise of issues along the lines of "please help me do X" (often without the "please") or "my use case doesn't work, figure out what's wrong" or "PyAV doesn't build on FOO"..

I know this pain from maintaining auto-editor. My recommendation is to move all these types of questions to the discussions. They don't belong in the bug tracker. And don't feel bad about not answering most of them, the broader community should tackle this, not one guy.

One last thing, I think @daveisfera will also make an excellent co-maintainer, but I'll let him make his own case.

[daveisfera]

Yes, I'll be glad to help out! I've been using WyattBlue's fork and would recommend merging it in wholesale or doing what's necessary to bring them over

[jlaine]

Thanks for the enthusiastic replies, I have invited @WyattBlue and will add permissions once he accepts the invitation.

@WyattBlue could we arrange a chat so we can put work out some basic rules / goals?

[WyattBlue]

@jlaine Thanks for inviting me on. I'm happy to chat with you.

For goals, I think in the medium-term, we should be #1110 and #1145 merged. I would also recommend dropping Python 3.7 and 3.8

What are the current blockers on those PRs? (besides of course the failing tests, but I think it's because our test are currently flakey)

[jlaine]

So far our policies have been:

• Support all non-EOL Python versions, so dropping Python 3.8 and 3.9 seems premature. To be honest Python versions are not much of a hassle.
• Support FFmpeg in Debian/stable and later. Since bookworm was released this would mean 5.1+ so we can cull some versions here.
• Provide binary wheels for Linux amd64 / aarch64 / i686, macOS x86_64 / arm64, Windows amd64.

In terms of priorities I would suggest for release ASAP:

• Any crashes. The repr methods are the usual suspects judging from reports.
• Get pyav-ffmpeg builds working again for all targets, and at the very least update vpx which suffers from a bad security vulnerability (which means our binary wheels do too). An update to FFmpeg 6 would be welcome too to allow us to include it in our CI matrix.
• Modernise our Cython versions and configuration.
• Any feature justifying your current fork.

Future versions:

• Investigate memory ownership model. Judging from reports against aiortc there may be memory leaks lurking related to packet data ownership.
• Revisit how we generate and host documentation. I don't have any way of updating the docs, @mikeboers ?
• Revisit or possibly eliminate the logging framework. To be honest, from my point of view its main value is populating exceptions. For the rest I'd leave it up to users to decide what they want to do: the fact it calls back into Python is the source of nasty deadlocks.
• Measure test coverage in CI, and refuse PRs which decrease test coverage. We have way too much untested code, which comes back to bite us in the ass during major FFmpeg updates or any kind of refactoring.

Regarding the release process, CI is set up to perform automatic uploads to PyPI as soon as a version tag is pushed.

[daveisfera]

Is there a reason to not merge the fork? It has a lot of good changes and resolves several issues

[hmaarrfk]

I'm willing to help if you would like.

I've done a lot of at conda-forge getting FFMPEG compiled from source, and thus also getting pyav to work well with conda-forge's FFMPEG.

I'm also guilty of having my own fork, but that is due to some performance edge cases I'm very concerned with.

I'm glad to help out where I can. keeping the code to high standards.

I also have a few projects (Picoscope for python) that are similar in their nature: creating a pythonic wrapper for some specialized (hardware) functionality. so I very much understand your challenges.

I for one believe that with the correct style and organization python can be made quite performant.

I'm also a core developer of scitkit image where my first PR was about getting their CIs up and running again.

happy to help where I can. let me know

[jlaine]

Is there a reason to not merge the fork? It has a lot of good changes and resolves several issues

I'm sure it does, so pick apart those good changes, make sure they are backed by automated tests, and let's have some PRs reviewed.

I appreciate the energy @WyattBlue is putting in here, but we haven't had a chance to talk yet and things are getting merged left and right which I'm not entirely confortable with. Nothing dramatic at this stage but I would like to make sure we're off to a good start before just handing over the keys to the house.

[hmaarrfk]

for what it's worth. I've taken on an other project maintenance. you can see my task list:

ukoethe/vigra#515

[jlaine]

for what it's worth. I've taken on an other project maintenance. you can see my task list:

ukoethe/vigra#515

Hi Mark! Please feel free to help out, help is always most welcome. Especially getting pull requests into shape so we can merge them would help a lot.

[FirefoxMetzger]

Nice! Awesome to see that things are moving again 🚀We have a pyAV wrapper/plugin in ImageIO (I'm the maintainer) that is set to replace imageio-ffmpeg once we reach feature parity.

If you, @jlaine, and @WyattBlue are interested we could meet and see if we could and/or want to integrate pyAV into ImageIO's CI/CD pipeline. We have automated weekly releases (except when AV releases a new version-major and breaks our CI :P) and it reduces the maintenance burden quite a bit. I have some DevOps experience and since ImageIO has a bit of funding we could use some of that money to set up a build farm either directly in GH Actions or on AWS as custom runners depending on how heavy the build is.

If that sounds interesting let me know. In that case I'll go, gather emails, and set up a video call.

[jlaine]

@WyattBlue @FirefoxMetzger any chance either of you is attending FOSDEM?

[WyattBlue]

@WyattBlue @FirefoxMetzger any chance either of you is attending FOSDEM?

Can't come. Atlanta is pretty far from Brussels, plus I'm currently busy with real life stuff.

[jlaine]

Can't come. Atlanta is pretty far from Brussels, plus I'm currently busy with real life stuff.

Ah yes, a bit far! I'm kind of cheating, it's a 10mn walk for me :)

[FirefoxMetzger]

Unfortunately, I won't make it neither. I'm Sweden based, but I won't be able to take time off during those days.

[jlaine]

I'm going to close this, as @WyattBlue has been doing a great job :)