`multipart` dependency conflicts with `python-multipart`, making impossible to install `treq` and `starlette` on the same system

[mgorny]

There are currently two multipart implementations on PyPI: multipart and python-multipart. Both install the same multipart package, and therefore cannot be installed simultaneously.

Both starlette and synapse use (apparently more popular) python-multipart. This is probably understandable, given that prior to June, the last release of multipart was in 2021. However, treq now started using multipart, which means it is no longer possible to install treq and starlette on the same system.

Please consider using python-multipart for the sake of compatibility instead.

[glyph]

Perhaps python-multipart should change its import name, since it postdates multipart? The apache license isn't a dealbreaker, but it's nice that multipart is MIT, has more recent commits, has a push parser in progress, and has currently-passing CI.

Neither appears to have any documentation, which isn't great, but multipart's README is more comprehensive and doesn't contain broken links

[mgorny]

python-multipart changing at this point would break all its reverse dependencies.

Honestly, this isn't my fight. All I'm pointing out that this change is making synapse not correctly installable, since it simultaneously depends on python-multipart and treq (which now depends on multipart).

Furthermore, since multipart is just a .py file, while python-multipart is a full package, if both are installed (pip doesn't do anything to prevent that) the latter will take precedence.

Yes, Python ecosystem is completely broken by design.

[glyph]

python-multipart changing at this point would break all its reverse dependencies.

Honestly, this isn't my fight. All I'm pointing out that this change is making synapse not correctly installable, since it simultaneously depends on python-multipart and treq (which now depends on multipart).

Yeah, thanks for letting us know.

Furthermore, since multipart is just a .py file, while python-multipart is a full package, if both are installed (pip doesn't do anything to prevent that) the latter will take precedence.

OK, good to know.

Yes, Python ecosystem is completely broken by design.

"completely" is a strong word there, but this is certainly a flaw.

[glyph]

Given that things are in a broken state currently, I am going to do the following:

• vendor multipart since (as you note) it is a single file, so we don't need to change the code for an immediate fix
• file the bug upstream with multipart, here: Name conflict with https://github.com/Kludex/python-multipart defnull/multipart#53
• file the bug upstream with python-multipart, here: package fight! rename to avoid import conflict with https://github.com/defnull/multipart/ Kludex/python-multipart#152

[glyph]

@mgorny OK, vendoring branch done. Perhaps if @twm has some time we can get a review & release soon.

[defnull]

I answered the request for renaming multipart.py in the issue linked above. tl;dr: no

Vendoring seems to be a sensible workaround for now, given how easy it is with a single-file no-dependency module. I'd suggest watching the github project for new releases though, as we have a pretty nice new sansio (non-blockling) parser almost ready that is significantly faster than the current implementation (or werkzeug) and also more predictable in memory and CPU consumption. Documentation will also improve.

[glyph]

@defnull For what it's worth, I'm inclined to agree with your reasoning, but we are in this situation due to decisions of others that I have relatively limited influence over :). We'll see what the python-multipart maintainers have to say. Best-case scenario here would be to merge the projects, since it's not clear to me that there's a great advantage of one over the other?

[defnull]

Merging both projects is also not an option, as the APIs are completely different. The only solution I can think of is for python-multipart to give up their conflicting package name and release a new version (with a major version bump, because it's a breaking change), then deprecate the old release and package. But no one can force anyone to do anything in this ecosystem.

Anyway, multipart is in a way better shape now. I'm actually happy enough with the current state that I'm planning a 1.0 release. That might change the 'popularity' aspect in the future a bit. Maybe. I'm really bad at promoting my own projects, so we'll see.

[Kludex]

This issue looks like a competition to see which project is doing better... "Which one has better documentation?" "Which one has a better parse?"

If it's possible to do what python-multipart did, then the problem is with the ecosystem itself. 🤷‍♂️

Anyway, it looks like this was solved. @mgorny we've interacted on many issues already, and I think I've always tried to be helpful. If you need something else here, let me know. 👍

[twm]

Hi @Kludex, treq maintainer here.

Anyway, it looks like this was solved. @mgorny we've interacted on many issues already, and I think I've always tried to be helpful. If you need something else here, let me know. 👍

To be clear, this has only been solved temporarily, for treq in the PyPI ecosystem. Vendoring multipart is not a long-term solution (there has already been a release — our vendored copy is out of date!). Nor is it a complete solution: when Debian goes to package treq I expect that they will un-vendor multipart and we'll be right back to the same conflict.

Can you offer some timeline for when you plan to merge python-multipart into Starlette? I don't see an open issue for doing so but I'd like to know when I can un-vendor multipart.

[Kludex]

I can't offer timeline. Sorry.