forklift splitting

[abitrolly]

There are comments in https://github.com/pypa/warehouse/tree/main/warehouse/forklift that need answers with a reference.

https://github.com/pypa/warehouse/blob/3943226cf1168f5cead40913d42603e9d1f25010/warehouse/forklift/__init__.py#L13-L15.

What does it meant? Make it a proper Django app? Or move it to requirements.txt? What happened to https://github.com/pypa/forklift which is a separate project, but now archived?

What's the problem this feature will solve?

Gain clarity into the state of Upload API and its development roadmap.

[dstufft]

FWIW Warehouse doesn't use Django, but rather Pyramid.

The idea behind that comment was that uploading lives at an entirely different domain/endpoint as the rest of PyPI, and it's request patterns are completely different (completely uncacheable, with fewer requests that are individually larger and longer, etc) that at some point it would probably make sense to pull that code out and make it a whole project of it's own. That would mean that it was "independently" developed, managed, and deployed from Warehouse itself.

That being said, it's a non trivial amount of work to do that, and nobody has begun to undertake it, so currently it's part of Warehouse and will remain so until/if someone undertakes that work.

[abitrolly]

FWIW Warehouse doesn't use Django, but rather Pyramid.

I was fast to judge if it is not Flask, then it must be Django. )

I think I can prove in this issue that it is not possible to split the upload code. By just counting checks that are made here against database and processing that is being done on the uploaded file.

While the upload payload is different, I don't see there the bottleneck here. Perhaps metrics could reveal what can be improved. At least that upfront collection is easier than spending time on writing code that will raise the complexity, but maybe won't bring any significant improvement in metrics.