Publishing to PyPI

[j2kun]

(Migrating from a short discord discussion)

Our group over at HEIR is adding experimental support for a torch frontend, using torch-mlir as one supported method to get from torch to linalg on tensors.

Our main issue is that it's a bit clumsy to get our community started with torch-mlir, since pip install torch-mlir doesn't work without finding the right dev wheel. Moreover, I can't seem to get bazel (with rules_python) to properly depend on torch-mlir's dev wheels. I'm not sure what the exact problem is, but I suspect it's expecting some extra manifest information that comes from PyPI by default.

This led me to ask: what is stopping torch-mlir from pushing the current dev wheels to PyPI? Is it a matter of bandwidth (and if so, might I provide some help)? And is torch-mlir at a decent enough point to start cutting non-dev releases?

[AlexanderViand]

Over at Belfort Labs, we've wanted this so much we've gone ahead and published our own (fork-based) package https://pypi.org/project/belfort-torch-mlir/ :)

Since we weren't familiar with the existing build workflow (and it seemed to have some significant issues, including no macOS support), we actually based this on the release workflow from HEIR, which is pretty proven and robust at this point: https://github.com/belfortlabs/torch-mlir/pull/3/changes

[j2kun]

I would be willing to try to port @AlexanderViand's PyPI release process to torch-mlir, if the torch-mlir maintainers are open to it.

[sahas3]

Oh this is great to know, I think publishing to pypi will be quite helpful for new users as it will lower the barrier to adoption. I'd prefer to use https://github.com/llvm/torch-mlir-release and integrate the PyPI release process as a workflow there -- adding macOS has been in my todo for a while but bandwidth is a problem 😄

cc: @marbre, @zjgarvey, @sjarus

[zjgarvey]

I personally don't have any context on the wheel releases. There is the torch-mlir-release repo, but why this is handled in a separate repo is unclear to me. I would be more than happy to have your team manage this part of the project if you have experience with PyPi releases and an active need to maintain them.

Sorry this took a while to come back to, thanks @sahas3 for the ping.

[j2kun]

There is the torch-mlir-release repo, but why this is handled in a separate repo is unclear to me.

The README on that repo says it's for access control /shrug. I can manage the split repo for the time being, and once the process is established we can migrate it if need be. Currently testing the release locally so if all goes well I should be able to post some PRs maybe tomorrow.

[j2kun]

Cf. llvm/torch-mlir-release#32 and #4639

[j2kun]

Also, does anyone have the ability to add me as a maintainer of https://pypi.org/project/torch-mlir/ ? I am https://pypi.org/user/j2kun/

[marbre]

@silvasean, do you still have access to https://pypi.org/project/torch-mlir/ and are willing to transfer or add new maintainers?