Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add wheels deploying workflow #43

Merged
merged 7 commits into from
Mar 2, 2024
Merged

Add wheels deploying workflow #43

merged 7 commits into from
Mar 2, 2024

Conversation

alecandido
Copy link
Member

If we're aiming for a release, we should also make sure to release something.

@alecandido
Copy link
Member Author

Since we're in an emergency situation, I dropped the generation of wheels for macos and windows, since (at least macos) has been proven to be incompatible with cuquantum.

It would be nice to have at least Quimb support for all platforms, but I'd face the problem after the release.

(I had a look to Qibojit, but there is no valuable solution in there: simply, the Cuquantum and CuPy dependencies are undeclared - they are only in a suitable Poetry group in Qibo, but it's definitely the wrong place)
https://github.com/qiboteam/qibojit/blob/5af642977fd66a4268608fbe2986e55160a88fe4/pyproject.toml#L22-L27

@alecandido
Copy link
Member Author

In order to install the dependencies, the cupy package has to build from source, and this can happen only in an environment where CUDA is available.
https://github.com/qiboteam/qibotn/actions/runs/8121865123/job/22200696123?pr=43#step:9:126

That would make all workflows failing in the GitHub CI, so I used a pre-built wheel, that can be installed in any supported platform (i.e. Linux), and the CUDA libraries are just dynamically discovered (so, if you don't run CUDA code, their absence won't be discovered as well).

To do this, I had to pick a given CUDA version (at least major), and I went for 12x. If you have any argument for which 11x is preferable, then just let me know.

@liweintu
Copy link
Contributor

liweintu commented Mar 2, 2024

Since we're in an emergency situation, I dropped the generation of wheels for macos and windows, since (at least macos) has been proven to be incompatible with cuquantum.

It would be nice to have at least Quimb support for all platforms, but I'd face the problem after the release.

(I had a look to Qibojit, but there is no valuable solution in there: simply, the Cuquantum and CuPy dependencies are undeclared - they are only in a suitable Poetry group in Qibo, but it's definitely the wrong place) https://github.com/qiboteam/qibojit/blob/5af642977fd66a4268608fbe2986e55160a88fe4/pyproject.toml#L22-L27

Could we use the CUDA availability from a preceding check job to guide the build job, similar to the workflow in rules.yml? I assume a new job for building a quimb-only package might be needed.

@alecandido
Copy link
Member Author

Could we use the CUDA availability from a preceding check job to guide the build job, similar to the workflow in rules.yml? I assume a new job for building a quimb-only package might be needed.

We could, but with no benefit in the short term: the purpose of that check right now is to prevent tests to run at all. If we prevent build to run, we'll be left with no released wheels.

@liweintu
Copy link
Contributor

liweintu commented Mar 2, 2024

We could, but with no benefit in the short term: the purpose of that check right now is to prevent tests to run at all. If we prevent build to run, we'll be left with no released wheels.

That's true as we only have one build job for both quimb and cuquantum. I was thinking if a new job (or step) could be added, such that when CUDA availability is detected as false, it could build a quimb-only package.

@alecandido
Copy link
Member Author

That's true as we only have one build job for both quimb and cuquantum. I was thinking if a new job (or step) could be added, such that when CUDA availability is detected as false, it could build a quimb-only package.

Yes, but I'd try to design anything more complex after the release, to minimize the risk of having no release at all by Tuesday.

Copy link

@yjmaxpayne yjmaxpayne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like silenceing extras is useful, thanks.

@liweintu
Copy link
Contributor

liweintu commented Mar 2, 2024

Yes, but I'd try to design anything more complex after the release, to minimize the risk of having no release at all by Tuesday.

Yup, that makes sense. Proceed to approve and merge this PR.

@liweintu liweintu merged commit 850934b into main Mar 2, 2024
6 checks passed
@liweintu liweintu deleted the wheels-workflow branch March 2, 2024 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants