Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sparse windows on CPU with MKL #102604

Closed
wants to merge 1 commit into from

Conversation

mantaionut
Copy link
Collaborator

@mantaionut mantaionut commented May 31, 2023

Fix #97352.
This PR changes the way the linking to intel MKL is done and updating MKL on Windows to mkl-2021.4.0 .
There are for both conda and pip packages MKL version with which you can link dynamically. mkl-devel contains the static versions of the dlls and MKL contains the needed dlls for the runtime. MKL dlls and static libs starting with 2021.4.0 have the version in their names( for MKL 2023 we have mkl_core.2.dll and for 2021.4.0 we have mkl_core.1.dll) so its possible to have multiple versions installed and it will work properly.
For the wheel build, I added dependency for whell MKL and on conda a dependecy for the conda MKL and on libtorch I copied the MKL binaries in libtorch.
In order to test this PR I have to use custom builder pytorch/builder#1467

cc @alexsamardzic @nikitaved @pearu @cpuhrsch @amjames @bhosmer @jcaip @peterjc123 @mszhanyi @skyline75489 @nbcsm @vladimir-aubrecht @iremyux @Blackhex @cristianPanaite

@pytorch-bot pytorch-bot bot added the release notes: releng release notes category label May 31, 2023
@pytorch-bot
Copy link

pytorch-bot bot commented May 31, 2023

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/102604

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 4156083 with merge base 6049998 (image):

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@mantaionut mantaionut force-pushed the Fix_sparse_windows branch 4 times, most recently from 06a2d02 to 987b456 Compare June 5, 2023 09:37
@mantaionut mantaionut added the ciflow/periodic Trigger jobs ran periodically on master (periodic.yml) on the PR label Jun 5, 2023
@mantaionut mantaionut force-pushed the Fix_sparse_windows branch 2 times, most recently from 4eb772e to b387cd9 Compare June 6, 2023 05:09
@mantaionut mantaionut added the ciflow/binaries Trigger all binary build and upload jobs on the PR label Jun 6, 2023
@mantaionut mantaionut force-pushed the Fix_sparse_windows branch 2 times, most recently from 3769714 to 097ea78 Compare June 11, 2023 17:00
@mantaionut mantaionut force-pushed the Fix_sparse_windows branch 2 times, most recently from f22be71 to fc8b4da Compare June 26, 2023 06:55
@mantaionut mantaionut force-pushed the Fix_sparse_windows branch 4 times, most recently from 85b363a to aaf3d35 Compare June 30, 2023 06:08
@mantaionut mantaionut force-pushed the Fix_sparse_windows branch 11 times, most recently from b2a7af2 to b72a468 Compare August 6, 2023 09:32
@mantaionut mantaionut force-pushed the Fix_sparse_windows branch 9 times, most recently from eb8a99d to b7a417d Compare December 22, 2023 14:15
@mantaionut mantaionut marked this pull request as ready for review December 22, 2023 14:15
@cpuhrsch
Copy link
Contributor

cpuhrsch commented Jan 9, 2024

@mantaionut @malfet - Do we still want to move forward with this?

@mantaionut
Copy link
Collaborator Author

@mantaionut @malfet - Do we still want to move forward with this?

I would like to move forward. But it also depends on pytorch/builder#1467 for which i had to make additional changes after rebasing.

@malfet
Copy link
Contributor

malfet commented Jan 22, 2024

@mantaionut @malfet - Do we still want to move forward with this?

I would like to move forward. But it also depends on pytorch/builder#1467 for which i had to make additional changes after rebasing.

Merged builder PR, please remove temporary changes from this PR and merge it

@mantaionut mantaionut force-pushed the Fix_sparse_windows branch 2 times, most recently from dfabdf7 to 6df47d7 Compare January 22, 2024 23:06
Added support for intel Sparse on Windows
@mantaionut
Copy link
Collaborator Author

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jan 23, 2024
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

pytorchmergebot pushed a commit that referenced this pull request Mar 4, 2024
Follow up on: #102604
Address this comment: #102604 (comment)

Whl metadata for all wheels published to pypi must match, otherwise poetry install will fail see this comment:
#88049 (comment)

Pull Request resolved: #121014
Approved by: https://github.com/malfet
Lourencom pushed a commit to Lourencom/pytorch that referenced this pull request Mar 6, 2024
Follow up on: pytorch#102604
Address this comment: pytorch#102604 (comment)

Whl metadata for all wheels published to pypi must match, otherwise poetry install will fail see this comment:
pytorch#88049 (comment)

Pull Request resolved: pytorch#121014
Approved by: https://github.com/malfet
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ciflow/binaries Trigger all binary build and upload jobs on the PR ciflow/periodic Trigger jobs ran periodically on master (periodic.yml) on the PR ciflow/trunk Trigger trunk jobs on your pull request Merged module: mkl Related to our MKL support module: sparse Related to torch.sparse module: windows Windows support for PyTorch open source release notes: releng release notes category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sparse is not available on Windows
8 participants