Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH, BLD: Add windows arm64 support #17804

Closed
wants to merge 2 commits into from

Conversation

ader1990
Copy link

@ader1990 ader1990 commented Nov 19, 2020

Added support for building numpy for Windows ARM64:

  • fix philox generator to use correct arm64 intrinsic __umulh
  • fix cpu recognition

Test results:

python.exe -c "import numpy; numpy.test(label='fast',verbose=3)

>> 11404 passed, 705 skipped, 1205 deselected, 19 failed in 497.70s (0:08:17)

You need a tweaked version of the msvc compiler for Python, which you can download from here: https://github.com/ader1990/CPython-Windows-ARM64/releases
Source code for Python 3.8.5: https://github.com/ader1990/cpython/tree/v3.8.5-win-arm64 (basically is just a hack to recognise the emulated build tools for ARM64). You can rebase the commit that fixes the build on recent Python versions too: ader1990/cpython@b8c59c9

This fix is an alternative solution for #17747

The macro _M_ARM64 can be used to detect ARM64 processor when using
Visual Studio 2017 or greater.

More info:
https://docs.microsoft.com/en-us/cpp/preprocessor/predefined-macros?view=msvc-160
@eric-wieser
Copy link
Member

This fix is an alternative solution for #17747

I'm not sure I understand the comparison. Are you suggesting that a different fix to #17747 would make this unecessary?

@ader1990
Copy link
Author

This fix is an alternative solution for #17747

I'm not sure I understand the comparison. Are you suggesting that a different fix to #17747 would make this unecessary?

Well, in the sense that there were multiple solutions presented on that issue (like future x64 emulation). Of course, I think the native build solution would be the best.

@charris charris changed the title Add windows arm64 support ENH, BLD: Add windows arm64 support Nov 22, 2020
* _M_ARM64 is defined by VS compiler >= 2017 to build native
* ARM64 packages on Windows
*/
#define NPY_CPU_ARMEL_AARCH64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why ARMEL_AARCH64 ? and not MSVC_AARCH64?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this is for endianess check, the closest was armel_aarch64.

#pragma intrinsic(__umulh)
static NPY_INLINE uint64_t mulhilo64(uint64_t a, uint64_t b, uint64_t *hip) {
uint64_t p0 = a * b;
uint64_t p1 = __umulh(a, b);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this an intrnisic newly defined for arm64 + MSVC? Is there a parallel intrinsic for arm64 + gcc?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the general definition: https://docs.microsoft.com/en-us/cpp/intrinsics/umulh?view=msvc-160
And it is present as an arm64 intrinsic: https://docs.microsoft.com/en-us/cpp/intrinsics/arm64-intrinsics?view=msvc-160

I do not know if there is one for arm+gcc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this better on AMD64 than using _umul128 directly?

@zooba
Copy link
Contributor

zooba commented Dec 3, 2020

FYI, there are "unofficial official" releases of CPython for ARM64 at https://dev.azure.com/Python/cpython/_build?definitionId=21 (browse to a build result, then "Artifacts" and you probably want "layout_full_arm64").

When numpy is ready to claim support for ARM64, I'll be happy to make them official CPython releases. But I don't want you guys to feel like you've been forced into it, which is why I'm holding off for now (and simultaneously trying to get some CI hardware for you... but that's a bit 😫...)

@ader1990
Copy link
Author

ader1990 commented Dec 3, 2020

FYI, there are "unofficial official" releases of CPython for ARM64 at https://dev.azure.com/Python/cpython/_build?definitionId=21 (browse to a build result, then "Artifacts" and you probably want "layout_full_arm64").

When numpy is ready to claim support for ARM64, I'll be happy to make them official CPython releases. But I don't want you guys to feel like you've been forced into it, which is why I'm holding off for now (and simultaneously trying to get some CI hardware for you... but that's a bit 😫...)

Unfortunately, I do not have access to those artifacts. The builds I created using github pipelines should be practically the same with those from that artifact store. Example of build produced using github actions for tag v3.9.0: https://github.com/ader1990/cpython-builder/actions/runs/377798403 . I have tested that build with this numpy patch and it works perfectly.

@zooba
Copy link
Contributor

zooba commented Dec 3, 2020

You can't even download them from the "..." at the right? That ought to work without signing in...

Either way, the main difference should just be that these ones are signed (and will match every release made from python.org).

@charris
Copy link
Member

charris commented Jan 25, 2021

@ader1990 Ping.

@zooba Any update on progress getting the ARM infrastructure in Azure?

@ader1990
Copy link
Author

@ader1990 Ping.

@zooba Any update on progress getting the ARM infrastructure in Azure?

Unfortunately, I cannot access the Azure infra, but the code looks great and it works great IMHO.

@zooba
Copy link
Contributor

zooba commented Jan 26, 2021

Any update on progress getting the ARM infrastructure in Azure?

Let me be careful here... there is progress, but there is no update... yeah I think I can say that much. :)

Base automatically changed from master to main March 4, 2021 02:05
@charris
Copy link
Member

charris commented Jul 27, 2021

@ader1990 I'm closing this in favor of #19513 which has been merged. Thanks for the initial work.

@charris charris closed this Jul 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants