Skip to content

Upgrade xnnpack to latest #25275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

fanchenkong1
Copy link
Contributor

Description

This change is based on #25135.

Upgrade xnnpack and several related third-party dependencies, including pthreadpool, cpuinfo, and kleidiai. This change also updates the xnnpack execution provider code to accommodate changes in the xnnpack api. Average pooling qu8 is removed as the corresponding microkernel seems no longer exist in xnnpack.

@fanchenkong1
Copy link
Contributor Author

@fs-eire , There seems to be quite a lot of download failures from vcpkg during the build. Do you have any suggestions on how to address those issues?

@snnn snnn closed this Jul 3, 2025
@snnn snnn reopened this Jul 3, 2025
@snnn snnn closed this Jul 3, 2025
@snnn snnn reopened this Jul 3, 2025
@snnn snnn closed this Jul 3, 2025
@snnn snnn reopened this Jul 3, 2025
@snnn snnn added the ep:Xnnpack issues related to XNNPACK EP label Jul 3, 2025
@snnn snnn marked this pull request as ready for review July 3, 2025 19:06
@snnn
Copy link
Member

snnn commented Jul 3, 2025

@fanchenkong1 , I have uploaded the files to the internal mirror. Now please check the remaining errors.

@fanchenkong1
Copy link
Contributor Author

@fanchenkong1 , I have uploaded the files to the internal mirror. Now please check the remaining errors.

Thanks, that’s very helpful!

@fs-eire
Copy link
Contributor

fs-eire commented Jul 6, 2025

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows x64 QNN CI Pipeline

Copy link

Azure Pipelines successfully started running 5 pipeline(s).

@fs-eire
Copy link
Contributor

fs-eire commented Jul 7, 2025

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows x64 QNN CI Pipeline

Copy link

Azure Pipelines successfully started running 5 pipeline(s).

@fanchenkong1 fanchenkong1 marked this pull request as draft July 7, 2025 08:59
@snnn
Copy link
Member

snnn commented Jul 7, 2025

Please format the code with clang-format

@fs-eire
Copy link
Contributor

fs-eire commented Jul 7, 2025

There are QNN CI pipeline failures:

Build Logs
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(56): error A2034: unknown opcode: [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(58): error A2034: unknown opcode: inch [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(63): error A2034: unknown opcode: cntw [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(73): error A2034: unknown opcode: cntw [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(75): error A2034: unknown opcode: cntw [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(81): error A2034: unknown opcode: ptrue [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(82): error A2034: unknown opcode: whilelt [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(84): error A2034: unknown opcode: whilelt [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(86): error A2034: unknown opcode: whilelt [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(87): error A2034: unknown opcode: whilelt [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(119): error A2034: unknown opcode: inch [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(128): error A2034: unknown opcode: inch [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(132): error A2034: unknown opcode: whilelt [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(161): error A2034: unknown opcode: [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(188): error A2034: unknown opcode: whilelt [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(189): error A2034: unknown opcode: inch [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(192): error A2034: unknown opcode: inch [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(194): error A2034: unknown opcode: [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(195): error A2034: unknown opcode: whilelt [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(222): error A2034: unknown opcode: [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(247): error A2034: unknown opcode: whilelt [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(251): error A2034: unknown opcode: inch [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(253): error A2034: unknown opcode: [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(254): error A2034: unknown opcode: inch [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(259): error A2034: unknown opcode: whilelt [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(270): error A2034: unknown opcode: [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(279): error A2034: unknown opcode: whilelt [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(280): error A2034: unknown opcode: whilelt [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(290): error A2034: unknown opcode: [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(293): error A2034: unknown opcode: whilelt [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-src\kai\ukernels\matmul\pack\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S(303): error A2034: unknown opcode: [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\VC\v170\BuildCustomizations\marmasm.targets(140,5): error MSB3721: The command "armasm64.exe kleidiai.dir\RelWithDebInfo\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.S -o "kleidiai.dir\RelWithDebInfo\kai_lhs_imatmul_pack_x16p2vlx2_x16p_sme_asm.obj" " exited with code 1. [D:\a\_work\1\b\RelWithDebInfo\_deps\kleidiai-build\kleidiai.vcxproj]

@fanchenkong1
Copy link
Contributor Author

Thanks, I will take a look.

@fanchenkong1
Copy link
Contributor Author

There are QNN CI pipeline failures:

Build Logs

I reproduced the ARM64 QNN CI run on my local device and find that the same “unknown opcode” errors appear as soon as I roll MSVC back to 19.39. It seems that the older toolset rejects the SVE/SVE2/SME assembly used in Kleidiai.

Kleidiai does not appear to expose a switch for selecting the kernel set. Could you let me know which way you’d prefer to resolve the failure?

  • Upgrade the CI bot to MSVC 19.40 or newer
  • Turn off Kleidiai in XNNPACK. So that we can probably stay on the current kleidiai version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ep:Xnnpack issues related to XNNPACK EP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants