Removing AMX build flag by dipanjans12 · Pull Request #16086 · microsoft/onnxruntime

dipanjans12 · 2023-05-24T23:03:58Z

Description

Replacing AMX intrinsics with machine code macro instructions in QGEMM kernel.
Removing AMX build flags for GCC in cmake file.

Motivation and Context

The additional AMX flag in cmake adds an extra layer of dependency on GCC version to use the feature.These changes should allow the usage of the AMX feature with just the CPU ID check.

…EMM kernel. 2. Removing amx build flags for gcc from the cmake file

…r file.

…nstrinsics are now defined in assembly

chenfucn · 2023-05-26T02:40:57Z

/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux OpenVINO CI Pipeline, Linux QNN CI Pipeline, MacOS CI Pipeline

chenfucn · 2023-05-26T02:41:20Z

/azp run ONNX Runtime Web CI Pipeline, Windows ARM64 QNN CI Pipeline, Windows CPU CI Pipeline, Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed

azure-pipelines · 2023-05-26T02:41:28Z

Azure Pipelines successfully started running 7 pipeline(s).

azure-pipelines · 2023-05-26T02:41:51Z

Azure Pipelines successfully started running 9 pipeline(s).

chenfucn · 2023-05-31T17:34:22Z

/azp run ONNX Runtime Web CI Pipeline, Windows ARM64 QNN CI Pipeline, Windows CPU CI Pipeline, Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed

chenfucn · 2023-05-31T17:34:34Z

/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux OpenVINO CI Pipeline, Linux QNN CI Pipeline, MacOS CI Pipeline

azure-pipelines · 2023-05-31T17:35:02Z

Azure Pipelines successfully started running 9 pipeline(s).

azure-pipelines · 2023-05-31T17:35:03Z

Azure Pipelines successfully started running 7 pipeline(s).

chenfucn · 2023-05-31T19:46:00Z

/azp run ONNX Runtime Web CI Pipeline, Windows ARM64 QNN CI Pipeline, Windows CPU CI Pipeline, Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed

chenfucn · 2023-05-31T19:46:15Z

/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux OpenVINO CI Pipeline, Linux QNN CI Pipeline, MacOS CI Pipeline

azure-pipelines · 2023-05-31T19:46:41Z

Azure Pipelines successfully started running 9 pipeline(s).

azure-pipelines · 2023-05-31T19:46:48Z

Azure Pipelines successfully started running 7 pipeline(s).

…ernel because of a gcc bug before gcc v10

yihonglyu · 2023-06-01T11:00:30Z

Please fix successful checks for merge

…config assembly to machine code for Linux

…config function calls for Windows

…of from a .s file. This avoids the need to enumerate all possible tile combinations

chenfucn · 2023-06-15T03:52:27Z

/azp run ONNX Runtime Web CI Pipeline, Windows ARM64 QNN CI Pipeline, Windows CPU CI Pipeline, Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed

chenfucn · 2023-06-15T03:52:43Z

/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux OpenVINO CI Pipeline, Linux QNN CI Pipeline, MacOS CI Pipeline

azure-pipelines · 2023-06-15T03:53:06Z

Azure Pipelines successfully started running 9 pipeline(s).

azure-pipelines · 2023-06-15T03:53:12Z

Azure Pipelines successfully started running 7 pipeline(s).

This reverts commit 681a0d0.

### Description This is to revert two PRs that aim at reducing AMX toolchain requirements. Unfortunately we still have some pipeline issues. #16390 #16086 ### Motivation and Context Looks like gcc link time optimization does not work very well with inline assembly in the above PRs.

dipanjans12 added 6 commits May 10, 2023 08:47

1. Replacing AMX intrinsics with embedded assembly instructions in QG…

024c403

…EMM kernel. 2. Removing amx build flags for gcc from the cmake file

Replacing the _tile_loadconfig intrinsic function with assembly code

9f7abac

Adding amx intrinsic definitions for windows and moving it to a heade…

ff8304f

…r file.

Removing the MLAS_AMX_SUPPORTED flag from the cmake file as the AMX i…

759fdf4

…nstrinsics are now defined in assembly

Removing MLAS_AMX_SUPPORTED flag from platform.cpp and mlasi.h

46ba78a

Replacing the _tile_storeconfig intrinsic function with assembly code

1e39098

dipanjans12 requested a review from a team as a code owner May 24, 2023 23:03

Merge branch 'microsoft:main' into amx-dsengupt

95931af

chenfucn reviewed May 25, 2023

View reviewed changes

Comment thread cmake/onnxruntime_mlas.cmake Outdated

Comment thread onnxruntime/core/mlas/lib/amx_common.h

Merge branch 'microsoft:main' into amx-dsengupt

b99edd5

chenfucn previously approved these changes May 26, 2023

View reviewed changes

removing the commented out build instructions

0e06f6a

dipanjans12 dismissed chenfucn’s stale review via a1e35ff May 30, 2023 23:39

dipanjans12 and others added 2 commits May 31, 2023 07:34

Adding the intrinsic flag for _mm512_loadu_epi32

a1e35ff

Merge branch 'microsoft:main' into amx-dsengupt

75269dd

replacing _mm512_loadu_epi32() with _mm512_loadu_si512() in the amx k…

a50b625

…ernel because of a gcc bug before gcc v10

Converting AMX TDP assembly to machine code for Linux

f2ddb42

dipanjans12 added 4 commits June 7, 2023 09:24

Converting AMX tile_load, tile_store, tile_loadconfig and tile_stored…

c7e3b9d

…config assembly to machine code for Linux

Adding the definitions for AMX TDP, load, store, loadconfig and store…

2e29833

…config function calls for Windows

Fixing the bug for tile_loadconfig and tile_storeconfig

b3f1dfa

Some code cleanup and separating out AMX macros to a new header file

f1dd21b

chenfucn suggested changes Jun 8, 2023

View reviewed changes

Comment thread onnxruntime/core/mlas/lib/amx_common.h Outdated

dipanjans12 added 3 commits June 9, 2023 01:34

Removing AMX macros from AssembleAvxVnni.h

d86ee8b

Calling the instruction macros directly from the header file instead …

9bd0457

…of from a .s file. This avoids the need to enumerate all possible tile combinations

Fixing the unaligned backslashed for a newline

1d5d8f9

chenfucn approved these changes Jun 15, 2023

View reviewed changes

Fixing some of the indentations in the instruction macros.

a7fb7a7

chenfucn merged commit 681a0d0 into microsoft:main Jun 15, 2023

mszhanyi added a commit that referenced this pull request Jun 16, 2023

Revert "Removing AMX build flag (#16086)"

3e3640f

This reverts commit 681a0d0.

chenfucn added a commit that referenced this pull request Jun 22, 2023

Revert "Removing AMX build flag (#16086)"

967442c

This reverts commit 681a0d0.

chenfucn mentioned this pull request Jun 22, 2023

Cfu revertamx #16455

Merged

Conversation

dipanjans12 commented May 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Uh oh!

Uh oh!

Uh oh!

chenfucn commented May 26, 2023

Uh oh!

chenfucn commented May 26, 2023

Uh oh!

azure-pipelines Bot commented May 26, 2023

Uh oh!

azure-pipelines Bot commented May 26, 2023

Uh oh!

chenfucn commented May 31, 2023

Uh oh!

chenfucn commented May 31, 2023

Uh oh!

azure-pipelines Bot commented May 31, 2023

Uh oh!

azure-pipelines Bot commented May 31, 2023

Uh oh!

chenfucn commented May 31, 2023

Uh oh!

chenfucn commented May 31, 2023

Uh oh!

azure-pipelines Bot commented May 31, 2023

Uh oh!

azure-pipelines Bot commented May 31, 2023

Uh oh!

yihonglyu commented Jun 1, 2023

Uh oh!

Uh oh!

chenfucn commented Jun 15, 2023

Uh oh!

chenfucn commented Jun 15, 2023

Uh oh!

azure-pipelines Bot commented Jun 15, 2023

Uh oh!

azure-pipelines Bot commented Jun 15, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dipanjans12 commented May 24, 2023 •

edited

Loading