Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow parallel pipeline/shader compilation #1601

Merged
merged 1 commit into from Dec 24, 2021

Conversation

kuhar
Copy link
Contributor

@kuhar kuhar commented Dec 22, 2021

Parallel compilation is controlled by the --num-threads flag.
We default to single-threaded compilation.

The main goal is to excercise the threading in the compiler without
having to run the full ICD on a machine with a GPU. With the thread
sanitizer being enabled in our public CI, we hope that this will catch
most of the potential synchronization issues in LLPC.

Parallel compilation and verbose output are disallowed because of
LLVM's ostream not being thread safe. Even if this wasn't the case,
verbose logs would be interleaved and not very useful.

Refactor stage processing such that ShaderModuleData mergin happens
at the end.

@kuhar
Copy link
Contributor Author

kuhar commented Dec 22, 2021

@amdvlk-admin
Copy link
Collaborator

Test summary for commit fc3c302

Driver commits used in build
  • CWPACK: amd-master 7387247eb9889ddcabbc1053b9c2052e253b088e
  • METROHASH: amd-master 3c566dd9cda44ca7fd97659e0b53ac953f9037d2
  • PAL: dev 44abc48d199dd28369bf7415eee1751c4bf616d2
  • SPVGEN: dev 580ade71d630a304805e9468ef22ad263c37d137
  • XGL: dev 38aa59bcf134d1cb340ddd2459b34b6eda798abe
  • LLVM-PROJECT: amd-gfx-gpuopen-dev c6950e03a4c97beadc88a8bf5bea3dc5aa2649ee
CTS tests (Failed: 0/225376)
  • Built with version 1.2.5.2
  • Rhel 8.2, Gfx10
    • Passed: 32512/56344 (57.7%)
    • Failed: 0/56344 (0.0%)
    • Not Supported: 23832/56344 (42.3%)
    • Warnings: 0/56344 (0.0%)
    Ubuntu 18.04, Gfx9
    • Passed: 32352/56344 (57.4%)
    • Failed: 0/56344 (0.0%)
    • Not Supported: 23991/56344 (42.6%)
    • Warnings: 1/56344 (0.0%)
    Ubuntu 20.04, Gfx8
    • Passed: 32688/56344 (58.0%)
    • Failed: 0/56344 (0.0%)
    • Not Supported: 23656/56344 (42.0%)
    • Warnings: 0/56344 (0.0%)
    Ubuntu 20.04, Gfx103
    • Passed: 35669/56344 (63.3%)
    • Failed: 0/56344 (0.0%)
    • Not Supported: 20675/56344 (36.7%)
    • Warnings: 0/56344 (0.0%)

llpc/tool/amdllpc.cpp Outdated Show resolved Hide resolved
llpc/tool/llpcCompilationUtils.h Outdated Show resolved Hide resolved
Parallel compilation is controlled by the `--num-threads` flag.
We default to single-threaded compilation.

The main goal is to excercise the threading in the compiler without
having to run the full ICD on a machine with a GPU. With the thread
sanitizer being enabled in our public CI, we hope that this will catch
most of the potential synchronization issues in LLPC.

Parallel compilation and verbose output are disallowed because of
LLVM's ostream not being thread safe. Even if this wasn't the case,
verbose logs would be interleaved and not very useful.

Refactor stage processing such that `ShaderModuleData` mergin happens
at the end.
@kuhar
Copy link
Contributor Author

kuhar commented Dec 23, 2021

V1. Fix typos. Rebase.

Copy link
Contributor

@vettoreldaniele vettoreldaniele left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks really good!

@amdvlk-admin
Copy link
Collaborator

Test summary for commit 87cf7c6

Driver commits used in build
  • CWPACK: amd-master 7387247eb9889ddcabbc1053b9c2052e253b088e
  • METROHASH: amd-master 3c566dd9cda44ca7fd97659e0b53ac953f9037d2
  • PAL: dev 44abc48d199dd28369bf7415eee1751c4bf616d2
  • SPVGEN: dev 580ade71d630a304805e9468ef22ad263c37d137
  • XGL: dev 38aa59bcf134d1cb340ddd2459b34b6eda798abe
  • LLVM-PROJECT: amd-gfx-gpuopen-dev c6950e03a4c97beadc88a8bf5bea3dc5aa2649ee
CTS tests (Failed: 0/225376)
  • Built with version 1.2.5.2
  • Rhel 8.2, Gfx10
    • Passed: 32512/56344 (57.7%)
    • Failed: 0/56344 (0.0%)
    • Not Supported: 23832/56344 (42.3%)
    • Warnings: 0/56344 (0.0%)
    Ubuntu 18.04, Gfx9
    • Passed: 32352/56344 (57.4%)
    • Failed: 0/56344 (0.0%)
    • Not Supported: 23991/56344 (42.6%)
    • Warnings: 1/56344 (0.0%)
    Ubuntu 20.04, Gfx8
    • Passed: 32688/56344 (58.0%)
    • Failed: 0/56344 (0.0%)
    • Not Supported: 23656/56344 (42.0%)
    • Warnings: 0/56344 (0.0%)
    Ubuntu 20.04, Gfx103
    • Passed: 35669/56344 (63.3%)
    • Failed: 0/56344 (0.0%)
    • Not Supported: 20675/56344 (36.7%)
    • Warnings: 0/56344 (0.0%)

@kuhar kuhar merged commit 3b8a84c into GPUOpen-Drivers:dev Dec 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants