-
Notifications
You must be signed in to change notification settings - Fork 25.6k
[vulkan][ci] Vulkan tests running on linux build via swiftshader (added to docker) #42614
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
[ghstack-poisoned]
💊 CI failures summary and remediationsAs of commit d6f20f6 (more details on the Dr. CI page):
🚧 1 fixed upstream failure:These were probably caused by upstream breakages that were already fixed.
Please rebase on the
|
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
@seemethere looks like another long compilation step for docker builders getting added again... |
aten/src/ATen/test/vulkan_test.cpp
Outdated
auto t_in = at::rand({1, C, H, W}, at::device(at::kCPU).dtype(at::kFloat)); | ||
auto t_w = at::rand({OC, C, KH, KW}, at::device(at::kCPU).dtype(at::kFloat)); | ||
auto t_b = at::zeros({OC}, at::device(at::kCPU).dtype(at::kFloat)); | ||
auto stride = c10::IntArrayRef{1}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oof, this definitely shouldn't have been written this way haha
This looks plausible. I'll let other people take a look. |
Yes, swiftshader build includes building llvm-10 that takes ~30 minutes with current configuration, but if to change docker build job's circleci Could we change it permanently to |
Is swiftshader something that could theoretically be built once and then used by multiple things? We recently just got rid of the Perhaps we can take a similar approach to |
I will try to do the same with swiftshader. |
…shader (added to docker)" Vulkan backend linux build (USE_VULKAN=1) and running Vulkan tests using software Vulkan implementation via [swiftshader](https://github.com/google/swiftshader) Vulkan linux build needs VulkanSdk and running tests needs Swiftshader. swiftshader needs to be compiled using clang toolchain, added them to bionic-clang-9 docker image. VulkanSdk will be downloaded from aws; Swiftshader is cloned from github, as it has many submodules , commit hash is fixed in install_swiftshader script. To pass all the tests: Disabled adaptive_avg_pool2d_2 as it needs at::view which will be landed in #42676 and after that can be enabled Change strides, padding, dilation params in tests to vector Docker image rebuild: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6700311 Vulkan Linux Build: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6701604 Vulkan Linux Test: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6703026 [ghstack-poisoned]
…shader (added to docker)" Vulkan backend linux build (USE_VULKAN=1) and running Vulkan tests using software Vulkan implementation via [swiftshader](https://github.com/google/swiftshader) Vulkan linux build needs VulkanSdk and running tests needs Swiftshader. swiftshader needs to be compiled using clang toolchain, added them to bionic-clang-9 docker image. VulkanSdk will be downloaded from aws; Swiftshader is cloned from github, as it has many submodules , commit hash is fixed in install_swiftshader script. To pass all the tests: Disabled adaptive_avg_pool2d_2 as it needs at::view which will be landed in #42676 and after that can be enabled Change strides, padding, dilation params in tests to vector Docker image rebuild: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6700311 Vulkan Linux Build: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6701604 Vulkan Linux Test: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6703026 [ghstack-poisoned]
I put prebuilt swiftshaders to aws (it is about 900mb) and unpacking of it instead of building works fine, tests passed. Is it ok to leave it on aws and download it from there instead of repeating the whole setup of llvm built on docker image in pytoch/builder? |
That should be fine as well as long as we have a way to recreate them if need be. |
…shader (added to docker)" Vulkan backend linux build (USE_VULKAN=1) and running Vulkan tests using software Vulkan implementation via [swiftshader](https://github.com/google/swiftshader) Vulkan linux build needs VulkanSdk and running tests needs Swiftshader. swiftshader needs to be compiled using clang toolchain, added them to bionic-clang-9 docker image. VulkanSdk will be downloaded from aws; Swiftshader is cloned from github, as it has many submodules , commit hash is fixed in install_swiftshader script. To pass all the tests: Disabled adaptive_avg_pool2d_2 as it needs at::view which will be landed in #42676 and after that can be enabled Change strides, padding, dilation params in tests to vector Docker image rebuild: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6700311 Vulkan Linux Build: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6701604 Vulkan Linux Test: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6703026 Differential Revision: [D23174038](https://our.internmc.facebook.com/intern/diff/D23174038) [ghstack-poisoned]
…shader (added to docker)" Vulkan backend linux build (USE_VULKAN=1) and running Vulkan tests using software Vulkan implementation via [swiftshader](https://github.com/google/swiftshader) Vulkan linux build needs VulkanSdk and running tests needs Swiftshader. swiftshader needs to be compiled using clang toolchain, added them to bionic-clang-9 docker image. VulkanSdk will be downloaded from aws; Swiftshader is cloned from github, as it has many submodules , commit hash is fixed in install_swiftshader script. To pass all the tests: Disabled adaptive_avg_pool2d_2 as it needs at::view which will be landed in #42676 and after that can be enabled Change strides, padding, dilation params in tests to vector Docker image rebuild: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6700311 Vulkan Linux Build: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6701604 Vulkan Linux Test: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6703026 Differential Revision: [D23174038](https://our.internmc.facebook.com/intern/diff/D23174038) [ghstack-poisoned]
…shader (added to docker)" Vulkan backend linux build (USE_VULKAN=1) and running Vulkan tests using software Vulkan implementation via [swiftshader](https://github.com/google/swiftshader) Vulkan linux build needs VulkanSdk and running tests needs Swiftshader. swiftshader needs to be compiled using clang toolchain, added them to bionic-clang-9 docker image. VulkanSdk will be downloaded from aws; Swiftshader is cloned from github, as it has many submodules , commit hash is fixed in install_swiftshader script. To pass all the tests: Disabled adaptive_avg_pool2d_2 as it needs at::view which will be landed in #42676 and after that can be enabled Change strides, padding, dilation params in tests to vector Docker image rebuild: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6700311 Vulkan Linux Build: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6701604 Vulkan Linux Test: https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6703026 Differential Revision: [D23174038](https://our.internmc.facebook.com/intern/diff/D23174038) ci-all check: #43364 [ghstack-poisoned]
@IvanKobzarev merged this pull request in ad8294d. |
Stack from ghstack:
Vulkan backend linux build (USE_VULKAN=1) and running Vulkan tests using software Vulkan implementation via swiftshader
Vulkan linux build needs VulkanSdk and running tests needs Swiftshader.
swiftshader needs to be compiled using clang toolchain, added them to bionic-clang-9 docker image.
VulkanSdk will be downloaded from aws;
Swiftshader is cloned from github, as it has many submodules , commit hash is fixed in install_swiftshader script.
To pass all the tests:
Disabled adaptive_avg_pool2d_2 as it needs at::view which will be landed in #42676 and after that can be enabled
Change strides, padding, dilation params in tests to vector
Docker image rebuild:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6700311
Vulkan Linux Build:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6701604
Vulkan Linux Test:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6703026
Differential Revision: D23174038
ci-all check: #43364