Skip to content

Conversation

IvanKobzarev
Copy link
Contributor

@IvanKobzarev IvanKobzarev commented Aug 5, 2020

Stack from ghstack:

Vulkan backend linux build (USE_VULKAN=1) and running Vulkan tests using software Vulkan implementation via swiftshader

Vulkan linux build needs VulkanSdk and running tests needs Swiftshader.
swiftshader needs to be compiled using clang toolchain, added them to bionic-clang-9 docker image.

VulkanSdk will be downloaded from aws;
Swiftshader is cloned from github, as it has many submodules , commit hash is fixed in install_swiftshader script.

To pass all the tests:
Disabled adaptive_avg_pool2d_2 as it needs at::view which will be landed in #42676 and after that can be enabled

Change strides, padding, dilation params in tests to vector

Docker image rebuild:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6700311
Vulkan Linux Build:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6701604
Vulkan Linux Test:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6703026

Differential Revision: D23174038

ci-all check: #43364

@dr-ci
Copy link

dr-ci bot commented Aug 5, 2020

💊 CI failures summary and remediations

As of commit d6f20f6 (more details on the Dr. CI page):



🚧 1 fixed upstream failure:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

Since your merge base is older than viable/strict, run these commands:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

Check out the recency history of this "viable master" tracking branch.


Extra GitHub checks: 1 failed


ci.pytorch.org: 1 failed


This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 181 times.

IvanKobzarev added a commit that referenced this pull request Aug 5, 2020
IvanKobzarev added a commit that referenced this pull request Aug 5, 2020
IvanKobzarev added a commit that referenced this pull request Aug 6, 2020
IvanKobzarev added a commit that referenced this pull request Aug 6, 2020
@IvanKobzarev IvanKobzarev requested a review from kostmo August 14, 2020 00:13
@ezyang ezyang requested review from malfet and seemethere August 14, 2020 03:14
@ezyang
Copy link
Contributor

ezyang commented Aug 14, 2020

@seemethere looks like another long compilation step for docker builders getting added again...

auto t_in = at::rand({1, C, H, W}, at::device(at::kCPU).dtype(at::kFloat));
auto t_w = at::rand({OC, C, KH, KW}, at::device(at::kCPU).dtype(at::kFloat));
auto t_b = at::zeros({OC}, at::device(at::kCPU).dtype(at::kFloat));
auto stride = c10::IntArrayRef{1};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oof, this definitely shouldn't have been written this way haha

@ezyang
Copy link
Contributor

ezyang commented Aug 14, 2020

This looks plausible. I'll let other people take a look.

@IvanKobzarev
Copy link
Contributor Author

IvanKobzarev commented Aug 14, 2020

@seemethere looks like another long compilation step for docker builders getting added again...

Yes, swiftshader build includes building llvm-10 that takes ~30 minutes with current configuration, but if to change docker build job's circleci resource-class to 2xlarge (currently it is large https://github.com/pytorch/pytorch/blob/master/.circleci/verbatim-sources/job-specs/docker_jobs.yml#L8) - It takes about ~15mins.

Could we change it permanently to 2xlarge?

@seemethere
Copy link
Member

Is swiftshader something that could theoretically be built once and then used by multiple things?

We recently just got rid of the llvm 9 build that we had in the docker builds since it only really needed to be built once. #43038

Perhaps we can take a similar approach to swiftshader

@IvanKobzarev
Copy link
Contributor Author

Is swiftshader something that could theoretically be built once and then used by multiple things?

We recently just got rid of the llvm 9 build that we had in the docker builds since it only really needed to be built once. #43038

Perhaps we can take a similar approach to swiftshader

I will try to do the same with swiftshader.

…shader (added to docker)"


Vulkan backend linux build (USE_VULKAN=1) and running Vulkan tests using software Vulkan implementation via [swiftshader](https://github.com/google/swiftshader)

Vulkan linux build needs VulkanSdk and running tests needs Swiftshader.
swiftshader needs to be compiled using clang toolchain, added them to bionic-clang-9 docker image.


VulkanSdk will be downloaded from aws;
Swiftshader is cloned from github, as it has many submodules , commit hash is fixed in install_swiftshader script.

To pass all the tests:
Disabled adaptive_avg_pool2d_2 as it needs at::view which will be landed in #42676 and after that can be enabled

Change strides, padding, dilation params in tests to vector

Docker image rebuild:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6700311
Vulkan Linux Build:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6701604
Vulkan Linux Test:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6703026

[ghstack-poisoned]
…shader (added to docker)"


Vulkan backend linux build (USE_VULKAN=1) and running Vulkan tests using software Vulkan implementation via [swiftshader](https://github.com/google/swiftshader)

Vulkan linux build needs VulkanSdk and running tests needs Swiftshader.
swiftshader needs to be compiled using clang toolchain, added them to bionic-clang-9 docker image.


VulkanSdk will be downloaded from aws;
Swiftshader is cloned from github, as it has many submodules , commit hash is fixed in install_swiftshader script.

To pass all the tests:
Disabled adaptive_avg_pool2d_2 as it needs at::view which will be landed in #42676 and after that can be enabled

Change strides, padding, dilation params in tests to vector

Docker image rebuild:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6700311
Vulkan Linux Build:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6701604
Vulkan Linux Test:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6703026

[ghstack-poisoned]
IvanKobzarev added a commit that referenced this pull request Aug 17, 2020
@IvanKobzarev
Copy link
Contributor Author

Is swiftshader something that could theoretically be built once and then used by multiple things?

We recently just got rid of the llvm 9 build that we had in the docker builds since it only really needed to be built once. #43038

Perhaps we can take a similar approach to swiftshader

@seemethere,

I put prebuilt swiftshaders to aws (it is about 900mb) and unpacking of it instead of building works fine, tests passed.

Is it ok to leave it on aws and download it from there instead of repeating the whole setup of llvm built on docker image in pytoch/builder?

@seemethere
Copy link
Member

Is swiftshader something that could theoretically be built once and then used by multiple things?
We recently just got rid of the llvm 9 build that we had in the docker builds since it only really needed to be built once. #43038
Perhaps we can take a similar approach to swiftshader

@seemethere,

I put prebuilt swiftshaders to aws (it is about 900mb) and unpacking of it instead of building works fine, tests passed.

Is it ok to leave it on aws and download it from there instead of repeating the whole setup of llvm built on docker image in pytoch/builder?

That should be fine as well as long as we have a way to recreate them if need be.

…shader (added to docker)"


Vulkan backend linux build (USE_VULKAN=1) and running Vulkan tests using software Vulkan implementation via [swiftshader](https://github.com/google/swiftshader)

Vulkan linux build needs VulkanSdk and running tests needs Swiftshader.
swiftshader needs to be compiled using clang toolchain, added them to bionic-clang-9 docker image.


VulkanSdk will be downloaded from aws;
Swiftshader is cloned from github, as it has many submodules , commit hash is fixed in install_swiftshader script.

To pass all the tests:
Disabled adaptive_avg_pool2d_2 as it needs at::view which will be landed in #42676 and after that can be enabled

Change strides, padding, dilation params in tests to vector

Docker image rebuild:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6700311
Vulkan Linux Build:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6701604
Vulkan Linux Test:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6703026

Differential Revision: [D23174038](https://our.internmc.facebook.com/intern/diff/D23174038)

[ghstack-poisoned]
IvanKobzarev added a commit that referenced this pull request Aug 20, 2020
…shader (added to docker)"


Vulkan backend linux build (USE_VULKAN=1) and running Vulkan tests using software Vulkan implementation via [swiftshader](https://github.com/google/swiftshader)

Vulkan linux build needs VulkanSdk and running tests needs Swiftshader.
swiftshader needs to be compiled using clang toolchain, added them to bionic-clang-9 docker image.


VulkanSdk will be downloaded from aws;
Swiftshader is cloned from github, as it has many submodules , commit hash is fixed in install_swiftshader script.

To pass all the tests:
Disabled adaptive_avg_pool2d_2 as it needs at::view which will be landed in #42676 and after that can be enabled

Change strides, padding, dilation params in tests to vector

Docker image rebuild:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6700311
Vulkan Linux Build:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6701604
Vulkan Linux Test:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6703026

Differential Revision: [D23174038](https://our.internmc.facebook.com/intern/diff/D23174038)

[ghstack-poisoned]
…shader (added to docker)"


Vulkan backend linux build (USE_VULKAN=1) and running Vulkan tests using software Vulkan implementation via [swiftshader](https://github.com/google/swiftshader)

Vulkan linux build needs VulkanSdk and running tests needs Swiftshader.
swiftshader needs to be compiled using clang toolchain, added them to bionic-clang-9 docker image.


VulkanSdk will be downloaded from aws;
Swiftshader is cloned from github, as it has many submodules , commit hash is fixed in install_swiftshader script.

To pass all the tests:
Disabled adaptive_avg_pool2d_2 as it needs at::view which will be landed in #42676 and after that can be enabled

Change strides, padding, dilation params in tests to vector

Docker image rebuild:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6700311
Vulkan Linux Build:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6701604
Vulkan Linux Test:
https://app.circleci.com/pipelines/github/pytorch/pytorch/200251/workflows/465f911f-f170-47e1-954e-b9605d91abd8/jobs/6703026

Differential Revision: [D23174038](https://our.internmc.facebook.com/intern/diff/D23174038)


ci-all check: #43364

[ghstack-poisoned]
IvanKobzarev added a commit that referenced this pull request Aug 21, 2020
@facebook-github-bot
Copy link
Contributor

@IvanKobzarev merged this pull request in ad8294d.

@facebook-github-bot facebook-github-bot deleted the gh/IvanKobzarev/71/head branch August 24, 2020 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants