Make CUDNN an alias of MIOPEN for HIP ops #12278

bddppq · 2018-10-03T04:54:30Z

This is mostly for reusing all the cudnn test cases in our python operator_tests.

bddppq · 2018-10-03T04:55:24Z

triggered tests https://ci.pytorch.org/jenkins/job/caffe2-builds/job/py2-clang7-rocmdeb-ubuntu16.04-trigger-test/193/

bddppq · 2018-10-03T17:55:37Z

@petrex @ashishfarmer There are some tests failing for the MIOPEN conv ops, could you help taking a look? error message, detailed jenkins log

petrex · 2018-10-03T17:59:24Z

@farmer, Ashish<mailto:Ashish.Farmer@amd.com> has a PR for group conv? Maybe we need to land that one first? From: bddppq <notifications@github.com> Reply-To: pytorch/pytorch <reply@reply.github.com> Date: Wednesday, October 3, 2018 at 10:57 AM To: pytorch/pytorch <pytorch@noreply.github.com> Cc: "Yeh, Peter" <Peter.Yeh@amd.com>, Mention <mention@noreply.github.com> Subject: Re: [pytorch/pytorch] Make CUDNN an alias of MIOPEN for HIP ops (#12278) @petrex<https://github.com/petrex> @ashishfarmer<https://github.com/ashishfarmer> There are some tests failing for the MIOPEN conv ops, could you help taking a look? error message<https://gist.github.com/bddppq/9d8db91f2d77ce58e4668cdea1b32739>, detailed jenkins log<https://ci.pytorch.org/jenkins/job/caffe2-builds/job/py2-clang7-rocmdeb-ubuntu16.04-test/33/consoleFull> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#12278 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAqt_63zlfYfh7EFk_wWTN5G0oi9bpWcks5uhPpwgaJpZM4XFTA0>.

ashishfarmer · 2018-10-03T18:00:03Z

This is a regression that was caused by my earlier check in for group conv optimizations. The changes in PR#12273 will fix this issue. Could we land that one first please

petrex · 2018-10-03T18:00:46Z

#12273 @bddppq

bddppq · 2018-10-03T18:15:41Z

@ashishfarmer @petrex ah I see. landing #12273

bddppq · 2018-10-03T21:43:44Z

New tests triggered https://ci.pytorch.org/jenkins/job/caffe2-builds/job/py2-clang7-rocmdeb-ubuntu16.04-trigger-test/265/

bddppq · 2018-10-03T22:49:16Z

@petrex @ashishfarmer there is a new error in test_convolution_layout https://gist.github.com/bddppq/b40b12637acb9b2554412525d1e78094

petrex · 2018-10-03T22:54:26Z

I am guessing { name: "order" s: "NHWC" } @farmer, Ashish<mailto:Ashish.Farmer@amd.com> Can you pls take a look ? thx From: bddppq <notifications@github.com> Reply-To: pytorch/pytorch <reply@reply.github.com> Date: Wednesday, October 3, 2018 at 3:50 PM To: pytorch/pytorch <pytorch@noreply.github.com> Cc: "Yeh, Peter" <Peter.Yeh@amd.com>, Mention <mention@noreply.github.com> Subject: Re: [pytorch/pytorch] Make CUDNN an alias of MIOPEN for HIP ops (#12278) @petrex<https://github.com/petrex> @ashishfarmer<https://github.com/ashishfarmer> there is a new error in test_convolution_layout https://gist.github.com/bddppq/b40b12637acb9b2554412525d1e78094 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#12278 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAqt_2MFry3swnEKm_HfOKLQds73i0wKks5uhT8hgaJpZM4XFTA0>.

ashishfarmer · 2018-10-03T23:17:28Z

@bddppq, It is trying to run NHWC using MIOpen. Currently we don't support NHWC layout in MIOpen.
In the conv_test.py, there needs a check

if _run_in_hip(gc, dc) and (engine == 'CUDNN') and (order != 'NCHW'):
            return

similar to the checks in spatial_bn_op_test.py

bddppq · 2018-10-04T04:59:51Z

@ashishfarmer Thanks for looking! Could you take this PR over to fix all the failed miopen tests?

ashishfarmer · 2018-10-04T16:15:54Z

Sure thing. Will check and enable appropriate tests for miopen

ashishfarmer · 2018-10-06T01:47:15Z

@pytorchbot retest this please

caffe2/python/operator_test/conv_test.py

-                engine_list.append('CUDNN')
+            if _run_in_hip(gc, dc):
+                if order == 'NCHW':
+                    engine_list.append('CUDNN')


bddppq · 2018-10-06T02:42:05Z

Triggered tests https://ci.pytorch.org/jenkins/job/caffe2-builds/job/py2-clang7-rocmdeb-ubuntu16.04-trigger-test/738/

ashishfarmer · 2018-10-09T19:00:27Z

@bddppq, triggered tests after the commits: https://ci.pytorch.org/jenkins/job/caffe2-builds/job/py2-clang7-rocmdeb-ubuntu16.04-trigger-test/1015/
The test that is failing is RNN cell test, and may have nothing to do with CUDNN aliasing. We are looking into the failure, but as far as changes for this PR are concerned, it looks done.

petrex · 2018-10-10T23:00:03Z

failing test : operator_test.rnn_cell_test.RNNCellTest.test_lstm_with_dot_attention_different_dim (from pytest)

IIRC , miopen does not support attention in lstm at this moment. Shall we skip this test?
@rohithkrn pls chime in.

rohithkrn · 2018-10-11T17:57:51Z

This test doesn't use miopen implementation of LSTM.

…s-cudnn-miopen

ashishfarmer · 2018-10-18T04:21:39Z

Triggered tests: https://ci.pytorch.org/jenkins/job/caffe2-builds/job/py2-clang7-rocmdeb-ubuntu16.04-trigger-test/2115/

@bddppq, the PR is ready for review

caffe2/python/operator_test/conv_test.py

@@ -19,6 +19,9 @@
 import unittest
 import os

+def _run_in_hip(gc, dc):


caffe2/python/operator_test/spatial_bn_op_test.py

    def test_spatialbn_test_mode_1d(
            self, size, input_channels, batch_size, seed, order, epsilon,
            inplace, engine, gc, dc):
+        # Currently MIOPEN SpatialBN only supports 2D
+        if _run_in_hip(gc, dc) and engine != "":


caffe2/python/operator_test/spatial_bn_op_test.py

    def test_spatialbn_train_mode_gradient_check_1d(
            self, size, input_channels, batch_size, seed, order, epsilon,
            momentum, engine, gc, dc):
+        # Currently MIOPEN SpatialBN only supports 2D
+        if _run_in_hip(gc, dc) and engine != "":
+            return


caffe2/python/operator_test/spatial_bn_op_test.py

    def test_spatialbn_test_mode_3d(
            self, size, input_channels, batch_size, seed, order, epsilon,
            inplace, engine, gc, dc):
+        # Currently MIOPEN SpatialBN only supports 2D
+        if _run_in_hip(gc, dc) and engine != "":
+            return


…s-cudnn-miopen

bddppq · 2018-10-24T20:56:16Z

triggered c2 rocm tests https://ci.pytorch.org/jenkins/job/caffe2-builds/job/py2-clang7-rocmdeb-ubuntu16.04-trigger-test/3118/

facebook-github-bot

bddppq has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

bddppq added 2 commits October 3, 2018 14:42

Make CUDNN an alias of MIOPEN for HIP ops

6de0ed2

skip 3d miopen conv

dc4efa1

bddppq force-pushed the alias-cudnn-miopen branch from 9ed9f5f to dc4efa1 Compare October 3, 2018 21:43

bddppq added the module: rocm AMD GPU support for Pytorch label Oct 4, 2018

Ashish added 2 commits October 5, 2018 14:57

checks for MIOpen functionality

94a8dc1

run only NCHW for MIOpen 1x1 conv test

2ae3a93

bddppq commented Oct 6, 2018

View reviewed changes

caffe2/python/operator_test/conv_test.py Outdated

engine_list.append('CUDNN')

if _run_in_hip(gc, dc):

if order == 'NCHW':

engine_list.append('CUDNN')

This comment was marked as off-topic.

Sign in to view

Ashish added 3 commits October 8, 2018 13:57

correct conditions for running MIOpen and HIP group conv

d552afd

append MIOPEN engine to the list

d5eecad

Checks for MIOpen support in pool tests

fe9d9dd

Ashish added 3 commits October 17, 2018 10:12

Merge branch 'master' of https://github.com/pytorch/pytorch into alia…

05079d8

…s-cudnn-miopen

Fallback to HIP for multi batch spatial BN and NHWC

4b38b4b

Enabled tests for non MIOpen 1d and 3d spatial BN

afa9554

bddppq commented Oct 24, 2018

View reviewed changes

Ashish added 4 commits October 24, 2018 10:23

Merge branch 'master' of https://github.com/pytorch/pytorch into alia…

e016d34

…s-cudnn-miopen

Moved hip device check code to util file

de57820

Remove unnecessary import

7739ef5

Use assume instead of return

9279a17

facebook-github-bot reviewed Oct 24, 2018

View reviewed changes

facebook-github-bot closed this in ccfaf46 Oct 25, 2018

petrex mentioned this pull request Oct 25, 2018

[Caffe2] Enable MIOPEN tests upstream (for CI) ROCm/pytorch#178

Closed

ezyang added the merged label Jun 26, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make CUDNN an alias of MIOPEN for HIP ops #12278

Make CUDNN an alias of MIOPEN for HIP ops #12278

bddppq commented Oct 3, 2018

bddppq commented Oct 3, 2018 •

edited

Loading

bddppq commented Oct 3, 2018

petrex commented Oct 3, 2018 via email

ashishfarmer commented Oct 3, 2018

petrex commented Oct 3, 2018

bddppq commented Oct 3, 2018

bddppq commented Oct 3, 2018

bddppq commented Oct 3, 2018

petrex commented Oct 3, 2018 via email

ashishfarmer commented Oct 3, 2018

bddppq commented Oct 4, 2018

ashishfarmer commented Oct 4, 2018

ashishfarmer commented Oct 6, 2018

This comment was marked as off-topic.

bddppq commented Oct 6, 2018

ashishfarmer commented Oct 9, 2018 •

edited

Loading

petrex commented Oct 10, 2018

rohithkrn commented Oct 11, 2018

ashishfarmer commented Oct 18, 2018

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

bddppq commented Oct 24, 2018

facebook-github-bot left a comment

Make CUDNN an alias of MIOPEN for HIP ops #12278

Make CUDNN an alias of MIOPEN for HIP ops #12278

Conversation

bddppq commented Oct 3, 2018

bddppq commented Oct 3, 2018 • edited Loading

bddppq commented Oct 3, 2018

petrex commented Oct 3, 2018 via email

ashishfarmer commented Oct 3, 2018

petrex commented Oct 3, 2018

bddppq commented Oct 3, 2018

bddppq commented Oct 3, 2018

bddppq commented Oct 3, 2018

petrex commented Oct 3, 2018 via email

ashishfarmer commented Oct 3, 2018

bddppq commented Oct 4, 2018

ashishfarmer commented Oct 4, 2018

ashishfarmer commented Oct 6, 2018

This comment was marked as off-topic.

bddppq commented Oct 6, 2018

ashishfarmer commented Oct 9, 2018 • edited Loading

petrex commented Oct 10, 2018

rohithkrn commented Oct 11, 2018

ashishfarmer commented Oct 18, 2018

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

bddppq commented Oct 24, 2018

facebook-github-bot left a comment

Choose a reason for hiding this comment

bddppq commented Oct 3, 2018 •

edited

Loading

ashishfarmer commented Oct 9, 2018 •

edited

Loading