Add the fusion of quantized batchnorm and relu #34795

lly-zero-one · 2020-03-16T06:45:24Z

As title, we want to support the BN2d_relu and BN3d_relu

Test to be added!

fix the test Add the batch_norm3d and batch_norm3d_relu fused quantized op fix linter make the type be consistent Add the fusion of quantized batchnorm and relu

dr-ci · 2020-03-16T06:47:16Z

💊 CircleCI build failures summary and remediations

As of commit 3554ef6 (more details on the Dr. CI page):

3/3 failures introduced in this PR

🕵️ 3 new failures recognized by patterns

The following build failures do not appear to be due to upstream breakages:

pytorch_linux_xenial_py3_6_gcc5_4_build (1/3)

Step: "Build" (full log | pattern match details)

Automatic merge failed; fix conflicts and then commit the result.

CONFLICT (add/add): Merge conflict in .circleci/config.yml 
Auto-merging .circleci/config.yml 
CONFLICT (add/add): Merge conflict in .circleci/cimodel/data/pytorch_build_definitions.py 
Auto-merging .circleci/cimodel/data/pytorch_build_definitions.py 
CONFLICT (add/add): Merge conflict in .circleci/cimodel/data/pytorch_build_data.py 
Auto-merging .circleci/cimodel/data/pytorch_build_data.py 
CONFLICT (add/add): Merge conflict in .circleci/cimodel/data/caffe2_build_data.py 
Auto-merging .circleci/cimodel/data/caffe2_build_data.py 
CONFLICT (add/add): Merge conflict in .circleci/cimodel/data/binary_build_data.py 
Auto-merging .circleci/cimodel/data/binary_build_data.py 
Automatic merge failed; fix conflicts and then commit the result.

pytorch_macos_10_13_py3_test (2/3)

Step: "Test" (full log | pattern match details)

Mar 19 14:51:37 RuntimeError: test_quantized failed! Received signal: SIGIOT

Mar 19 14:51:37 frame #0: c10::Error::Error(c10::SourceLocation, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 135 (0x1026ceee7 in libc10.dylib) 
Mar 19 14:51:37 frame #1: at::Context::setQEngine(c10::QEngine) + 487 (0x116816367 in libtorch_cpu.dylib) 
Mar 19 14:51:37 frame #2: THPModule_setQEngine(_object*, _object*) + 75 (0x115c71cdb in libtorch_python.dylib) 
Mar 19 14:51:37 <omitting python frames> 
Mar 19 14:51:37  
Mar 19 14:51:37 Traceback (most recent call last): 
Mar 19 14:51:37   File "test/run_test.py", line 674, in <module> 
Mar 19 14:51:37     main() 
Mar 19 14:51:37   File "test/run_test.py", line 667, in main 
Mar 19 14:51:37     raise RuntimeError(message) 
Mar 19 14:51:37 RuntimeError: test_quantized failed! Received signal: SIGIOT 
Mar 19 14:51:38 + cleanup 
Mar 19 14:51:38 + retcode=1 
Mar 19 14:51:38 + set +x

pytorch_windows_vs2019_py36_cuda10.1_build (3/3)

Step: "Build" (full log | pattern match details)

C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\algorithm(2749): error C2780: '_OutIt std::move(_InIt,_InIt,_OutIt)': expects 3 arguments - 1 provided

        with
        [
            T=int64_t,
            _Ty=c10::IValue,
            _RanIt=c10::impl::ListIterator<int64_t,std::_Vector_iterator<std::_Vector_val<std::_Simple_types<c10::IValue>>>>
        ]
..\caffe2\operators\experimental\c10\cpu\expand_dims_cpu.cc(17): note: while compiling class template member function 'void caffe2::`anonymous-namespace'::expand_dims_cpu<float>::operator ()(const at::Tensor &,const at::Tensor &,c10::List<int64_t>)'
C:\Users\circleci\project\aten\src\ATen/core/boxing/kernel_functor.h(276): note: see reference to function template instantiation 'void caffe2::`anonymous-namespace'::expand_dims_cpu<float>::operator ()(const at::Tensor &,const at::Tensor &,c10::List<int64_t>)' being compiled
..\caffe2\operators\experimental\c10\cpu\expand_dims_cpu.cc(60): note: see reference to class template instantiation 'caffe2::`anonymous-namespace'::expand_dims_cpu<float>' being compiled
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\algorithm(2749): error C2672: 'std::move': no matching overloaded function found
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\algorithm(2749): error C2780: '_OutIt std::move(_InIt,_InIt,_OutIt)': expects 3 arguments - 1 provided
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\xutility(3939): note: see declaration of 'std::move'
Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28610.4 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

AVE_AVX2_CPU_DEFINITION /MD /O2 /Ob2 /DNDEBUG /w /EHa /bigobj -DNDEBUG   -DCUDA_HAS_FP16=1 -DUSE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD /Z7 /EHa /DNOMINMAX /wd4267 /wd4251 /wd4522 /wd4838 /wd4305 /wd4244 /wd4190 /wd4101 /wd4996 /wd4275 /bigobj -O2 -DCAFFE2_BUILD_MAIN_LIB -DONNX_BUILD_MAIN_LIB -std:c++14 /showIncludes /Focaffe2\CMakeFiles\torch_cpu.dir\operators\rnn\recurrent_network_op.cc.obj /Fdcaffe2\CMakeFiles\torch_cpu.dir\ /FS -c ..\caffe2\operators\rnn\recurrent_network_op.cc 
Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28610.4 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

 -DHAVE_AVX2_CPU_DEFINITION /MD /O2 /Ob2 /DNDEBUG /w /EHa /bigobj -DNDEBUG   -DCUDA_HAS_FP16=1 -DUSE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD /Z7 /EHa /DNOMINMAX /wd4267 /wd4251 /wd4522 /wd4838 /wd4305 /wd4244 /wd4190 /wd4101 /wd4996 /wd4275 /bigobj -O2 -DCAFFE2_BUILD_MAIN_LIB -DONNX_BUILD_MAIN_LIB -std:c++14 /showIncludes /Focaffe2\CMakeFiles\torch_cpu.dir\quantization\server\conv_relu_op.cc.obj /Fdcaffe2\CMakeFiles\torch_cpu.dir\ /FS -c ..\caffe2\quantization\server\conv_relu_op.cc 
Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28610.4 for x64

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

This comment has been revised 24 times.

facebook-github-bot

@lly-zero-one has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot

@lly-zero-one has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

supriyar · 2020-03-16T17:00:10Z

Do we not need changes in the python side to do the fusion of these ops as well?

lly-zero-one · 2020-03-16T17:45:39Z

Do we not need changes in the python side to do the fusion of these ops as well?

Yes, I have another PR in my sandbox. It will be together with conv3d and the quantization flow change.

supriyar · 2020-03-16T22:15:16Z

Can we merge this PR with the one that has the test for it? I'm wary of enabling fused Relu without testing it.

lly-zero-one · 2020-03-16T22:23:55Z

Can we merge this PR with the one that has the test for it? I'm wary of enabling fused Relu without testing it.

I am adding a bn3drelu test to this PR.

facebook-github-bot

@lly-zero-one has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

test/test_quantized.py

supriyar

LGTM

facebook-github-bot

@lly-zero-one is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

mruberry · 2020-03-20T01:24:28Z

Unlanding. This appears to have broken the the Mac build:

Mar 19 16:57:02 Generating XML reports...
Mar 19 16:57:02 Running test_quantized ... [2020-03-19 16:57:02.241228]
Mar 19 16:57:03 
Mar 19 16:57:03 Running tests...
Mar 19 16:57:03 ----------------------------------------------------------------------
Mar 19 16:58:15 s.ssssssssss............libc++abi.dylib: terminating with uncaught exception of type c10::Error: quantized engine FBGEMM is not supported (setQEngine at ../aten/src/ATen/Context.cpp:113)
Mar 19 16:58:15 frame #0: c10::Error::Error(c10::SourceLocation, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 135 (0x10c3b0ee7 in libc10.dylib)
Mar 19 16:58:15 frame #1: at::Context::setQEngine(c10::QEngine) + 487 (0x11f7e8dc7 in libtorch_cpu.dylib)
Mar 19 16:58:15 frame #2: THPModule_setQEngine(_object*, _object*) + 75 (0x11ec2b72b in libtorch_python.dylib)
Mar 19 16:58:15 <omitting python frames>
Mar 19 16:58:15 
Mar 19 16:58:15 Traceback (most recent call last):
Mar 19 16:58:15   File "test/run_test.py", line 675, in <module>
Mar 19 16:58:15     main()
Mar 19 16:58:15   File "test/run_test.py", line 668, in main
Mar 19 16:58:15     raise RuntimeError(message)
Mar 19 16:58:15 RuntimeError: test_quantized failed! Received signal: SIGIOT
Mar 19 16:58:15 + cleanup
Mar 19 16:58:15 + retcode=1
Mar 19 16:58:15 + set +x

facebook-github-bot · 2020-03-20T14:29:47Z

@lly-zero-one merged this pull request in 733b631.

Add the 3d upsample op for video model

05b03ff

fix the test Add the batch_norm3d and batch_norm3d_relu fused quantized op fix linter make the type be consistent Add the fusion of quantized batchnorm and relu

lly-zero-one changed the base branch from master to fbcode/warm March 16, 2020 06:46

lly-zero-one changed the base branch from fbcode/warm to master March 16, 2020 06:47

facebook-github-bot reviewed Mar 16, 2020

View reviewed changes

resolve conflict

7fc86f9

lly-zero-one requested a review from supriyar March 16, 2020 07:01

facebook-github-bot reviewed Mar 16, 2020

View reviewed changes

add the op test

d238eea

facebook-github-bot reviewed Mar 19, 2020

View reviewed changes

lly-zero-one requested a review from dskhudia March 19, 2020 18:06

supriyar reviewed Mar 19, 2020

View reviewed changes

test/test_quantized.py Outdated Show resolved Hide resolved

rm the qnnpack engine

3554ef6

lly-zero-one requested a review from supriyar March 19, 2020 21:41

supriyar approved these changes Mar 19, 2020

View reviewed changes

facebook-github-bot reviewed Mar 19, 2020

View reviewed changes

facebook-github-bot closed this in 733b631 Mar 19, 2020

facebook-github-bot added the merged label Mar 20, 2020

mruberry added the Merged label Oct 28, 2020

Add the fusion of quantized batchnorm and relu #34795

Add the fusion of quantized batchnorm and relu #34795

Uh oh!

Conversation

lly-zero-one commented Mar 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Mar 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CircleCI build failures summary and remediations

🕵️ 3 new failures recognized by patterns

pytorch_linux_xenial_py3_6_gcc5_4_build (1/3)

pytorch_macos_10_13_py3_test (2/3)

pytorch_windows_vs2019_py36_cuda10.1_build (3/3)

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

supriyar commented Mar 16, 2020

Uh oh!

lly-zero-one commented Mar 16, 2020

Uh oh!

supriyar commented Mar 16, 2020

Uh oh!

lly-zero-one commented Mar 16, 2020

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

supriyar left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

mruberry commented Mar 20, 2020

Uh oh!

facebook-github-bot commented Mar 20, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

lly-zero-one commented Mar 16, 2020 •

edited

Loading

dr-ci bot commented Mar 16, 2020 •

edited

Loading