This repository was archived by the owner on Jul 1, 2025. It is now read-only.
Update NNPI backend to v0.6.0.5 #4539
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hi @omromano! Thank you for your pull request and welcome to our community.We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks! |
|
Hi @jfix71 , @arunm-git |
jfix71
suggested changes
May 21, 2020
lib/Backends/NNPI/Importer.cpp
Outdated
There was a problem hiding this comment.
nit: Instead of calling new, can we just keep this on the stack/local to this scope?
lib/Backends/NNPI/Importer.cpp
Outdated
There was a problem hiding this comment.
nit: Instead of calling new, can we just keep this on the stack/local to this scope?
There was a problem hiding this comment.
typo: "inferContex" missing t at the end
lib/Backends/NNPI/NNPI.cpp
Outdated
Comment on lines
467
to
469
There was a problem hiding this comment.
Tiniest of nits 🙂
Suggested change
| case Kinded::Kind::NNPICustomDSPNodeKind: | |
| return true; | |
| case Kinded::Kind::NNPICustomIANodeKind: | |
| return true; | |
| case Kinded::Kind::NNPICustomDSPNodeKind: | |
| case Kinded::Kind::NNPICustomIANodeKind: | |
| return true; |
lib/Backends/NNPI/NNPIOptions.cpp
Outdated
There was a problem hiding this comment.
We try to avoid using exceptions. Can you use strtof instead with appropriate error checking?
lib/Backends/NNPI/NNPIOptions.h
Outdated
There was a problem hiding this comment.
spelling: "decleration" -> "declaration"
lib/Backends/NNPI/NNPITracing.cpp
Outdated
lib/Backends/NNPI/NNPITracing.h
Outdated
There was a problem hiding this comment.
nit: triple /// to match other doxygen-style comments
lib/Backends/NNPI/Importer.cpp
Outdated
There was a problem hiding this comment.
This was the change we didn't include from 5.2.5, because it broke A-step, right? Has it been fixed? I noticed that we blacklisted the RWQ-FC test on A-step but not this one. In case we run into issues when integrating CC: @arunm-git @hl475
There was a problem hiding this comment.
@jfix71 We have disabled the failing for A-step (added to block list).
See lib/Backends/NNPI/tests/NNPIOperatorTest.cpp:140
{"rowwiseQuantizedFCTestSymmetric/0", TestBlacklist::A0AnyEngine},
There was a problem hiding this comment.
@hl475 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
yinghai
reviewed
May 22, 2020
| !(glowRowwiseFC->getOffsets()) || | ||
| importer.zeroes(nodeValueName(glowRowwiseFC->getOffsets()).c_str()), | ||
| "Bad offset value", NNPI_INVALID_PARAM); | ||
|
|
There was a problem hiding this comment.
Why we removed all these code?
| uint32_t dilation[SPATIAL_DIMS2] = {1, 1}; // No dilation, default values | ||
|
|
||
| // Add internal tensor for Symlowp input. | ||
| std::string symlowpInputName = |
There was a problem hiding this comment.
In both cases the change is no longer needed (conversion to symlowp before PCQ).
That the support for gemmlowp inputs/outputs for PCQ conv that was enabled for Jack.
facebook-github-bot
pushed a commit
to pytorch/pytorch
that referenced
this pull request
May 29, 2020
Summary: Updating to NNPI Backend version v0.6.0.5 Pull Request resolved: pytorch/glow#4539 Test Plan: Imported from GitHub, without a `Test Plan:` line. **Glow Dev Mode Testing ICEREF** ``` buck test //glow: -- NNPI ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/844425092762063/ **Glow Opt Mode Testing ICEREF** ``` buck test mode/opt //glow: -- NNPI ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/8444249313953808/ **Glow Opt Mode Testing On Device** ``` buck test mode/opt -c glow.nnpi_use_inf_api=true //glow: -- NNPI -j 1 ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/5629499560791922/ **Net_runner defaults OPT ICEREF** ``` buck test mode/opt //glow/fb/test:net_runner_nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/1970324865910264/ **Net_runner defaults OPT CARD** ``` USE_INF_API=1 buck test mode/opt //glow/fb/test:net_runner_nnpi ``` FAIL https://www.internalfb.com/intern/testinfra/testconsole/testrun/6192449501601838/ **Net_runner tiny ctr_mbl_feed_2020q1 OPT CARD** ``` USE_INF_API=1 LD_LIBRARY_PATH=third-party-buck/platform007/build/fb-nnpi/lib ./buck-out/opt/gen/glow/fb/test/net_runner_nnpi --logfiledb ~/test/161676462_0.predictor --opt_net ~/test/debug_optimized_net_0.pb_txt --use_input ~/test/inputs.pb.recordio --glow-nnpi-memory=13000000 --glow-num-devices=2 --ref_impl=glow --test_impls=glow,c2_fp16 --caffe2_fbgemm_fake_fp16_clamp --glow_global_fp16 --glow_clip_fp16 --glow_global_fused_scale_offset_fp16 --fbgemm_deserialize_to_original_format --inference_threads 16 --load_model_by_blob --glow_global_fp16_placeholders --glow_global_fp16_constants --glow_clip_fp16_skip_inputs --glow_nnpi_lower_all_batch_matmul=false --glow_nnpi_num_parallel_chunks=6 --print_latency ``` success **Net_runner FP16 OPT CARD** ``` USE_INF_API=1 LD_LIBRARY_PATH=third-party-buck/platform007/build/fb-nnpi/lib ./buck-out/opt/gen/glow/fb/test/net_runner_nnpi --opt_net buck-out/opt/gen/glow/fb/test/instagram_ctr_model_debug_optimized_net/debug_optimized_net_0.pb_txt --logfiledb buck-out/opt/gen/glow/fb/test/instagram_ctr_model_tiny/105533872_0.predictor --use_input buck-out/opt/gen/glow/fb/test/instagram_ctr_model_inputs_pb/inputs.pb --inference_threads=1 --glow_global_fp16 --glow_global_fused_scale_offset_fp16=1 --glow_clip_fp16 ``` FAIL P131738351 **numerics tests on IceRef** ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_batchmatmul_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/1688849890354625/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_batchnorm_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/1125900069449763/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_fc_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/5910974535833584/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_op_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/5910974535833516/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_int8_ops_nnpinnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/2533274820486771/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_sls_4bit_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/5910974536132077/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_sls_8bit_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/4222124677262742/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_sls_8bit_nnpi_fp32nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/1688849890355133/ Previously disabled tests `test_slws_fused_8bit_rowwise_acc32_nnpi` and `test_small_sls_acc32` still FAIL https://www.internalfb.com/intern/testinfra/testconsole/testrun/7599824383821851/ **numerics tests on Card** ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_batchmatmul_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/4222124677263984/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_batchnorm_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/1125900069450391/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_fc_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/1688849890354230/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_op_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/5910974535833691/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_int8_ops_nnpinnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/3659174723845865/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_sls_4bit_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/5910974536135102/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_sls_8bit_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/2251799842675543/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_sls_8bit_nnpi_fp32nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/2533274820487658/ Previously disabled tests `test_slws_fused_8bit_rowwise_acc32_nnpi` and `test_small_sls_acc32` still FAIL https://www.internalfb.com/intern/testinfra/testconsole/testrun/1407375046178756/ Reviewed By: arunm-git Differential Revision: D21697616 Pulled By: hl475 fbshipit-source-id: 3732324986eb40e644686cdd10e44951678508a7
vdantu
pushed a commit
to vdantu/glow
that referenced
this pull request
Jul 12, 2020
Summary: Updating to NNPI Backend version v0.6.0.5 Pull Request resolved: pytorch#4539 Test Plan: Imported from GitHub, without a `Test Plan:` line. **Glow Dev Mode Testing ICEREF** ``` buck test //glow: -- NNPI ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/844425092762063/ **Glow Opt Mode Testing ICEREF** ``` buck test mode/opt //glow: -- NNPI ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/8444249313953808/ **Glow Opt Mode Testing On Device** ``` buck test mode/opt -c glow.nnpi_use_inf_api=true //glow: -- NNPI -j 1 ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/5629499560791922/ **Net_runner defaults OPT ICEREF** ``` buck test mode/opt //glow/fb/test:net_runner_nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/1970324865910264/ **Net_runner defaults OPT CARD** ``` USE_INF_API=1 buck test mode/opt //glow/fb/test:net_runner_nnpi ``` FAIL https://www.internalfb.com/intern/testinfra/testconsole/testrun/6192449501601838/ **Net_runner tiny ctr_mbl_feed_2020q1 OPT CARD** ``` USE_INF_API=1 LD_LIBRARY_PATH=third-party-buck/platform007/build/fb-nnpi/lib ./buck-out/opt/gen/glow/fb/test/net_runner_nnpi --logfiledb ~/test/161676462_0.predictor --opt_net ~/test/debug_optimized_net_0.pb_txt --use_input ~/test/inputs.pb.recordio --glow-nnpi-memory=13000000 --glow-num-devices=2 --ref_impl=glow --test_impls=glow,c2_fp16 --caffe2_fbgemm_fake_fp16_clamp --glow_global_fp16 --glow_clip_fp16 --glow_global_fused_scale_offset_fp16 --fbgemm_deserialize_to_original_format --inference_threads 16 --load_model_by_blob --glow_global_fp16_placeholders --glow_global_fp16_constants --glow_clip_fp16_skip_inputs --glow_nnpi_lower_all_batch_matmul=false --glow_nnpi_num_parallel_chunks=6 --print_latency ``` success **Net_runner FP16 OPT CARD** ``` USE_INF_API=1 LD_LIBRARY_PATH=third-party-buck/platform007/build/fb-nnpi/lib ./buck-out/opt/gen/glow/fb/test/net_runner_nnpi --opt_net buck-out/opt/gen/glow/fb/test/instagram_ctr_model_debug_optimized_net/debug_optimized_net_0.pb_txt --logfiledb buck-out/opt/gen/glow/fb/test/instagram_ctr_model_tiny/105533872_0.predictor --use_input buck-out/opt/gen/glow/fb/test/instagram_ctr_model_inputs_pb/inputs.pb --inference_threads=1 --glow_global_fp16 --glow_global_fused_scale_offset_fp16=1 --glow_clip_fp16 ``` FAIL P131738351 **numerics tests on IceRef** ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_batchmatmul_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/1688849890354625/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_batchnorm_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/1125900069449763/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_fc_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/5910974535833584/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_op_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/5910974535833516/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_int8_ops_nnpinnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/2533274820486771/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_sls_4bit_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/5910974536132077/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_sls_8bit_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/4222124677262742/ ``` buck test //caffe2/caffe2/contrib/fakelowp/test:test_sls_8bit_nnpi_fp32nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/1688849890355133/ Previously disabled tests `test_slws_fused_8bit_rowwise_acc32_nnpi` and `test_small_sls_acc32` still FAIL https://www.internalfb.com/intern/testinfra/testconsole/testrun/7599824383821851/ **numerics tests on Card** ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_batchmatmul_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/4222124677263984/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_batchnorm_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/1125900069450391/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_fc_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/1688849890354230/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_op_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/5910974535833691/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_int8_ops_nnpinnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/3659174723845865/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_sls_4bit_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/5910974536135102/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_sls_8bit_nnpi_fp16nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/2251799842675543/ ``` buck test -c glow.nnpi_use_inf_api=true //caffe2/caffe2/contrib/fakelowp/test:test_sls_8bit_nnpi_fp32nnpi ``` https://www.internalfb.com/intern/testinfra/testconsole/testrun/2533274820487658/ Previously disabled tests `test_slws_fused_8bit_rowwise_acc32_nnpi` and `test_small_sls_acc32` still FAIL https://www.internalfb.com/intern/testinfra/testconsole/testrun/1407375046178756/ Reviewed By: arunm-git Differential Revision: D21697616 Pulled By: hl475 fbshipit-source-id: 3732324986eb40e644686cdd10e44951678508a7
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: Updating to NNPI Backend version v0.6.0.5