[TF:TRT] Enable TensorRT explicit precision (QDQ/QAT) support

[christopherbate]

Adds TensorRT QDQ support ("explicit precision mode"), also sometimes referred to as "QAT support", referring to the training algorithm where QDQ nodes are used. From here on, we refer to the existing non-explicit precision pathway in the code base as the "dynamic range INT8" mode (DR INT8) and the new mode as the QDQ INT8 mode.

In the new mode, TF QuantizeAndDequantize operations are converted to TensorRT quantization scaling layers. Both the new QDQ mode logic as well as existing DR mode logic (ConvertQuantize) are moved into the file convert/ops/quantization_ops.cc.

In addition, in QDQ mode it is necessary to prevent the existing Grappler optimizations invoked in trt_convert.py on the loaded SavedModel from folding frozen QuantizeAndDequantizeV2 operations between weighted ops (Conv, Matmul ,etc) and the weight constants. Thus, we depend on the experimental Grappler rewriter config option experimental_disable_folding_quantization_emulation and will be affected if it is removed. The alternative is to allow Grappler folding of the QDQ and constant weights and inserting identity QDQ scale factors manually during TensorRT network construction, but the logic becomes extremely verbose .

A test suite is added in convert/ops/quantization_ops_test.cc. It builds a variety of sub-graph patterns and tests for conversion success. Because TRT QDQ mode has evolved significantly in terms of robustness and features between TRT 7 and TRT8, a set of test waive/skip policies are added indicating which patterns of use are appropriate for TRT7 vs TRT8

[gbaned]

@christopherbate Can you please resolve conflicts? Thanks!

[christopherbate]

Conflicts resolved

[christopherbate]

All done. Let me know when you want me to squash.

[bixia1]

I have two questions regarding this:
(1) we don't consider the TensorRT version to enable use_explicit_precision, this looks wrong to me.
For example, a user has a model that uses QuantizeAndDequantizeV2 + TensorRT 7. It was working but now with this PR, he sees an error produced by TensorRT, similar to that in quantization_ops_test.cc line 435 & 440. Am I right here?
(2) A user has a model that uses all the OPs in kQuantizationOpNames + TensorRT8. It was working fine. But now with your change, we only convert one out of the four OPs in kQuantizationOpNames . Will this be a problem?

[christopherbate]

OK yes, I let me disable this for TRT < 8.0. We can use TRT 7 for very limited use cases, but likely EP mode won't be useful in the context of TF-TRT with TRT7 at this time.

[christopherbate]

fixed

[christopherbate]

Completely disabled explicit QDQ tests and grappler options (disable_emulate_quantization_folding) for TRT < 8.0.

[christopherbate]

@bixia1 Good to squash?

[gbaned]

@christopherbate Can you please resolve conflicts? Thanks!

[christopherbate]

@christopherbate Can you please resolve conflicts? Thanks!

rebased, thanks

[bixia1]

@christopherbate please squash and I will approve.

[christopherbate]

squashed