[TF-TRT] Cast Converter Re-Engineered

[DEKHTIARJonathan]

This PR changes the way TF-TRT deal with Cast nodes. TensorRT engineers advised us to treat Cast as an Identity node and let TensorRT decides on which compute precision to use according to precision_mode=....

This behavior can be deactivated using TF_TRT_EXPERIMENTAL_FEATURES=reject_fp32_fp16_cast if needed to work around any unforeseen issue

This PR also adds converter.summary() to TF-TRT test files in order to ease test debugability

[DEKHTIARJonathan]

@bixia1 for review
CC: @tfeher

[bixia1]

I have two general comment on this PR:
(1) the existing support is cast from fp16 to fp32, the feature name added here is called reject_fp32_fp16_cast. Shouldn't it be reject_fp16_fp32_cast?
(2) the existing code only supports cast from fp16 to fp32, the change here make us support fp32 to fp16 as well. Is this an intensionally change? If yes, please add test and add this to the PR description.

[gbaned]

Hi @DEKHTIARJonathan Can you please check @bixia1's comments and keep us posted ? Thank you!

[gbaned]

@DEKHTIARJonathan Any update on this PR? Please. Thank you!

[DEKHTIARJonathan]

@bixia1

(1) the existing support is cast from fp16 to fp32, the feature name added here is called reject_fp32_fp16_cast. Shouldn't it be reject_fp16_fp32_cast?

What I mean by reject_fp32_fp16_cast is to reject all FP Cast (fp32 => fp16 and fp16 => fp32), I am renaming the feature to reject_all_fp_cast_ops, I think it's more self explanatory.

(2) the existing code only supports cast from fp16 to fp32, the change here make us support fp32 to fp16 as well. Is this an intensionally change? If yes, please add test and add this to the PR description.

Yes absolutely, it is intentional. This PR changes the way Cast Operation in handled by TRT to and from FP16/32 precision. I am updating the PR description

[DEKHTIARJonathan]

@bixia1 please review. This PR is good to go ;)