[Intel MKL] Changes in common files to enable MKL Quantized ops with native format

[mahmoud-abuzaina]

This PR adds the common changes to enable MKL Quantized ops in native format mode. It removes the dependency on the duplicate number of inputs/outputs for quantized ops.

Added by @penpornk for API review:

• The ops that are changed in API Golden files all have hidden visibility.
• The attribute data_format is added because MKL ops support both TF's native NHWC format and its own blocked format.
• The attributes is_weight_const and is_filter_const are added for caching purposes.
• The changes should be backward compatible.

[penpornk]

Thank you for the PR and I'm so sorry for the delay! The overall PR looks good to me.
I'm a bit hesitant on adding oneDNN-specific parameters (is_*_const) to the non-MKL-prefixed Quantized ops (which are visible to C++ API users). But I'll leave this part to the API review. (Will tag them after I approve this PR.)

[mahmoud-abuzaina]

Thank you for reviewing the PR. I have made the requested changed.
For adding an attribute to the ops, I thought these ops are not visible in the API as we mark them as hidden, for e.g. here. These ops usually get used by frozen graphs generated by our offline tool.

[penpornk]

Thank you for the fixes!

For adding an attribute to the ops, I thought these ops are not visible in the API as we mark them as hidden, for e.g. here.

Oh, you're right. Sorry I forgot to check before. :)

However, this PR will still require an API review anyway because it modifies the API golden files (//tensorflow/tools/api/golden/...). Tagging API review now. (It will be reviewed next week.)

[gbaned]

@mahmoud-abuzaina Can you please check @penpornk's comments and keep us posted ? Thanks!

[penpornk]

@mahmoud-abuzaina Sorry for the delay! The API review committee would like to see how the added attributes will be used. Can we put these attribute changes in the PRs that actually have the implementation that use them? We can keep other general changes in this PR.

[mahmoud-abuzaina]

@penpornk no worries. This PR and other related pending PRs don't add any new implementation to use the newly added attributes. Rather, the implementation that uses these attributes has been there for long time.
Here is the background: all Quantized* ops are hidden API ops and get generated/added to frozen pb file via offline tool. They were considered dummy ops. At runtime those ops will be rewritten to corresponding _MklQuantized*. The purpose of _MklQuantized* is to handle the extra number of inputs and outputs that we used to require. With native format support, we are getting rid of all extra inputs and outputs. So I thought we can reuse Quantized* ops (that are usable only via oneDNN integration code) instead of creating new ops. While doing that I noticed that these ops are missing few required attributes. These attributes have been used for a while with existing implementation, but the missing attributes were not causing a failure before because Quantized* ops were dummy ops and will be always rewritten to _Mkl ones which have the those attributes. Now since we are using Quantized* named ops, we are adding the missing attributes but they will use existing implementation. As an example of existing attribute usage see here. Sorry for my verbosity and please let me if I am missing something.

[mahmoud-abuzaina]

Closing for now. I will do some changes and reopen once ready.

[penpornk]

@mahmoud-abuzaina So sorry for my delayed reply and thank you very much for the thorough explanation! I didn't remember that these attributes are already used in the already checked-in kernels.

Re: Hidden ops: The API review committee reminded me that op names not prefixed by _ are still callable. Marking them hidden just puts them in tf.raw_ops, so they are still part of the API golden.

Edited to add: We synced offline about closing the PR for now. Looking forward to the changes!

[penpornk]

I'm so sorry for the long delay. Thank you again for the PR!