This repository was archived by the owner on Jul 1, 2024. It is now read-only.
Add checks, and refine optimize_graph_with_openvino_tfx APIs; Disable cost based fallback when Cluster Fallback is disabled #318
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs
sspintel
force-pushed
the
sspintel/graph_optimizations
branch
from
3d55e41 to
2cdffa2
Compare
ck-intel
added a commit
that referenced
this pull request
Jun 28, 2022
* Remove older CustomGraphOptimizer methods * Update builder version to 2.0 * Enable Classification Sample through Grappler * Add AnalyticalCostEstimator that predicts and prints per-op costs in TF Logs * Change item to grappler_item * Refine graph optimization python API * Use cluster object from Grappler for device GOps * Working classification sample with grappler pass - Fix fetch nodes list - Do not consider feed, and fetch nodes for marking * Add translation for NMS ops and enable support in OCM for SquaredDifference,LessEqual,NotEqual,Cumsum * Update NMS implementation for transforming OV output shapes as per TF implementation * Disabling NMSV3 test for GPU, it fails for some trivial test fn checks, but passes when tested as a separate fn * Remove get_shape calls used for VLOG in Transpose * Replace get_shape with get_partial_shape wherever only rank is required * Added Op translations for CTCGreedyDecoder and BatchMatMulV2 (#301) * added translations for `CTCGreedyDecoder` and `BatchMatMulV2` * enable tests for `CTCGreedyDecoder` * enable tests for `BatchMatMulV2` * added custom test cases for `CTCGreedyDecoder` * fixed function name * Update OCM submodule to ocm master for the added ops * Implement SparseToDense OP for CPU * Fixed abuge introduced in fixing get_shape in TpKV2 * Restore min non trivial node threshold to right shift by 5, and update cluster busting comments * Cluster wise cost estimation using Grappler - Add GraphDef creation for each cluster at the deassign_clusters stage - Temporarily operate with the heuristic of scheduling only the top-1 costly cluster on OpenVINO * Fix translation of Select OP, and right shift condition in deassign clusters * Added Op translations and extended type support (#304) * added translation for `BatchMatMul` * added test for `Tile` * added test for `Reshape` * added test for `ExpandDims` * added tests for `BatchMatMul` and `BatchMatMulV2` * enable tests for `BatchMatMul` * disable float64 tests for GPU * update OCM submodule * Added fused translation for `CTCGreedyDecoder` (#305) * added translation for fused CTC greedy decoder * added tests for fused CTC greedy decoder * fixed typos * Update squeeze op translation to use input shape only when required * Add env variable to select TOP K clusters for execution, update omit cluster logic in deassign clusters * Update translation to handle dynamic shape for Range, and Zeroslike * Comment out zeroslike from dynamic to static flow, as translation is modified to handle it * Comment math_grad_test.SquaredDifferenceOpTest.testGrad test, failing with minor tolerance error after enabling double for certain OPs * Update Range, Fill, Prod and OneHot op translations and static inputs to handle dynamic input shapes * First cut of conv2d and fused conv2d with dynamic shape input * fixed bug in layout transformation * Cosmetic changes for dynamic conv2d support, add temp fix for protobuf version error in TF-2.8.0 * Add Cost based cluster de-assignment and fallback - Added grappler/costs/cost_analyzer.cc, CostAnalyzer->PredictCosts computes and tags node-wise runtime costs onto the graph as an attribute - DeassignClusters uses node-wise costs to add up cluster-wise costs and keep only top-3 clusters and busts the rest - At runtime, this cluster wise TF cost is compared against OV costs. Clusters which are costly yet perform poorly on OV fallsback to native session. * Enable size translation to handle dynamic input shapes, and concat translation to handle zero dimension inputs * Fix float to bool casting bug (#307) * Update ocm; Add infer time measurement in OV * Add OPENVINO_TF_MAX_CLUSTERS for TopK Costs * Rewrite Pass API call change * - Build both RewritePass and GrapplerOptimizationPass - Add API that controls if RewritePass is run in a given scope; Gets disabled when optimize_graph_with_openvino() is called - Add cluster_cost attr to nGraphEncapsulate NodeDef - Set NumWarmupSteps for TF Profiler to 1 * Revert EvictAllClusters from OVTFOptimizer * Enable rewrite pass to work alongside grappler pass (#310) * Modify TopKV2 tranlsation to handle dynamic shapes, and remove the corresponding static input condition. Code formatting done * Enable Rewrite Pass with Grappler pass *Modify deassign_clusters.cc to merge Min non trivial nodes changes with cost based deassignment logic *Enable Dynamic to Static flow for GPU with reduced cosntraints *Calculate cluster cost with native TF in compute kernel to decide run time assignment of cluster with Rewrite pass *Disable some of the NMS Ops in both grappler as well as Rewrite pass, as they are impacting performance of some OD models * Fix IsRewritePassEnabled api call syntax at 2 places * Add support for Frozen models with Grappler Pass (#311) * Add TFV1 OptimizeGraph API for Frozen models * Enable GrapplerOptimization pass for frozen models - Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs * Remove Grappler Related Build Flags; For UTs assume no Grappler * Remove Grappler pass tests; New tests TDB * Removed call to `update_config`; fixes failing unit tests (#312) * removed call to `update_config` * added missing attr `cluster_cost` * Fixes CI build failures (#314) * fixed protobuf version bug * cleanup: removed references to `infer_duration_in_ms` * Revert optimize_graph call in classification sample * cleanup post-processing Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add support for _MklSwish Op, generated when OneDNN optimizations are enabled (#316) * Add env variable based disable option for cost based runtime cluster assignment (#317) * Add checks, and refine optimize_graph_with_openvino_tfx APIs; Disable cost based fallback when Cluster Fallback is disabled (#318) * Add TFV1 OptimizeGraph API for Frozen models * Enable GrapplerOptimization pass for frozen models - Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs * Remove Grappler Related Build Flags; For UTs assume no Grappler * Fallback to TF only when Dynfallback is enabled in OpKernel Compute * Add checks, refinements, and multi-input support for the optimize_graph API * Add API Usage for optimize_graph_with_openvino_tf1 * Remove repeated protobuf install step Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Srihari Humbarwadi <srihari.humbarwadi@intel.com>
ck-intel
added a commit
that referenced
this pull request
Jul 14, 2022
* Update opset support to opset8 (#300) * Update opset support to opset8 * Update maxpool op params bas per pset8 in transpose_sinking cpp test * Performance optimizations (#313) * Remove older CustomGraphOptimizer methods * Update builder version to 2.0 * Enable Classification Sample through Grappler * Add AnalyticalCostEstimator that predicts and prints per-op costs in TF Logs * Change item to grappler_item * Refine graph optimization python API * Use cluster object from Grappler for device GOps * Working classification sample with grappler pass - Fix fetch nodes list - Do not consider feed, and fetch nodes for marking * Add translation for NMS ops and enable support in OCM for SquaredDifference,LessEqual,NotEqual,Cumsum * Update NMS implementation for transforming OV output shapes as per TF implementation * Disabling NMSV3 test for GPU, it fails for some trivial test fn checks, but passes when tested as a separate fn * Remove get_shape calls used for VLOG in Transpose * Replace get_shape with get_partial_shape wherever only rank is required * Added Op translations for CTCGreedyDecoder and BatchMatMulV2 (#301) * added translations for `CTCGreedyDecoder` and `BatchMatMulV2` * enable tests for `CTCGreedyDecoder` * enable tests for `BatchMatMulV2` * added custom test cases for `CTCGreedyDecoder` * fixed function name * Update OCM submodule to ocm master for the added ops * Implement SparseToDense OP for CPU * Fixed abuge introduced in fixing get_shape in TpKV2 * Restore min non trivial node threshold to right shift by 5, and update cluster busting comments * Cluster wise cost estimation using Grappler - Add GraphDef creation for each cluster at the deassign_clusters stage - Temporarily operate with the heuristic of scheduling only the top-1 costly cluster on OpenVINO * Fix translation of Select OP, and right shift condition in deassign clusters * Added Op translations and extended type support (#304) * added translation for `BatchMatMul` * added test for `Tile` * added test for `Reshape` * added test for `ExpandDims` * added tests for `BatchMatMul` and `BatchMatMulV2` * enable tests for `BatchMatMul` * disable float64 tests for GPU * update OCM submodule * Added fused translation for `CTCGreedyDecoder` (#305) * added translation for fused CTC greedy decoder * added tests for fused CTC greedy decoder * fixed typos * Update squeeze op translation to use input shape only when required * Add env variable to select TOP K clusters for execution, update omit cluster logic in deassign clusters * Update translation to handle dynamic shape for Range, and Zeroslike * Comment out zeroslike from dynamic to static flow, as translation is modified to handle it * Comment math_grad_test.SquaredDifferenceOpTest.testGrad test, failing with minor tolerance error after enabling double for certain OPs * Update Range, Fill, Prod and OneHot op translations and static inputs to handle dynamic input shapes * First cut of conv2d and fused conv2d with dynamic shape input * fixed bug in layout transformation * Cosmetic changes for dynamic conv2d support, add temp fix for protobuf version error in TF-2.8.0 * Add Cost based cluster de-assignment and fallback - Added grappler/costs/cost_analyzer.cc, CostAnalyzer->PredictCosts computes and tags node-wise runtime costs onto the graph as an attribute - DeassignClusters uses node-wise costs to add up cluster-wise costs and keep only top-3 clusters and busts the rest - At runtime, this cluster wise TF cost is compared against OV costs. Clusters which are costly yet perform poorly on OV fallsback to native session. * Enable size translation to handle dynamic input shapes, and concat translation to handle zero dimension inputs * Fix float to bool casting bug (#307) * Update ocm; Add infer time measurement in OV * Add OPENVINO_TF_MAX_CLUSTERS for TopK Costs * Rewrite Pass API call change * - Build both RewritePass and GrapplerOptimizationPass - Add API that controls if RewritePass is run in a given scope; Gets disabled when optimize_graph_with_openvino() is called - Add cluster_cost attr to nGraphEncapsulate NodeDef - Set NumWarmupSteps for TF Profiler to 1 * Revert EvictAllClusters from OVTFOptimizer * Enable rewrite pass to work alongside grappler pass (#310) * Modify TopKV2 tranlsation to handle dynamic shapes, and remove the corresponding static input condition. Code formatting done * Enable Rewrite Pass with Grappler pass *Modify deassign_clusters.cc to merge Min non trivial nodes changes with cost based deassignment logic *Enable Dynamic to Static flow for GPU with reduced cosntraints *Calculate cluster cost with native TF in compute kernel to decide run time assignment of cluster with Rewrite pass *Disable some of the NMS Ops in both grappler as well as Rewrite pass, as they are impacting performance of some OD models * Fix IsRewritePassEnabled api call syntax at 2 places * Add support for Frozen models with Grappler Pass (#311) * Add TFV1 OptimizeGraph API for Frozen models * Enable GrapplerOptimization pass for frozen models - Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs * Remove Grappler Related Build Flags; For UTs assume no Grappler * Remove Grappler pass tests; New tests TDB * Removed call to `update_config`; fixes failing unit tests (#312) * removed call to `update_config` * added missing attr `cluster_cost` * Fixes CI build failures (#314) * fixed protobuf version bug * cleanup: removed references to `infer_duration_in_ms` * Revert optimize_graph call in classification sample * cleanup post-processing Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add support for _MklSwish Op, generated when OneDNN optimizations are enabled (#316) * Add env variable based disable option for cost based runtime cluster assignment (#317) * Add checks, and refine optimize_graph_with_openvino_tfx APIs; Disable cost based fallback when Cluster Fallback is disabled (#318) * Add TFV1 OptimizeGraph API for Frozen models * Enable GrapplerOptimization pass for frozen models - Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs * Remove Grappler Related Build Flags; For UTs assume no Grappler * Fallback to TF only when Dynfallback is enabled in OpKernel Compute * Add checks, refinements, and multi-input support for the optimize_graph API * Add API Usage for optimize_graph_with_openvino_tf1 * Remove repeated protobuf install step Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Srihari Humbarwadi <srihari.humbarwadi@intel.com> * Upgrade TensorFlow Support to v2.9.1 (#319) * update tensorflow dependencies * `cxx11_abi_version` flag now defaults to 1; added bazel version check * upgrade to TF v2.9.1; removed flags `build_tf_from_source` and `disable_cpp_api` * disabled unit tests that fail on native TF too * change default `tf_version` to v2.9.1 * fixed TF wheel path for windows * export missing symbols * use `shutil.copytree` to copy artifacts on windows * updated TF wheel URL for windows * update ovtf version to `2.1.0` * Renamed patch file * Add fixes for issues from static code analysis tool (#320) * Updated OCM submodule and fixed formatting (#322) * Updated OCM * Fixed formatting * TF Upgrade in Dockerfiles (#321) * Update TF version in notebooks and dockerfiles * - Changes in dockerfile used for wheel builds to support new TF CXX11 Upgrade - Rename Dockerfiles from 2.0.0 to 2.1.0 - Change OVTF version in Dockerfiles * Make Docker as default container registry to pull openvino dev images * Change crosstop_top toolchain to support manylinux2014 in TF build * Remove ADD rename_wheel.sh * Fix merge conflict in Ubuntu18 CI Dockerfile * Upgrade OVTF version in sample notebooks * Revert to working TF version in serving dockerfiles; Change OVTF version in notebook comments * Revert OVTF version in Serving dockerfiles * Remove env var from examples to enable ONEDNN, as it is default enabled from TF-2.9.x (#324) * Remove env var from examples to enable ONEDNN, as it is default enabled from TF-2.9.x * Update warm-up iterations in python samples and ipynb's * Fix variable def scope error, occured after addition of warm-up iterations * Code formatting * Updated TF Serving Dockerfiles to support TensorFlow 2.9.1 (#325) * Upgraded TF version; Skip building OVTF from source * Fixed TF install bug; Made OV base image tags configurable * Avoid installing TF again * Add changes to Docker READMe. Remove unused TF_VERSION ARG * Clarify headings; Fix 'Disable OpenVINO' * Replace occurrences of OVTF Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Adapt ZerosLike translation for dynamic input shape (#327) * Fix macOS CI issues; Remove installed packages checking in Ipynb notebooks (#329) * Upgrade TF Version to 2.9.1 in CI yml for MacOS * Add enclosing brackets for average inference time calculation * Upgrade TF and OV ABIs in macOS CI yml file * Change build ABI flag of CI 0 * Remove package checking in notebooks - Install TF and OVTF packages by default instead * Use PyPi TF by default * Use ABI1 by default * Change to ABI0 build as PyPi TF is ABI0 on macOS * Remove unwanted INSTALL step; Remove always() condition for samples * Use test_runner instead of tf_unittest_runner for MYD TF Python UTs Launch ut runner with os.system() on macOS to ignore exception * Fix const op translation (#332) * Fix copying values from const nodes, bug caught in Windows * Update ocm submodule, test cases list updated for Myriad * Remove unused commented code * Fix TF backwards compatibility issues (#333) * Enable Grappler only for TF version >=2.6 * Fix type conversion errors for TF 2.4.4 * Fix type conversion issues for TF 2.4.4 * Make base version of TF for Grappler support 2.8 * Fix code formatting * Fix virtualenv issue in object detection notebook; Fix Windows build error for absl::GetCurrentTimeNanos() (#334) * Add virtualenv step to isolate yolov4 model conversion * Fix Windows build error for absl::GetCurrentTimeNanos() * Revert metadata changes in object detection notebook * Fix detection notebook issue on docker (#335) * Update documentation for releases 2.1.0 (#323) * Update documentation for releases 2.1.0 *Update Installation and Build docs *Update main Readme and Python Readme *Update all other files for TF, OVTF versions, and Links if required * Minor cosmetic changes in Build and Install.md files * Reorganize env var documentation order, and add details of the OPENVINO_TF_MAX_CLUSTERS var in Usage.md * Fix Build Instructions for macOS (#330) * Update mandarin files for main readme, docker readme, and usage.md *Prelim attempt as the changes were not much in these files *These changes would be reviewed by the team handling the Translations * Update installation table * Update build and install mandarin docs * Fix review comments * Fix a typo in mandarin install doc Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Update windows tf wheel link (#336) * Remove optimize_graph API for v2.1.0 (#337) Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> Co-authored-by: Srihari Humbarwadi <srihari.humbarwadi@intel.com>
cavusmustafa
pushed a commit
that referenced
this pull request
Sep 1, 2022
* Remove older CustomGraphOptimizer methods * Update builder version to 2.0 * Enable Classification Sample through Grappler * Add AnalyticalCostEstimator that predicts and prints per-op costs in TF Logs * Change item to grappler_item * Refine graph optimization python API * Use cluster object from Grappler for device GOps * Working classification sample with grappler pass - Fix fetch nodes list - Do not consider feed, and fetch nodes for marking * Add translation for NMS ops and enable support in OCM for SquaredDifference,LessEqual,NotEqual,Cumsum * Update NMS implementation for transforming OV output shapes as per TF implementation * Disabling NMSV3 test for GPU, it fails for some trivial test fn checks, but passes when tested as a separate fn * Remove get_shape calls used for VLOG in Transpose * Replace get_shape with get_partial_shape wherever only rank is required * Added Op translations for CTCGreedyDecoder and BatchMatMulV2 (#301) * added translations for `CTCGreedyDecoder` and `BatchMatMulV2` * enable tests for `CTCGreedyDecoder` * enable tests for `BatchMatMulV2` * added custom test cases for `CTCGreedyDecoder` * fixed function name * Update OCM submodule to ocm master for the added ops * Implement SparseToDense OP for CPU * Fixed abuge introduced in fixing get_shape in TpKV2 * Restore min non trivial node threshold to right shift by 5, and update cluster busting comments * Cluster wise cost estimation using Grappler - Add GraphDef creation for each cluster at the deassign_clusters stage - Temporarily operate with the heuristic of scheduling only the top-1 costly cluster on OpenVINO * Fix translation of Select OP, and right shift condition in deassign clusters * Added Op translations and extended type support (#304) * added translation for `BatchMatMul` * added test for `Tile` * added test for `Reshape` * added test for `ExpandDims` * added tests for `BatchMatMul` and `BatchMatMulV2` * enable tests for `BatchMatMul` * disable float64 tests for GPU * update OCM submodule * Added fused translation for `CTCGreedyDecoder` (#305) * added translation for fused CTC greedy decoder * added tests for fused CTC greedy decoder * fixed typos * Update squeeze op translation to use input shape only when required * Add env variable to select TOP K clusters for execution, update omit cluster logic in deassign clusters * Update translation to handle dynamic shape for Range, and Zeroslike * Comment out zeroslike from dynamic to static flow, as translation is modified to handle it * Comment math_grad_test.SquaredDifferenceOpTest.testGrad test, failing with minor tolerance error after enabling double for certain OPs * Update Range, Fill, Prod and OneHot op translations and static inputs to handle dynamic input shapes * First cut of conv2d and fused conv2d with dynamic shape input * fixed bug in layout transformation * Cosmetic changes for dynamic conv2d support, add temp fix for protobuf version error in TF-2.8.0 * Add Cost based cluster de-assignment and fallback - Added grappler/costs/cost_analyzer.cc, CostAnalyzer->PredictCosts computes and tags node-wise runtime costs onto the graph as an attribute - DeassignClusters uses node-wise costs to add up cluster-wise costs and keep only top-3 clusters and busts the rest - At runtime, this cluster wise TF cost is compared against OV costs. Clusters which are costly yet perform poorly on OV fallsback to native session. * Enable size translation to handle dynamic input shapes, and concat translation to handle zero dimension inputs * Fix float to bool casting bug (#307) * Update ocm; Add infer time measurement in OV * Add OPENVINO_TF_MAX_CLUSTERS for TopK Costs * Rewrite Pass API call change * - Build both RewritePass and GrapplerOptimizationPass - Add API that controls if RewritePass is run in a given scope; Gets disabled when optimize_graph_with_openvino() is called - Add cluster_cost attr to nGraphEncapsulate NodeDef - Set NumWarmupSteps for TF Profiler to 1 * Revert EvictAllClusters from OVTFOptimizer * Enable rewrite pass to work alongside grappler pass (#310) * Modify TopKV2 tranlsation to handle dynamic shapes, and remove the corresponding static input condition. Code formatting done * Enable Rewrite Pass with Grappler pass *Modify deassign_clusters.cc to merge Min non trivial nodes changes with cost based deassignment logic *Enable Dynamic to Static flow for GPU with reduced cosntraints *Calculate cluster cost with native TF in compute kernel to decide run time assignment of cluster with Rewrite pass *Disable some of the NMS Ops in both grappler as well as Rewrite pass, as they are impacting performance of some OD models * Fix IsRewritePassEnabled api call syntax at 2 places * Add support for Frozen models with Grappler Pass (#311) * Add TFV1 OptimizeGraph API for Frozen models * Enable GrapplerOptimization pass for frozen models - Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs * Remove Grappler Related Build Flags; For UTs assume no Grappler * Remove Grappler pass tests; New tests TDB * Removed call to `update_config`; fixes failing unit tests (#312) * removed call to `update_config` * added missing attr `cluster_cost` * Fixes CI build failures (#314) * fixed protobuf version bug * cleanup: removed references to `infer_duration_in_ms` * Revert optimize_graph call in classification sample * cleanup post-processing Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add support for _MklSwish Op, generated when OneDNN optimizations are enabled (#316) * Add env variable based disable option for cost based runtime cluster assignment (#317) * Add checks, and refine optimize_graph_with_openvino_tfx APIs; Disable cost based fallback when Cluster Fallback is disabled (#318) * Add TFV1 OptimizeGraph API for Frozen models * Enable GrapplerOptimization pass for frozen models - Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs * Remove Grappler Related Build Flags; For UTs assume no Grappler * Fallback to TF only when Dynfallback is enabled in OpKernel Compute * Add checks, refinements, and multi-input support for the optimize_graph API * Add API Usage for optimize_graph_with_openvino_tf1 * Remove repeated protobuf install step Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Srihari Humbarwadi <srihari.humbarwadi@intel.com>
cavusmustafa
added a commit
that referenced
this pull request
Sep 20, 2022
* Update opset support to opset8 (#300) * Update opset support to opset8 * Update maxpool op params bas per pset8 in transpose_sinking cpp test * Performance optimizations (#313) * Remove older CustomGraphOptimizer methods * Update builder version to 2.0 * Enable Classification Sample through Grappler * Add AnalyticalCostEstimator that predicts and prints per-op costs in TF Logs * Change item to grappler_item * Refine graph optimization python API * Use cluster object from Grappler for device GOps * Working classification sample with grappler pass - Fix fetch nodes list - Do not consider feed, and fetch nodes for marking * Add translation for NMS ops and enable support in OCM for SquaredDifference,LessEqual,NotEqual,Cumsum * Update NMS implementation for transforming OV output shapes as per TF implementation * Disabling NMSV3 test for GPU, it fails for some trivial test fn checks, but passes when tested as a separate fn * Remove get_shape calls used for VLOG in Transpose * Replace get_shape with get_partial_shape wherever only rank is required * Added Op translations for CTCGreedyDecoder and BatchMatMulV2 (#301) * added translations for `CTCGreedyDecoder` and `BatchMatMulV2` * enable tests for `CTCGreedyDecoder` * enable tests for `BatchMatMulV2` * added custom test cases for `CTCGreedyDecoder` * fixed function name * Update OCM submodule to ocm master for the added ops * Implement SparseToDense OP for CPU * Fixed abuge introduced in fixing get_shape in TpKV2 * Restore min non trivial node threshold to right shift by 5, and update cluster busting comments * Cluster wise cost estimation using Grappler - Add GraphDef creation for each cluster at the deassign_clusters stage - Temporarily operate with the heuristic of scheduling only the top-1 costly cluster on OpenVINO * Fix translation of Select OP, and right shift condition in deassign clusters * Added Op translations and extended type support (#304) * added translation for `BatchMatMul` * added test for `Tile` * added test for `Reshape` * added test for `ExpandDims` * added tests for `BatchMatMul` and `BatchMatMulV2` * enable tests for `BatchMatMul` * disable float64 tests for GPU * update OCM submodule * Added fused translation for `CTCGreedyDecoder` (#305) * added translation for fused CTC greedy decoder * added tests for fused CTC greedy decoder * fixed typos * Update squeeze op translation to use input shape only when required * Add env variable to select TOP K clusters for execution, update omit cluster logic in deassign clusters * Update translation to handle dynamic shape for Range, and Zeroslike * Comment out zeroslike from dynamic to static flow, as translation is modified to handle it * Comment math_grad_test.SquaredDifferenceOpTest.testGrad test, failing with minor tolerance error after enabling double for certain OPs * Update Range, Fill, Prod and OneHot op translations and static inputs to handle dynamic input shapes * First cut of conv2d and fused conv2d with dynamic shape input * fixed bug in layout transformation * Cosmetic changes for dynamic conv2d support, add temp fix for protobuf version error in TF-2.8.0 * Add Cost based cluster de-assignment and fallback - Added grappler/costs/cost_analyzer.cc, CostAnalyzer->PredictCosts computes and tags node-wise runtime costs onto the graph as an attribute - DeassignClusters uses node-wise costs to add up cluster-wise costs and keep only top-3 clusters and busts the rest - At runtime, this cluster wise TF cost is compared against OV costs. Clusters which are costly yet perform poorly on OV fallsback to native session. * Enable size translation to handle dynamic input shapes, and concat translation to handle zero dimension inputs * Fix float to bool casting bug (#307) * Update ocm; Add infer time measurement in OV * Add OPENVINO_TF_MAX_CLUSTERS for TopK Costs * Rewrite Pass API call change * - Build both RewritePass and GrapplerOptimizationPass - Add API that controls if RewritePass is run in a given scope; Gets disabled when optimize_graph_with_openvino() is called - Add cluster_cost attr to nGraphEncapsulate NodeDef - Set NumWarmupSteps for TF Profiler to 1 * Revert EvictAllClusters from OVTFOptimizer * Enable rewrite pass to work alongside grappler pass (#310) * Modify TopKV2 tranlsation to handle dynamic shapes, and remove the corresponding static input condition. Code formatting done * Enable Rewrite Pass with Grappler pass *Modify deassign_clusters.cc to merge Min non trivial nodes changes with cost based deassignment logic *Enable Dynamic to Static flow for GPU with reduced cosntraints *Calculate cluster cost with native TF in compute kernel to decide run time assignment of cluster with Rewrite pass *Disable some of the NMS Ops in both grappler as well as Rewrite pass, as they are impacting performance of some OD models * Fix IsRewritePassEnabled api call syntax at 2 places * Add support for Frozen models with Grappler Pass (#311) * Add TFV1 OptimizeGraph API for Frozen models * Enable GrapplerOptimization pass for frozen models - Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs * Remove Grappler Related Build Flags; For UTs assume no Grappler * Remove Grappler pass tests; New tests TDB * Removed call to `update_config`; fixes failing unit tests (#312) * removed call to `update_config` * added missing attr `cluster_cost` * Fixes CI build failures (#314) * fixed protobuf version bug * cleanup: removed references to `infer_duration_in_ms` * Revert optimize_graph call in classification sample * cleanup post-processing Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add support for _MklSwish Op, generated when OneDNN optimizations are enabled (#316) * Add env variable based disable option for cost based runtime cluster assignment (#317) * Add checks, and refine optimize_graph_with_openvino_tfx APIs; Disable cost based fallback when Cluster Fallback is disabled (#318) * Add TFV1 OptimizeGraph API for Frozen models * Enable GrapplerOptimization pass for frozen models - Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs * Remove Grappler Related Build Flags; For UTs assume no Grappler * Fallback to TF only when Dynfallback is enabled in OpKernel Compute * Add checks, refinements, and multi-input support for the optimize_graph API * Add API Usage for optimize_graph_with_openvino_tf1 * Remove repeated protobuf install step Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Srihari Humbarwadi <srihari.humbarwadi@intel.com> * Add fixes for issues from static code analysis tool (#320) * TF Upgrade in Dockerfiles (#321) * Update TF version in notebooks and dockerfiles * - Changes in dockerfile used for wheel builds to support new TF CXX11 Upgrade - Rename Dockerfiles from 2.0.0 to 2.1.0 - Change OVTF version in Dockerfiles * Make Docker as default container registry to pull openvino dev images * Change crosstop_top toolchain to support manylinux2014 in TF build * Remove ADD rename_wheel.sh * Fix merge conflict in Ubuntu18 CI Dockerfile * Upgrade OVTF version in sample notebooks * Revert to working TF version in serving dockerfiles; Change OVTF version in notebook comments * Revert OVTF version in Serving dockerfiles * Upgrade to TF 2.8 version and update documentation (#267) * Initial attempt for translation to OV 2.0 * Backend related updates for OV 2.0 translation * Additional OV2.0 updates and VADM OV2.0 * TF-2.8 upgrade, and partial doc updates * Fix C++ test utilities for OV 2.0, update yml files for now to use ABI1 builds * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Remove -march=native option and update OV build flag * Disabling some of the OV build flags temporarily * Remove # typo * mac doesn't support working with openvino binary package * Fix mac yml Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add OVTF runtime Dockerfiles (#266) * Initial attempt for translation to OV 2.0 * Backend related updates for OV 2.0 translation * Additional OV2.0 updates and VADM OV2.0 * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Upgrade TF version to 2.8 in code * Update dockerfiles to include only OVTF build and runtime prep stages * Add steps to download GPL sources * Update README; Change tensorflow pkg link * Modify OpenVINO base image version tag * Initial commit for TF Serving enabled with OVTF * Adding dockerfiles with openvino tensorflow as base, readme * Add TF Serving Dockerfile * Update Serving Dockerfile * Remove Colab notebooks * Fix mac yml * Add Ubuntu20 TF Serving Dockerfile * Update README Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Laxmi Ganesan <laxmi.ganesan@intel.com> * Update cmakelist to filter OV libs for OVTF wheel (#272) * Update cmakelist for filtering the required OV libs for wheel generation * Correct cmake as gpu plugin is not available for mac * Update comments in cmakelist * Update CI yml, and dockerfiles (#271) * Initial attempt for translation to OV 2.0 * Additional OV2.0 updates and VADM OV2.0 * Fix C++ test utilities for OV 2.0, update yml files for now to use ABI1 builds * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Move CI Dockerfiles * Remove older tensorflow cxx11 ABI build dockerfile * Add new TF CXX11 build Dockerfile * Add steps to build cc lib and prepare tf build package * Add CXX_11 flag to cc libs build * Update CI Dockerfiles * Update yml and dockerfiles * Change default user in dockerfile * Change image descriptions * ubuntu18, ubuntu20, mac yml update * Update mac yml * Update mac agent pool * Fix pip version * Fix mac openvino_c.dylib and libtbb.dylib not found * Change TFHub Cache Dir Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> * Error check for backend type in cpp api (#243) * Fix set_backend python api to throw the right exception (#276) * Update wheel platform tags; ABI1 build is now manylinux_2_27 compatible (#275) * Initial attempt for translation to OV 2.0 * Additional OV2.0 updates and VADM OV2.0 * Add manylinux build tools * Update tag for manylinux2014 * Add platform check for manylinux tags Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> * OVTF releases 2.0.0 doc updates (#278) * Modify the readme for pre-release * Update build doc for 2.0 release and usage doc for convert variables to constants instructions * Update build doc for 2.0 release and usage doc for convert variables to constants instructions * Update docker documentation for more clarity * Update build and install docs for backward compatibility * Update Docker related documentation; Fix test-cases in notebooks * Enable OneDNN and VAR2CONST conversion in examples and notebooks * Fix CI * Update mandarin documentation files * Clarify docker documentation Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * TFFE merged into releases/2.0 (resolved conficts) * Removed rebase conflicts with TFFE branch * Updates to build with OV 2.0 master branch (#252) * Update cmakelists and build script to enable OVTF with OV 2.0 master branch * Add fix for BatchNDAndSpaceND op TF UT * Add support for OV 2022.1 version * Build OpenVINO from sources in CI * Remove ngraph lib dependency from CPP tests build Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add support for failing OMZ models (#256) * Comment ParseNodeAttributes call in Ngraphencapsulate, as it is unused and causing error for some models in windows * Update build script for symbolic linking of TF python unit test folder * Code formating for build_ovtf.py updates Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> * Disabling TopKV2 op (#257) * TopKV2 op disabled * Upgrade to TF 2.8 version and update documentation (#267) * Initial attempt for translation to OV 2.0 * Backend related updates for OV 2.0 translation * Additional OV2.0 updates and VADM OV2.0 * TF-2.8 upgrade, and partial doc updates * Fix C++ test utilities for OV 2.0, update yml files for now to use ABI1 builds * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Remove -march=native option and update OV build flag * Disabling some of the OV build flags temporarily * Remove # typo * mac doesn't support working with openvino binary package * Fix mac yml Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add OVTF runtime Dockerfiles (#266) * Initial attempt for translation to OV 2.0 * Backend related updates for OV 2.0 translation * Additional OV2.0 updates and VADM OV2.0 * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Upgrade TF version to 2.8 in code * Update dockerfiles to include only OVTF build and runtime prep stages * Add steps to download GPL sources * Update README; Change tensorflow pkg link * Modify OpenVINO base image version tag * Initial commit for TF Serving enabled with OVTF * Adding dockerfiles with openvino tensorflow as base, readme * Add TF Serving Dockerfile * Update Serving Dockerfile * Remove Colab notebooks * Fix mac yml * Add Ubuntu20 TF Serving Dockerfile * Update README Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Laxmi Ganesan <laxmi.ganesan@intel.com> * Update CI yml, and dockerfiles (#271) * Initial attempt for translation to OV 2.0 * Additional OV2.0 updates and VADM OV2.0 * Fix C++ test utilities for OV 2.0, update yml files for now to use ABI1 builds * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Move CI Dockerfiles * Remove older tensorflow cxx11 ABI build dockerfile * Add new TF CXX11 build Dockerfile * Add steps to build cc lib and prepare tf build package * Add CXX_11 flag to cc libs build * Update CI Dockerfiles * Update yml and dockerfiles * Change default user in dockerfile * Change image descriptions * ubuntu18, ubuntu20, mac yml update * Update mac yml * Update mac agent pool * Fix pip version * Fix mac openvino_c.dylib and libtbb.dylib not found * Change TFHub Cache Dir Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> * Error check for backend type in cpp api (#243) * Fix set_backend python api to throw the right exception (#276) * Update wheel platform tags; ABI1 build is now manylinux_2_27 compatible (#275) * Initial attempt for translation to OV 2.0 * Additional OV2.0 updates and VADM OV2.0 * Add manylinux build tools * Update tag for manylinux2014 * Add platform check for manylinux tags Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> * OVTF releases 2.0.0 doc updates (#278) * Modify the readme for pre-release * Update build doc for 2.0 release and usage doc for convert variables to constants instructions * Update build doc for 2.0 release and usage doc for convert variables to constants instructions * Update docker documentation for more clarity * Update build and install docs for backward compatibility * Update Docker related documentation; Fix test-cases in notebooks * Enable OneDNN and VAR2CONST conversion in examples and notebooks * Fix CI * Update mandarin documentation files * Clarify docker documentation Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * TFFE merged into releases/2.0 (resolved conficts) * Correct the conflicts during rebase * Update patch for Protobuf revert changes * Opset issue fix for TFFE * Temporary fix for control inputs (#328) * Temporary fix for control inputs * Ignoring additional outputs created by frontend * Comment lines and cleanup related to control edge fix * _MKLSwish op support for TFFE (#341) * Concat&NonMaxSupression conversion update, zero-dim fix (#343) * Slice size minus one support added (#347) * Slice size minus one support added * Hoisted parameter bug fix * Cast translation fix for bool * Temp. fix for hoisted param bug (#348) * Temp. fix for FusedBatchNorm duplicate parameters * Changed default OV to TF-FE dev. branch * Apply changes for protobuf from openvino (#352) * Apply changes for protobuf from openvino * Fix _Arg translator to use decoder for getting tensor * Remove function that is not needed anymore * Remove protobuf workaround * Switch to 2022.2 openvino branch * Input setting perf. fix (#356) * Input setting perf. fix * fix for convert variables to constant * Kotreshr/ctc greedy decoder conversion extension added (#360) * ctc_greedy_decoder tf_conversion_extension added * custom tf conversion extension for sparse to dense added * conv2d_kernels_check unit test modified as per the exception info changes * Mcavus/fix remaining model issues (#359) * Fix for squeeze_dim attribute issue * Temporary fix for DepthwiseConv2dNative issue * Fix for hoisted params issue * Handling non-existing attributes before translation * Revert temporary fix for DepthwiseConv2dNative issue * Bug fix for hoisted parameter handling * Fixed remaining rebase issues * Fix UTs, macOS Build Fix, and Samples SegFault; Apply code formatting (mcavus/tffe_releases_2.2) (#363) * Apply code formatting changes * Remove problematic call to EvictMRUClusters * Disable testFloorDivGrad as it's an unsupported training op * Rethrow NodeValidationFailure exception as InvalidArgument for TF Handling * Update exception handling of conv2D_KernelChecks * Add working fix to build TFFE Extensions on macOS * Enabled base OVTF translations as optional (#364) * Enabled base OVTF translations as optional * Temp. fix for skipped inputs bug & segfault fix * Apply code formatting * Handle OPENVINO_TF_DISABLE_TFFE envvar better Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * TF FE device related disabling & TS turned off (#365) * TF FE device related disabling & TS turned off * Code formating for TF FE device based disabling * TFFE build fixes for windows (#366) * build fixes for windows * tf windows build patch updated * Enable building OpenVINO from source for Windows CI * Comment OpenVINO Download to recover VM disk space * wrapped blocks with win32 check Co-authored-by: unknown <intel@BGAOPTIPLEX001.gar.corp.intel.com> Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Option to enable performance counters for debugging with OPENVINO_TF_ENABLE_PERF_COUNT (#369) * TF FE device related disabling & TS turned off * Code formating for TF FE device based disabling * Enabling OPENVINO_TF_ENABLE_PERF_COUNT prints performance counters at VLOG_LEVEL=1 * Handle setting envvars in BackendManager * Add back TFFrontendDisabled * Add OVTFProfilingEnabled checks for encapsulate_op Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: Chandrakant Khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> Co-authored-by: Srihari Humbarwadi <srihari.humbarwadi@intel.com> Co-authored-by: Laxmi Ganesan <laxmi.ganesan@intel.com> Co-authored-by: Athina Saha <athina.saha@intel.com> Co-authored-by: Maxim Vafin <maxim.vafin@intel.com> Co-authored-by: kotreshr <111343878+kotreshr@users.noreply.github.com> Co-authored-by: unknown <intel@BGAOPTIPLEX001.gar.corp.intel.com>
sspintel
added a commit
that referenced
this pull request
Nov 7, 2022
* Update opset support to opset8 (#300) * Update opset support to opset8 * Update maxpool op params bas per pset8 in transpose_sinking cpp test * Performance optimizations (#313) * Remove older CustomGraphOptimizer methods * Update builder version to 2.0 * Enable Classification Sample through Grappler * Add AnalyticalCostEstimator that predicts and prints per-op costs in TF Logs * Change item to grappler_item * Refine graph optimization python API * Use cluster object from Grappler for device GOps * Working classification sample with grappler pass - Fix fetch nodes list - Do not consider feed, and fetch nodes for marking * Add translation for NMS ops and enable support in OCM for SquaredDifference,LessEqual,NotEqual,Cumsum * Update NMS implementation for transforming OV output shapes as per TF implementation * Disabling NMSV3 test for GPU, it fails for some trivial test fn checks, but passes when tested as a separate fn * Remove get_shape calls used for VLOG in Transpose * Replace get_shape with get_partial_shape wherever only rank is required * Added Op translations for CTCGreedyDecoder and BatchMatMulV2 (#301) * added translations for `CTCGreedyDecoder` and `BatchMatMulV2` * enable tests for `CTCGreedyDecoder` * enable tests for `BatchMatMulV2` * added custom test cases for `CTCGreedyDecoder` * fixed function name * Update OCM submodule to ocm master for the added ops * Implement SparseToDense OP for CPU * Fixed abuge introduced in fixing get_shape in TpKV2 * Restore min non trivial node threshold to right shift by 5, and update cluster busting comments * Cluster wise cost estimation using Grappler - Add GraphDef creation for each cluster at the deassign_clusters stage - Temporarily operate with the heuristic of scheduling only the top-1 costly cluster on OpenVINO * Fix translation of Select OP, and right shift condition in deassign clusters * Added Op translations and extended type support (#304) * added translation for `BatchMatMul` * added test for `Tile` * added test for `Reshape` * added test for `ExpandDims` * added tests for `BatchMatMul` and `BatchMatMulV2` * enable tests for `BatchMatMul` * disable float64 tests for GPU * update OCM submodule * Added fused translation for `CTCGreedyDecoder` (#305) * added translation for fused CTC greedy decoder * added tests for fused CTC greedy decoder * fixed typos * Update squeeze op translation to use input shape only when required * Add env variable to select TOP K clusters for execution, update omit cluster logic in deassign clusters * Update translation to handle dynamic shape for Range, and Zeroslike * Comment out zeroslike from dynamic to static flow, as translation is modified to handle it * Comment math_grad_test.SquaredDifferenceOpTest.testGrad test, failing with minor tolerance error after enabling double for certain OPs * Update Range, Fill, Prod and OneHot op translations and static inputs to handle dynamic input shapes * First cut of conv2d and fused conv2d with dynamic shape input * fixed bug in layout transformation * Cosmetic changes for dynamic conv2d support, add temp fix for protobuf version error in TF-2.8.0 * Add Cost based cluster de-assignment and fallback - Added grappler/costs/cost_analyzer.cc, CostAnalyzer->PredictCosts computes and tags node-wise runtime costs onto the graph as an attribute - DeassignClusters uses node-wise costs to add up cluster-wise costs and keep only top-3 clusters and busts the rest - At runtime, this cluster wise TF cost is compared against OV costs. Clusters which are costly yet perform poorly on OV fallsback to native session. * Enable size translation to handle dynamic input shapes, and concat translation to handle zero dimension inputs * Fix float to bool casting bug (#307) * Update ocm; Add infer time measurement in OV * Add OPENVINO_TF_MAX_CLUSTERS for TopK Costs * Rewrite Pass API call change * - Build both RewritePass and GrapplerOptimizationPass - Add API that controls if RewritePass is run in a given scope; Gets disabled when optimize_graph_with_openvino() is called - Add cluster_cost attr to nGraphEncapsulate NodeDef - Set NumWarmupSteps for TF Profiler to 1 * Revert EvictAllClusters from OVTFOptimizer * Enable rewrite pass to work alongside grappler pass (#310) * Modify TopKV2 tranlsation to handle dynamic shapes, and remove the corresponding static input condition. Code formatting done * Enable Rewrite Pass with Grappler pass *Modify deassign_clusters.cc to merge Min non trivial nodes changes with cost based deassignment logic *Enable Dynamic to Static flow for GPU with reduced cosntraints *Calculate cluster cost with native TF in compute kernel to decide run time assignment of cluster with Rewrite pass *Disable some of the NMS Ops in both grappler as well as Rewrite pass, as they are impacting performance of some OD models * Fix IsRewritePassEnabled api call syntax at 2 places * Add support for Frozen models with Grappler Pass (#311) * Add TFV1 OptimizeGraph API for Frozen models * Enable GrapplerOptimization pass for frozen models - Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs * Remove Grappler Related Build Flags; For UTs assume no Grappler * Remove Grappler pass tests; New tests TDB * Removed call to `update_config`; fixes failing unit tests (#312) * removed call to `update_config` * added missing attr `cluster_cost` * Fixes CI build failures (#314) * fixed protobuf version bug * cleanup: removed references to `infer_duration_in_ms` * Revert optimize_graph call in classification sample * cleanup post-processing Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add support for _MklSwish Op, generated when OneDNN optimizations are enabled (#316) * Add env variable based disable option for cost based runtime cluster assignment (#317) * Add checks, and refine optimize_graph_with_openvino_tfx APIs; Disable cost based fallback when Cluster Fallback is disabled (#318) * Add TFV1 OptimizeGraph API for Frozen models * Enable GrapplerOptimization pass for frozen models - Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs * Remove Grappler Related Build Flags; For UTs assume no Grappler * Fallback to TF only when Dynfallback is enabled in OpKernel Compute * Add checks, refinements, and multi-input support for the optimize_graph API * Add API Usage for optimize_graph_with_openvino_tf1 * Remove repeated protobuf install step Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Srihari Humbarwadi <srihari.humbarwadi@intel.com> * Add fixes for issues from static code analysis tool (#320) * TF Upgrade in Dockerfiles (#321) * Update TF version in notebooks and dockerfiles * - Changes in dockerfile used for wheel builds to support new TF CXX11 Upgrade - Rename Dockerfiles from 2.0.0 to 2.1.0 - Change OVTF version in Dockerfiles * Make Docker as default container registry to pull openvino dev images * Change crosstop_top toolchain to support manylinux2014 in TF build * Remove ADD rename_wheel.sh * Fix merge conflict in Ubuntu18 CI Dockerfile * Upgrade OVTF version in sample notebooks * Revert to working TF version in serving dockerfiles; Change OVTF version in notebook comments * Revert OVTF version in Serving dockerfiles * Upgrade to TF 2.8 version and update documentation (#267) * Initial attempt for translation to OV 2.0 * Backend related updates for OV 2.0 translation * Additional OV2.0 updates and VADM OV2.0 * TF-2.8 upgrade, and partial doc updates * Fix C++ test utilities for OV 2.0, update yml files for now to use ABI1 builds * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Remove -march=native option and update OV build flag * Disabling some of the OV build flags temporarily * Remove # typo * mac doesn't support working with openvino binary package * Fix mac yml Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add OVTF runtime Dockerfiles (#266) * Initial attempt for translation to OV 2.0 * Backend related updates for OV 2.0 translation * Additional OV2.0 updates and VADM OV2.0 * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Upgrade TF version to 2.8 in code * Update dockerfiles to include only OVTF build and runtime prep stages * Add steps to download GPL sources * Update README; Change tensorflow pkg link * Modify OpenVINO base image version tag * Initial commit for TF Serving enabled with OVTF * Adding dockerfiles with openvino tensorflow as base, readme * Add TF Serving Dockerfile * Update Serving Dockerfile * Remove Colab notebooks * Fix mac yml * Add Ubuntu20 TF Serving Dockerfile * Update README Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Laxmi Ganesan <laxmi.ganesan@intel.com> * Update cmakelist to filter OV libs for OVTF wheel (#272) * Update cmakelist for filtering the required OV libs for wheel generation * Correct cmake as gpu plugin is not available for mac * Update comments in cmakelist * Update CI yml, and dockerfiles (#271) * Initial attempt for translation to OV 2.0 * Additional OV2.0 updates and VADM OV2.0 * Fix C++ test utilities for OV 2.0, update yml files for now to use ABI1 builds * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Move CI Dockerfiles * Remove older tensorflow cxx11 ABI build dockerfile * Add new TF CXX11 build Dockerfile * Add steps to build cc lib and prepare tf build package * Add CXX_11 flag to cc libs build * Update CI Dockerfiles * Update yml and dockerfiles * Change default user in dockerfile * Change image descriptions * ubuntu18, ubuntu20, mac yml update * Update mac yml * Update mac agent pool * Fix pip version * Fix mac openvino_c.dylib and libtbb.dylib not found * Change TFHub Cache Dir Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> * Error check for backend type in cpp api (#243) * Fix set_backend python api to throw the right exception (#276) * Update wheel platform tags; ABI1 build is now manylinux_2_27 compatible (#275) * Initial attempt for translation to OV 2.0 * Additional OV2.0 updates and VADM OV2.0 * Add manylinux build tools * Update tag for manylinux2014 * Add platform check for manylinux tags Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> * OVTF releases 2.0.0 doc updates (#278) * Modify the readme for pre-release * Update build doc for 2.0 release and usage doc for convert variables to constants instructions * Update build doc for 2.0 release and usage doc for convert variables to constants instructions * Update docker documentation for more clarity * Update build and install docs for backward compatibility * Update Docker related documentation; Fix test-cases in notebooks * Enable OneDNN and VAR2CONST conversion in examples and notebooks * Fix CI * Update mandarin documentation files * Clarify docker documentation Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * TFFE merged into releases/2.0 (resolved conficts) * Removed rebase conflicts with TFFE branch * Updates to build with OV 2.0 master branch (#252) * Update cmakelists and build script to enable OVTF with OV 2.0 master branch * Add fix for BatchNDAndSpaceND op TF UT * Add support for OV 2022.1 version * Build OpenVINO from sources in CI * Remove ngraph lib dependency from CPP tests build Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add support for failing OMZ models (#256) * Comment ParseNodeAttributes call in Ngraphencapsulate, as it is unused and causing error for some models in windows * Update build script for symbolic linking of TF python unit test folder * Code formating for build_ovtf.py updates Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> * Disabling TopKV2 op (#257) * TopKV2 op disabled * Upgrade to TF 2.8 version and update documentation (#267) * Initial attempt for translation to OV 2.0 * Backend related updates for OV 2.0 translation * Additional OV2.0 updates and VADM OV2.0 * TF-2.8 upgrade, and partial doc updates * Fix C++ test utilities for OV 2.0, update yml files for now to use ABI1 builds * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Remove -march=native option and update OV build flag * Disabling some of the OV build flags temporarily * Remove # typo * mac doesn't support working with openvino binary package * Fix mac yml Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add OVTF runtime Dockerfiles (#266) * Initial attempt for translation to OV 2.0 * Backend related updates for OV 2.0 translation * Additional OV2.0 updates and VADM OV2.0 * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Upgrade TF version to 2.8 in code * Update dockerfiles to include only OVTF build and runtime prep stages * Add steps to download GPL sources * Update README; Change tensorflow pkg link * Modify OpenVINO base image version tag * Initial commit for TF Serving enabled with OVTF * Adding dockerfiles with openvino tensorflow as base, readme * Add TF Serving Dockerfile * Update Serving Dockerfile * Remove Colab notebooks * Fix mac yml * Add Ubuntu20 TF Serving Dockerfile * Update README Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Laxmi Ganesan <laxmi.ganesan@intel.com> * Update CI yml, and dockerfiles (#271) * Initial attempt for translation to OV 2.0 * Additional OV2.0 updates and VADM OV2.0 * Fix C++ test utilities for OV 2.0, update yml files for now to use ABI1 builds * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Move CI Dockerfiles * Remove older tensorflow cxx11 ABI build dockerfile * Add new TF CXX11 build Dockerfile * Add steps to build cc lib and prepare tf build package * Add CXX_11 flag to cc libs build * Update CI Dockerfiles * Update yml and dockerfiles * Change default user in dockerfile * Change image descriptions * ubuntu18, ubuntu20, mac yml update * Update mac yml * Update mac agent pool * Fix pip version * Fix mac openvino_c.dylib and libtbb.dylib not found * Change TFHub Cache Dir Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> * Error check for backend type in cpp api (#243) * Fix set_backend python api to throw the right exception (#276) * Update wheel platform tags; ABI1 build is now manylinux_2_27 compatible (#275) * Initial attempt for translation to OV 2.0 * Additional OV2.0 updates and VADM OV2.0 * Add manylinux build tools * Update tag for manylinux2014 * Add platform check for manylinux tags Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> * OVTF releases 2.0.0 doc updates (#278) * Modify the readme for pre-release * Update build doc for 2.0 release and usage doc for convert variables to constants instructions * Update build doc for 2.0 release and usage doc for convert variables to constants instructions * Update docker documentation for more clarity * Update build and install docs for backward compatibility * Update Docker related documentation; Fix test-cases in notebooks * Enable OneDNN and VAR2CONST conversion in examples and notebooks * Fix CI * Update mandarin documentation files * Clarify docker documentation Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * TFFE merged into releases/2.0 (resolved conficts) * Correct the conflicts during rebase * Update patch for Protobuf revert changes * Opset issue fix for TFFE * Temporary fix for control inputs (#328) * Temporary fix for control inputs * Ignoring additional outputs created by frontend * Comment lines and cleanup related to control edge fix * _MKLSwish op support for TFFE (#341) * Concat&NonMaxSupression conversion update, zero-dim fix (#343) * Slice size minus one support added (#347) * Slice size minus one support added * Hoisted parameter bug fix * Cast translation fix for bool * Temp. fix for hoisted param bug (#348) * Temp. fix for FusedBatchNorm duplicate parameters * Changed default OV to TF-FE dev. branch * Apply changes for protobuf from openvino (#352) * Apply changes for protobuf from openvino * Fix _Arg translator to use decoder for getting tensor * Remove function that is not needed anymore * Remove protobuf workaround * Switch to 2022.2 openvino branch * Input setting perf. fix (#356) * Input setting perf. fix * fix for convert variables to constant * Kotreshr/ctc greedy decoder conversion extension added (#360) * ctc_greedy_decoder tf_conversion_extension added * custom tf conversion extension for sparse to dense added * conv2d_kernels_check unit test modified as per the exception info changes * Mcavus/fix remaining model issues (#359) * Fix for squeeze_dim attribute issue * Temporary fix for DepthwiseConv2dNative issue * Fix for hoisted params issue * Handling non-existing attributes before translation * Revert temporary fix for DepthwiseConv2dNative issue * Bug fix for hoisted parameter handling * Fixed remaining rebase issues * Fix UTs, macOS Build Fix, and Samples SegFault; Apply code formatting (mcavus/tffe_releases_2.2) (#363) * Apply code formatting changes * Remove problematic call to EvictMRUClusters * Disable testFloorDivGrad as it's an unsupported training op * Rethrow NodeValidationFailure exception as InvalidArgument for TF Handling * Update exception handling of conv2D_KernelChecks * Add working fix to build TFFE Extensions on macOS * Enabled base OVTF translations as optional (#364) * Enabled base OVTF translations as optional * Temp. fix for skipped inputs bug & segfault fix * Apply code formatting * Handle OPENVINO_TF_DISABLE_TFFE envvar better Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * TF FE device related disabling & TS turned off (#365) * TF FE device related disabling & TS turned off * Code formating for TF FE device based disabling * TFFE build fixes for windows (#366) * build fixes for windows * tf windows build patch updated * Enable building OpenVINO from source for Windows CI * Comment OpenVINO Download to recover VM disk space * wrapped blocks with win32 check Co-authored-by: unknown <intel@BGAOPTIPLEX001.gar.corp.intel.com> Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Option to enable performance counters for debugging with OPENVINO_TF_ENABLE_PERF_COUNT (#369) * TF FE device related disabling & TS turned off * Code formating for TF FE device based disabling * Enabling OPENVINO_TF_ENABLE_PERF_COUNT prints performance counters at VLOG_LEVEL=1 * Handle setting envvars in BackendManager * Add back TFFrontendDisabled * Add OVTFProfilingEnabled checks for encapsulate_op Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: Chandrakant Khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> Co-authored-by: Srihari Humbarwadi <srihari.humbarwadi@intel.com> Co-authored-by: Laxmi Ganesan <laxmi.ganesan@intel.com> Co-authored-by: Athina Saha <athina.saha@intel.com> Co-authored-by: Maxim Vafin <maxim.vafin@intel.com> Co-authored-by: kotreshr <111343878+kotreshr@users.noreply.github.com> Co-authored-by: unknown <intel@BGAOPTIPLEX001.gar.corp.intel.com>
sspintel
added a commit
that referenced
this pull request
Nov 11, 2022
* Update opset support to opset8 (#300) * Update opset support to opset8 * Update maxpool op params bas per pset8 in transpose_sinking cpp test * Performance optimizations (#313) * Remove older CustomGraphOptimizer methods * Update builder version to 2.0 * Enable Classification Sample through Grappler * Add AnalyticalCostEstimator that predicts and prints per-op costs in TF Logs * Change item to grappler_item * Refine graph optimization python API * Use cluster object from Grappler for device GOps * Working classification sample with grappler pass - Fix fetch nodes list - Do not consider feed, and fetch nodes for marking * Add translation for NMS ops and enable support in OCM for SquaredDifference,LessEqual,NotEqual,Cumsum * Update NMS implementation for transforming OV output shapes as per TF implementation * Disabling NMSV3 test for GPU, it fails for some trivial test fn checks, but passes when tested as a separate fn * Remove get_shape calls used for VLOG in Transpose * Replace get_shape with get_partial_shape wherever only rank is required * Added Op translations for CTCGreedyDecoder and BatchMatMulV2 (#301) * added translations for `CTCGreedyDecoder` and `BatchMatMulV2` * enable tests for `CTCGreedyDecoder` * enable tests for `BatchMatMulV2` * added custom test cases for `CTCGreedyDecoder` * fixed function name * Update OCM submodule to ocm master for the added ops * Implement SparseToDense OP for CPU * Fixed abuge introduced in fixing get_shape in TpKV2 * Restore min non trivial node threshold to right shift by 5, and update cluster busting comments * Cluster wise cost estimation using Grappler - Add GraphDef creation for each cluster at the deassign_clusters stage - Temporarily operate with the heuristic of scheduling only the top-1 costly cluster on OpenVINO * Fix translation of Select OP, and right shift condition in deassign clusters * Added Op translations and extended type support (#304) * added translation for `BatchMatMul` * added test for `Tile` * added test for `Reshape` * added test for `ExpandDims` * added tests for `BatchMatMul` and `BatchMatMulV2` * enable tests for `BatchMatMul` * disable float64 tests for GPU * update OCM submodule * Added fused translation for `CTCGreedyDecoder` (#305) * added translation for fused CTC greedy decoder * added tests for fused CTC greedy decoder * fixed typos * Update squeeze op translation to use input shape only when required * Add env variable to select TOP K clusters for execution, update omit cluster logic in deassign clusters * Update translation to handle dynamic shape for Range, and Zeroslike * Comment out zeroslike from dynamic to static flow, as translation is modified to handle it * Comment math_grad_test.SquaredDifferenceOpTest.testGrad test, failing with minor tolerance error after enabling double for certain OPs * Update Range, Fill, Prod and OneHot op translations and static inputs to handle dynamic input shapes * First cut of conv2d and fused conv2d with dynamic shape input * fixed bug in layout transformation * Cosmetic changes for dynamic conv2d support, add temp fix for protobuf version error in TF-2.8.0 * Add Cost based cluster de-assignment and fallback - Added grappler/costs/cost_analyzer.cc, CostAnalyzer->PredictCosts computes and tags node-wise runtime costs onto the graph as an attribute - DeassignClusters uses node-wise costs to add up cluster-wise costs and keep only top-3 clusters and busts the rest - At runtime, this cluster wise TF cost is compared against OV costs. Clusters which are costly yet perform poorly on OV fallsback to native session. * Enable size translation to handle dynamic input shapes, and concat translation to handle zero dimension inputs * Fix float to bool casting bug (#307) * Update ocm; Add infer time measurement in OV * Add OPENVINO_TF_MAX_CLUSTERS for TopK Costs * Rewrite Pass API call change * - Build both RewritePass and GrapplerOptimizationPass - Add API that controls if RewritePass is run in a given scope; Gets disabled when optimize_graph_with_openvino() is called - Add cluster_cost attr to nGraphEncapsulate NodeDef - Set NumWarmupSteps for TF Profiler to 1 * Revert EvictAllClusters from OVTFOptimizer * Enable rewrite pass to work alongside grappler pass (#310) * Modify TopKV2 tranlsation to handle dynamic shapes, and remove the corresponding static input condition. Code formatting done * Enable Rewrite Pass with Grappler pass *Modify deassign_clusters.cc to merge Min non trivial nodes changes with cost based deassignment logic *Enable Dynamic to Static flow for GPU with reduced cosntraints *Calculate cluster cost with native TF in compute kernel to decide run time assignment of cluster with Rewrite pass *Disable some of the NMS Ops in both grappler as well as Rewrite pass, as they are impacting performance of some OD models * Fix IsRewritePassEnabled api call syntax at 2 places * Add support for Frozen models with Grappler Pass (#311) * Add TFV1 OptimizeGraph API for Frozen models * Enable GrapplerOptimization pass for frozen models - Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs * Remove Grappler Related Build Flags; For UTs assume no Grappler * Remove Grappler pass tests; New tests TDB * Removed call to `update_config`; fixes failing unit tests (#312) * removed call to `update_config` * added missing attr `cluster_cost` * Fixes CI build failures (#314) * fixed protobuf version bug * cleanup: removed references to `infer_duration_in_ms` * Revert optimize_graph call in classification sample * cleanup post-processing Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add support for _MklSwish Op, generated when OneDNN optimizations are enabled (#316) * Add env variable based disable option for cost based runtime cluster assignment (#317) * Add checks, and refine optimize_graph_with_openvino_tfx APIs; Disable cost based fallback when Cluster Fallback is disabled (#318) * Add TFV1 OptimizeGraph API for Frozen models * Enable GrapplerOptimization pass for frozen models - Cluster GraphDefs aren't destroyed for Grappler mode as CPU is the only backend made available for Grappler, creating a new Backend() should preserve any previously cached GraphDefs * Remove Grappler Related Build Flags; For UTs assume no Grappler * Fallback to TF only when Dynfallback is enabled in OpKernel Compute * Add checks, refinements, and multi-input support for the optimize_graph API * Add API Usage for optimize_graph_with_openvino_tf1 * Remove repeated protobuf install step Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Srihari Humbarwadi <srihari.humbarwadi@intel.com> * Add fixes for issues from static code analysis tool (#320) * TF Upgrade in Dockerfiles (#321) * Update TF version in notebooks and dockerfiles * - Changes in dockerfile used for wheel builds to support new TF CXX11 Upgrade - Rename Dockerfiles from 2.0.0 to 2.1.0 - Change OVTF version in Dockerfiles * Make Docker as default container registry to pull openvino dev images * Change crosstop_top toolchain to support manylinux2014 in TF build * Remove ADD rename_wheel.sh * Fix merge conflict in Ubuntu18 CI Dockerfile * Upgrade OVTF version in sample notebooks * Revert to working TF version in serving dockerfiles; Change OVTF version in notebook comments * Revert OVTF version in Serving dockerfiles * Upgrade to TF 2.8 version and update documentation (#267) * Initial attempt for translation to OV 2.0 * Backend related updates for OV 2.0 translation * Additional OV2.0 updates and VADM OV2.0 * TF-2.8 upgrade, and partial doc updates * Fix C++ test utilities for OV 2.0, update yml files for now to use ABI1 builds * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Remove -march=native option and update OV build flag * Disabling some of the OV build flags temporarily * Remove # typo * mac doesn't support working with openvino binary package * Fix mac yml Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add OVTF runtime Dockerfiles (#266) * Initial attempt for translation to OV 2.0 * Backend related updates for OV 2.0 translation * Additional OV2.0 updates and VADM OV2.0 * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Upgrade TF version to 2.8 in code * Update dockerfiles to include only OVTF build and runtime prep stages * Add steps to download GPL sources * Update README; Change tensorflow pkg link * Modify OpenVINO base image version tag * Initial commit for TF Serving enabled with OVTF * Adding dockerfiles with openvino tensorflow as base, readme * Add TF Serving Dockerfile * Update Serving Dockerfile * Remove Colab notebooks * Fix mac yml * Add Ubuntu20 TF Serving Dockerfile * Update README Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Laxmi Ganesan <laxmi.ganesan@intel.com> * Update cmakelist to filter OV libs for OVTF wheel (#272) * Update cmakelist for filtering the required OV libs for wheel generation * Correct cmake as gpu plugin is not available for mac * Update comments in cmakelist * Update CI yml, and dockerfiles (#271) * Initial attempt for translation to OV 2.0 * Additional OV2.0 updates and VADM OV2.0 * Fix C++ test utilities for OV 2.0, update yml files for now to use ABI1 builds * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Move CI Dockerfiles * Remove older tensorflow cxx11 ABI build dockerfile * Add new TF CXX11 build Dockerfile * Add steps to build cc lib and prepare tf build package * Add CXX_11 flag to cc libs build * Update CI Dockerfiles * Update yml and dockerfiles * Change default user in dockerfile * Change image descriptions * ubuntu18, ubuntu20, mac yml update * Update mac yml * Update mac agent pool * Fix pip version * Fix mac openvino_c.dylib and libtbb.dylib not found * Change TFHub Cache Dir Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> * Error check for backend type in cpp api (#243) * Fix set_backend python api to throw the right exception (#276) * Update wheel platform tags; ABI1 build is now manylinux_2_27 compatible (#275) * Initial attempt for translation to OV 2.0 * Additional OV2.0 updates and VADM OV2.0 * Add manylinux build tools * Update tag for manylinux2014 * Add platform check for manylinux tags Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> * OVTF releases 2.0.0 doc updates (#278) * Modify the readme for pre-release * Update build doc for 2.0 release and usage doc for convert variables to constants instructions * Update build doc for 2.0 release and usage doc for convert variables to constants instructions * Update docker documentation for more clarity * Update build and install docs for backward compatibility * Update Docker related documentation; Fix test-cases in notebooks * Enable OneDNN and VAR2CONST conversion in examples and notebooks * Fix CI * Update mandarin documentation files * Clarify docker documentation Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * TFFE merged into releases/2.0 (resolved conficts) * Removed rebase conflicts with TFFE branch * Updates to build with OV 2.0 master branch (#252) * Update cmakelists and build script to enable OVTF with OV 2.0 master branch * Add fix for BatchNDAndSpaceND op TF UT * Add support for OV 2022.1 version * Build OpenVINO from sources in CI * Remove ngraph lib dependency from CPP tests build Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add support for failing OMZ models (#256) * Comment ParseNodeAttributes call in Ngraphencapsulate, as it is unused and causing error for some models in windows * Update build script for symbolic linking of TF python unit test folder * Code formating for build_ovtf.py updates Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> * Disabling TopKV2 op (#257) * TopKV2 op disabled * Upgrade to TF 2.8 version and update documentation (#267) * Initial attempt for translation to OV 2.0 * Backend related updates for OV 2.0 translation * Additional OV2.0 updates and VADM OV2.0 * TF-2.8 upgrade, and partial doc updates * Fix C++ test utilities for OV 2.0, update yml files for now to use ABI1 builds * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Remove -march=native option and update OV build flag * Disabling some of the OV build flags temporarily * Remove # typo * mac doesn't support working with openvino binary package * Fix mac yml Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Add OVTF runtime Dockerfiles (#266) * Initial attempt for translation to OV 2.0 * Backend related updates for OV 2.0 translation * Additional OV2.0 updates and VADM OV2.0 * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Upgrade TF version to 2.8 in code * Update dockerfiles to include only OVTF build and runtime prep stages * Add steps to download GPL sources * Update README; Change tensorflow pkg link * Modify OpenVINO base image version tag * Initial commit for TF Serving enabled with OVTF * Adding dockerfiles with openvino tensorflow as base, readme * Add TF Serving Dockerfile * Update Serving Dockerfile * Remove Colab notebooks * Fix mac yml * Add Ubuntu20 TF Serving Dockerfile * Update README Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Laxmi Ganesan <laxmi.ganesan@intel.com> * Update CI yml, and dockerfiles (#271) * Initial attempt for translation to OV 2.0 * Additional OV2.0 updates and VADM OV2.0 * Fix C++ test utilities for OV 2.0, update yml files for now to use ABI1 builds * Update ymls for sourcing OV, and updating cmake build option for OV2.0 * Move CI Dockerfiles * Remove older tensorflow cxx11 ABI build dockerfile * Add new TF CXX11 build Dockerfile * Add steps to build cc lib and prepare tf build package * Add CXX_11 flag to cc libs build * Update CI Dockerfiles * Update yml and dockerfiles * Change default user in dockerfile * Change image descriptions * ubuntu18, ubuntu20, mac yml update * Update mac yml * Update mac agent pool * Fix pip version * Fix mac openvino_c.dylib and libtbb.dylib not found * Change TFHub Cache Dir Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: chandrakant khandelwal <chandrakant.khandelwal@intel.com> * Error check for backend type in cpp api (#243) * Fix set_backend python api to throw the right exception (#276) * Update wheel platform tags; ABI1 build is now manylinux_2_27 compatible (#275) * Initial attempt for translation to OV 2.0 * Additional OV2.0 updates and VADM OV2.0 * Add manylinux build tools * Update tag for manylinux2014 * Add platform check for manylinux tags Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> * OVTF releases 2.0.0 doc updates (#278) * Modify the readme for pre-release * Update build doc for 2.0 release and usage doc for convert variables to constants instructions * Update build doc for 2.0 release and usage doc for convert variables to constants instructions * Update docker documentation for more clarity * Update build and install docs for backward compatibility * Update Docker related documentation; Fix test-cases in notebooks * Enable OneDNN and VAR2CONST conversion in examples and notebooks * Fix CI * Update mandarin documentation files * Clarify docker documentation Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * TFFE merged into releases/2.0 (resolved conficts) * Correct the conflicts during rebase * Update patch for Protobuf revert changes * Opset issue fix for TFFE * Temporary fix for control inputs (#328) * Temporary fix for control inputs * Ignoring additional outputs created by frontend * Comment lines and cleanup related to control edge fix * _MKLSwish op support for TFFE (#341) * Concat&NonMaxSupression conversion update, zero-dim fix (#343) * Slice size minus one support added (#347) * Slice size minus one support added * Hoisted parameter bug fix * Cast translation fix for bool * Temp. fix for hoisted param bug (#348) * Temp. fix for FusedBatchNorm duplicate parameters * Changed default OV to TF-FE dev. branch * Apply changes for protobuf from openvino (#352) * Apply changes for protobuf from openvino * Fix _Arg translator to use decoder for getting tensor * Remove function that is not needed anymore * Remove protobuf workaround * Switch to 2022.2 openvino branch * Input setting perf. fix (#356) * Input setting perf. fix * fix for convert variables to constant * Kotreshr/ctc greedy decoder conversion extension added (#360) * ctc_greedy_decoder tf_conversion_extension added * custom tf conversion extension for sparse to dense added * conv2d_kernels_check unit test modified as per the exception info changes * Mcavus/fix remaining model issues (#359) * Fix for squeeze_dim attribute issue * Temporary fix for DepthwiseConv2dNative issue * Fix for hoisted params issue * Handling non-existing attributes before translation * Revert temporary fix for DepthwiseConv2dNative issue * Bug fix for hoisted parameter handling * Fixed remaining rebase issues * Fix UTs, macOS Build Fix, and Samples SegFault; Apply code formatting (mcavus/tffe_releases_2.2) (#363) * Apply code formatting changes * Remove problematic call to EvictMRUClusters * Disable testFloorDivGrad as it's an unsupported training op * Rethrow NodeValidationFailure exception as InvalidArgument for TF Handling * Update exception handling of conv2D_KernelChecks * Add working fix to build TFFE Extensions on macOS * Enabled base OVTF translations as optional (#364) * Enabled base OVTF translations as optional * Temp. fix for skipped inputs bug & segfault fix * Apply code formatting * Handle OPENVINO_TF_DISABLE_TFFE envvar better Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * TF FE device related disabling & TS turned off (#365) * TF FE device related disabling & TS turned off * Code formating for TF FE device based disabling * TFFE build fixes for windows (#366) * build fixes for windows * tf windows build patch updated * Enable building OpenVINO from source for Windows CI * Comment OpenVINO Download to recover VM disk space * wrapped blocks with win32 check Co-authored-by: unknown <intel@BGAOPTIPLEX001.gar.corp.intel.com> Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> * Option to enable performance counters for debugging with OPENVINO_TF_ENABLE_PERF_COUNT (#369) * TF FE device related disabling & TS turned off * Code formating for TF FE device based disabling * Enabling OPENVINO_TF_ENABLE_PERF_COUNT prints performance counters at VLOG_LEVEL=1 * Handle setting envvars in BackendManager * Add back TFFrontendDisabled * Add OVTFProfilingEnabled checks for encapsulate_op Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com> Co-authored-by: Chandrakant Khandelwal <chandrakant.khandelwal@intel.com> Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com> Co-authored-by: Srihari Humbarwadi <srihari.humbarwadi@intel.com> Co-authored-by: Laxmi Ganesan <laxmi.ganesan@intel.com> Co-authored-by: Athina Saha <athina.saha@intel.com> Co-authored-by: Maxim Vafin <maxim.vafin@intel.com> Co-authored-by: kotreshr <111343878+kotreshr@users.noreply.github.com> Co-authored-by: unknown <intel@BGAOPTIPLEX001.gar.corp.intel.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.