[Good First Issue][TF FE]: Support MatrixInverse operation for TensorFlow

[rkazants]

Context

OpenVINO component responsible for support of TensorFlow models is called as TensorFlow Frontend (TF FE). TF FE converts a model represented in TensorFlow opset to a model in OpenVINO opset.

In order to infer TensorFlow models with MatrixInverse operation by OpenVINO, TF FE needs to be extended with this operation support.

What needs to be done?

For MatrixInverse operation support, you need to implement the corresponding loader into TF FE op directory and to register it into the dictionary of Loaders. One loader is responsible for conversion (or decomposition) of one type of TensorFlow operation.

Here is an example of loader implementation for TensorFlow Einsum operation:

OutputVector translate_einsum_op(const NodeContext& node) { 
     auto op_type = node.get_op_type(); 
     TENSORFLOW_OP_VALIDATION(node, op_type == "Einsum", "Internal error: incorrect usage of translate_einsum_op."); 
     auto equation = node.get_attribute<std::string>("equation"); 
  
     OutputVector inputs; 
     for (size_t input_ind = 0; input_ind < node.get_input_size(); ++input_ind) { 
         inputs.push_back(node.get_input(input_ind)); 
     } 
  
     auto einsum = make_shared<Einsum>(inputs, equation); 
     set_node_name(node.get_name(), einsum); 
     return {einsum}; 
 } 

In this example, translate_einsum_op converts TF Einsum into OV Einsum. NodeContext object passed into the loader packs all info about inputs and attributes of Einsum operation. The loader retrieves an attribute of the equation by using the NodeContext::get_attribute() method, prepares input vector, creates Einsum operation from OV opset and returns a vector of outputs.

Responsibility of a loader is to parse operation attributes, prepare inputs and express TF operation via OV operations sub-graph. Example for Einsum demonstrates the resulted sub-graph with one operation. In PR #19007 you can see operation decomposition into multiple node sub-graph.

Once you are done with implementation of the translator, you need to implement the corresponding layer tests test_tf_MatrixInverse.py and put it into layer_tests/tensorflow_tests directory. Example how to run some layer test:

export TEST_DEVICE=CPU
cd openvino/tests/layer_tests/tensorflow_tests
pytest test_tf_Shape.py

Hint

Use newly added operation Inverse into OV opset-14

Example Pull Requests

• [TF FE][TF Hub] Support DivNoNan and EnsureShape operations #19007

Resources

• What is OpenVINO?
• How to Build OpenVINO
• Developer documentation for TensorFlow Frontend
• Contribution guide - start here!
• Intel DevHub Discord channel - engage in discussions, ask questions and talk to OpenVINO developers

Contact points

• @openvinotoolkit/openvino-tf-frontend-maintainers
• @rkazants in GitHub
• rkazants in Discord

Ticket

No response

[koookieee]

.take

[github-actions]

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

[koookieee]

Hello @rkazants , I will be working on this issue. Let me know if you have any comments or suggestions !

[rkazants]

Hi @koookieee, any update on the task?

Best regards,
Roman

[dyogaharshitha]

.take

[github-actions]

Thanks for being interested in this issue. It looks like this ticket is already assigned to a contributor. Please communicate with the assigned contributor to confirm the status of the issue.

[rkazants]

No update for long time, I have to release this task.

Best regards,
Roman

[hongbo-wei]

.take

[github-actions]

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

[hongbo-wei]

Hello @rkazants, I've built OpenVINO locally, does this mean I can start coding and contributing? OpenVINO is a software toolkit, compared to other simpler applications, it's hard to have a direct visual of what is going on there, hence it is easier for developers to know what to do next. So I wonder if I'm on the right track : )

[rkazants]

Hello @rkazants, I've built OpenVINO locally, does this mean I can start coding and contributing? OpenVINO is a software toolkit, compared to other simpler applications, it's hard to have a direct visual of what is going on there, hence it is easier for developers to know what to do next. So I wonder if I'm on the right track : )

Hi @hongbo-wei,

Now you need to implement translator for TF MatrixInverse to OV opset Inverse. Please check: https://docs.openvino.ai/2024/documentation/openvino-ir-format/operation-sets/operation-specs/matrix/Inverse_14.html

Should be simple translator. Check translators for other operations and description above.

Best regards,
Roman

[hongbo-wei]

Hello, Roman @rkazants, I want to build OpenVINO runtime to test my contributed program, and I follow the instructions in Mac (ARM).

In the second last step, I run cmake -G "Ninja Multi-Config" -DENABLE_SYSTEM_PUGIXML=ON -DENABLE_SYSTEM_SNAPPY=ON -DENABLE_SYSTEM_PROTOBUF=ON .., and was told that couldn't find Protobuf even it is installed using brew. I therefore concatenated -DPROTOBUF_LIBRARY=~/opt/homebrew/opt/protobuf before .. in the command, and run it. No error.

In the last step I encountered an error that says 'protobuf::libprotobuf-NOTFOUND'.

ninja: error: 'protobuf::libprotobuf-NOTFOUND', needed by '/Users/hongbo_wei/downloads/GitHub/openvino/bin/arm64/Release/libopenvino_tensorflow_frontend.2024.2.0.dylib', missing and no known rule to make it

I then set the environment path for CMake using the right path to protobuf, but the build system still cannot find it

export CMAKE_PREFIX_PATH=/opt/homebrew/opt/protobuf
export CMAKE_LIBRARY_PATH=/opt/homebrew/opt/protobuf

I also asked in Intel Community.

I'm so happy I'm moving closer to complete my Good First Issue.

[hongbo-wei]

Mac (ARM)

Correct path of Protobuf found and fixed. Encounter error when building.

/opt/homebrew/include/absl/functional/internal/any_invocable.h:380:28: error: no member named 'in_place_type_t' in namespace 'absl'
struct IsInPlaceType<absl::in_place_type_t<T>> : std::true_type {};
                     ~~~~~~^
/opt/homebrew/include/absl/functional/internal/any_invocable.h:380:44: error: 'T' does not refer to a value
struct IsInPlaceType<absl::in_place_type_t<T>> : std::true_type {};
                                           ^
/opt/homebrew/include/absl/functional/internal/any_invocable.h:379:17: note: declared here
template <class T>
                ^
/opt/homebrew/include/absl/functional/internal/any_invocable.h:380:46: error: expected unqualified-id
struct IsInPlaceType<absl::in_place_type_t<T>> : std::true_type {};
                                             ^
/opt/homebrew/include/absl/functional/internal/any_invocable.h:476:27: error: no template named 'in_place_type_t' in namespace 'absl'
  explicit CoreImpl(absl::in_place_type_t<QualTRef>, Args&&... args) {
                    ~~~~~~^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
ninja: build stopped: subcommand failed.