Add tensorflow tensor support

[darshil929]

Changes

• Added support for ord=0 in tf_linalg.py's norm function
• Fixed the keepdims parameter for nuclear norm case
• Added tests that check norm with keepdims=True for both CPU and GPU classes
• Added tests for lstsq function with multiple right-hand sides (rank-2 tensors)

Reason for changes

Right now TensorFlow tensors don't work with nncf.Tensor, which is blocking issue #3041 from being done. This PR adds TensorFlow backend support so we can use TensorFlow tensors with NNCF operations. I'm continuing the work from PR #3106 but fixing some things reviewers pointed out and adding stuff that was missing.

Related tickets

#3041
#3106

Tests

• Added a test_norm_keepdims function to both CPU and GPU classes to check if keepdims works right
• Added test_lstsq_rank2 to make sure least squares works with multiple columns
• Note: I've added the requested tests, but encountered environment setup issues that prevented me from running them successfully. The error appears to be related to function signature registration in the TensorFlow backend integration, which I think is beyond the scope of my changes

[darshil929]

@kshpv

I've implemented the suggested change to the save_file.register decorator. I've also fixed string literals in exceptions in tf_linalg.py and tf_numeric.py to address the pre-commit linting issues that were causing CI failures. The PR should be ready for review now.

[kshpv]

@kshpv

I've implemented the suggested change to the save_file.register decorator. I've also fixed string literals in exceptions in tf_linalg.py and tf_numeric.py to address the pre-commit linting issues that were causing CI failures. The PR should be ready for review now.

Hi @darshil929
Could you please fix precommit issues at first?

[darshil929]

@kshpv
I've implemented the suggested change to the save_file.register decorator. I've also fixed string literals in exceptions in tf_linalg.py and tf_numeric.py to address the pre-commit linting issues that were causing CI failures. The PR should be ready for review now.

Hi @darshil929 Could you please fix precommit issues at first?

I'm analyzing the precommit issues now and will get them fixed as soon as possible.

[darshil929]

Am I supposed to create a new PR to complete the work on issue #3041 or it should be done in the current PR?

It would be easier for reviewers to split the work into parts. I suggest creating a new PR

Hi @kshpv
I'm currently working on preparing my proposal draft for GSOC project 14 - "Accelerating Inference of NNCF-Compressed LLMs with Triton," for OpenVINO-NNCF. I'll raise the PR for issue #3041 as soon as my proposal work is completed, which should be within a couple of days. I would really appreciate a review of my proposal and suggestions for improvement. What would be the best way to share my Google Doc with you? Would you prefer I send it to your email, or should I contact you through Discord instead?
Thank you for your time and guidance.

You can send your proposal to alexander.suslov@intel.com

Sure sir, I am working on a example-prototype to add as a part of my proposal and demonstrate my understandings which will take a little time to complete. As soon as it is done, I will email you the draft.

[kshpv]

@kshpv

Sir, I've made some minor variable name changes and re-applied tf.identity() to the functions that were facing device preservation issues initially to maintain consistency in code. Sorry for the delay.

Could you check that adding tf.identity() resolves device issues? For all tests where it resolves, please remove the skipping device check. If it does not resolve, then please roll back adding tf.identity()

[darshil929]

@kshpv
Sir, I've made some minor variable name changes and re-applied tf.identity() to the functions that were facing device preservation issues initially to maintain consistency in code. Sorry for the delay.

Could you check that adding tf.identity() resolves device issues? For all tests where it resolves, please remove the skipping device check. If it does not resolve, then please roll back adding tf.identity()

Sir, I have made the required changes.

[darshil929]

Hi @alexsu52
Sir, I've implemented comprehensive test coverage for the norm function as requested:

• Added tests for all norm types on tensors from 1D to 4D
• Fixed the bug with nuclear norm and keepdims parameter for higher dimensions
• Added tests for ord=0 implementation
• Included edge case tests for empty tensors and extreme values (NaN, Inf)

All tests now pass. Could you please review when you have a chance.
Thank you!

[darshil929]

Hi @alexsu52

Sir, I've successfully moved norm tests to the TemplateTestNNCFTensorOperators class. All tests are running perfectly on the TensorFlow backend.

However, there are some failures on PyTorch and NumPy backends.

PyTorch backend failures:

NumPy backend failures:

Could you please advice if I should modify the backend implementations to handle these edge cases?
Or is there another approach you'd recommend?

Thank you!

[alexsu52]

Hi @alexsu52

Sir, I've successfully moved norm tests to the TemplateTestNNCFTensorOperators class. All tests are running perfectly on the TensorFlow backend.

However, there are some failures on PyTorch and NumPy backends.

PyTorch backend failures:

NumPy backend failures:

Could you please advice if I should modify the backend implementations to handle these edge cases? Or is there another approach you'd recommend?

Thank you!

Hi @darshil929,

Thank you for the contribution!

Please, update tests regarding the description of the norm function https://github.com/alexsu52/nncf/blob/develop/nncf/tensor/functions/linalg.py#L19. For example, the test test_norm_order_zero failed calling result = fns.linalg.norm(tensor_2d, ord=0), but the norm function does not supports ord=0 for 2d tensor https://github.com/alexsu52/nncf/blob/develop/nncf/tensor/functions/linalg.py#L38.

Some insights, numpy behavior is the default behavior unless otherwise noted. If you find a bug in the numpy, pytorch or openvino backends, please feel free to fix it. Please also keep the function description up to date.

[alexsu52]

Please, fix the pre-commit tests.

[darshil929]

Sir,

I've updated the norm tests to align with the documented behavior in linalg.py where ord=0 is officially supported only for vectors, not matrices. The tests now properly validate vector norm behavior while maintaining compatibility across all backends (TensorFlow, PyTorch, and NumPy).

I've also preserved the TensorFlow implementation's extended capabilities for ord=0

All tests are now passing. Could you please review these changes?

Thanks!

[alexsu52]

LGTM