tf.keras.backend.eye crash (abort) with large input

[DNXie]

Click to expand!

Issue Type

Bug

Source

binary

Tensorflow Version

2.11.0-dev20220914

Custom Code

No

OS Platform and Distribution

Ubuntu 18.04.4 LTS (x86_64)

Mobile device

No response

Python version

3.7.6

Bazel version

No response

GCC/Compiler version

No response

CUDA/cuDNN version

N/A

GPU model and memory

No response

Current Behaviour?

tf.keras.backend.eye and tf.eye crash (abort) with large input

Also reproduced in this gist

Standalone code to reproduce the issue

import tensorflow as tf
tf.keras.backend.eye(size=2752212975)

import tensorflow as tf
tf.eye(2752212975)

Relevant log output

2022-09-15 18:51:32.477313: F tensorflow/core/framework/tensor_shape.cc:572] Check failed: size >= 0 (0 vs. -1542754321)
Aborted (core dumped)

[tilakrayal]

@sachinprasadhs,
I was able to reproduce the issue on tensorflow v2.8, v2.9 and nightly. Kindly find the gist of it here.

[sachinprasadhs]

This is due to the very large input which is causing OOM/ memory overflow with large input, when you try large value like tf.int32.max, you will get the error output.
Below is the error output.

import tensorflow as tf
tf.eye(2147483647)

ResourceExhaustedError: OOM when allocating tensor with shape[2147483647,2147483647] and type float on /job:localhost/replica:0/task:0/device:CPU:0 by allocator cpu [Op:MatrixDiagV3] name: diag 

[DNXie]

@sachinprasadhs Hi, Thanks for looking into this. With the input I provided, I see a crash (abortion) instead of an OOM error.

As public APIs, it would be great to have the functions kindly throw exceptions for these cases instead of crashing.

[yongtang]

Added a PR #57790 for the fix.

[tilakrayal]

@DNXie,
When I tried to execute the code on tf-nightly, I observed that the crash happened in colab where in the run time logs it mentioned the OOM error as the warning for the large input. Kindly find the screenshots for the reference

[github-actions]

This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you.

[github-actions]

This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No