Skip to content

tf.linalg.det gradient crashes on singular matrices #119477

@wuyii8941

Description

@wuyii8941

Issue type

Bug

Have you reproduced the bug with TensorFlow Nightly?

Yes

Source

source

TensorFlow version

TensorFlow: 2.22.0-dev20260508

Custom code

Yes

OS platform and distribution

No response

Mobile device

No response

Python version

No response

Bazel version

No response

GCC/compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current behavior?

Summary

Computing the gradient of tf.linalg.det via tf.GradientTape crashes with InvalidArgumentError: Input is not invertible when the input matrix is singular (det=0). The forward pass correctly returns 0, but the backward pass fails because the gradient implementation internally calls matrix_inverse.

Root cause

The gradient of det(A) is currently implemented as det(A) * A^{-T}, which requires computing A^{-1}. When det(A) = 0, both factors are problematic: det(A) = 0 and A^{-1} doesn't exist. The gradient is actually 0 * ∞, which is undefined.

The mathematically correct gradient is the adjugate matrix (transpose of cofactor matrix):

d/dA det(A) = adj(A)^T

For A = [[1, 2], [2, 4]], the correct gradient is [[4, -2], [-2, 1]]. The adjugate is always well-defined, even for singular matrices.

Impact

This crash occurs in any optimization that involves determinants of matrices that can become singular during training, such as:

  • Normalizing flow models (det of Jacobian)
  • Gaussian process models (det of covariance matrix)
  • Any regularization term involving log(det(A)) where A approaches singularity

Also affects tf.linalg.slogdet

tf.linalg.slogdet has the same crash: the forward pass returns sign=0, logabsdet=-inf correctly, but the gradient crashes with InvalidArgumentError: Input is not invertible because it also uses MatrixInverse internally.

m = tf.constant([[1.0, 2.0], [2.0, 4.0]], dtype=tf.float64)
with tf.GradientTape() as tape:
    tape.watch(m)
    sign, logabsdet = tf.linalg.slogdet(m)
g = tape.gradient(logabsdet, m)  # CRASH: InvalidArgumentError

Environment

  • TensorFlow: 2.22.0-dev20260508
  • OS: Ubuntu 20.04
  • Affects both CPU and GPU

Standalone code to reproduce the issue

### Reproduction


import tensorflow as tf

m = tf.constant([[1.0, 2.0], [2.0, 4.0]], dtype=tf.float64)
print("det:", tf.linalg.det(m).numpy())  # 0.0

with tf.GradientTape() as tape:
    tape.watch(m)
    d = tf.linalg.det(m)
g = tape.gradient(d, m)  # CRASH: InvalidArgumentError


**Error:**

InvalidArgumentError: Input is not invertible. [Op:MatrixInverse]

Relevant log output

Metadata

Metadata

Assignees

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions