Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
source
TensorFlow version
TensorFlow: 2.22.0-dev20260508
Custom code
Yes
OS platform and distribution
No response
Mobile device
No response
Python version
No response
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
Summary
Computing the gradient of tf.linalg.det via tf.GradientTape crashes with InvalidArgumentError: Input is not invertible when the input matrix is singular (det=0). The forward pass correctly returns 0, but the backward pass fails because the gradient implementation internally calls matrix_inverse.
Root cause
The gradient of det(A) is currently implemented as det(A) * A^{-T}, which requires computing A^{-1}. When det(A) = 0, both factors are problematic: det(A) = 0 and A^{-1} doesn't exist. The gradient is actually 0 * ∞, which is undefined.
The mathematically correct gradient is the adjugate matrix (transpose of cofactor matrix):
For A = [[1, 2], [2, 4]], the correct gradient is [[4, -2], [-2, 1]]. The adjugate is always well-defined, even for singular matrices.
Impact
This crash occurs in any optimization that involves determinants of matrices that can become singular during training, such as:
- Normalizing flow models (det of Jacobian)
- Gaussian process models (det of covariance matrix)
- Any regularization term involving
log(det(A)) where A approaches singularity
Also affects tf.linalg.slogdet
tf.linalg.slogdet has the same crash: the forward pass returns sign=0, logabsdet=-inf correctly, but the gradient crashes with InvalidArgumentError: Input is not invertible because it also uses MatrixInverse internally.
m = tf.constant([[1.0, 2.0], [2.0, 4.0]], dtype=tf.float64)
with tf.GradientTape() as tape:
tape.watch(m)
sign, logabsdet = tf.linalg.slogdet(m)
g = tape.gradient(logabsdet, m) # CRASH: InvalidArgumentError
Environment
- TensorFlow: 2.22.0-dev20260508
- OS: Ubuntu 20.04
- Affects both CPU and GPU
Standalone code to reproduce the issue
### Reproduction
import tensorflow as tf
m = tf.constant([[1.0, 2.0], [2.0, 4.0]], dtype=tf.float64)
print("det:", tf.linalg.det(m).numpy()) # 0.0
with tf.GradientTape() as tape:
tape.watch(m)
d = tf.linalg.det(m)
g = tape.gradient(d, m) # CRASH: InvalidArgumentError
**Error:**
InvalidArgumentError: Input is not invertible. [Op:MatrixInverse]
Relevant log output
Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
source
TensorFlow version
TensorFlow: 2.22.0-dev20260508
Custom code
Yes
OS platform and distribution
No response
Mobile device
No response
Python version
No response
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
Summary
Computing the gradient of
tf.linalg.detviatf.GradientTapecrashes withInvalidArgumentError: Input is not invertiblewhen the input matrix is singular (det=0). The forward pass correctly returns 0, but the backward pass fails because the gradient implementation internally callsmatrix_inverse.Root cause
The gradient of
det(A)is currently implemented asdet(A) * A^{-T}, which requires computingA^{-1}. Whendet(A) = 0, both factors are problematic:det(A) = 0andA^{-1}doesn't exist. The gradient is actually0 * ∞, which is undefined.The mathematically correct gradient is the adjugate matrix (transpose of cofactor matrix):
For
A = [[1, 2], [2, 4]], the correct gradient is[[4, -2], [-2, 1]]. The adjugate is always well-defined, even for singular matrices.Impact
This crash occurs in any optimization that involves determinants of matrices that can become singular during training, such as:
log(det(A))where A approaches singularityAlso affects
tf.linalg.slogdettf.linalg.slogdethas the same crash: the forward pass returnssign=0, logabsdet=-infcorrectly, but the gradient crashes withInvalidArgumentError: Input is not invertiblebecause it also usesMatrixInverseinternally.Environment
Standalone code to reproduce the issue
Relevant log output