corrdiag

nolds corr_dim estimates the correlation dimension as the slope of log C(r) against log r, where C(r) is the Grassberger-Procaccia correlation sum, the fraction of distinct embedded-vector pairs within r. It builds the pairwise-distance matrix as an all-zeros array and fills only the off-diagonal entries, so the n diagonal entries stay at zero, then counts np.sum(dists <= r), which for every r > 0 includes those n diagonal zeros. The denominator n(n-1) excludes self-pairs and the code's own comment says self-matches should be excluded from both, so the numerator and the denominator disagree. Each C(r) is inflated by n / (n(n-1)) = 1/(n-1). Because that is an additive constant on C(r) before the log, it flattens the log-log curve and biases the fitted correlation dimension downward, worst where C(r) is small.

The code

nolds, nolds/measures.py, corr_dim, installed and current main (identical), lines 1947-1965:

dists = np.zeros((len(orbit), len(orbit)), dtype=np.float64)   # diagonal left at zero
for i in range(len(orbit)):
    d = dist(orbit[i+1:], orbit[i])
    dists[i+1:, i] = d      # fill column i (off-diagonal)
    dists[i, i+1:] = d      # fill row i    (off-diagonal)
...
for r in rvals:
    # "if we exclude self-matches in the numerator, it makes sense to
    #  also exclude self-matches from the denominator."
    s = 1.0 / (n * (n - 1)) * np.sum(dists <= r)   # counts the n diagonal zeros for every r > 0

The sibling sampen in the same file excludes self-matches correctly: it compares template i only with templates j > i, so it never forms a self-pair. The fix in corr_dim is to drop the diagonal before counting, np.fill_diagonal(dists, np.inf).

What it does

The correlation dimension with the diagonal counted (as nolds does) and with the self-pairs excluded (the definition), on the logistic map at r = 4, whose attractor has correlation dimension near one:

emb_dim=3: diagonal counted 0.9294, self-pairs excluded 0.9859, bias -0.0564
emb_dim=5: diagonal counted 0.8569, self-pairs excluded 1.0492, bias -0.1924
emb_dim=8: diagonal counted 0.4787, self-pairs excluded 1.2268, bias -0.7481
additive floor on C(r) from the counted diagonal: 1/(n-1) = 2.020e-03

Driving the real library:

real nolds.corr_dim 0.919048 == diagonal-counted model 0.919048: True

correlation dimension biased downward (logistic map, true near 1):
  emb_dim=3: nolds 0.9294 vs correct 0.9859  bias -0.0564
  emb_dim=5: nolds 0.8569 vs correct 1.0492  bias -0.1924
  emb_dim=8: nolds 0.4787 vs correct 1.2268  bias -0.7481

sibling sampen: nolds 0.223144 == exact self-excluded 0.223144: True

The real nolds.corr_dim matches the diagonal-counted correlation sum to machine precision, so the counted diagonal is the whole story; excluding the self-pairs recovers a dimension near one. The bias grows with the embedding dimension, from -0.06 at emb_dim=3 to -0.75 at emb_dim=8, where the estimate is nearly halved. That growth is the signature of a slope error, not the harmless intercept offset the code's comment discusses: the multiplicative 1/N^2-versus-1/(N(N-1)) prefactor only shifts the log-log intercept, but the 1/(n-1) from the counted diagonal is additive on C(r) before the log, so it changes the slope. The sibling sampen, which never forms a self-pair, matches the exact self-excluded count.

Scope

The call is fully default, corr_dim(data, emb_dim) with the default rvals, lag, dist, and RANSAC fit, on a standard chaotic series. The failure is silent: a plausible finite dimension is returned, and it is systematically too small, more so at the higher embedding dimensions where one looks to see the estimate converge. The error is a deterministic count of exactly n diagonal self-matches, not floating-point round-off. The fix is to exclude the diagonal from the count, matching the n(n-1) denominator and the code's own comment.

Layout

• excerpt.py: the distance-matrix construction and the correlation sum quoted with the flags that name the fault, and the correct sibling sampen.
• corrsum.py: the correlation sum and the fitted dimension with the diagonal self-matches counted or excluded, so the counted version reproduces nolds and the excluded version is the definition.
• consequence.py: the real nolds.corr_dim matching the diagonal-counted model, the dimension biased downward across embedding dimensions, and the correct sibling sampen.
• test_corrdiag.py: the counted diagonal adds exactly 1/(n-1) to C(r), the bias is negative and grows with embedding dimension, the real dimension matches the diagonal-counted model and is nearly halved at emb_dim=8, and sampen is correct.

Reproduce

python corrsum.py
python consequence.py
python test_corrdiag.py

The distance-matrix construction is quoted from current main; the dimensions are produced by the real nolds.corr_dim. The fix is np.fill_diagonal(dists, np.inf) before the count.