Skip to content

v-code01/detrendwin

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

detrendwin

spectrum's speriodogram detrends by subtracting the mean, but it does so after applying the window. It computes the transform of x * w - m where a detrended, windowed periodogram is the transform of (x - m) * w. The two differ by m * (w - 1), a window-shaped signal whose transform is large, so on the default detrend=True path the mean is never removed from the windowed data and a nonzero DC component leaks a window-shaped spike across the low-frequency bins. For a constant input, whose periodogram must be zero after detrending, the result is the full power of that constant.

The code

spectrum, spectrum/periodogram.py, speriodogram (detrend=True default), 0.10.0 and current main (identical), lines 121-135:

if detrend == True:
    m = np.mean(x, axis=axis)
...
res = (abs(rfft(x*w - m, NFFT, axis=-1))) ** 2. / r      # mean subtracted after windowing

The docstring states the mean is removed before the FFT. The correct detrended, windowed periodogram is (abs(rfft((x - m)*w, ...))) ** 2. / r, the mean removed before the window. The detrend=False branch sets m = 0 and computes the correct un-detrended periodogram |rfft(x*w)|^2 / N, so the fault is the detrend=True branch alone, not the window or the normalization.

What it does

The windowed periodogram with the mean removed before and after windowing, on a constant input:

constant input x = 5*ones(256):
  detrend before windowing: total power 0.00e+00 (zero)
  detrend after windowing:  total power 1692.8000, DC bin 1354.2400
  residual m*(w-1) max magnitude: 4.6000

Driving the real library:

constant input x=5*ones(256), detrend=True: spectrum total power 1704.7128 (DC 1364.8407), correct total 0.00e+00
mixed x=3+cos: spectrum DC bin 491.3426 vs correct 5.32e-32, max|diff| first 10 bins 491.3426
spectrum == wrong-order formula |rfft(x*w - m)|^2/N: True
detrend=False == correct un-detrended periodogram: True

A constant x = 5 should have a zero periodogram after detrending, but the real speriodogram returns a total power of 1704.7 with a window-shaped low-frequency spike. A mean-plus-tone signal keeps a DC bin of 491.3 where correct detrending gives 5e-32. The library output equals |rfft(x*w - m)|^2 / N exactly, which pins the fault to the order of the mean removal, and the detrend=False branch matches the correct un-detrended periodogram, which isolates the fault to the detrend=True branch. The residual the wrong order leaves in the windowed data is m * (w - 1).

Scope

detrend=True is the default, and the docstring's own narrative promises that the mean is removed. The failure is silent: a plausible spectrum is returned, but the detrending does nothing and the low-frequency bins carry a window-shaped image of the mean that a real detrend would remove, so any low-frequency reading near DC is contaminated. The error is a wrong order of two operations, not floating-point round-off, and it is decisively exposed by a constant input. The fix is to detrend before windowing, (x - m) * w, in all four branches.

Layout

  • excerpt.py: the mean subtraction and the four x*w - m transforms quoted with the flags that name the fault, and the correct (x - m)*w form.
  • dcremoval.py: the windowed periodogram with the mean removed before or after windowing, so the before order is zero on a constant and the after order keeps its power, plus the residual m*(w-1).
  • consequence.py: the real speriodogram, the constant input keeping its power, the mixed-signal DC leak, the exact match to the wrong-order formula, and the correct detrend=False branch.
  • test_detrendwin.py: the model before order is zero and the after order is not, the residual is m*(w-1), the real constant input keeps its power, the mixed signal leaks, the output matches the wrong-order formula, and detrend=False is correct.

Reproduce

python dcremoval.py
python consequence.py
python test_detrendwin.py

The transform lines are quoted from current main; the spectra are produced by the real speriodogram. The fix is to detrend before windowing, (x - m) * w.

About

spectrum's `speriodogram` detrends by subtracting the mean, but it does so after applying the window. It computes the transform of `x * w - m` where a detrended, windowed periodogram is the transform of `(x - m) * w`.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages