Automatic window length selection for FIR-based methods

Window length selection seems to play an important role in the results of moving averages/statistics, STFT, FIR filters, and autoregressive models.

All methods rely on the convolution of a set of weights/impulse response/kernel with the signal, and then optimizing for some metric on the resulting filtered signal. The optimal window length selection in this work is based on the idea of multiplying the window weights by a continuous (Gaussian) with a certain scale length, serving as a proxy for window length.

Moving average filter with window length selection

A moving average filter is basically a FIR filter with all weights equal to 1 divided by the window length.

• Experiment: Which window length is learnt when minimizing the variance of the filtered sine wave?
  • The output of the filter should be zero when the window length is the same as the period of the sine wave.
• Result: The window length is learnt to have roughly double the period of the sine wave.

FIR Filter with window length selection

Fir filter coefficients can be learnt to maximize the kurtosis of the squared, filtered signals.

• Experiment: Apply FIR model maximising the kurtosis of the squared, filtered signals to a carrier signal modulated by an impulsive wave.
  • The filter response should match the carrier frequency. Furthermore, the window length should be similar to the period of the modulating sine wave.
• Result: Seems to learn a somewhat appropriate window length and filter response, but optimisation is not robust and the provided example is somewhat contrived.

Other findings

• There are many cases where the model is tempted to learn infinite or zero, or negative window lengths.
• For the lower dimensional problems, I think that getting stuck in local minima is common.
• Despite normalization of the filter, the window often fights with the filter weights, with filter weights taking very large values; only to be multiplied by a very small window value.
  • Some regularization of the filter weights might be necessary.
  • An alternative is to use the window instead to inform the strength of regularization of each filter weight and not use the window as a multiplier.
• The window length parameter has a bigger "say" in the optimization problem and might require a different learning rate/rescaling.
• The FIR filter needs to be long enough to capture possibly low carrier frequencies.

Implementation notes

• Idea is to keep the optimized parameters for window length between 0 and 1: This is why the free parameter is a percentage of the maximum window length.
• All the FIR-related methods can inherit from the same base class with the same optimizers and loss functions.