Resampling vs Subsampling (frequential scattering)

[OverLordGoldDragon]

Origin; split for focus. Still relevant with implems 1, 2, & 3.

2. Resampling vs subsampling

Since shorter padding here corresponds to a trimming, not subsampling, the 'observation time' per row is the same (i.e., row-to-row stays dw, not 2*dw).

1. Resample: keeps filters tailored for the same frequential scale F across all paddings
2. Subsample: recalibrates filters to scales F/2, F/4, ..., or x2, x4 frequencies

Seems there's a use case for each:

• 1 enforces same degree of frequential invariance across all psi2. Disadvantages:
  • a. Suffers at highest scales: if e.g. min padded is 32 and max in 128, then lowest frequency wavelets will target long-term structures that do not exist and incur significant boundary effects
  • b. Deprioritizes sharp dynamics (less filters dedicated to high frequencies with tight temporal support)
• 2 better exploits the "zoom" advantage by recalibrating as if less-padded is the intended input size. Disadvantages:
  • Less than user-set frequential invariance

Exact code on padding, resampling.

I'm inclined toward subsampling.

[lostanlen]

answered in lostanlen#1 ?

[OverLordGoldDragon]

Yes, but here I'd like to post a demo of resample_* = False's advantage over = True, along a summary of our discussions.

[OverLordGoldDragon]

There's a problem with resample_phi_fr = True: shortest filter ends up doing global average pooling, and most filters deviate significantly from original's time behavior (defeating purpose of resampling):

It's clear from first plot that we can downsize by up to 2^1 without significantly altering the lowpass; subsampling does not suffer from this.

This means we cannot subsample (in _frequency_scattering) beyond said bound before _joint_lowpass(), defeating the entire purpose of variable padding. If the MATLAB implementation pads variably and resamples... it also suffers. Thoughts? @lostanlen

[lostanlen]

the problem only arises for large values of F, i.e. above one octave.
It's a difficult corner case in between global averaging pooling and small F (equal or lower than one octave).
In the MATLAB implementation F defaults to one octave.

Global average pooling can be made numerically correct (and fast) even with a jagged array like the one we have.
Small F does not suffer much from this problem because the support of phi fits even the smallest j2-gram.

Large F is more complicated and requires more padding, i.e. ~64 coefficients in your case. Which will slow down the code but is the only solution to be numerically correct and keep some amount of frequential sensitivity. (as opposed to global average pooling)

I guess that the nicest workaround would be to account for that in the padding condition? I.e. prioritize fitting the p*i_fr filterbank and then consider the j1<j2 trimming heuristic if possible

[OverLordGoldDragon]

Yes, it's what I figured, but I'm saying with the default F set per 2**J_fr, there'll basically be no subsampling before _joint_lowpass.

I'll account for it, then.

[OverLordGoldDragon]

Turns out the padlen(S1) > padlen(Y2) case isn't accounted correctly with "resampling": resampling with more samples does not permit subsampling more. This forces all coefficients to double in size (i.e. the "longest input" reference now shifts from 2**J_pad_max to 2**(J_pad_max+1)), if we are to remain correct.

This seems too great a price to pay for sparing a bit extra boundary effects; it afflicts a fraction (2 / (len(psi2_f) + 2)) of coefficients, and the corrective price is all joint coefficients packing more air. Instead I say we force padlen(S1) <= padlen(Y2) with resample_*=True. (The =False variant doesn't suffer from this).

[OverLordGoldDragon]

I've decided to scrap accounting for padlen(S1) != padlen(Y2) (< is also possible) entirely and simply force them equal; it adds much complexity without much benefit. If one insists on marginal gains, code at this stage can be built off of.

Self note: sc_freq.ind_start_fr[-1] is for J_pad_fr_max not J_pad_fr_fo

[OverLordGoldDragon]

More trouble than I realized: since subsampling by k doesn't only sigma *= k but also xi *= k, we end up with duplicate wavelets, where subsampling simply "resets" parameters to an earlier combination:

Red doesn't follow blue, orange does; that's the duplication, a later wavelet overlaps an earlier.

This'd be less trouble if not for exact duplication, though a complicated re-ordering would be requires to guarantee xi[n + 1] < x[n]. My original motivation for subsampling only concerned reducing temporal width; to this end, I propose doing exactly that. Instead of subsampling by k, just do sigma *= k, and call it recalibrate=True instead of resample=False:

example wavelets (red, cyan above):

[lostanlen]

i don't see why we'd need 3) same total number of wavelets

[OverLordGoldDragon]

Only for 3D+. But there's other advantages; I realized there is dropping of wavelets happening after all, it's forced by shorter filter length which limits maximum scale. Difference is, drop = use fewer existing filters, more sparse; recalibrate = redistribute existing filters, finer distribution for new max scale:

• Greater xi_max -> greater discriminability of small FDTS
• More xi between higher xi_min and new xi_max -> more 2D wavelet orientations in directions relevant to input length
• Greater sigma for all (but last) wavelets -> more appropriate lesser width for lesser input length

xi discrepancy may be bad news for 4D convs, but 3D with one separable wavelet along channels permits independent treatment.

[OverLordGoldDragon]

Previous filterbanks but with resampling of different parameters at original length, for perspective:

[OverLordGoldDragon]

One way to do variable n1_fr per n2 is to keep all parameters the same but drop those whose sigma are beneath threshold, namely drop <= sigma_min * 2**downsampling_factor. Included full length for reference:

Is this what you had in mind? @lostanlen

[lostanlen]

@OverLordGoldDragon yes that's exactly what i had i mind! and also my preferred option because it keeps the n1_fr's xis (when expressed in radians or cycles per octaves) aligned across different n2