Fix smoothing bug when signal is shorter than a window

[tskir]

Closes #587. Thank you @tetedange13 for an amazingly detailed investigation which helped a lot!

In Savitzky-Golay smoothing, for certain combinations of parameters, a (wing-padded) signal length can turn out to be shorter than the smoothing window requested, which is mathematically impossible to compute and causes an index mismatch exception downstream.

This change avoids this by checking for this condition and reducing the smoothing window length and, if necessary, also the polynomial order. This ensures (compared to e.g. returning the original signal) that at least some sensible smoothing is still being performed.

[tskir]

@etal I'm reasonably certain that this should be a sound solution (and I also verified that it definitely fixes #587), but perhaps it would be useful if you could double-check that the math checks out, as it's not quite my area of expertise

[etal]

Interesting, I think I meant for check_inputs to do this operation internally, but apparently it doesn't.

Would you like to attempt merging check_inputs and check_smoothing_parameters into a single function? If not, and you're happy with the results of this function, then I'm fine with merging this PR as it is.

[tetedange13]

This PR should also fix a tinu bug I got from running cnvkit.py scatter --by-bin --trend my_patient.cnr, where chrY crashes due to --trend flag

Full traceback here: traceback.txt
=> It is a bit different from #587, IMHO core problem is the same

Here is a minimal".cnr" that reproduce the bug: sample.cnr.txt

Hope this helps.
Thanks again.
Felix.

[tskir]

@etal Thank you for your comments! I've now investigated this further and here's what I found.

You are right: check_inputs already checks for this condition via _width2wing, and for short signals the wing size is correctly reduced when necessary:

cnvkit/cnvlib/smoothing.py

Lines 41 to 44 in 9dd1e7c

	elif width >= 2 and int(width) == width:
	# Ensure window width <= len(x) to avoid TypeError
	width = min(width, len(x) - 1)
	wing = int(width // 2)

The problem is how these results are then used. After calling check_inputs, all functions except savgol() recalculate the final window width as 2 * total_wing + 1, which will always work.

Because savgol() was not taking that into account, this could potentially cause two distinct problems:

• For short signals, the originally requested total_width is too big;
• And for really short signals, even the one-iteration window width (7 by default) is too big.

I have now rewritten this PR to properly address both of them. An additional function is no longer required, as well as any modifications to check_inputs.

[tskir]

@tetedange13 Thank you for the additional test file! I've verified that the latest version of this PR should indeed fix the scatter crash.

[etal]

This is good. Thanks!