Spectral adaptive samplers

[vsnever]

This adds the following new samplers:

• MaskedFrameSampler2D,
• SpectralAdaptiveSampler1D,
• SpectralAdaptiveSampler2D,
• MaskedSpectralAdaptiveSampler2D.

Also, parameter validation is added for all adaptive samplers (I hope this doesn't break anything).

SpectralAdaptiveSampler2D is useful when calculating in a single run multiple filtered camera images with different filters. It has three options for how to reduce the spectral array of normalized errors to a single value for a pixel. I'll be happy to add more options if needed.

There are also two small changes in documentation:

• description of ratio parameter of adaptive samplers is added,
• all previously missing pipelines and samplers are added to API reference.

[mattngc]

Wow, had a quick look through and its really nice code Vlad. I think its one Alex would probably want to review.

[CnlPepper]

Thanks for fixing up some of the existing classes, they were in need of completing. I have the following general comments I'd like to see addressed before these are merged, in addition to the specific comments in the code:

• I've generally only highlighted a single instance of a change in the code comments, please apply those comments to all equivalent instances in the merge request.

• It seems unnecessary to have Masked and non-Masked samplers. Just add masking to the existing samplers and ignore the mask if it is set to None (default). Easy to implement, just an 'if mask and mask[i][j]:' test at the point of evaluation.

• Some of the functions are getting overlong.

• Don't force users to supply data in a specific form when it is trivial to handle and convert consistently.

Otherwise, looks good. I like the choice of weighting methods for the spectral samplers.

If you can make these changes I'll merge them for the next release.

[CnlPepper]

Legibility: All error messages should be proper sentences. Start with a capital letter and end with a full stop.

[vsnever]

Fixed.

[CnlPepper]

User convenience: I would just convert the mask array to dtype=np.bool and state that in the document. Any value != to 0 will map to True.

[vsnever]

Fixed.

[CnlPepper]

See previous comment on masks.

[CnlPepper]

These options should really be in their own functions, or made plugable classes (maybe a later improvement).

[vsnever]

I moved them to their own functions.

[vsnever]

@CnlPepper, thank you very much for the review. I hope that I fixed all the issues you indicated. Below are some comments.

• The mask attribute is added to all 2d samplers. Existing Masked samplers are converted to wrappers and marked as deprecated (they raise respective warning message).
• The setter converts mask array to np.bool whatever the original type is. If mask is not provided, the all-true mask is created at the first call of generate_tasks(). This simplifies the checks. Also, a memoryview object is created for the mask (it turns out that numpy boolean arrays are stored in np.uint8).
• Reduction methods of Spectral Adaptive samplers are moved to their own functions. An additional method called 'power_percentile' is added. If percentile=x, this method will abort extra sampling when x% of spectral bins with the highest spectral power all have normalised errors lower than cutoff. This method should be more useful than the simple 'percentile' method.

[vsnever]

There was a small formatting error in the docstring of Spectral Adaptive samplers which is fixed now.

[CnlPepper]

Like the new improvements, just spotted 2 more items to consider then I'll merge.

[CnlPepper]

Thanks for the work on this. There are a few tweaks I'll make the the mask interface to make it more user friendly, but I'll leave them until we get closer to release. At present, if you rerun a job with a different pixel size the old mask will be used and error at the frame size check.

[vsnever]

Yes, this is a bug. If the mask was not specified by the user, the error should not be raised. Probably the easiest fix is to replace:

if pixels != (self._mask.shape[0], self._mask.shape[1]):
    raise ValueError('The pixel geometry passed to the frame sampler is inconsistent with the mask frame size.')

with

if pixels != (self._mask.shape[0], self._mask.shape[1]):
    if np.all(self._mask):
        self.mask = np.ones(pixels, dtype=np.bool)
    else:
        raise ValueError('The pixel geometry passed to the frame sampler is inconsistent with the mask frame size.')