SVD data processing

[eggerdj]

Summary

This PR implements a

• SVD node and an IQ averaging node for the data processor,
• a data processor training mechanism,
• run-time options that can be given to a data processor, and
• an error propagation mechanism.

Details and comments

The SVD allows the user to project the IQ data on the main axis of the data. This also means that this axis must be determined before using the SVD. Furthermore, this training is not done on a single datum but either on all the data or a subset of the data (the data processing calibration data).

DataProcessors can now be trained one node at a time using the train method. Each node has a is_trained property which by default is True. Nodes that require a training process must overwrite this property.

The error propagation and run-time options are implemented by changing the signature of DataAction calls from

def __call__(self, data: Any) -> Any:

to

def __call__(self, data: Any, error: Optional[Any] = None, **options) -> Tuple[Any, Any]:

and making the required changes in DataProcessor.

TODO:

• DataProcessor training mechanisme
• IQ averaging node
• Run-time options
• Error propagation for SVD

[eggerdj]

This is needed for #31

[blakejohnson]

Can we have an option to "self-train" this from the data source? This is often good enough if your data set has a reasonable mixture of 0s and 1s.

[eggerdj]

In #22 we decided to have the data processor apply to an individual datum, i.e. processor(experiment_data.data(index)). It therefore only sees one point of the data at the time. Self-fitting is thus done in the analysis class, see e.g. #46. This also gives us the possibility to train on either all or part of the data. E.g. an experiment might include a few ground-state and excited-state calibration circuits (which we can single out for training) before measuring the actual thing. PR #48 introduces a mechanism that allows nodes and data processors to self train on the data they are given.

[nkanazawa1989]

Overall this looks good.

[chriseclectic]

I'm not sure about the changes to the base class here which seem to be very tied to this specific SVD function. Training doesn't seem like it is a general enough property for the base class.

Also do we need to allow runtime options for these objects, or can we take it in a direction of specifically setting options for node and data processors at initialization or using set_options? Passing generic kwargs through the entire processing chain might lead to some issues later on similar to the current issues in experiment classes with the various layers of options.

[eggerdj]

Here is some rational for the current implementation. Consider the case of spectroscopy. When running spectroscopy you have no pi-pulse and cannot run any calibrations. Therefore, you need to train your SVD (or whatever projection method you use) on the data you measure in the spectroscopy experiment. Therefore the data processor needs to self train on the data it is given. This would look like this:

processor = DataProcessor("memory", [SVD()])

processor.train(experiment_data.data())

y_sigmas = np.array([processor(datum) for datum in experiment_data.data()])

[nkanazawa1989]

LGTM

[chriseclectic]

Looks better with the subclasses. One question is does is_trained need to be an abstract method? Or can the object just have an attribute like self._trained = False that this property returns, and the subclasses should set this to True as part of the train abstract method implementation?

[eggerdj]

One advantage with the abstract property is that it forces the person implementing the node to carefully think about what it means for the node to be trained.