Data storage objects

[odunbar]

Purpose

Fixes #90
This PR adds DataStorage.jl containing objects to store singleton, or paired, data samples. The idea is to help ensure dimensionally consistent data (as these always store a 2D array where the samples are columns, so if in doubt - the data are columns), which has lead to numerous issues in the past. They also ensure that your input-output pairs for the emulator are safeguarded (e.g it checks consistent dimensions of samples).

Users will need to specify orientation when they provide arrays (as to whether the data stored as a list of rows, or a list of columns), then internally they are arranged always as columns. Effectively therefore we are really just adding more rigid constructors for data arrays.

Contains:

• The DataContainer object stores data as columns
• The PairedDataContainer object stores 2 DataContainer objects, one for input data, one for output data. The number of samples is checked to be the same length in inputs and outputs, while the data dimensions can differ.
• Unit tests for the object and related functions, also updated unit tests for the other objects to use these
• Small changes to examples (Lorenz 96/Cloudy/LossMinimization/GaussianProcessEmulator) to work with new data storage
• To obtain training points from the EKP: in src/Utilities there is a get_training_points function which returns a PairedDataContainer
• The GaussianProcessEmulator only takes in PairedDataContainer now

Also...

• I required some larger changes to the Cloudy example run (even without the new data storage))
• I removed some confusing vcat([X.first, X.last]...) type lines in GP setup and replaced with interpretable loop
• getters and setters for EKP, I really push for us to use getters/setters as an interface
• MCMC stores posterior array with data-as-columns, and utilities work with this too.

Construction example

DataContainer

To create a Data container simply create an array, and specify whether you consider data within as a list of columns, or not. e.g for 2D data and 10 samples

x = rand(2,10)
column_store = DataContainer(x,data_are_columns=true)

or for 10D data and 2 samples

column_store2 = DataContainer(x,data_are_columns=false)

Remember, the data is always stored as columns so

@assert(get_data(column_store) == get_data(column_store2))

returns true

PairedDataContainer

Can be constructed from 2 DataContainers of the same sample size.

input_output_pairs = PairedDataContainer(column_store,column_store)

Or from arrays themselves, (so long as they have the same "samples" dimension)

input_output_pairs2 = PairedDataContainer(x,x,data_are_columns=true) 

and,

PairedDataContainer(column_store,column_store2) 

throws a dimension mismatch error.

[odunbar]

bors r+

[bors]

Build succeeded:

• test-linux
• test-macos
• test-windows