LaTiS Operations

There are a number of operations that can be performed on a LaTiS Dataset. The names may evolve, especially as we evolve towards version 3 which will have a more fundamental basis in functional programming, typed lambda calculus, category theory, and abstract mathematics in general (e.g. sets, monads, functors, morphisms, natural transformations). Many of these are inspired from functional programming as implemented in Scala, relational algebra and the emerging LaTiS algebra, not to mention all the usual things people like to do with their data.

Operations that only modify data values (without reducing the number of samples or modifying the Dataset model)

basic arithmetic LaTiS Arithmetic

unit conversion (e.g. inches to cm)

replace (e.g. missing value with NaN)

smooth: Reduce the noise in the data.

Operations that reduce the number of Function samples without changing data values (Filters)

selection: Exclude samples based on simple boolean expression (t > 0).

limit(n): Keep only first n samples.

subset: Drops samples based domain index value constraints.

slice: Subset that also reduces the dimensionality of the domain (thus altering the model).

first, last, min, max: Filters that reduce a Function to a Tuple (thus altering the model).

Operations that reduce the number of Function samples that may change data values

resample: Represent the range values on a different domain set. This could be simple and equivalent to a subset or as complex as a coordinate system transformation.

bin: Partition the domain into equal sized bins and average the range values. This may also preserve the min, max, and standard deviation of the values in each bin.

sum, total: Sum all the samples for a given dimension(s). This reduces the dimensionality of the domain (thus altering the model).

mean, standard deviation: likewise

integrate: Like sum but accounting for irregularly spaced domain values.

Operations that primarily modify the dataset model

rename: Simply changes the name of a parameter without modifying the structure of the Dataset.

projection: Exclude parameters.

t -> (a,b,c) project(t,a,c) => t -> (a,c)

An implicit Index parameter may be substituted to preserve the Function.

t -> (a,b,c) project(a,c) => i -> (a,c)
t -> (a,b,c) project(t)   => i -> t

factorization: Restructuring the Dataset to emphasize certain functional relationships.

i -> (t,a,b,c) factorOut(t) => t -> (a,b,c)

In the general case where the factor is not a suitable domain parameter (e.g. duplicate values), this will introduce a nested Function:

i -> (t,a,b,c) factorOut(a) => a -> (i -> (t,b,c))

pivot: Turn a set of Function samples into a Tuple with one element for each sample.

i -> a pivot => (a1, a2, a3,...)
x -> (y -> a) pivot(y) => x -> (a_y1, a_y2, a_y3,...)

Operations that aggregate (merge) Datasets

TODO

Function Evaluation

Like other functions, LaTiS Functions support the notion of being evaluated. The argument (LaTiS Variable) of a Function evaluation must be the same type as the domain of the Function. The result of the evaluation has the type of the range and is equivalent to resampling the Function at the given domain value.