3. Statistics Nodes (csp.stats)

This page contains the documentation for the csp.stats library. The stats library contains functions to calculate statistics on time series data over rolling windows.

API Reference

Base Statistics
1. count: counts the number of data ticks within a given interval
2. unique: counts the number of unique values within a given interval
3. sum: rolling sum of values within a given interval
4. prod: rolling product of values within a given interval
5. first: the earliest value still within the interval
6. last: the last value of the interval
7. mean: the mean of values within the interval
8. gmean: the geometric mean of values within the interval
Order Statistics
1. max: the maximum value within the interval
2. min: the minimum value within the interval
3. median: the median value within the interval
4. quantile: the quantile value within the interval
5. argmin: the time at which the minimum interval value ticked
6. argmax: the time at which the maximum interval value ticked
7. rank: the time series rank of the most recent tick in the interval
Moment-Based Statistics
1. var: variance of the time series within the interval
2. stddev: standard deviation within the interval
3. sem: standard error within the interval
4. cov: covariance between two in-sequence time series within the interval
5. corr: correlation between two in-sequence time series within the interval
6. skew: skewness of the time series within the interval
7. kurt: kurtosis (or excess kurtosis) of the time series within the interval
Exponential Moving Statistics
1. ema: exponential moving average, with numerous different variations available
2. ema_var: exponential moving variance
3. ema_std: exponential moving standard deviation
4. ema_cov: exponential moving covariance between two in-sequence time series
NumPy Specific Statistics
1. cov_matrix: covariance matrix between N time-series (in a NumPy array) over a rolling time interval
2. corr_matrix: normalized correlation matrix between N time-series (in a NumPy array) a rolling time interval
3. list_to_numpy: converts a listbasket of time-series into a NumPy array
4. numpy_to_list: converts a NumPy array time-series into a listbasket
Cross-Sectional Statistics
1. cross_sectional: receive all data within the current window for a cross-sectional calculation

Getting started

Introduction

The csp.stats library provides rolling window calculations on time series data in csp. The goal of the library is to provide a uniform, robust interface for statistical calculations in csp. Each computation is a csp.graph which consists of one or more nodes that perform a given computation. Users can treat these graphs as a "black box" with specified inputs and outputs as provided in the API reference. Example statistics graphs for mean and standard deviation are provided below to give a rough idea of how the graphs work.

Mean using a tick-specified interval 437686747

Standard deviation using a tick-specified interval 437686748

Rolling windows can either be specified by the number of ticks in the window or the time duration of the window. Users can specify minimum window sizes for results as well as the minimum number of data points for a valid computation. Standard NaN handling is provided with two different options. Weighting is available for relevant stats functions such as sums, mean, covariance, and skew.

Working with a single-valued time series

Time series of float and int types can be used for all stats functions, except those listed as "NumPy Specific". Internally, all values are cast to float-type. NaN values in the series (if applicable) are allowed and will be handled as specified by the ignore_na flag.

If you are performing the same calculation on many different series, it is highly recommended that you use a NumPy array. NumPy array inputs result in a much smaller csp graph which can drastically improve performance. If different series tick asynchronously, then sometimes using single-input calculations cannot be avoided. However, you can consider sampling your data at regularly specified intervals, and then using the sampled values to create a NumPy array which is provided to the calculation.

Working with a NumPy time series

All statistics functions work on both single-input time series and time series of NumPy arrays. NumPy arrays provide the ability to perform the same calculation on many different elements within the same csp.node, and therefore drastically reduce the overall size of the csp graph. The performance benefits of using NumPy arrays for large-scale computations (i.e. thousands of symbols) is order of magnitudes faster, per benchmarking. To convert a list of individual series into a NumPy array, use the csp.stats.list_to_numpy conversion node. To convert back to a basket of series, use the csp.stats.numpy_to_list converter.

All calculations on NumPy arrays are performed element-wise, with exception of cov_matrix and corr_matrix which are defined in the statistical sense. Arrays of arbitrary dimension are supported, as well as array views such as transposes and slices. The data type of arrays must be of float-type, not an int. If your data is integer valued, convert the array to a float-type using the astype function in the NumPy library. Basic mathematical operations (such as addition, multiplication etc.) are defined on NumPy array time series using NumPy's built-in functions, which allow for proper broadcasting rules.

Working with a basket of time series

There are two ways that users can run stats function on a listbasket of time series. If the data in the time series ticks together (or relatively together) then users can convert their listbasket data into a NumPy array time series using the list_to_numpy node, run the calculations they want, and then convert back to a listbasket using the numpy_to_list node. Since NumPy arrays only require one node per computation, whereas a list of N time series will require N nodes, this method is highly efficient even for small graphs. Below is a diagram of the workflow for a listbasket with 2 elements.

A sum over a listbasket with 2 elements 437687654

If the data does not tick (or is sampled) at the same time or the computations are fundamentally different in nature (i.e. different intervals), then the NumPy method will not provide the desired functionality. Instead, if users wish to store all their individual time series in a listbasket, then they must use single input stats with standard csp listbasket syntax. This method is significantly slower than using NumPy arrays, since the graphs must be much larger. However, depending on your use case, this may be unavoidable. If possible, it is highly recommended that you consider transformations to your data that allow it to be stored in NumPy arrays, such as sampling at given intervals.

Cross-sectional statistics

The stats library also exposes an option to compute cross-sectional statistics. Cross-sectional statistics are statistics which are computed using every value in the window at each iteration. These computations are less efficient than rolling window functions that employ smart updating. However, some computations may have to be applied cross-sectionally, and some users may want to apply cross-sectional statistics for small window calculations that require high numerical stability.

To use cross-sectional statistics, use the csp.stats.cross_sectional utility to receive all data in the current window. Then, use csp.apply to use your own function on the cross-sectional data. The cross_sectional function allows for the same user options as standard stats functions (such as triggering and sampling). An example of using csp.stats.cross_sectional is shown below:

# Starttime: 2020-01-01 00:00:00
x = {'2020-01-01': 1, '2020-01-01': 2, '2020-01-01': 3, '2020-01-01': 4, '2020-01-01': 5}
cs = cross_sectional(x, interval=3, min_window=2)
cs

{'2020-01-02': [1,2], '2020-01-03': [1,2,3], '2020-01-04': [2,3,4], '2020-01-05': [3,4,5]}

# Calculate a cross-sectional mean
cs_mean = csp.apply(cs, lambda v: sum(v)/len(v), float)
cs_mean

{'2020-01-02': 1.5, '2020-01-03': 2.0, '2020-01-04': 3.0, '2020-01-05': 4.0}

Expanding window statistics

An expanding window holds all ticks of its underlying time series - in other words, the window grows unbounded as you receive more data points. To use an expanding window, either don't specify an interval or set interval=None. An example of an expanding window sum is shown below:

# Starttime: 2020-01-01 00:00:00
x = {'2020-01-01': 1, '2020-01-01': 2, '2020-01-01': 3, '2020-01-01': 4, '2020-01-01': 5}
sum(x)

{'2020-01-01': 1, '2020-01-02': 3, '2020-01-03': 6, '2020-01-04': 10, '2020-01-05': 15}

Common user options

Intervals

Intervals can be specified as a tick window or a time window. Tick windows are int arguments while time windows are timedelta arguments. For example,

csp.stats.mean(x, interval=4) will calculate a rolling mean over the last 4 ticks of data.
csp.stats.mean(x, interval=timedelta(seconds=4)) will calculate a rolling mean over the last 4 seconds of data

Time intervals are inclusive at the right endpoint but exclusive at the left endpoint. For example, if x ticks every one second with a value of 1, and I call csp.stats.sum(x, timedelta(seconds=1))then my output will be 1 at all times. It will not be 2, since the left endpoint value (which ticked exactly one second ago) is not included.

Tick intervals include NaN values. For example, a tick interval of size 10 with 9 NaN values in the interval will only use the single non-nan value for computations. For more information on NaN handling, see the "NaN handling" section.

If no interval is specified, then the calculation will be treated as an expanding window statistic and all data will be cumulative (see the above section on Expanding Window Statistics).

Triggers, samplers and resets

Triggers are optional arguments which trigger a computation of the statistic. If no trigger is provided as an argument, the statistic will be computed every time x ticks i.e. x becomes the trigger.

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3}
trigger = {'2020-01-02': True}

sum(x, interval=2)

{'2020-01-02': 3, '2020-01-03': 5}

sum(x, interval=2, trigger=trigger)

# No result at day 3
{'2020-01-02': 3}

Samplers are optional arguments which sample the data. Samplers are used to signify when the data, x, should tick. If no sampler is provided, the data is sampled whenever x ticks i.e. x becomes the sampler.

If the sampler ticks and x does as well, then the tick is treated as valid data
If the sampler ticks but x does not, then the tick is treated as NaN data
If the sampler does not tick but x does, then the tick is ignored completely

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3}
sampler = {'2020-01-01': True, '2020-01-03': True}

sum(x, interval=2)

{'2020-01-02': 3, '2020-01-03': 5}

sum(x, interval=2, sampler=sampler)

# Tick on day 2 is ignored
{'2020-01-03': 4}

Resets are optional arguments which reset the interval, clearing all existing data. Whenever reset ticks, the data is cleared. If no reset is provided, then the data is never reset.

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3}
reset = {'2020-01-02 12:00:00': True}

sum(x, interval=2)

{'2020-01-02': 3, '2020-01-03': 5}

sum(x, interval=2, reset=reset)

# Data is reset after day 2
{'2020-01-02': 3, '2020-01-03': 3}

Important: the order of operations between all three actions is as follows: reset, sample, trigger. If all three series were to tick at the same time: the data is first reset, then sampled, and then a computation is triggered.

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3}
reset = {2020-01-03: True}

# Trigger = sampler = x. Reset, trigger and sampler therefore all tick at 2020-01-03

sum(x, interval=2, reset=reset)

# the data is first reset, then 3 is sampled, and then the sum is computed
{'2020-01-02': 3, '2020-01-03': 3}

Data validity

Minimum window size (min_window) is the smallest allowable window before returning a computation. If a time window interval is used, then min_window must also be a timedelta. If a tick interval is used, then min_window must also be an int. Minimum window is a startup condition: once the minimum window size is reached, it will never go away. For example, if you have a minimum window of 5 ticks with a 10 tick interval, once 5 ticks of data have occurred computations will always be returned when triggered. By default, the minimum window size is equal to the interval itself.

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3}
sum(x, interval=2, min_window=1}

{'2020-01-01': 1, '2020-01-02': 3, '2020-01-03': 5}

sum(x, interval=timedelta(days=2), min_window=timedelta(days=1))

# Assuming graph start time is 2020-01-01
{'2020-01-02': 3, '2020-01-03': 5}

Minimum data points (min_data_points) is the number of valid (non-nan) data points that must exist in the current window for a valid computation. By default, min_data_points is 0. However, in most applications, if you are dealing with frequently NaN data you may want to ensure that stats computations provide meaningful results. Thus, if the interval has fewer than min_data_points values, the computation is too noisy and thus NaN is returned instead.

x = {'2020-01-01': 1, '2020-01-02': nan, '2020-01-03': 3}

sum(x, interval=2)

{'2020-01-02': 1, '2020-01-03': 3}

sum(x, interval=2, min_data_points=2)

# We only have 1 valid data point
{'2020-01-02': nan, '2020-01-03': nan}

NaN handling

The stats library provides a uniform interface for NaN handling. Functions have an ignore_na parameter which is a bool argument (default value is True).

If ignore_na=True, then NaN values are "ignored" in the computation but still included in the interval
If ignore_na=False, then NaN values make the whole computation NaN ("poison" the interval) as long as they are present in the interval

x = {'2020-01-01': 1, '2020-01-02': nan, '2020-01-03': 3, '2020-01-04': 4}

sum(x, interval=2, ignore_na=True}

{'2020-01-02': 1, '2020-01-03': 3, '2020-01-04': 7}

sum(x, interval=2, ignore_na=False)

# NaN at t=2 only out of interval by t=4
{'2020-01-02': nan, '2020-01-03': nan, '2020-01-04': 7}

For exponential moving calculations, EMA NaN handling is slightly different. If ignore_na=True, then NaN values are completely discarded. If ignore_na=False, then NaN values do not poison the interval, but rather count as a tick with no data. This affects the reweighting of past data points when the next tick with valid data is added. For a detailed explanation, see the EMA section.

Weighted statistics

Weights is an optional time-series which gives a relative weight to each data point. Weighted statistics are available for: sum(), mean(), var(), cov(), stddev(), sem(), corr(), skew(), kurt(), cov_matrix() and corr_matrix(). Since weights are relative, they do not need to be normalized by the user. Weights also do not need to tick at the same time as the data, necessarily: the weights are sampled whenever the data sampler ticks. For higher-order statistics such as variance, covariance, correlation, standard deviation, standard error, skewness and kurtosis, weights are interpreted as frequency weights. This means that a weight of 1 corresponds to that observation occurring once and a weight of 2 signifies that observation occurring twice.

If either the data or its corresponding weight is NaN, then the weighted data point is collectively treated as NaN.

# Single valued time series

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': 4}
weights = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-04': 1}

sum(x, interval=2, weights=weights)

# Weight of 2 applied to x=3, as it is sampled
{'2020-01-02': 5, '2020-01-03': 10, '2020-01-04': 10}

mean(x, interval=2, weights=weights)

# Weighted mean
{'2020-01-02': 1.667, '2020-01-03': 2.5, '2020-01-04': 3.333}

If the time-series is of type float, then the weights series is also of type float. If the time-series is of type np.ndarray, then the weights series is sometimes of type np.ndarray and sometimes of type float. For element-wise statistics sum(), mean(), var(), stddev(), sem(), skew(), kurt() the weights are element-wise as well. For cov_matrix() and corr_matrix(), the weights are of type float since they apply to the data vector collectively. Consult the individual function references for more details.

# NumPy applied element-wise

x = {'2020-01-01': [1,1], '2020-01-02': [2,2], '2020-01-03': [3,3]}
weights = {'2020-01-01': [1,2], '2020-01-02': [2,1], '2020-01-03': [1,1]}

sum(x, interval=2, weights=weights)

{'2020-01-02': [5,4], '2020-01-03': [7,5]}

mean(x, interval=2, weights=weights)

# Weighted mean
{'2020-01-02': [1.667, 1.333], '2020-01-03': [2.333, 2.5]}

Numerical stability

Stats functions are not guaranteed to be numerically stable due to the nature of a rolling window calculation. These functions implement online algorithms which have increased risk of floating point precision errors, especially when the data is ill-conditioned. Users are recommended to apply their own data cleaning before calling these functions. Data cleaning may include clipping large, erroneous values to be NaN or normalizing data based on historical ranges. Cleaning can be implemented using the csp.apply node (see baselib documentation) with your cleaning pipeline expressed within a callable object (function). If numerical stability is paramount, then cross-sectional calculations can be used at the cost of efficiency (see the section below on Cross-Sectional Statistics).

Where possible, csp.stats algorithms are chosen to maximize stability while maintaining their online efficiency. For example, rolling variance is calculated using Welford's online algorithm and rolling sums are calculated using Kahan's algorithm if precise=True is set. Floating-point error can still accumulate when the functions are used on large data streams, especially if the interval used is small in comparison to the quantity of data. Each stats method that is prone to floating-point error exposes a recalc parameter which is an optional time-series argument to trigger a clean recalculation of the statistic. The recalculation clears any accumulated floating-point error up to that point.

The `recalc` parameter

The recalc parameter is an optional time-series argument designed to stop unbounded floating-point error accumulation in rolling csp.stats functions. When recalc ticks, the next calculation of the desired statistic will be computed with all data in the window. This clears any accumulated error from prior intervals. The parameter is meant to be used heuristically for use cases involving large data streams and small interval sizes, causing values to be continuously added and removed from the window. Periodically triggering a recalculation will limit the floating-point error accumulation caused by these updates; for example, a user could set recalc to tick every 100 intervals of their data. The cost of triggering a recalculation is efficiency: since all data in the window must be processed, it is not as fast as doing the calculation in the standard online fashion.

A basic example using the recalc parameter is provided below.

x = {'2020-01-01': 0.1, '2020-01-02': 0.2, '2020-01-03': 0, '2020-01-04': 0}
sum(x, interval=2)

# floating-point error has caused the sum to not perfectly go to zero
{'2020-01-02': 0.3, '2020-01-03': 0.19999999, '2020-01-03': -0.00000001}

recalc = {'2020-01-04': True}
sum(x, interval=2, recalc=recalc)

# at day 4, a clean recalculation clears the floating-point error from the previous data
{'2020-01-02': 0.3, '2020-01-03': 0.19999999, '2020-01-04': 0}

Example use cases

Below are some examples of how to use the stats library.

Single valued time series

The first example we use computes an exponential moving price, rolling VWAP and cumulative volume for a single symbol. Additionally, we trigger a computation every 1 minute and we reset the data every 5 minutes. VWAP is calculated using a 2 minute interval which is first valid after 1 minute. Below is the code to calculate VWAP, EMA and volume for a single time series.

Example 1: Single input stats

st = datetime(2020,1,1)
prices_data = [
    (st+timedelta(minutes=1.3), 12.653), (st+timedelta(minutes=2.3), 14.210), (st+timedelta(minutes=3.8), 13.099),
    (st+timedelta(minutes=4.1), 12.892), (st+timedelta(minutes=4.4), 17.328), (st+timedelta(minutes=5.1), 18.543),
    (st+timedelta(minutes=5.3), 17.564), (st+timedelta(minutes=6.3), 19.023), (st+timedelta(minutes=8.7), 19.763)
]

volume_data = [
    (st+timedelta(minutes=1.3), 100), (st+timedelta(minutes=2.3), 115), (st+timedelta(minutes=3.8), 85),
    (st+timedelta(minutes=4.1), 90), (st+timedelta(minutes=4.4), 95), (st+timedelta(minutes=5.1), 185),
    (st+timedelta(minutes=5.3), 205), (st+timedelta(minutes=6.3), 70), (st+timedelta(minutes=8.7), 65)
]

@csp.graph
def graph():
    price = csp.curve(typ=float, data=prices_data)
    volume = csp.curve(typ=float, data=volume_data)

    # Trigger a computation every minute
    trigger = csp.timer(timedelta(minutes=1))

    # Reset the data every 5 minutes, but only after computing values
    reset = csp.delay(csp.timer(timedelta(minutes=5)), timedelta(microseconds=1))

    # calculate rolling 2 minute VWAP with our first data at 1 minute
    vwap = csp.stats.mean(price, interval=timedelta(minutes=2), min_window=timedelta(minutes=1), trigger=trigger,
            weights=volume, reset=reset)

    # calculate a time-weighted EMA of halflife 2 minutes for the price data
    ewm_price = csp.stats.ema(price, halflife=timedelta(minutes=2), trigger=trigger, reset=reset)

    # calculate cumulative volume over the entire execution without reset
    total_vol = csp.stats.sum(volume, interval=None, min_window=timedelta(minutes=1), trigger=trigger)

    csp.add_graph_output('vwap', vwap)
    csp.add_graph_output('ewm_price', ewm_price)
    csp.add_graph_output('total_vol', total_vol)

results = csp.run(graph, starttime=st, endtime=st+timedelta(minutes=10))

for i in range(10):
    print(f"Time: {results['vwap'][i][0]}", end="\t")
    print(f"VWAP: {round(results['vwap'][i][1], 4)}", end="\t")
    print(f"Exp Price: {round(results['ewm_price'][i][1], 4)}", end="\t")
    print(f"Cum. Vol: {results['total_vol'][i][1]}")

This provides the following output:

Output: Example 1

Time: 2020-01-01 00:01:00   VWAP: nan   Exp Price: nan  Cum. Vol: 0.0
Time: 2020-01-01 00:02:00   VWAP: 12.653    Exp Price: 12.653   Cum. Vol: 100.0
Time: 2020-01-01 00:03:00   VWAP: 13.4858   Exp Price: 13.4315  Cum. Vol: 215.0
Time: 2020-01-01 00:04:00   VWAP: 13.7378   Exp Price: 13.3207  Cum. Vol: 300.0
Time: 2020-01-01 00:05:00   VWAP: 14.518    Exp Price: 14.0364  Cum. Vol: 485.0
Time: 2020-01-01 00:06:00   VWAP: 18.0284   Exp Price: 18.0535  Cum. Vol: 875.0
Time: 2020-01-01 00:07:00   VWAP: 18.1797   Exp Price: 18.3767  Cum. Vol: 945.0
Time: 2020-01-01 00:08:00   VWAP: 19.023    Exp Price: 18.3767  Cum. Vol: 945.0
Time: 2020-01-01 00:09:00   VWAP: 19.763    Exp Price: 18.9312  Cum. Vol: 1010.0
Time: 2020-01-01 00:10:00   VWAP: 19.763    Exp Price: 18.9312  Cum. Vol: 1010.0

NumPy time series

The second example uses a NumPy array of prices sampled at regular intervals and computes the mean, geometric mean, and exponential moving average for each symbol. It also computes correlation matrix between all 3 symbols. Below is the code to implement this behaviour.

Example 2: NumPy

st = datetime(2020,1,1)

# Prices are sampled every 1 minute
symb1_prices = [8.65, 8.67, 8.72, 8.68, 8.90, 9.1, 9.04, 9.11, 9.34, 9.36]
symb2_prices = [314.34, 315.67, 316.70, 316.45, 320.10, 323.84, 322.76, 328.56, 328.60, 329.60]
symb3_prices = [23.75, 23.55, 23.23, 23.98, 22.10, 21.89, 21.78, 20.50, 21.00, 21.23]

prices_data = [(st+timedelta(minutes=i+1), np.array([symb1_prices[i], symb2_prices[i], symb3_prices[i]], dtype=float))
                for i in range(10)]

@csp.graph
def graph():
    price = csp.curve(typ=np.ndarray, data=prices_data)

    # Trigger a computation every minute
    trigger = csp.timer(timedelta(minutes=1))

    # calculate rolling 5 minute mean price for each symbol, starting at 1 minute of data
    avg_price = csp.stats.mean(price, interval=timedelta(minutes=5), min_window=timedelta(minutes=1), trigger=trigger)

    # calculate rolling 5 minute geometric mean price for each symbol
    geom_avg_price = csp.stats.gmean(price, interval=timedelta(minutes=5), min_window=timedelta(minutes=1), trigger=trigger)

    # calculate an adjusted EMA for the prices
    ewm_price = csp.stats.ema(price, alpha=0.1, adjust=True, trigger=trigger)

    # calculate 5 minute correlation between all 3 symbols, starting at 3 minutes
    corr_matrix = csp.stats.corr_matrix(price, interval=timedelta(minutes=5), min_window=timedelta(minutes=3), trigger=trigger)

    csp.add_graph_output('avg_price', avg_price)
    csp.add_graph_output('geom_avg_price', geom_avg_price)
    csp.add_graph_output('ewm_price', ewm_price)
    csp.add_graph_output('corr_matrix', corr_matrix)

results = csp.run(graph, starttime=st, endtime=st+timedelta(minutes=10))

print("Price Averages\n")
for i in range(10):
    print(f"Time: {results['avg_price'][i][0]}", end="\t")
    print(f"Mean Price: {results['avg_price'][i][1]}", end="\t")
    print(f"Geom. Mean Price: {results['geom_avg_price'][i][1]}", end="\t")
    print(f"Exp. Price: {results['ewm_price'][i][1]}")

print("\nCorrelation\n")
for i in range(7):
    print(f"Time: {results['corr_matrix'][i][0]}", end="\t")
    print(f"Corr. Matrix:\n{results['corr_matrix'][i][1]}")

This produces the following output:

Output: Example 2

Price Averages

Time: 2020-01-01 00:01:00   Mean Price: [  8.65 314.34  23.75]  Geom. Mean Price: [  8.65 314.34  23.75]    Exp. Price: [  8.65 314.34  23.75]
Time: 2020-01-01 00:02:00   Mean Price: [  8.66  315.005  23.65 ]   Geom. Mean Price: [  8.65999423 315.00429807  23.64978858]  Exp. Price: [  8.66052632 315.04        23.64473684]
Time: 2020-01-01 00:03:00   Mean Price: [  8.68 315.57  23.51]  Geom. Mean Price: [  8.67995013 315.56852083  23.50902287]  Exp. Price: [  8.68247232 315.65254613  23.49169742]
Time: 2020-01-01 00:04:00   Mean Price: [  8.68   315.79    23.6275]    Geom. Mean Price: [  8.6799626  315.78866016  23.62589277]  Exp. Price: [  8.68175342 315.88443152  23.63368712]
Time: 2020-01-01 00:05:00   Mean Price: [  8.724 316.652  23.322]   Geom. Mean Price: [  8.72353051 316.64625745  23.31251011]  Exp. Price: [  8.73504798 316.91384923  23.2591695 ]
Time: 2020-01-01 00:06:00   Mean Price: [  8.814 318.552  22.95 ]   Geom. Mean Price: [  8.8124637  318.53747102  22.93535502]  Exp. Price: [  8.81293615 318.39203037  22.96696094]
Time: 2020-01-01 00:07:00   Mean Price: [  8.888 319.97   22.596]   Geom. Mean Price: [  8.88642762 319.95566723  22.579736  ]  Exp. Price: [  8.85645973 319.22928237  22.73944438]
Time: 2020-01-01 00:08:00   Mean Price: [  8.966 322.342  22.05 ]   Geom. Mean Price: [  8.96453127 322.31694075  22.02215374]  Exp. Price: [  8.90097697 320.86759325  22.3462371 ]
Time: 2020-01-01 00:09:00   Mean Price: [  9.098 324.772  21.454]   Geom. Mean Price: [  9.09689154 324.75483388  21.44538332]  Exp. Price: [  8.97264489 322.12986314  22.12647182]
Time: 2020-01-01 00:10:00   Mean Price: [  9.19  326.672  21.28 ]   Geom. Mean Price: [  9.18904091 326.65997908  21.27381398]  Exp. Price: [  9.03211707 323.27678308  21.98883292]

Correlation

Time: 2020-01-01 00:03:00   Corr. Matrix:
[[ 1.          0.95054061 -0.9939441 ]
 [ 0.95054061  1.         -0.97891514]
 [-0.9939441  -0.97891514  1.        ]]
Time: 2020-01-01 00:04:00   Corr. Matrix:
[[ 1.          0.86504218 -0.66950353]
 [ 0.86504218  1.         -0.29371476]
 [-0.66950353 -0.29371476  1.        ]]
Time: 2020-01-01 00:05:00   Corr. Matrix:
[[ 1.          0.96714874 -0.96061921]
 [ 0.96714874  1.         -0.88441561]
 [-0.96061921 -0.88441561  1.        ]]
Time: 2020-01-01 00:06:00   Corr. Matrix:
[[ 1.          0.9978151  -0.92710194]
 [ 0.9978151   1.         -0.90858578]
 [-0.92710194 -0.90858578  1.        ]]
Time: 2020-01-01 00:07:00   Corr. Matrix:
[[ 1.          0.998662   -0.93410762]
 [ 0.998662    1.         -0.91916838]
 [-0.93410762 -0.91916838  1.        ]]
Time: 2020-01-01 00:08:00   Corr. Matrix:
[[ 1.          0.91208488 -0.91005387]
 [ 0.91208488  1.         -0.95709548]
 [-0.91005387 -0.95709548  1.        ]]
Time: 2020-01-01 00:09:00   Corr. Matrix:
[[ 1.          0.84663341 -0.62080623]
 [ 0.84663341  1.         -0.93385468]
 [-0.62080623 -0.93385468  1.        ]]
...

Performance benchmarking

Benchmark 1: csp.stats calculations on a single symbol compared to an equivalent cross-sectional NumPy calculation

437694455 437694456 437694457

Benchmark 2: csp.stats functions applied to multiple symbols; using a single computation node with stats.list_to_numpy conversion compared to a node per symbol.

437694458 437694459

API reference

Base Statistics

Count

count(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data.
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
- By default, the min_window is equal to the interval
ignore_na: if True, ignores NaN values in the window (does not count them). If false, NaN values make the count NaN.
- By default, ignore_na is True
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
- By default, there is no reset series.
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_points, NaN is returned.

Returns:

A time-series of how many data points are currently in the interval. If a tick count is used, then it is necessarily less than or equal to the interval.

Examples: `count`

Starttime: 2020-01-01 00:00:00

1. Default behavior

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': nan, '2020-01-05': 5}
count(x, interval=3)

# NaN is not counted
{'2020-01-03': 3, '2020-01-04': 2, '2020-01-05': 2}

2. Including NaN

count(x, interval=3, ignore_na=False)

{'2020-01-03': 3, '2020-01-04': nan, '2020-01-05': nan}

3. Triggering

trigger = {'2020-01-03': True, '2020-01-05': True}
count(x, interval=timedelta(days=3), min_window=timedelta(days=2), ignore_na=True, trigger=trigger)

{'2020-01-03': 3, '2020-01-05': 2}

4. Sampling

sampler = {'2020-01-01': True, '2020-01-02': True, '2020-01-03': True, '2020-01-05': True, '2020-01-06': True}
count(x, interval=timedelta(days=3), min_window=timedelta(days=2), sampler=sampler)

{'2020-01-03': 3, '2020-01-05': 2}

Note: the x value at 2020-01-04 is ignored completely since sampler does not tick, while the value at 2020-01-06 is treated as NaN.

5. Reset

reset = {'2020-01-04': True}
count(x, interval=timedelta(days=3), min_window=timedelta(days=2), reset=reset)

{'2020-01-03': 3, '2020-01-04': 0, '2020-01-05': 1}

Note: the window data is reset at 2020-01-04, and its value is NaN, so the count is 0

6. NumPy

x_np = {'2020-01-01': [1,1], '2020-01-02': [2,np.nan], '2020-01-03': [3,3]}
count(x_np, interval=3, min_window=1)

{'2020-01-01': [1,1], '2020-01-02': [2,1], '2020-01-03': [3,2]} # count is per element

Unique

unique(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    min_data_points: int = 0,
    precision: int = 10
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
- By default, the min_window is equal to the interval
trigger: another optional time-series which can be use to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
- By default, there is no reset series.
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.
precision: the decimal place precision at which two floats are considered non-unique. For example, if precision=2, then 2.001 and 2.002 would be considered non-unique.
- By default, precision is set to 10 decimal places.

Returns:

a time-series of how many unique (excluding nan) values are currently in the interval

Examples: `unique`

Starttime: 2020-01-01 00:00:00

1. Default behavior

x = {'2020-01-01': 2, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': nan, '2020-01-05': 3}
unique(x, interval=3, min_window=2)

{'2020-01-02': 1, '2020-01-03': 2, '2020-01-04': 2, '2020-01-05': 1}

2. Triggering

trigger = {'2020-01-03': True, '2020-01-05': True}
unique(x, interval=timedelta(days=3), min_window=timedelta(days=2), trigger=trigger)

{'2020-01-03': 2, '2020-01-05': 1}

3. NumPy

x_np = {'2020-01-01': [1,1], '2020-01-02': [2,np.nan], '2020-01-03': [3,1]}
unique(x_np, interval=3, min_window=1)

{'2020-01-01': [1,1], 2020-01-02: [2,1], '2020-01-03': [3,1]} # unique is per element

Sum

sum(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    precise: bool = False,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    weights: ts[Union[float, np.ndarray]] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data. Can either be a ts[Union[float, np.ndarray]] or ts[np.ndarray].
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
- By default, the min_window is equal to the interval
precise: if True we use a more numerically stable implementation (Kahan) which is less efficient
ignore_na: if True, does not include any nan values in the window. If false, nan values are included and will make the entire window value nan.
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
weights: a time-series of weights for each observation in x, used to calculate a weighted sum (optional).
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset": another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the window statistic, and in doing so clears any accumulated floating-point error
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling sums over the interval

Examples: `sum`

Starttime: 2020-01-01 00:00:00

1. Default behavior

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': nan, '2020-01-05': 5}
sum(x, interval=3)

{'2020-01-03': 6, '2020-01-04: 5', '2020-01-05': 8}

2. Including NaNs

sum(x, interval=3, min_window=2, ignore_na=False)

{'2020-01-02': 3, '2020-01-03': 6, '2020-01-04': nan, '2020-01-05': nan}

3. Weighted single input

weights = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-04': 3}
sum(x_np, interval=3, weights=weights)

{'2020-01-03': 11, '2020-01-04': 10, '2020-01-05': 21} # 21 = 5x3 + 3x2

4. NumPy

x_np = {'2020-01-01': [1,1], '2020-01-02': [2,np.nan], '2020-01-03': [3,1]}
sum(x_np, interval=3, min_window=1)

{'2020-01-01': [1,1], '2020-01-02': [3,1], '2020-01-03': [4,2]}

5. NumPy weighted sum

np_weights = {'2020-01-01': [1,2], '2020-01-02': [2,1}
sum(x_np, interval=3, min_window=1, weights=np_weights)

{'2020-01-01': [1,2], '2020-01-02': [5,2], '2020-01-03': [11,3]} # weights applied elementwise

Product

prod(
    x: ts[Union[float, np.ndarray]],
    interval : Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
- By default, the min_window is equal to the interval
ignore_na: if True, does not include any nan values in the window. If false, nan values are included and will make the entire window value nan.
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the window statistic, and in doing so clears any accumulated floating-point error
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling products over the interval. The computation is unstable for large products and windows.

Examples: `prod`

Starttime: 2020-01-01 00:00:00

1. Default behavior

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': nan, '2020-01-05': 5}
prod(x, interval=3, min_window=2, ignore_na=True)

{'2020-01-02': 2, '2020-01-03': 6 '2020-01-04': 6, '2020-01-05': 15}

2. NumPy

x_np = {'2020-01-01': [1,2], '2020-01-02': [3,4], '2020-01-03': [5,6]}
prod(x_np, 3, 2)

{'2020-01-02': [3,8], '2020-01-03': [15,24]}

First

first(
    x: ts[Union[float, np.ndarray]],
    interval : Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    min_data_points: int = 0,
    ignore_na: bool = True
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
- By default, the min_window is equal to the interval
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.
ignore_na: if True, will return the first non-nan value in the window. If False, will return the first value in the window

Returns:

a time-series of the earliest (non-nan) value still within the given interval

Examples: `first`

See last

Last

last(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
- By default, the min_window is equal to the interval
ignore_na: if True, will return the last non-nan value in the window. If False, will return the last value in the window
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of the most recent value within the given interval

Examples: `first` and `last`

Starttime: 2020-01-01 00:00:00

1. Default - first

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': nan, '2020-01-05': 5}
first(x, interval=3)

{'2020-01-03': 1, '2020-01-04': 2, '2020-01-05': 3}

2. Including NaN - last

last(x, interval=3, ignore_na=False)

{'2020-01-03': 3, '2020-01-04': nan, '2020-01-05': nan}

3. Triggering - last

trigger = {'2020-01-03': True, '2020-01-04': True}
last(x, interval=timedelta(days=3), ignore_na=True, trigger=trigger)

{'2020-01-03': 3, '2020-01-04': 3}

4. NumPy - first

x_np = {'2020-01-01': [1,1], '2020-01-02': [2,np.nan], '2020-01-03': [3,3]}
first(x_np, interval=2)

# first non-nan value
{'2020-01-02': [1,1], '2020-01-03': [2,3]}

Mean

mean(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    weights: ts[Union[float, np.ndarray]] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[float, np.ndarray]]

x: the time-series data. Can either be a ts[Union[float, np.ndarray]] or a ts[np.ndarray].
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
- By default, the min_window is equal to the interval
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
weights: a time-series of weights for each observation in x, used to calculate a weighted mean (optional). Weights do not need to be normalized.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the window statistic, and in doing so clears any accumulated floating-point error
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling means over the interval. Computation uses smart updating so overflow is not an issue, since no sums are kept

Examples: `mean`

See gmean

Geometric Mean

gmean(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    min_data_points: int = 0
)→ ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
- By default, the min_window is equal to the interval
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling geometric means over the interval. Requires a strictly positive-valued input.

Examples: `mean` and `gmean`

Starttime: 2020-01-01 00:00:00

1. Default behavior

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': nan, '2020-01-05': 5}
mean(x, interval=3, min_window=2)

{'2020-01-02': 1.5, '2020-01-03': 2.0, '2020-01-04': 2.5, '2020-01-05': 4.0}

2. Including NaN

mean(x, interval=3, min_window=2, ignore_na=False)

{'2020-01-02': 1.5, '2020-01-03': 2.0, '2020-01-04': nan, '2020-01-05': nan}

3. Geometric mean

trigger = {'2020-01-03': True, '2020-01-05': True}
gmean(x, interval=timedelta(days=3), min_window=timedelta(days=2), ignore_na=True, trigger=trigger)

{'2020-01-03': 1.817, '2020-01-05': 3.873}

4. Weighted mean

weights = {'2020-01-01': 1, '2020-01-03': 2}
mean(x, interval=3, min_window=2, ignore_na=True, weights=weights)

{'2020-01-02': 1.5, '2020-01-03': 2.25, '2020-01-04': 2.667, '2020-01-05': 4.0}

Note: the first two observations get relative weight of 1, then the last three get relative weight of 2

5. NumPy weighted mean

x_np = {'2020-01-01': [1., 1., 1.], '2020-01-02': [2., 2., 2.], '2020-01-03': [3., 3., 3.]}
np_weights = {'2020-01-01': [1., 1., 1.], '2020-01-02': [2., 1., 2.], '2020-01-03': [3., 1., 3.]}
mean(x_np, 3, 2)

{'2020-01-02': [1.667, 1.5, 1.667], '2020-01-03': [2.667, 2.0, 2.6667]}

Order Statistics

Maximum

max(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
- By default, the min_window is equal to the interval
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling maximums over the interval.

Examples: `max`

See min

Minimum

min(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling minimums over the interval.

Examples: `max` and `min`

Starttime: 2020-01-01 00:00:00

1. Default behavior

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': nan, '2020-01-05': 5}
min(x, interval=3, min_window=2)

{'2020-01-02': 1, '2020-01-03': 1, '2020-01-04': 2, '2020-01-05': 3}

2. Including NaN

max(x, interval=3, min_window=2, ignore_na=False)

{'2020-01-02': 2, '2020-01-03': 3, '2020-01-04': nan, '2020-01-05': nan}

3. NumPy example

x_np= {'2020-01-01': [2,3], '2020-01-02': [6,1], '2020-01-03': [1,9]}
min(x, interval=timedelta(days=3), min_window=timedelta(days=1))

{'2020-01-02': [2,1], '2020-01-03': [1,1]}

Median

median( x: ts[Union[float, np.ndarray]], interval: Union[timedelta, int] = None, min_window: Union[timedelta, int] = None, ignore_na: bool = True, trigger: ts[object] = None, sampler: ts[object] = None, reset: ts[object] = None, min_data_points: int = 0 ) → ts[Union[float, np.ndarray]]

Args:
- **x**: the time-series data
- **interval**: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
    - if an int, represents the number of ticks to use
    - if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
- **min_window**: the minimum allowable interval to use before outputting data
    - If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
    - If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
- **ignore_na**: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
- **trigger**: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned. 
    - *By default*, the trigger is the series itself.
- **sampler**: another optional time-series which specifies when x *should* tick. The behavior is as follows:
    - If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
    - If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
    - If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
    - *By default*, the sampler is the series itself.
- **reset**: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
- **min_data_points**: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:
- a time-series of rolling medians over the interval. Uses midpoint interpolation if there are an even number of samples.

#### Examples: `median`
See `quantile`


### Quantile

quantile( x: ts[Union[float, np.ndarray]], interval: Union[timedelta, int] = None, quant: Union[float, List[float]] = None, min_window: Union[timedelta, int] = None, interpolate: str = "linear", ignore_na: bool = True, trigger: ts[object] = None, sampler: ts[object] = None, reset: ts[object] = None, min_data_points: int = 0 ) → Union[ts[Union[float, np.ndarray]], [ts[Union[float, np.ndarray]]]

Args:
- **x**: the time-series data
- **interval**: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
    - if an int, represents the number of ticks to use
    - if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
- **quant:** the quantile to calculate, which must be between 0 and 1
    - If provided a list, then all quantiles will be calculated for the list. 
- **min_window**: the minimum allowable interval to use before outputting data
    - If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
    - If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks.
- **interpolate**: the interpolation method to use when the quantile does not correspond to an individual value. Must be one of the following options:
    - **"linear"**: interpolates linearly between the two closest values. For example, the 0.333 quantile of (1,2) with linear interpolation is 1.333.
    - **"lower"**: returns the lower of the two closest values.
    - **"higher"**: returns the higher of the two closest values.
    - **"midpoint"**: returns the midpoint between the two closest values. For example, the 0.333 quantile of (1,2) with midpoint interpolation is 1.5.
    - **"nearest"**: returns the value at the nearest position.  For example, the 0.333 quantile of (1,2) with nearest interpolation is 1. In cases of ties, the higher value is returned.
- **ignore_na**: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
- **trigger**: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned. 
    - *By default*, the trigger is the series itself.
- **sampler**: another optional time-series which specifies when x *should* tick. The behavior is as follows:
    - If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
    - If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
    - If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
    - *By default*, the sampler is the series itself.
- **reset**: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
- **min_data_points**: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:
- a time-series *or* list-basket of time-series of rolling quantiles over the interval. 
    - If the quant parameter is a list then a list-basket will be returned.
    - If it is a float then a time-series will be returned.
    - The order of quantiles in the list-basket is equal to the order of the input.


#### Examples: `median` and `quantile`

Starttime: `2020-01-01 00:00:00`

**1. Median**

```python
x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': nan, '2020-01-05': 5}
median(x, interval=3, min_window=2)

{'2020-01-02': 1.5, '2020-01-03': 2, '2020-01-04': 2.5, '2020-01-05': 4}

2. Quantile with multiple values

quantile(x, interval=3, quant=[0.25, 0.5, 0.75], min_window=2, ignore_na=False)

[
    {'2020-01-02': 1.25, '2020-01-03': 1.5, '2020-01-04': nan, '2020-01-05': nan},
    {'2020-01-02': 1.5, '2020-01-03': 2.0, '2020-01-04': nan, '2020-01-05': nan},
    {'2020-01-02': 1.75, '2020-01-03': 2.5, '2020-01-04': nan, '2020-01-05': nan}
]

3. Quantile with trigger

trigger = {'2020-01-03': True, '2020-01-05': True}
quantile(x, interval=timedelta(days=3), quant=0.333, min_window=timedelta(days=2), interpolate="midpoint", ignore_na=True, trigger=trigger)

{'2020-01-03': 1.5, '2020-01-05': 4}

4. NumPy array with multiple quantiles

x_np = {'2020-01-01': [1,2,3], '2020-01-02': [2,3,4], '2020-01-03': [3,4,5]}
quantile(x_np, interval=3, quant=[0.25,0.5,0.75], min_window=1)

# this is a listbasket of NumPy array time series
[
    {'2020-01-01': [1,2,3], '2020-01-02': [1.25, 2.25, 3.25], '2020-01-03': [1.5, 2.5, 3.5]},
    {'2020-01-01': [1,2,3], '2020-01-02': [1.5, 2.5, 3.5], '2020-01-03': [2., 3., 4.]},
    {'2020-01-01': [1,2,3], '2020-01-02': [1.75, 2.75, 3.75], '2020-01-03': [2.5, 3.5, 4.5]}
]

Argmin

argmin(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    return_most_recent: bool = True,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[datetime, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
return_most_recent: if True, in the case of a tie, the most recent time will be returned. If false, the least recent time will be returned.
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid.

Returns:

a time-series of rolling argmin values over the interval, returned as a datetime or NumPy array of np.datetime64 objects. If no data is present or NaN invalidation occurs, the default time '1970-1-1 00:00:00' is returned.

Examples: `argmin`

See argmax

Argmax

argmax(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    return_most_recent: bool = True,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[datetime, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
return_most_recent: if True, in the case of a tie, the most recent time will be returned. If false, the least recent time will be returned.
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling argmax values over the interval, returned as a datetime or NumPy array of np.datetime64 objects. If no data is present or NaN invalidation occurs, the default time '1970-1-1 00:00:00' is returned.

Examples: `argmax and` argmin`

Starttime: 2020-01-01 00:00:00

1. Default behavior

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 1, '2020-01-04': nan, '2020-01-05': 4}
argmax(x, 3)

{'2020-01-03': '2020-01-02', '2020-01-04': '2020-01-02', '2020-01-05': '2020-01-05'}

argmin(x, 3)

{'2020-01-03': '2020-01-03', '2020-01-04': '2020-01-03', '2020-01-05': '2020-01-03'}

2. NumPy example

x_np = {'2020-01-01': [1,2], '2020-01-02': [2,1], '2020-01-03': [3,0]}
argmax(x_np, 3, 2)

{'2020-01-02': ['2020-01-02', '2020-01-01'], '2020-01-03': ['2020-01-03', '2020-01-01']}

argmin(x_np, 3, 1)

{'2020-01-02': ['2020-01-01', '2020-01-02'], '2020-01-03': ['2020-01-01', '2020-01-03']}

3. return_most_recent=False

argmin(x, 3, return_most_recent=False)

{'2020-01-03': '2020-01-01', '2020-01-04': '2020-01-03', 2020-01-05: '2020-01-03'} # Note how the first element is '2020-01-01', not '2020-01-03'

Rank

rank(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    method: str = "min",
    ignore_na: bool = True,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    min_data_points: int = 0,
    na_option: str = "keep"
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use 100-tick rolling interval with no output until we have 50 ticks
method: the method to use to rank groups of records that have the same value
- "min": the lowest rank in the group is returned i.e. if the window data is [1,2,2,3] and the last tick is 2, then rank=1
- "max": the highest rank in the group is returned i.e. if the window data is [1,2,2,3] and the last tick is 2, then rank=3
- "avg": the average rank in the group is returned i.e. if the window data is [1,2,2,3] and the last tick is 2, then rank=2
- By default, the "min" method is used.
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, nan is returned.
na_option: how to rank a nan value when it is the last value to be ranked
- "keep": return a nan rank for a nan value
- "last": rank the last non-nan value present in the interval
- By default, the "keep" option is used.
Output: a time-series of rolling ranks over the interval, where a rank of 0 means that the current (last) ticked value is the smallest in the given interval.

Examples: `rank`

Starttime: 2020-01-01 00:00:00

1. Default behavior

x = {'2020-01-01': 1, '2020-01-02': 3, '2020-01-03': 2, '2020-01-04': 5, '2020-01-05': 4}
rank(x, 5, min_window=3)

{'2020-01-03': 1, '2020-01-04': 3, '2020-01-05': 3}

2. NumPy example

x_np = {'2020-01-01': [1,2], '2020-01-02': [3,2], '2020-01-03': [2,1]}
rank(x_np, 3, 2)

# Note how the second element at '2020-01-02' is 0, not 1, as by default the "min" method is used
{'2020-01-02': [1, 0], '2020-01-03': [1, 0]}

3. "keep" vs "last" NaN option

x = {'2020-01-01': 1, '2020-01-02': 3, '2020-01-03': 2, '2020-01-04': nan, '2020-01-05': 4}
rank(x, 5, min_window=3, na_option="keep")

{'2020-01-03': 1, '2020-01-04': nan, '2020-01-05': 3}

rank(x, 5, min_window=3, na_option="last")

# the last valid value, 1, is ranked at '2020-01-04'
{'2020-01-03': 1, '2020-01-04': 1, '2020-01-05': 3}

Moment-Based Statistics

Variance

var(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ddof: int = 1,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    weights: ts[Union[float, np.ndarray]] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
ddof: delta degrees of freedom. Example: if ddof=1, then normalization term is 1/(N-1). If ddof=0, then 1/N.
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
weights: a time-series of weights for each observation in x, used to calculate a weighted variance (optional). Weights do not need to be normalized.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the window statistic, and in doing so clears any accumulated floating-point error
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling variance over the interval. If insufficient samples for given ddof, then no value output is generated. Since the smart mean is being used, overflow is not a problem.

Examples: `var`

See Standard Error.

Standard Deviation

stddev(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ddof: int = 1,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    weights: ts[Union[float, np.ndarray]] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
ddof: delta degrees of freedom
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
weights: a time-series of weights for each observation in x, used to calculate a weighted standard deviation (optional). Weights do not need to be normalized.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the window statistic, and in doing so clears any accumulated floating-point error
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling standard deviations over the interval. If insufficient samples for given ddof, then no value output is generated.

Examples: `stddev`

See Standard Error.

Standard Error

sem(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ddof: int = 1,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    weights: ts[Union[float, np.ndarray]] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
): → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
ddof: delta degrees of freedom
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another optional time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned.
- By default, the trigger is the series itself.
weights: a time-series of weights for each observation in x, used to calculate a weighted standard error (optional). Weights do not need to be normalized.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the window statistic, and in doing so clears any accumulated floating-point error
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling standard errors

Examples: Variance, Standard Deviation, Standard Error

Starttime: 2020-01-01 00:00:00

1. Variance

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': nan, '2020-01-05': 5}
var(x, interval=3, min_window=2)

{'2020-01-02': 0.5, '2020-01-03': 1.0, '2020-01-04': 0.5, '2020-01-05': 2.0}

2. Biased variance

var(x, interval=3, min_window=2, ddof=0, ignore_na=True) # biased

{'2020-01-02': 0.25, '2020-01-03': 0.666, '2020-01-04': 0.25, '2020-01-05': 1.0}

3. Standard deviation including NaNs

stddev(x, interval=3, min_window=2, ignore_na=False)

{'2020-01-02': 0.707, '2020-01-03': 1.0, '2020-01-04': nan, '2020-01-05': nan}

4. Standard error with triggering

trigger = {'2020-01-03': True, '2020-01-05': True}
sem(x, interval=timedelta(days=3), min_window=timedelta(days=2), trigger=trigger)

{'2020-01-03': 0.707, '2020-01-05': 1.0}

Covariance

cov(
    x: ts[Union[float, np.ndarray]],
    y: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ddof: int = 1,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    weights: ts[Union[float, np.ndarray]] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
): → ts[Union[float, np.ndarray]]

Args:

x: time-series data. If x is of type np.ndarray, then the covariance calculation is performed element-wise with the corresponding values in y.
y: time-series data that ticks in sequence with x
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
ddof: delta degrees of freedom
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned
- By default, the trigger is the series itself.
weights: a time-series of weights for each observation in x, used to calculate a weighted covariance (optional). Weights do not need to be normalized.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the window statistic, and in doing so clears any accumulated floating-point error
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling covariances between x and y

Examples: `cov`

See Correlation.

Correlation

corr(
    x: ts[Union[float, np.ndarray]],
    y: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    weights: ts[Union[float, np.ndarray]] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
): → ts[Union[float, np.ndarray]]

Args:

x: time-series data. If x is of type np.ndarray, then the correlation calculation is performed element-wise with the corresponding values in y.
y: time-series data that ticks in sequence with x
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned
- By default, the trigger is the series itself.
weights: a time-series of weights for each observation in x, used to calculate a weighted correlation (optional). Weights do not need to be normalized.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the window statistic, and in doing so clears any accumulated floating-point error
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling Pearson correlation coefficients between x and y

Examples: Covariance and Correlation

Starttime: 2020-01-01 00:00:00

1. Covariance

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': 4, '2020-01-05': 5}
y = {'2020-01-01': 5, '2020-01-02': 4, '2020-01-03': 3, '2020-01-04': 2, '2020-01-05': 1}
cov(x, y, interval=3, min_window=2)

{'2020-01-02': -0.5, '2020-01-03': -1.0, '2020-01-04': -1.0, '2020-01-05': -1.0}

2. Correlation

corr(x, y, interval=3)

{'2020-01-03': -1.0, '2020-01-04': -1.0, '2020-01-05': -1.0}

Skewness

skew(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ignore_na: bool = True,
    bias: bool = False,
    trigger: ts[object] = None,
    weights: ts[Union[float, np.ndarray]] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
): → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
bias: if True, calculates a biased (unadjusted) skew. If false (default), calculates a Gaussian-unbiased measure.
trigger: another time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned
- By default, the trigger is the series itself.
weights: a time-series of weights for each observation in x, used to calculate a weighted skew (optional). Weights do not need to be normalized.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the window statistic, and in doing so clears any accumulated floating-point error
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling sample skew measures, using the adjusted Fisher–Pearson standardized moment coefficient.

Examples: `skew`

See Kurtosis.

Kurtosis

kurt(
    x: ts[Union[float, np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ignore_na: bool = True,
    excess: bool = True,
    bias: bool = False,
    trigger: ts[object] = None,
    weights: ts[Union[float, np.ndarray]] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
): → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
excess: if True (default) uses the definition of excess kurtosis (kurt - 3). If false, uses the standard definition.
bias: if True, calculates a biased (unadjusted) kurtosis. If false (default), calculates a Gaussian-unbiased measure.
trigger: another time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned
- By default, the trigger is the series itself.
weights: a time-series of weights for each observation in x, used to calculate a weighted kurtosis (optional). Weights do not need to be normalized.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the window statistic, and in doing so clears any accumulated floating-point error
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of rolling sample kurtosis measures, using the adjusted Fisher–Pearson standardized moment coefficient.

Examples: `skew` and `kurt`

Starttime: 2020-01-01 00:00:00

1. Skew

x = {'2020-01-01': 1, '2020-01-02': 2, ..., 2020-01-10: 10}
skew(x, interval=7)

{2020-01-07: 0, 2020-01-08: 0, 2020-01-09: 0, 2020-01-10: 0}

2. Kurtosis

kurt(x, interval=7) # excess kurtosis

{2020-01-07: -1.2, 2020-01-08: -1.2, 2020-01-09: -1.2, 2020-01-10: -1.2}

Exponential Moving Statistics

Exponential Moving Average

ema(
    x: ts[Union[float, np.ndarray]],
    min_periods: int = 1,
    alpha: Optional[float] = None,
    span: Optional[float] = None,
    com: Optional[float] = None,
    halflife: Optional[timedelta] = None,
    adjust: bool = True,
    horizon: int = None,
    ignore_na: bool = False,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
min_periods: the minimum allowable number of ticks to use before outputting data. The default is 1 for any EMA function.
alpha: the EMA weight parameter specified directly. If adjust = True, EMA is calculated such that

$$EMA(t) = \frac{\sum\limits_{t=-n}^{0} (1-\alpha)^{-t} x(-t)}{\sum\limits_{t=-n}^{0} (1-\alpha)^{-t}}$$

If adjust = False, EMA is calculated such that

$$EMA(t) = (1-\alpha)EMA(t-1) + \alpha x(t)$$ $$EMA(t=0) = x(0)$$

By default, adjust = True, to give better estimates for starting intervals.

The following are alternative methods to specify the $\alpha$ parameter.
- span: specify alpha in terms of span, such that
  
  $$\alpha = \frac{2}{span+1}$$
- com: specify alpha in terms of centre of mass, such that
  
  $$\alpha = \frac{1}{1+com}$$
- halflife: Halflife is different from the other parameters. Half-life is a timedelta argument that specifies the half-life of observation weights. Half-life is useful when observations are irregularly spaced and a better estimate is needed to properly weight more recent data. Let $t_{-1}$ be the time of the last observation.
  
  Then:
  
  $$\lambda(t) = 1 - \exp(\frac{-(t-t_{-1})*\ln(2)}{halflife})$$ $$EMA(t) = \frac{ \lambda(t)*EMA(t-1) + x(t)}{\text{normalization constant}}$$
  
  Something to note is that the ignore_na flag does not matter if a halflife interval is specified. The behavior would be the same in both cases, since an absolute time interval is being used to re-weight the moving average, not a tick interval.
  
  Exactly one of alpha, span, com, halflife must be given
adjust: if True, early observations are adjusted to give a more "smoothed" estimate of the EMA. The difference is that if adjust=True, then each new observation receives a relative weight of 1. If adjust = False, each new observation receives a relative weight of alpha.
- adjust=True means that:
$$EMA(t) = \frac{x(t)+(1-\alpha)x(t-1)+(1-\alpha)^2 x(t-2) + ... + (1-\alpha)^n x(t-n)}{1+(1-\alpha)+(1-\alpha)^ 2 + ... + (1-\alpha)^n}$$
- adjust=False means that:
$$EMA(t) = \frac{\alpha * x(t) + \alpha * (1-\alpha) * x(t-1) + \alpha * (1-\alpha)^2 * x(t-2) + ... + \boldsymbol{(1-\alpha)^n x(0)}}{\alpha+\alpha*(1-\alpha)+\alpha*(1-\alpha)^ 2 + ... + (1-\alpha)^n}$$

$$\text{and thus } EMA(t=0) = x(0)$$

Adjust only applies with tick specified intervals, not time specified intervals. Time specified intervals (i.e. half-life) do not need adjustment as they are, by definition, already adjusted.
horizon: the maximum number of ticks to use in the computation. For example, if horizon = 10, then only the 10 most recent data points are used. If not specified, all data points for x are used, with early ticks decaying exponentially in weighting. Horizon will be ignored with a half-life (time-based) interval.
- If horizon is set to h, then even if x has more than h ticks the EMA will computed as such if adjust=True.
$$EMA(t) = \frac{\sum_{t=-h}^{0} (1-\alpha)^{-t} x(t)}{\sum_{t=-h}^{0} (1-\alpha)^{-t}}$$
- The only difference if adjust=False is that the first ever tick, while in the window, receives weight 1 at the start instead of weight $\alpha$ like the rest of the values.
ignore_na: if True, nan values will be "ignored" meaning weights will be placed on relative position. If False (default), weights are based on global position, and renormalized as such.
- For example, let us consider a dataset (1,nan,2) using adjust=True.
  - If ignore_na=True then the weighting is based on relative position as such: $$EMA(t=2) = \frac{(1-\alpha)*1 + 2}{(1-\alpha)+1}$$
  - If ignore_na=False then the weighting is based on global position as such: $$EMA(t=2) = \frac{(1-\alpha)^2*1 + 2}{(1-\alpha)^2+1}$$
trigger: another time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation.
recalc: another optional time-series which triggers a clean recalculation of the EMA, and in doing so clears any accumulated floating-point error.
- Note: only valid when a finite-horizon EMA is used.
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of exponentially-weighted moving averages over the interval.

Examples: `ema`

Starttime: 2020-01-01 00:00:00

1. Unadjusted EMA

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': 4, '2020-01-05': 5}
ema(x, alpha=0.1, adjust=False) # unadjusted

{'2020-01-01': 1.0, '2020-01-02': 1.1, '2020-01-03': 1.29, '2020-01-04': 1.561, '2020-01-05': 1.9049}

2. Adjusted EMA

ema(x, alpha=0.1, adjust=True)  # adjusted, default method

{'2020-01-01': 1.0, '2020-01-02': 1.5263, '2020-01-03': 2.0701, '2020-01-04': 2.6313, '2020-01-05': 3.20971}

3. Finite horizon EMA

ema(x, alpha=0.1, adjust=True, horizon=2) # finite horizon

{'2020-01-01': 1.0, '2020-01-02': 1.5263, '2020-01-03': 2.5263, '2020-01-04': 3.5263, '2020-01-05': 4.5263}

4. Time-based decay EMA

ema(x, halflife=timedelta(days=1)) # time-based

{'2020-01-01': 1.0, '2020-01-02': 1.6666, '2020-01-03': 2.4286, '2020-01-04': 3.2666, '2020-01-05': 4.1613}

5. Unadjusted EMA for NumPy array

x_np = {'2020-01-01': [1,2], '2020-01-02': [4,5], '2020-01-03': [7,8]}
ema(x_np, alpha=0.1, adjust=False)

{'2020-01-01': [1,2], '2020-01-02': [1.3,2.3], '2020-01-03': [1.87,2.87] }

Exponential Moving Variance

ema_var(
    x: ts[Union[float, np.ndarray]],
    min_periods: int = 1,
    alpha: Optional[float] = None,
    span: Optional[float] = None,
    com: Optional[float] = None,
    halflife: Optional[Union[float, timedelta]] = None,
    adjust: bool = True,
    horizon: int = None,
    bias: bool = False,
    ignore_na: bool = False,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
min_periods: the minimum allowable number of ticks to use before outputting data. The default is 1 for any EMA function.
alpha, span, com, halflife: as described in EMA
adjust: as specified in EMA
horizon: as specified in EMA.
bias: if True, uses a biased population weighted variance. If false, normalized by a proper debiasing factor.
ignore_na: if True, nan values will be "ignored" meaning weights will be placed on relative position. If False (default), weights are based on global position.
trigger: another time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the EMA, and in doing so clears any accumulated floating-point error.
- Note: only valid when a finite-horizon EMA is used.
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of exponentially-weighted moving variances over the interval.

Examples: `ema_var`

See Exponential Moving Standard Deviation

Exponential Moving Standard Deviation

ema_std(
    x: ts[Union[float, np.ndarray]],
    min_periods: int = 1,
    alpha: Optional[float] = None,
    span: Optional[float] = None,
    com: Optional[float] = None,
    halflife: Optional[Union[float, timedelta]] = None,
    adjust: bool = True,
    horizon: int = None,
    bias: bool = False,
    ignore_na: bool = False,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0,
) → ts[Union[float, np.ndarray]]

Args:

x: the time-series data
min_periods: the minimum allowable number of ticks to use before outputting data. The default is 1 for any EMA function.
alpha, span, com, halflife: as described in EMA
adjust: as specified in EMA
horizon: as specified in EMA.
bias: if True, uses a biased population weighted variance. If false, normalized by debiasing factor
ignore_na: if True, nan values will be "ignored" meaning weights will be placed on relative position. If False (default), weights are based on global position.
trigger: another time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the EMA, and in doing so clears any accumulated floating-point error.
- Note: only valid when a finite-horizon EMA is used.
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of exponentially-weighted moving standard deviations over the interval.

Examples: Exp. Moving Variance and Standard Deviation

Starttime: 2020-01-01 00:00:00

1. Exp. Moving Standard Deviation

x = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3, '2020-01-04': nan, '2020-01-05': 5}
ema_std(x, min_periods=2, span=20, adjust=False, bias=False, ignore_na=False)

{'2020-01-02': 0.707, '2020-01-03': 1.11636, '2020-01-04': 1.11636, '2020-01-05': 1.937005}

2. Exp. Moving Variance

ema_var(x, min_periods=2, span=20, adjust=False, bias=True, ignore_na=False)

{'2020-01-02': 0.086168, '2020-01-03': 0.390588 '2020-01-04': 0.390588, '2020-01-05': 1.644124}

Exponential Moving Covariance

ema_cov(
    x: ts[Union[float, np.ndarray]],
    y: ts[Union[float, np.ndarray]],
    min_periods: int = 1,
    alpha: Optional[float] = None,
    span: Optional[float] = None,
    com: Optional[float] = None,
    halflife: Optional[Union[float, timedelta]] = None,
    adjust: bool = True,
    horizon: int = None,
    bias: bool = False,
    ignore_na: bool = False,
    trigger: ts[object] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
) → ts[Union[float, np.ndarray]]

Args:

x: time-series data. If x is of type np.ndarray, the exponential-moving covariance is calculated element-wise with the corresponding values in y.
y: time-series data which ticks in-sequence with x
min_periods: the minimum allowable number of ticks to use before outputting data. The default is 1 for any EMA function.
alpha, span, com, halflife: as described in EMA
adjust: as specified in EMA
horizon: as specified in EMA.
bias: if True, uses a biased population weighted covariance. If false, normalized by debiasing factor
ignore_na: if True, nan values will be "ignored" meaning weights will be placed on relative position. If False (default), weights are based on global position.
trigger: another time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the EMA, and in doing so clears any accumulated floating-point error.
- Note: only valid when a finite-horizon EMA is used.
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of exponentially-weighted moving covariance over the interval.

NumPy Specific Statistics

Covariance Matrix

cov_matrix(
    x: ts[np.ndarray],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ddof: int = 1,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    weights: ts[Union[float, np.ndarray]] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
) → ts[np.ndarray]

Args:

x: the time-series of dimension (N,) arrays which represent N variables
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
ddof: delta degrees of freedom
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned
- By default, the trigger is the series itself.
weights: a time-series of weights for each observation in x, used to calculate a weighted covariance matrix (optional). Weights do not need to be normalized.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the statistic, and in doing so clears any accumulated floating-point error.
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of (potentially weighted) covariance matrices, each of which is a NumpyNDArray of dimensionality (N,N)

Correlation Matrix

corr_matrix(
    x: ts[np.ndarray],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    ignore_na: bool = True,
    trigger: ts[object] = None,
    weights: ts[Union[float, np.ndarray]] = None,
    sampler: ts[object] = None,
    reset: ts[object] = None,
    recalc: ts[object] = None,
    min_data_points: int = 0
) → ts[np.ndarray]

Args:

x: the time-series of dimension (N,) arrays which represent N variables
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
ignore_na: if True, does not include any nan values in the window. If false, nan values in the window will make the entire window value nan.
trigger: another time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned
- By default, the trigger is the series itself.
weights: a time-series of weights for each observation in x, used to calculate a weighted correlation matrix (optional). Weights do not need to be normalized.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation
recalc: another optional time-series which triggers a clean recalculation of the statistic, and in doing so clears any accumulated floating-point error.
min_data_points: the minimum number of valid (non-nan) data points that must exist in the interval for a calculation to be valid. If there are fewer than min_data_point, NaN is returned.

Returns:

a time-series of (potentially weighted) correlation matrices, each of which is a NumpyNDArray of dimensionality (N,N)

Examples: Covariance and Correlation Matrices

Starttime: 2020-01-01 00:00:00

1. Covariance

x = {'2020-01-01': np.array([0., 0., 0.]), '2020-01-02': np.array([1., -1., 2.]), '2020-01-03': np.array([2., -2., 4.])}
cov_matrix(x, 3, ddof=0)

{'2020-01-03': np.array([1, -1, 2],
                     [-1, 1, -2],
                      [2, -2, 4])}

2. Correlation

corr_matrix(x, 3)

{'2020-01-03': np.array([1, -1, 1],
                     [-1, 1, -1],
                      [1, -1, 1])}

NumPy Conversions

list_to_numpy(x: [ts[float]], fillna: bool = False) → ts[np.ndarray]

Args:

x: a listbasket of time series
fillna: If False, unticked elements are treated as NaN. If True, unticked elements will hold their previous value in the array.

Returns:

a NumPy 1D array where each value corresponds to the element of the listbasket with the same index

numpy_to_list(x: ts[np.ndarray], n: int) → [ts[float]]

Args:

x: a NumPy array valued time series
n: the number of output channels in the listbasket Returns:
a listbasket where each value corresponds to the element of the array with the same index

Examples: NumPy Conversions

Starttime: 2020-01-01 00:00:00

1. List to NumPy

x1 = {'2020-01-01': 1, '2020-01-02': 2, '2020-01-03': 3}
x2 = {'2020-01-01': 1.5, '2020-01-03': 3.5}
list_to_numpy([x1,x2], fillna=False)

{'2020-01-01': [1, 1.5], '2020-01-02': [2, np.nan], '2020-01-03': [3, 3.5]} # no x2 tick on day 2

list_to_numpy([x1,x2], fillna=True)

{'2020-01-01': [1, 1.5], '2020-01-02': [2, 1.5], '2020-01-03': [3, 3.5]} # holds x2 value for day 2

2. NumPy to list

x_np = {'2020-01-01': [1,2], '2020-01-02': [3,4], '2020-01-03': [5,6]}
numpy_to_list(x_np, 2)

[
    {'2020-01-01': 1, '2020-01-02': 3, '2020-01-03': 5},
    {'2020-01-01': 2, '2020-01-02': 4, '2020-01-03': 6}
]

Cross-Sectional Statistics

Cross Sectional

cross_sectional(
    x: ts[Union[float,np.ndarray]],
    interval: Union[timedelta, int] = None,
    min_window: Union[timedelta, int] = None,
    trigger: ts[object] = None,
    as_numpy: bool = False,
    sampler: ts[object] = None,
    reset: ts[object] = None
) → ts[Union[np.ndarray, List[float], List[np.ndarray]]]

Args:

x: the time-series data
interval: the rolling interval over which to use data. If unspecified or set to None, an expanding (unbounded) window will be used.
- if an int, represents the number of ticks to use
- if a timedelta, represents the time interval to keep data (non-inclusive at left endpoint)
min_window: the minimum allowable interval to use before outputting data
- If the interval is a timedelta then this must also be a timedelta. Example: interval=60s, min_window=30s means to use a 60s rolling interval with no output for the first 30s.
- If the interval is a tick count then this must also be a tick count. Example: interval=100, min_window=50 means to use a 100-tick rolling interval with no output until we have 50 ticks
as_numpy: if True, the data will be returned as a NumPy array instead of a list.
- For a single-valued time series, this is a one-dimensional NumPy array
- For a NumPy array time series, this is a NumPy array of one extra dimension
trigger: another time-series which can be used to externally trigger computations. Whenever the trigger ticks, the given statistic will be updated and returned
- By default, the trigger is the series itself.
sampler: another optional time-series which specifies when x should tick. The behavior is as follows:
- If x ticks *and *sampler ticks, then the x tick is considered valid and is used.
- If x ticks but sampler does not tick, then the x tick is considered invalid and is ignored.
- If x does not tick but sampler ticks, then the x tick is considered NaN and is handled based on the ignore_na flag.
- By default, the sampler is the series itself.
reset: another optional time-series which, when ticked, will clear all data in the interval and "reset" the calculation

Returns:

a time-series where each tick contains all the data of x currently within the interval. Use this for custom cross-sectional calculations

Examples: Cross-sectional calculations

Starttime: 2020-01-01 00:00:00

x = {'2020-01-01': 1, '2020-01-01': 2, '2020-01-01': 3, '2020-01-01': 4, '2020-01-01': 5}
cs = cross_sectional(x, interval=3, min_window=2)
cs

{'2020-01-02': [1,2], '2020-01-03': [1,2,3], '2020-01-04': [2,3,4], '2020-01-05': [3,4,5]}

Calculate a cross-sectional mean

cs_mean = csp.apply(cs, lambda v: sum(v)/len(v), float)
cs_mean

{'2020-01-02': 1.5, '2020-01-03': 2.0, '2020-01-04': 3.0, '2020-01-05': 4.0}

Get the results as a NumPy array

cs = cross_sectional(x, interval=3, min_window=2, as_numpy=True)
cs

{'2020-01-02': np.array([1,2]), '2020-01-03': np.array([1,2,3]), '2020-01-04': np.array([2,3,4]), '2020-01-05': np.array([3,4,5])}

3. Statistics Nodes (csp.stats)

API Reference

Getting started

Introduction

Working with a single-valued time series

Working with a NumPy time series

Working with a basket of time series

Cross-sectional statistics

Expanding window statistics

Common user options

Intervals

Triggers, samplers and resets

Data validity

NaN handling

Weighted statistics

Numerical stability

The recalc parameter

Example use cases

Single valued time series

NumPy time series

Performance benchmarking

API reference

Base Statistics

Count

Examples: count

Unique

Examples: unique

Sum

Examples: sum

Product

Examples: prod

First

Examples: first

Last

Examples: first and last

Mean

Examples: mean

Geometric Mean

Examples: mean and gmean

Order Statistics

Maximum

Examples: max

Minimum

Examples: max and min

Median

Argmin

Examples: argmin

Argmax

Examples: argmax and argmin`

Rank

Examples: rank

Moment-Based Statistics

Variance

Examples: var

Standard Deviation

Examples: stddev

Standard Error

Examples: Variance, Standard Deviation, Standard Error

Covariance

Examples: cov

Correlation

Examples: Covariance and Correlation

Skewness

Examples: skew

Kurtosis

Examples: skew and kurt

Exponential Moving Statistics

Exponential Moving Average

Examples: ema

Exponential Moving Variance

Examples: ema_var

Exponential Moving Standard Deviation

Examples: Exp. Moving Variance and Standard Deviation

Exponential Moving Covariance

NumPy Specific Statistics

Covariance Matrix

Correlation Matrix

Examples: Covariance and Correlation Matrices

NumPy Conversions

Examples: NumPy Conversions

The `recalc` parameter

Examples: `count`

Examples: `unique`

Examples: `sum`

Examples: `prod`

Examples: `first`

Examples: `first` and `last`

Examples: `mean`

Examples: `mean` and `gmean`

Examples: `max`

Examples: `max` and `min`

Examples: `argmin`

Examples: `argmax and` argmin`

Examples: `rank`

Examples: `var`

Examples: `stddev`

Examples: `cov`

Examples: `skew`

Examples: `skew` and `kurt`

Examples: `ema`

Examples: `ema_var`