new pat_timeAverageDiagnostics() function

[jonathancallahan]

I cloned the source code for openair and tool a look at the implementation of the timeAverage() function.

That function is waaaaay to tied to the openair data model. We can create our own function that does what we want, runs faster and has much more readable code. I've gotten a start in local_examples/PROTOTYPE_pat_timeAverage.R.

The feature set for this function is:

• accept a pat pat and return a tibble with new data columns on a new time axis
• accept a unit parameter specifying the new time axis period
• the returned tibble with have columns with mean, sd and count statistics for each of pm25_A, pm25_B, temperature and humidity. Columns will be named <parameter>_<statistic>
• the returned tibble will have additional t-test parameters: pm25_t, pm25_df and pm25_p
• be sure to convert any NaN, Inf or NULL values generated by mean or sd into NA
• if it takes less than 4 hours, add support for a data.thresh parameter

[hmrtn]

I think that any sort of statistical analysis should exist independently of the this function. Perhaps a pat_ttest() or in a pat_statistics()?

[jonathancallahan]

I disagree. This function is all about applying statistical functions to consecutive chunks of the overall time series and returning vector that represents the value of that statistic calculated for each chunk. Currently, the statical functions you have enabled are mean, sd and count, etc.

The idea for the t-test is to have another function that would behave similar to sd in that it takes the data within a chunk of time, applies an algorithm and returns a number. So you will need to implement another internal function that looks like this:

if ( stats == "ttest_qc" ) {
func <- function(x) {
# use t.test with pm25_A and pm25_B and appropriate parameters
# return the t test statistic or p value as the result
}

The use case is to generate clean, hourly pm25 data with the following steps:

• remove outliers from raw data
• pm25 <- calculate AB mean
• calculate t_statistic
• badQCIndices <- t_statistic > someThreshold
• pm25[baQCIndices] <- NA