Intermittent flow - proper support

[nicholascarey]

Ok, time to start thinking about supporting this properly. Big intermittent flow experiments, and looping inspect, calc_rate, etc. through a big dataset at regular (or irregular) intervals.

Maybe not for 2.0, but let's aim for 2.1.

How i see it working:

Specialised loop fns for calc_rate, auto_rate etc. with starts and ends and by inputs. These will be vectors of start and end locations of replicates in by = "time" or "row". For very regular experiments, no real need for these, just need a interval or something input to indicate the regularity or length of each rep. Although it would need a start point...?

Also add buffer inputs to exclude period of time at starts/ends of each rep to exclude

User can create the rep locations themselves using usual R syntax, or we build in some sort of identification of reps. My peaks function would be ideal for this. Make it into a rep_id function, output location vectors go into the loop functions.

Have some error catching so that an error doesn't stop the whole loop. This is a pain when running auto_rate across huge numbers of reps, and i have had quite a few people getting in touch with this specific problem.

Output will be some sort of summary list. Probably no need to save every object, just extract the rate(s), locations, coefficients etc. Basically each loop is a row of the summary table.

Although, what if you want to plot a particular one? Maybe make saving every object an option, or just have a simple output.

Add in subset_rate for subsetting this object, or some sort of other summary function.

[multimeric]

I wonder if you could also segment the data using just a threshold. The data I'm working with has the probe taken out at various times, at which point the concentration measurement drops noticeably. If the user could just say "when the concentration drops below 10 umol/L, the probe isn't in anymore", it would be quite intuitive to work with. Of course this would only work for a certain subset of experiments, but it could be an option.

[nicholascarey]

Thanks for the suggestion. Ok, that is one option to keep in mind. Have only the starting ideas of how it will work, but basically there will be lots of ways the user can specify the structure, as well as automatic detection.

[nicholascarey]

Another older suggestion. Here so we don't forget

#90

[multimeric]

Also I had a look into some regression changepoint tools like segmented, strucchange, and EnvCpt. Basically they treat segmentations of the regression model as another parameter and evaluate if the improvement in fit is "worth" the extra parameters, using measures like AIC/BIC. I had a bit of trouble getting good results from them, but that's another potential angle.

[multimeric]

For, example I had this data:

Then using strucchange I basically just ran breaks = breakpoints(sensor ~ time + 1, data=df, h=0.05) and I got these breakpoints:
.

Do you think this is an integration you could see working with your package?

[nicholascarey]

Thanks for the suggestion - i'll definitely check out that package.

Honestly, to build in support for data like this - unlikely. The vast majority of intermittent-flow respirometry data is very regular, both in period and oxygen range. We are concentrating on ease of use for typical use cases. Anything as idiosyncratic and unique as this is best left to individuals to wrangle themselves.

I am envisioning an input of replicate locations as row numbers and/or time values, so one you have them, however you choose to get them, you should be able to enter them and the function will loop through the data. For actual automatic or assisted location detection though, we will be concentrating on regular structures.

It's early days though, so we'll see.

[nicholascarey]

Added in v2.1

[multimeric]

Nice! How did you implement it?

[nicholascarey]

Quite simple when you look at it but very complicated to implement. Sorry, no automatic rep detection. Maybe in the future.

https://januarharianto.github.io/respR/articles/calc_rate.int.html
https://januarharianto.github.io/respR/articles/auto_rate.int.html
https://januarharianto.github.io/respR/articles/intermittent_long.html