Increase reading speed of for read_secuTrial()

[PatrickRWright]

Is your feature request related to a problem? Please describe.
Very big exports (i.e. tens of thousands of entries) need long to read. Maybe its possible to boost the performance.

[aghaynes]

I tried a little profiling... It seems to be converting the dates that's primarily causing the lag (at least in the dataset I've looked at)

[aghaynes]

Specifically, it's a merge in .convert_dates

[PatrickRWright]

What's the tool you are using there? Looks super useful.

[aghaynes]

profvis - it integrates with RStudio's IDE
see here for tutorial on RStudio's support forum

[aghaynes]

devtools::load_all()
profvis::profvis(read_secuTrial(path_to_export))

I find it easier to look at the Data tab

[PatrickRWright]

[PatrickRWright]

We could benchmark the tidyverse reading functions to see if its worthwhile switching.

[PatrickRWright]

In the spirit of structured procrastination I prepared a small benchmark:
https://gist.github.com/PatrickRWright/4ed5d4e5b5aed03b7a1aa5b593dd9b64

readr is faster but its not exactly light speed either.

[aghaynes]

There is also data.table::fread and vroom::vroom which would be worth looking at. vroom is apparently the fastest, although I've never used it... (screenshot from the vroom readme)

[PatrickRWright]

If it were just the numbers I agree. Lets have a look at the dependency consequences it has. The speedup looks pretty impressive though.