Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/cache flywire leaves #71

Merged
merged 13 commits into from
Jan 19, 2021
Merged

Feature/cache flywire leaves #71

merged 13 commits into from
Jan 19, 2021

Conversation

jefferis
Copy link
Collaborator

Basic implementation of #67

* so that we can be sure it doesn't change when we refactor
* includes brotli which is most performant but perhaps not completely standard as an option
* in fact the higher level function will choose brotli if available
* these could result in duplicates in the cache
* also fix bug in handling more than 1 rootid
* with tests
* and remove matrixStats dependency
* uses max.cols (which I had not spotted before) and pmax
* not a massive effect, but still 580->412 ms for my test query

> bench::mark(current=flywire_ntpred("720575940615237849"), min_time = 5)
# A tibble: 1 x 13
  expression    min  median `itr/sec` mem_alloc `gc/sec` n_itr  n_gc total_time result  memory  time  gc
  <bch:expr> <bch:> <bch:t>     <dbl> <bch:byt>    <dbl> <int> <dbl>   <bch:tm> <list>  <list>  <lis> <lis>
1 current     557ms   586ms      1.22    16.2MB    0.486     5     2      4.11s <ntpre… <Rprof… <bch… <tib…

> bench::mark(p3 <- flywire_ntpred("720575940615237849"))
# A tibble: 1 x 13
  expression                                   min median `itr/sec` mem_alloc `gc/sec` n_itr  n_gc
  <bch:expr>                                 <bch> <bch:>     <dbl> <bch:byt>    <dbl> <int> <dbl>
1 p3 <- flywire_ntpred("720575940615237849") 412ms  412ms      2.43    16.7MB     2.43     1     1
@jefferis jefferis merged commit ffdc0ec into master Jan 19, 2021
@jefferis jefferis deleted the feature/cache-flywire_leaves branch January 19, 2021 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant