Performance fixes for wide data frames

[romainfrancois]

This was going through the SymbolVector::push_back method.

void push_back(const SymbolString& s) {
    v.push_back(s.get_string());
  }

and v is a CharacterVector 😱

This is still expensive e.g. compared to [, I'll look for other bottlenecks.

library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
library(purrr)

n_samples <- 500
n_features <- 100000

df <- bind_cols(
  tibble(sample_id = paste0("sample_", 1:n_samples)),
  1:n_features %>%
    map(rnorm, n = n_samples) %>%
    map(as_tibble) %>%
    bind_cols()
)

system.time(fetched_sample <- filter(df, sample_id == "sample_1"))
#>    user  system elapsed 
#>   1.720   0.029   1.755
system.time(fetched_sample <- df[which(df$sample_id == "sample_1"),])
#>    user  system elapsed 
#>   0.048   0.001   0.048

For reference in the original issue, these were the timings:

> system.time(fetched_sample <- filter(df, sample_id == "sample_1"))
#>.   user  system elapsed
#> 165.060   0.110 165.446

[krlmlr]

Is there a corresponding issue? Please merge with a corresponding NEWS bullet point, I've created the r-0.7.5 branch just in case.

[romainfrancois]

I can merge but there might be some more work in there. The issue is #3335

[krlmlr]

Do you mean you can make the code even faster? How? Do you want to leave the PR open, or merge, leave the issue open then, and revisit later?

[romainfrancois]

I’m pretty sure this can be faster. Might br a good example to benchmark the SlicingIndex bounds checks for example.

But i guess it does not hurt to merge and leave the issue open

[krlmlr]

Can you please add a bullet point to NEWS.md?

[lock]

This old issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with reprex) and link to this issue. https://reprex.tidyverse.org/