[[ subsetting much slower than $

[ebein]

Starting with tibble 3.0.0, column subsetting using [[ is much slower than $. This causes slowdowns in functions that call [[ many times, for example data.matrix on a wide tibble.

df <- tibble::tibble(x = 1)

bench::mark(
  dollar = df$x,
  bracket = df[["x"]],
  iterations = 1000
)
#> # A tibble: 2 x 6
#>   expression      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 dollar        6.8us    8.1us   100956.    7.96KB      0  
#> 2 bracket     190.3us  211.2us     3998.  165.09KB     12.0

^{Created on 2020-06-03 by the reprex package (v0.3.0)}

[hadley]

The call to vectbl_as_col_location2() is responsible for the speed difference, probably due to its use of tryCatch(), which is slow.

[md0u80c9]

Out of interest, do you know if $ was significantly quicker prior to tibble 3 - or was performance more equal?

[ebein]

From tibble 2.1.1 on a different machine. So it seems like $ was ~2x faster on 2.1.1 and is 25-30x faster on 3.0.1.

df <- tibble::tibble(x = 1)

bench::mark(
  dollar = df$x,
  bracket = df[["x"]],
  iterations = 1000
)
#> # A tibble: 2 x 6
#>   expression      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 dollar     682.65ns   1.36us   514262.    6.28KB        0
#> 2 bracket      1.02us   1.36us   472464.     5.3KB        0

^{Created on 2020-06-10 by the reprex package (v0.2.1)}

[md0u80c9]

Thanks @ebein (all my machines were on tibble 3 and the hassle of doing a full reinstall to check it meant I was cheeky and just asked the question!): very interesting that from a practical perspective it may be better to train my muscle memory to use $ where possible (obviously [[ has benefits where the column name isn't a constant!)

[krlmlr]

Once we remove the vectbl_as_col_location2() call and the associated overhead, run time drops to 9 µs. Still way too much, compared to 130 ns for base lists.

[krlmlr]

Pure S3 dispatch without doing actual work is already 1.3 µs. Oh well...

[krlmlr]

Now:

df <- tibble::tibble(x = 1)

bench::mark(
  dollar = df$x,
  bracket = df[["x"]],
  iterations = 1000
)
#> # A tibble: 2 x 6
#>   expression      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 dollar       4.29µs   4.67µs   199184.    17.6KB        0
#> 2 bracket      3.91µs   4.25µs   219202.    90.6KB        0

^{Created on 2020-06-14 by the reprex package (v0.3.0)}

[krlmlr]

We can strive for even faster processing (closer to 2 µs), I suspect this needs a full rewrite in C. This should be fast enough for most use cases.

[github-actions]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue and link to this old issue if necessary.