New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Very slow subsetting for character #544

Closed
krlmlr opened this Issue Nov 30, 2018 · 3 comments

Comments

Projects
None yet
2 participants
@krlmlr
Copy link
Member

krlmlr commented Nov 30, 2018

@krlmlr krlmlr added this to the 2.0.0 milestone Nov 30, 2018

@yutannihilation

This comment has been minimized.

Copy link
Member

yutannihilation commented Dec 4, 2018

Here's a quick profile of [.tbl_df on my laptop. It seems safe_match() in check_names_df.character() is the main cause of slow subsetting for character. So, I think you can fix this somehow.

But, as these checks seem basically necessary, I guess [.tbl_df will stay several times slower than [.data.frame for a while. Can I expect [.tbl_df will get as fast as [.data.frame in future? (or in version 2.0.0?)

Codes:

library(tibble)

d <- data.frame(a = 1, b = 2, c = 3)
t <- as_tibble(d)

profvis::profvis({for (i in 1:1e5) {t[, "c"]}})
profvis::profvis({for (i in 1:1e5) {t[, 3]}})

Results:
2018-12-04 12 16 28
2018-12-04 12 17 02

@krlmlr

This comment has been minimized.

Copy link
Member

krlmlr commented Dec 4, 2018

Thanks. That's the same profile I see. I think we might be able to make it ~8x faster if necessary, not sure if that's fast enough.

@yutannihilation

This comment has been minimized.

Copy link
Member

yutannihilation commented Dec 5, 2018

Yeah, the problem is we don't know how fast is fast enough... but ~8x faster sounds great.

@krlmlr krlmlr closed this in #551 Dec 19, 2018

krlmlr added a commit that referenced this issue Dec 19, 2018

Merge pull request #551 from tidyverse/f-544-fast-subset
- Faster subsetting with the `[` operator (#544).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment