-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TPC-H benchmarks #33
TPC-H benchmarks #33
Conversation
R/bm-tpc-h.R
Outdated
# all queries take an input_func which is a function that will return a dplyr tbl | ||
# referencing the table needed. | ||
#' @export | ||
tpc_h_queries <- list( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could also do this as:
tpc_h_queries <- list()
tpc_h_queries[[1]] <- function(input_func) ...
tpc_h_queries[[6]] <- function(input_func) ...
Could be more intuitive than stringified number names
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, yes of course. This is much better
R/bm-tpc-h.R
Outdated
if (format == "parquet") { | ||
input_functions[["arrow"]] <- function(name) { | ||
file <- tpch_files[[name]] | ||
return(arrow::read_parquet(file, as_data_frame = FALSE)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not open_dataset()? I wonder if that has different/better properties, even if it is a single file. (Hey we should benchmark that!) It at least shouldn't be memory constrained.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll add that as an option and see if there are better properties. You're right we probably should always do open_dataset()
here even if these are typically one-file datasets.
} else if (format == "native") { | ||
# native is different: read the feather file in first, and pass the table | ||
tab <- list() | ||
for (name in names(tpch_files)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So we have to read in all tpch_files now? Do these queries share the same dataset, or do we get 22 datasets for 22 queries?
Since each process is only running a single query (right?) it seems like you should be able to only read in the ones that the current query requires
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is what this is doing, yeah. Each query uses the same dataset (well dataset of multiple tables). This query only needs one table (lineitem), so we really only need to read that in.
I can make a query to table map so that we only read in the tables that are necessary for each query.
Ultimately, I don't think this will have all that much of an impact, lineitem is the largest table by far, so reading the others in shouldn't have a huge impact even if they aren't used.
Additionally, right now what happens is that each of the files are memory mapped by this, but aren't actually paged into memory until the first run (this is why for arrow the first run in this condition is so much longer than the others). I'm planning to turn memory mapping off so that we really are starting with a truly in-memory table (though we could parameterize that if we wanted to!)
Well, it turns out I'm also pretty surprised that Any thoughts about an explanation for these oddities @bkietz? With datasets, our performance against parquet files is in line with DuckDB's, though our native query processing is considerably longer. Turning off memory mapping has the impacts I expected:
We have a few questions that we need to anser:
|
@nealrichardson I'm planning to merge this today, unless you have any other things you would like to change or more time to look at it. |
Possible scope creeps (that probably aren't worth doing right now, if we're going to separate out and have a separate library for managing sources)