[VL] Add config to velox's file read #3990
Conversation
|
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/oap-project/gluten/issues Then could you also rename commit message and pull request title in the following format? See also: |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Velox PR created. facebookincubator/velox#7978 |
|
With directinputbuffer, velox fetches data in iothreads (number is configed by spark.gluten.sql.columnar.backend.velox.IOThreads), the task's memory pool is passed from query context, which is used to allocate memory for fetched data. There is a risk of memory leak check may fails. The new PR #4005 disabled the executor crash on memory leak, it put a warning instead |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
details are shown here: facebookincubator/velox#8041 |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
solve conflict
|
Run Gluten Clickhouse CI |
|
Velox facebookincubator/velox#7978 merged, waiting for rebase |
cpp/velox/compute/VeloxBackend.cc
Outdated
| conf->get<std::string>(kFilePreloadThreshold, "1048576")); // 1M | ||
|
|
||
| // set cache_prefetch_min_pct = 0 to force all loads are prefetched in DirectBufferInput. | ||
| FLAGS_cache_prefetch_min_pct = 0; |
There was a problem hiding this comment.
Could you help add gluten conf for this property also instead of hard code 0 here?
FelixYBW
changed the title
|
Run Gluten Clickhouse CI |
FelixYBW
force-pushed
the
addfilescanconfig
branch
from
5680032
to
cac0ddc
Compare
|
Run Gluten Clickhouse CI |
|
===== Performance report for TPCH SF2000 with Velox backend, for reference only ====
|
Velox PR7217(facebookincubator/velox#7217) added directbufferinput, which leads to performance regression seriously. The root cause is that the default config in the PR is not optimal for remote storage. You may find more talk here: facebookincubator/velox#7873
The PR added 3 config:
loadQuantum: 256M (make sure it's larger than row group size, parquet default is 128M)
maxCoalesceDistance: 1M ( in case the columns are not load contieneously, like select a, c from table_with_column_a_b_c. If b is mall column than 1M, then we can load it to make a large block
CoalesceBytes: 64M, break the row group fetches into small chunks
With these configuration, here is the final traceview: You can see the S3 read is totally in parallel with data processing.