Skip to content

RepartitionExec is missing BaselineMetrics #18218

@2010YOUY01

Description

@2010YOUY01

Is your feature request related to a problem or challenge?

BaselineMetrics is a set of common metrics for almost all operators, now RepartitionExec is not tracking it.

The BaselineMetrics output_rows and elapsed_compute are not showed in RepartitionExec in the below datafusion-cli reproducer:

> explain analyze
select v1, count(*)
from generate_series(1,100) as t1(v1)
group by v1;
+-------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type         | plan                                                                                                                                                                                                                                     |
+-------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Plan with Metrics | ProjectionExec: expr=[v1@0 as v1, count(Int64(1))@1 as count(*)], metrics=[output_rows=100, elapsed_compute=9.332µs]                                                                                                                     |
|                   |   AggregateExec: mode=FinalPartitioned, gby=[v1@0 as v1], aggr=[count(Int64(1))], metrics=[output_rows=100, elapsed_compute=4.043955ms, spill_count=0, spilled_bytes=0.0 B, spilled_rows=0, peak_mem_used=67328]                         |
|                   |     CoalesceBatchesExec: target_batch_size=8192, metrics=[output_rows=100, elapsed_compute=60.834µs]                                                                                                                                     |
|                   |       RepartitionExec: partitioning=Hash([v1@0], 14), input_partitions=14, metrics=[fetch_time=7.515374ms, repartition_time=45.428µs, send_time=14.348µs]                                                                                |
|                   |         AggregateExec: mode=Partial, gby=[v1@0 as v1], aggr=[count(Int64(1))], metrics=[output_rows=100, elapsed_compute=5.612461ms, spill_count=0, spilled_bytes=0.0 B, spilled_rows=0, skipped_aggregation_rows=0, peak_mem_used=7328] |
|                   |           RepartitionExec: partitioning=RoundRobinBatch(14), input_partitions=1, metrics=[fetch_time=17.459µs, repartition_time=1ns, send_time=11.471µs]                                                                                 |
|                   |             ProjectionExec: expr=[value@0 as v1], metrics=[output_rows=100, elapsed_compute=1.667µs]                                                                                                                                     |
|                   |               LazyMemoryExec: partitions=1, batch_generators=[generate_series: start=1, end=100, batch_size=8192], metrics=[output_rows=100, elapsed_compute=10.458µs]                                                                   |
|                   |                                                                                                                                                                                                                                          |
+-------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.011 seconds.

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions