-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Description
**Is your feature request related to a problem or challenge?
Relate #2965
https://github.com/apache/arrow-datafusion/blob/193fc3b4549ed40a08218b744388f9211605ab53/datafusion/core/src/physical_plan/file_format/file_stream.rs#L218-L228
Before i add the cost time metric here. But there are two sub future or stream under self.poll_inner(cx)
one is a future to read parquet metadata
https://github.com/apache/arrow-datafusion/blob/193fc3b4549ed40a08218b744388f9211605ab53/datafusion/core/src/physical_plan/file_format/file_stream.rs#L162-L165
another is: for read and decode data
https://github.com/apache/arrow-datafusion/blob/193fc3b4549ed40a08218b744388f9211605ab53/datafusion/core/src/physical_plan/file_format/file_stream.rs#L177-L181
If these future are not ready will return Poll::Pending, which will make the task blocked and need someone wake itself.
So we miss the read_meta time and reading time.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.