Add per-file I/O speed reporting#271
Merged
Merged
Conversation
AltraMayor
added
Enhancement
Feature request
AltraMayor
force-pushed
the
per-file-stats
branch
2 times, most recently
from
49c3df4 to
a8a6b29
Compare
AltraMayor
changed the title
AltraMayor
force-pushed
the
per-file-stats
branch
6 times, most recently
from
ebda501 to
c6dc3d2
Compare
Add macro-level timing using clock_gettime() around the file processing loop in f3write.c. This accurately measures the wall-clock time spent on each file, independent of libflow's throttling and chunking logic, and prints the resulting average speed at the end of each file.
Modified libflow:measure() to return the blocks and time of a completed measurement window. Updated f3write.c to capture these measurements and track the minimum and maximum speeds observed during the processing of each file. These metrics are now printed alongside the average speed.
Introduce parameter measurement_boundary to end_measurement(), pass true in f3write.c and f3read.c since they make measurements on files (i.e., a file is a measurement boundary), and pass false for everyone else. Reaching a boundary forces any leftover processed_blocks and acc_delay_ns to be committed to the global statistics. This prevents measurement data from bleeding across boundaries.
AltraMayor
force-pushed
the
per-file-stats
branch
from
c6dc3d2 to
f13c9b2
Compare
This was referenced Apr 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces per-file I/O speed reporting to
f3writeandf3readwithout disruptinglibflow's throttling logic.Problem:
Previously,
f3writeandf3readreported a cumulative average speed at the start of each file. Becauselibflowgroups blocks dynamically, measurements often crossed file boundaries — especially on drives exceeding 1GB/s where a 1GB file finishes in a fraction of a second. This architecture meant that per-file speed and internal performance metrics, such as min and max speeds, were not available.Solution:
This pull request introduces three complementary changes to separate statistics reporting from flow control:
f3writeandf3readwithclock_gettime()to accurately measure the wall-clock time spent on each file independently oflibflow, providing a per-file average speed.inst_speed) fromlibflow'smeasure()function, allowingf3writeandf3readto track the minimum and maximum speeds observed during the processing of each file.bool measurement_boundarytoend_measurement()inlibflowto commit any leftover block progress and time measurements at the end of a file without invoking the flow state machine, ensuring measurement data does not bleed across file boundaries.This pull request closes #134 and closes #251.