Skip to content

Support independent chunk and block sizes for both Read and Write FixedDuration microbenchmarks#779

Merged
ankitaluthra1 merged 2 commits intofsspec:mainfrom
ankitaluthra1:read-benchmark
Mar 31, 2026
Merged

Support independent chunk and block sizes for both Read and Write FixedDuration microbenchmarks#779
ankitaluthra1 merged 2 commits intofsspec:mainfrom
ankitaluthra1:read-benchmark

Conversation

@yuxin00j
Copy link
Copy Markdown
Collaborator

@yuxin00j yuxin00j commented Mar 16, 2026

  • Support chunk_sizes_mb and block_sizes_mb independently in ReadFixedDurationConfigurator.
  • Rename block_sizes_mb to chunk_sizes_mb in read_fixed_duration/configs.yaml.
  • Support block_sizes_mb in WriteFixedDurationConfigurator and WriteFixedDurationBenchmarkParameters.
  • Pass block_size to gcs_kwargs in test_write_multi_process to enable testing various block sizes for writes.
  • Update test_configs.py to match the refined logic for both read and write configurators.

Benchmark run result

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 16, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.18%. Comparing base (d1bb2cc) to head (b194de5).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #779      +/-   ##
==========================================
+ Coverage   75.84%   76.18%   +0.34%     
==========================================
  Files          14       14              
  Lines        2645     2645              
==========================================
+ Hits         2006     2015       +9     
+ Misses        639      630       -9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@yuxin00j yuxin00j marked this pull request as ready for review March 16, 2026 04:17
@ankitaluthra1
Copy link
Copy Markdown
Collaborator

/gcbrun

Comment thread gcsfs/tests/perf/microbenchmarks/read_fixed_duration/configs.py
…map parameters.

- Rename block_sizes_mb to chunk_sizes_mb in configs.yaml.
- Update configs.py to support chunk_sizes_mb and block_sizes_mb independently.
- Update test_configs.py to match refined logic.
@yuxin00j yuxin00j changed the title Refine ReadFixedDurationConfigurator to read from YAML and correctly map parameters. Support independent chunk and block sizes for both Read and Write FixedDuration microbenchmarks Mar 26, 2026
@yuxin00j yuxin00j requested a review from jasha26 March 26, 2026 06:58
@ankitaluthra1
Copy link
Copy Markdown
Collaborator

/gcbrun

@ankitaluthra1 ankitaluthra1 merged commit 1ec7a98 into fsspec:main Mar 31, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants