Skip to content

Update benchmarks to remove custom env for prefetcher - #996

Merged
ankitaluthra1 merged 2 commits into
fsspec:mainfrom
ankitaluthra1:microbenchmark-fix
Aug 7, 2026
Merged

Update benchmarks to remove custom env for prefetcher#996
ankitaluthra1 merged 2 commits into
fsspec:mainfrom
ankitaluthra1:microbenchmark-fix

Conversation

@ankitaluthra1

Copy link
Copy Markdown
Collaborator
  • Prefetcher is by default ON when cache_type is not SET, custom settings to use prefetcher with none is not needed anymore.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request removes experimental read configurations from the benchmark Cloud Build pipeline and removes the explicit cache_type="none" parameter from several read operations in the performance microbenchmarks. The reviewer feedback highlights that removing cache_type="none" from random, mixed, and reopen read operations will cause them to default to readahead caching. This default behavior will trigger inefficient background prefetching and unnecessary GCS requests, which will introduce significant overhead and skew the benchmark results. It is recommended to retain cache_type="none" for these non-sequential read patterns.

start_time = time.perf_counter()
files_it = itertools.cycle(file_paths)
open_kwargs = {"block_size": block_size, "cache_type": "none"}
open_kwargs = {"block_size": block_size}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Removing cache_type="none" from random reads (_read_op_rand) will cause them to use the default cache_type="readahead". For random read patterns, readahead caching is highly inefficient and will trigger unnecessary GCS requests, severely degrading benchmark performance and skewing the results. We should keep cache_type="none" for random reads.

Suggested change
open_kwargs = {"block_size": block_size}
open_kwargs = {"block_size": block_size, "cache_type": "none"}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect, the new default is none

start_time = time.perf_counter()
files_it = itertools.cycle(file_paths)
open_kwargs = {"block_size": block_size, "cache_type": "none"}
open_kwargs = {"block_size": block_size}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Removing cache_type="none" from reopen reads (_read_op_reopen) will cause them to use the default cache_type="readahead". For reopen benchmarks (which repeatedly open a file, read a single chunk, and close it), readahead caching will trigger background prefetching that is immediately wasted when the file is closed, adding significant overhead and skewing the connection overhead measurements. We should keep cache_type="none" for reopen reads.

Suggested change
open_kwargs = {"block_size": block_size}
open_kwargs = {"block_size": block_size, "cache_type": "none"}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect, new default is none

while time.perf_counter() - start_time < runtime:
path = next(files_it)
with gcs.open(path, "rb", cache_type="none") as f:
with gcs.open(path, "rb") as f:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Removing cache_type="none" from mixed reads (_read_op_mixed) will cause them to use the default cache_type="readahead". Since mixed reads include random seeks, readahead caching can introduce significant overhead and skew the benchmark results. It is better to keep cache_type="none" for mixed reads.

Suggested change
with gcs.open(path, "rb") as f:
with gcs.open(path, "rb", cache_type="none") as f:

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect, new default is none

@codecov

codecov Bot commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.03%. Comparing base (8880f63) to head (59f8524).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #996   +/-   ##
=======================================
  Coverage   90.03%   90.03%           
=======================================
  Files          16       16           
  Lines        3593     3593           
=======================================
  Hits         3235     3235           
  Misses        358      358           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ankitaluthra1
ankitaluthra1 marked this pull request as ready for review August 6, 2026 13:09
@ankitaluthra1
ankitaluthra1 merged commit afcaa35 into fsspec:main Aug 7, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants