[feature](file cache) Add page prefetch and I/O coalescing for cold queries

[bobhan1]

What problem does this PR solve?

Issue Number: None

Related PR: #65658

Problem Summary:

Cold cloud scans currently discover compressed data-page misses when each FileColumnIterator reaches the page that it needs. Because columns and pages are consumed in order, adjacent remote reads cannot be prepared across columns before consumption, remote latency is exposed on the query thread, and nearby page reads are not coalesced even when one bounded range could satisfy them.

The page-read unit and the file-cache persistence unit are intentionally different. Segment data pages are variable-sized and may be unaligned, while cache persistence accepts complete fixed-size cache blocks. Moving variable-range planning, hole completion, or I/O coalescing into the fixed-block write service would mix query read semantics with cache persistence and would require the service to accept a new task shape.

This PR adds a separate, opt-in query page-prefetch pipeline. It prepares the pages needed by all participating columns, coalesces them under explicit amplification limits, reads exact ranges asynchronously, lets PageIO decode directly from prefetched slices, and optionally completes dense cache blocks for background writeback. The existing fixed-block write service continues to receive one complete fixed-capacity cache-block task at a time.

The feature is disabled by default. Admission pressure, allocation failure, pool rejection, remote-read failure, and prefetched-page decode failure fall back to the existing page-read path. Query cancellation and genuine page metadata or checksum corruption retain their existing error semantics.

Goals

• Overlap remote data-page I/O across pages and across the physical columns that a SegmentIterator will actually consume.
• Coalesce nearby page reads while bounding range size, page count, gap size, and read amplification.
• Preserve exact sparse-rowid reads without adding speculative lookahead pages.
• Reuse downloaded file-cache blocks and completed inflight buffers without creating cache cells, waiting for downloading blocks, or scheduling writes from the prefetch read itself.
• Decode prefetched page slices through the same footer, checksum, decompression, pre-decode, and page-cache insertion logic as ordinary page reads.
• Complete and persist only useful full cache blocks without adding variable-size tasks or range-merging logic to the fixed-block write service.
• Bound speculative memory and active work independently at query and BE scope, with non-blocking admission and complete rollback on rejection.
• Keep fallback behavior local so speculative prefetch does not change SQL results.

Non-goals

• Changing BlockFileCache layout, block size, LRU policy, cache admission semantics, or the fixed-block async write queue contract.
• Adding cross-query singleflight for page-prefetch ranges. Existing downloaded and inflight cache-block reuse still applies.
• Fetching sparse holes after a page has already been consumed.
• Prefetching segment indexes, warm-up or dry-run reads, compaction reads, external Parquet or ORC pages, peer-cache workflows, or non-query readers.
• Supporting Variant physical-column propagation in this change.
• Implementing adaptive window sizing. The current selector uses a fixed window.
• Claiming that every page decode is zero-copy. Compressed pages can consume the prefetched compressed slice directly, while uncompressed pages still take ownership through the existing PageIO representation.

Performance validation

This is still a Draft PR and does not claim a measured performance improvement yet. No cloud regression or end-to-end benchmark has been run for the current branch.

The planned validation will compare the feature disabled and enabled on the same cold cloud dataset and query set after clearing relevant caches. It will record end-to-end query latency, remote request count, remote bytes, requested page bytes, fetched bytes, coalesced gap bytes, completed-block fill bytes, prefetch wait time, fallback rate, and cache writeback behavior. The implementation already enforces the configured read-amplification bound; the benchmark is intended to determine whether the reduction in serialized remote requests outweighs the additional bounded bytes for representative continuous and sparse scans.

Architecture

SegmentIterator performs a prepare pass for every physical column that will be read before it starts the consume pass. A per-column PagePrefetcher owns page state and range references. Pure planning remains separate from asynchronous execution, page decoding, and cache persistence.

flowchart LR
    SI[SegmentIterator two-pass planning]
    CI[Column iterators and nested rowid mapping]
    PF[PagePrefetcher]
    PC[Storage page-cache precheck]
    RP[PageReadPlanner]
    WC[FileCacheWritebackCoordinator]
    IOS[PagePrefetchIOService]
    CR[CachedRemoteFileReader exact NO_WRITE unaligned read]
    LOCAL[Downloaded cache blocks and inflight buffers]
    REMOTE[Remote object storage]
    RANGE[PrefetchRange and variable-size buffer]
    PIO[PageIO slice decode]
    CONSUME[FileColumnIterator consume]
    USEFUL[Consumed-page writeback claim]
    COPY[Background complete-block copy]
    SUBMIT[FixedBlockAsyncWriteSubmitter]
    QUEUE[Existing fixed-block async write queue]
    CACHE[BlockFileCache]

    SI --> CI
    CI --> PF
    PF --> PC
    PC -->|uncached candidates| RP
    RP --> WC
    WC --> IOS
    IOS --> CR
    CR --> LOCAL
    CR --> REMOTE
    CR --> RANGE
    RANGE --> PIO
    PIO --> CONSUME
    CONSUME --> USEFUL
    USEFUL --> COPY
    COPY --> SUBMIT
    SUBMIT --> QUEUE
    QUEUE --> CACHE

Loading

Component responsibilities

Component	Responsibility	Important guarantee
SegmentIterator	Builds the actual column set for normal, predicate, selected-row, and lazy-pruned phases, then calls prepare_page_prefetch for all of them before consuming any column.	Remote I/O for multiple participating columns can overlap without changing column-consumption order.
Complex column iterators	Propagate logical row requirements to physical child iterators. Struct children reuse parent rowids; array and map iterators read outer offsets first and translate selected rows into item-space rowids.	All key/value or item ranges are prepared before their physical children are consumed, while inactive branches are skipped.
PagePrefetcher	Owns the fixed window, sparse-page selection, page-cache precheck, page state, range submission, wait-at-consumption, statistics, cancellation, and fallback decisions for one physical FileColumnIterator.	A page is decoded from at most one tracked prefetched range, and rejected or unusable ranges fall back locally.
PageReadPlanner	Validates immutable page metadata and creates file-offset-ordered ranges under the configured gap, size, page-count, and amplification bounds.	Planning is pure and performs no I/O or cache mutation.
FileCacheWritebackCoordinator	Computes true EOF-aware cache-block coverage, adds bounded holes only for sufficiently dense blocks, invalidates blocks associated with a bad page, and schedules writeback only after useful consumption.	Failure to complete a block removes only the optional block completion; required page reads remain in the plan.
PagePrefetchIOService	Applies query and global admission, allocates range buffers, submits reads and writeback copies to the shared segment-prefetch pool, publishes terminal range state, propagates cancellation, supports live budget updates, and drains its own tasks during shutdown.	Admission never waits. Every unsuccessful stage releases all acquired range, byte, memory, and outstanding-task ownership.
CachedRemoteFileReader	Serves exact NO_WRITE + UNALIGNED ranges from downloaded cache blocks, completed inflight buffers, and remote spans while preserving REMOTE_ONLY_ON_MISS.	The prefetch read does not create cache cells, take downloader ownership, wait for DOWNLOADING, append or finalize cache files, or submit a cache write.
PageIO	Validates and decodes an already-owned encoded-page slice through the common page implementation.	Footer parsing, checksum validation, decompression, pre-decoding, page-cache insertion, and corruption behavior stay aligned with the ordinary read path.
FixedBlockAsyncWriteSubmitter	Converts one already-complete cache-block payload into the existing fixed-capacity task after a final cache probe, epoch check, inflight ownership decision, allocation, and rollback-safe queue submission.	The write service contract remains fixed-size and unchanged; the physical EOF block uses a short valid prefix in a fixed-capacity task.

Buffer and budget separation

Buffer	Shape and lifetime	Budget	Consumer
PagePrefetchBuffer	Variable-sized coalesced range. It remains resident while a range or page slice still references it.	Per-query and per-BE resident-byte limits. The active-range slot is released when I/O reaches a terminal state, while the byte reservation remains until the buffer is destroyed.	PageIO and the background complete-block copy.
Fixed async write buffer	Exactly one cache-block capacity, with a possibly short valid prefix only at physical EOF.	Existing per-cache-disk pending-byte admission owned by the fixed-block write service.	Cache persistence worker.

The background complete-block copy reserves another cache-block worth of query and BE resident bytes before it runs. This prevents a query thread from copying a large block and accounts for the short interval where both the variable range and the fixed task buffer may coexist.

Important flows

1. Prepare all columns before consuming any column

For continuous scans, the selector covers every page required by the current ordinal range and then looks ahead in scan direction until the fixed target window is reached. It refills when the number of unconsumed planned pages is at or below half of the target. Reverse scans select candidates in reverse consumption order but sort the final I/O candidates by file offset before planning.

For sparse rowids, the selector maps only the supplied rowids to data pages, deduplicates page indexes, excludes already tracked pages, and does not add lookahead. This keeps point and highly selective reads from expanding into a continuous window.

Before range planning, each candidate is checked for minimum encoded size, file bounds, monotonically increasing file offsets and page indexes, non-overlapping ordinal ranges, and valid ordinal coverage. A storage page-cache hit is recorded as skipped and does not consume prefetch I/O admission.

sequenceDiagram
    participant S as SegmentIterator
    participant C as Column iterators
    participant P as PagePrefetcher
    participant I as PagePrefetchIOService
    participant R as Remote reader
    participant D as PageIO

    S->>S: determine actual columns and read phase
    loop every participating physical column
        S->>C: prepare_page_prefetch row requirements
        C->>C: map nested rows to child rowids when needed
        C->>P: prepare candidate pages
        P->>I: non-blocking range submissions
        I->>R: exact asynchronous range read
    end
    loop existing column-consumption order
        S->>C: read rows
        C->>P: acquire only the page now being consumed
        P-->>C: ready slice or fallback decision
        C->>D: decode prefetched slice
        D-->>C: parsed page
    end

Loading

2. Bounded range coalescing and block completion

The base planner merges file-ordered page candidates only when all four conditions remain true:

• The inter-page gap is no larger than query_page_prefetch_max_gap_bytes.
• The resulting range is no larger than query_page_prefetch_max_range_bytes.
• The resulting range contains no more than query_page_prefetch_max_pages_per_range pages.
• fetched_bytes / requested_page_bytes does not exceed query_page_prefetch_max_read_amplification_ratio.

The writeback coordinator starts from that valid page-only plan. It computes cache-block coverage using the real file size, so the final physical block uses its true valid prefix. A block is considered only when covered page bytes meet query_page_prefetch_writeback_min_block_coverage. The coordinator tentatively adds the missing bytes needed to complete that block and keeps the completion only if every range and amplification limit still holds. Otherwise it discards that optional completion and preserves the page-only plan.

Each final range tracks requested page bytes, coalesced gap bytes, optional block-fill bytes, page slice offsets, and complete-block slice offsets. These categories are disjoint and sum to the fetched range size.

3. Exact mixed-source range read

The I/O worker forces CacheWriteMode::NO_WRITE and CacheAlignMode::UNALIGNED for the requested range. CachedRemoteFileReader first reuses complete inflight buffers, then probes existing downloaded cache blocks, and finally issues remote reads for uncovered exact intersections. Adjacent remote intersections are joined, but the request is never expanded merely to align with a cache block.

A DOWNLOADING cache block is treated as remotely readable for this path rather than making the prefetch worker wait. A missing local cache file triggers the existing asynchronous self-heal removal and the affected bytes are read remotely. REMOTE_ONLY_ON_MISS keeps its existing behavior and does not become eligible for block completion or writeback.

This path returns one fully initialized range buffer and source byte statistics, but it never mutates file-cache ownership. Cache population is a separate decision after successful page consumption.

4. Range state, wait, decode, and fallback

PrefetchRange has the following monotonic lifecycle:

CREATED -> QUEUED -> RUNNING -> READY
        -> REJECTED          -> FAILED
                             -> CANCELLED

The range slot is released as soon as the range becomes terminal. The resident-byte reservation remains attached to the reference-counted buffer, allowing PageIO or a background writeback copy to safely outlive the worker task.

The query thread does not wait during planning or admission. It waits only when it reaches a page still covered by an accepted range. At consumption it checks the storage page cache again, waits for the covering range if necessary, obtains the exact page slice, and invokes PageIO::decode_page_from_slice. A ready range avoids another file read.

If the range was rejected, failed, cancelled for a non-query reason, or unavailable, the iterator reads the page through the existing path. If prefetched bytes fail checksum, footer, decompression, or pre-decode validation, associated complete blocks are invalidated and the iterator retries through the existing read path. Only failure of that authoritative path is returned to the query.

5. Useful-only fixed-block writeback

A complete block is not submitted merely because its bytes were fetched. It becomes claimable only when at least one associated source page is successfully decoded and consumed. A decode failure invalidates every complete block associated with that page. Each block can be claimed once.

The writeback copy runs on the shared segment-prefetch pool, not on the query thread. Before and after obtaining the block slice it rechecks service acceptance, both feature gates, query cancellation, REMOTE_ONLY_ON_MISS, block eligibility, and the cache write epoch. It then calls FixedBlockAsyncWriteSubmitter, which performs the final cache-state probe, allocates a fixed-capacity buffer, copies the complete payload, acquires inflight ownership, submits one fixed-block task, and rolls back on every skip or rejection.

A writeback allocation failure, stale epoch, existing cache owner, backpressure rejection, or persistence failure cannot retroactively fail the query. The already-consumed page remains valid and a later read may populate the cache again.

6. Admission, cancellation, and shutdown

Range admission reserves in this order: query range and byte budget, BE range and byte budget, range-buffer allocation, outstanding-task ownership, and shared-pool submission. Writeback-copy admission reserves bytes but not another range slot. All failures unwind acquired ownership without waiting for capacity.

The safe worker I/O context owns copied query ID and cache-admission values and clears query-thread-only pointers. Query contexts keep weak ownership of the runtime query and cancel ranges when the query is cancelled or destroyed.

The service reuses segment_prefetch_thread_pool and does not own or shut down that pool. During ExecEnv shutdown it stops accepting work, waits for registered submitters, cancels live query contexts, waits for only its own outstanding tasks, and is destroyed before the shared pool and file-cache objects that those tasks reference.

Configuration

Configuration	Default	Purpose
enable_query_page_prefetch	false	Master gate for the query page-prefetch path.
enable_async_file_cache_write	false	Existing prerequisite because complete-block persistence uses the fixed-block async write path. If either gate is off, iterators use the legacy page-read path.
query_page_prefetch_window_pages	16	Fixed target page window for continuous forward and reverse scans.
query_page_prefetch_min_window_pages	1	Validated lower bound carried by prefetch options for future adaptive sizing.
query_page_prefetch_max_window_pages	64	Validated upper bound carried by prefetch options for future adaptive sizing.
query_page_prefetch_max_gap_bytes	65536	Maximum gap allowed when coalescing adjacent page components.
query_page_prefetch_max_range_bytes	4194304	Maximum bytes in one submitted range. It must be at least one file-cache block.
query_page_prefetch_max_pages_per_range	32	Maximum requested pages represented by one range.
query_page_prefetch_max_read_amplification_ratio	2.0	Maximum ratio of fetched range bytes to requested page bytes.
query_page_prefetch_max_inflight_ranges_per_query	16	Maximum active prefetch ranges for one query.
query_page_prefetch_max_inflight_ranges	64	Maximum active prefetch ranges for one BE.
query_page_prefetch_max_inflight_bytes_per_query	67108864 (64 MiB)	Maximum resident prefetch and writeback-copy bytes for one query.
query_page_prefetch_max_inflight_bytes_per_be	536870912 (512 MiB)	Maximum resident prefetch and writeback-copy bytes for one BE.
query_page_prefetch_writeback_min_block_coverage	0.5	Minimum requested-page coverage before a cache block is considered for bounded completion.
enable_query_page_prefetch_adaptive_window	false	Reserved switch for future adaptive sizing. This change implements fixed-window selection only.

Validators enforce positive sizes and counts, min_window <= window <= max_window, max_gap < max_range, per-query limits no larger than BE limits, amplification at least 1.0, and writeback coverage in (0, 1]. The four query and BE inflight range and byte limits update the service, global budget, and existing query contexts online. Planning options are snapshotted when a physical column creates its PagePrefetcher.

Observability

PagePrefetcher maintains internal accounting for candidate, submitted, consumed, page-cache-skipped, ready-hit, fallback, throttled, and cancelled work, together with wait time, remote I/O time, requested and fetched bytes, coalesced gaps, block-fill bytes, local or inflight bytes, remote bytes, and writeback-eligible blocks. The focused unit tests assert these counters and budget transitions.

Remote and local bytes plus remote I/O time continue to flow through the existing CachedRemoteFileReader and file-cache read statistics. This Draft does not yet add new user-facing RuntimeProfile counters or page-prefetch-specific bvars, so the PR does not claim production profile visibility that is not present in the current implementation.

Compatibility and current scope

• The path is eligible only in cloud mode for READER_QUERY data-page reads with a live query context, positive tablet ID, CachedRemoteFileReader, file cache, accepting prefetch service, and fixed-block async write service.
• Storage page-cache hits remain first-class and skip speculative I/O. Index and metadata page behavior is unchanged.
• Continuous forward and reverse reads, exact sparse rowids, normal reads, predicate reads, selected-row reads, and lazy-pruned reads are wired through the two-pass planner.
• Scalar and nullable physical columns plus struct, array, and map propagation are implemented. Variant is intentionally left on the existing path.
• REMOTE_ONLY_ON_MISS, non-query readers, pruned columns, disabled features, unavailable services, and ineligible readers retain the existing behavior.
• The current implementation has no cross-query page-range singleflight, no after-consumption sparse-hole fetch, and no adaptive window adjustment.

Tests

• Focused ASAN BE unit tests: 264 passed, 3 existing disabled tests skipped, 0 failed across 267 discovered tests in 13 suites.
  • Exact unaligned cached and remote reads, downloaded and inflight reuse, DOWNLOADING behavior, self-heal, REMOTE_ONLY_ON_MISS, and fixed-block submission rollback.
  • Slice decode checksum, footer, compression, pre-decode, page-cache behavior, and corruption fallback.
  • Candidate validation, forward and reverse fixed windows, exact sparse rowids, coalescing limits, amplification limits, EOF handling, block coverage, and optional completion rollback.
  • Query and BE range and byte admission, live option updates, allocation and pool rejection, cancellation races, state transitions, safe I/O context lifetime, writeback-copy gates, and shutdown draining.
  • FileColumnIterator, scalar and complex-column propagation, two-pass SegmentIterator planning, selected rows, predicate phases, lazy-pruned phases, and fallback.
• Full BE build completed with the current Release build configuration: ./build.sh --be -j100.
• Style check passed: build-support/check-format.sh.
• Patch whitespace check passed: git diff --check b586dd1f429..HEAD.
• clang-tidy was intentionally not run for this change.

Pending validation before leaving Draft

• Cloud regression with both gates disabled and enabled, including runtime switching and cache reuse on a second query.
• End-to-end cold-query performance comparison for continuous, reverse, sparse, selected-row, and complex-column scans.
• Production-facing profile or bvar integration if the final performance-validation workflow requires page-prefetch-specific visibility.

Release note

Add an opt-in BE query page-prefetch and remote I/O coalescing path for cold cloud scans. The feature prepares data pages across participating columns, reads bounded exact ranges asynchronously, decodes pages from prefetched slices, and can complete useful fixed cache blocks for background persistence. It is controlled by enable_query_page_prefetch, requires enable_async_file_cache_write, and remains disabled by default.

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

• Behavior changed:

  • No.
  • Yes. When both feature gates are explicitly enabled for eligible cloud query reads, data pages can be prefetched and coalesced before consumption and useful complete cache blocks can be persisted in the background. Default behavior is unchanged.

• Does this need documentation?

  • No. The feature is experimental, disabled by default, and the implementation contract is documented in this Draft PR.
  • Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?