[flink] Avoid redundant HTTP BLOB existence probes

[wwj6591812]

Purpose

#8219 introduced an HTTP existence preflight for blob-write-null-on-missing-file. Its final design performs the check in FlinkRowWrapper, because the writer-side fallback considered at that time could not safely turn an open failure into NULL after record bytes had been emitted.

#8412 later changed the BLOB writer to open the source before writing record bytes and added safe open-time classification for HTTP 404 and other fetch failures. The Flink preflight remained, so a materialized HTTP descriptor currently performs:

1. HEAD
2. if HEAD is not 200, a range GET
3. the full GET required to copy the payload into a managed blob file

Each operation has its own 429/503 retry loop. This causes redundant requests for successful resources and can amplify load while an HTTP origin is throttling.

Changes

• Compute which root BLOB fields are actually materialized into managed blob files.
• For HTTP(S) descriptors of those fields, defer existence and open-failure handling directly to the writer's required full GET.
• Preserve the preflight for inline blob-descriptor-field / blob-view-field values, because those fields have no later writer fetch.
• Preserve the preflight for non-HTTP descriptors.
• Match HTTP schemes case-insensitively, consistent with the URI reader and writer.

The writer remains the single final classifier for materialized HTTP descriptors:

• HTTP 404 becomes NULL only when blob-write-null-on-missing-file=true.
• Other open/fetch failures become NULL only when blob-write-null-on-fetch-failure=true.
• A body read failure after output has started still fails the write.
• HttpClientUtils.exists, retry status codes, retry counts, timeout defaults, and the HTTP 416 zero-length-resource contract are unchanged.

Request reduction

With both write-null options enabled:

Final result	Before	After
Successful HTTP resource	HEAD + full GET	full GET
GET-only HTTP resource	HEAD + range GET + full GET	full GET
HTTP 404	HEAD + range GET	full GET
Non-retryable HTTP error	up to 3 request chains	1 request chain
Persistent HTTP 429/503	up to 3 retry chains	1 retry chain

This reduces the persistent 429/503 upper bound from 18 attempts to 6 with the current default of five additional retries, without adding configuration or changing final row semantics.

Implementation

• FlinkSinkBuilder derives materialized BLOB positions from the table row type and excludes CoreOptions.blobInlineField().
• The positions are passed to FlinkRowWrapper.
• FlinkRowWrapper skips only the redundant HTTP(S) preflight for those positions.
• Existing overloads default to an empty materialized set, preserving their previous behavior.

Tests

Added unit and end-to-end coverage for:

• materialized HTTP 200: one full GET and payload round-trip;
• materialized HTTP 404: one full GET and NULL with the missing-file option;
• materialized HTTP 400: one full GET and NULL with missing-file plus fetch-failure options;
• HTTP 429 and 503 followed by 200: two full GET attempts, no HEAD or range request, and payload round-trip;
• inline descriptor HTTP 404: HEAD plus range GET is retained and NULL is written;
• uppercase HTTP scheme;
• a non-BLOB column before a BLOB column, to verify field-position handling.

Focused verification completed with 19 tests passing (13 wrapper tests and 6 HTTP BLOB integration cases), together with Checkstyle, Spotless, and git diff --check.

[wwj6591812]

@JingsongLi hi,please cc, thx

[JingsongLi]

+1