[Umbrella][FIP-49] Implement the Fluss Gateway REST API

[beryllw]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

FIP-49 has been accepted to add a stateless, multi-cluster REST Gateway for Apache Fluss. The Gateway exposes metadata discovery, DDL, writes, primary-key lookups, and prefix lookups to HTTP clients while preserving Fluss authentication and authorization semantics.

• FIP: https://cwiki.apache.org/confluence/spaces/FLUSS/pages/444334561/FIP-49+Fluss+Gateway+REST+API
• Discussion: https://lists.apache.org/thread/s5yjysvoj8o6nxdj49m99hg8yn51tgjw
• Vote: https://lists.apache.org/thread/t5jkcvo8b07oxjywbckotbsxl8fhcd76
• Vote result: https://lists.apache.org/thread/f4217jbyy1dv139ml9t5ybwkhsymy2xm

A proof of concept implements most of the proposal, but it is too large to review as one pull request. This umbrella tracks capability-sized issues and review units that can be tested and merged independently.

Sub-tasks

The work is split along three parallel lines so each unit can be reviewed and merged independently: capability units run against the GatewayBackend port with the testing backend and do not wait for native connectivity; native-connectivity units are config-gated and land in parallel; delivery items trail the rest.

Foundation

• [gateway] Introduce the fluss-gateway module #4002
• [FIP-49][gateway] Add the Gateway runtime, common HTTP layer, lifecycle, and test infrastructure #3958
• [FIP-49][gateway] Validate Gateway configuration, restrict Fluss client options, and redact secrets #3969

Capability line (testing backend, no native dependency)

• [FIP-49][gateway] Add the GatewayBackend port with the domain model (types, row encoding, input decoding) and the testing backend, plus the cluster discovery REST API
• [FIP-49][gateway] Add HTTP-layer REST authentication (only needs the runtime foundation; independent of the backend port and Fluss connectivity)
• [FIP-49][gateway] Add catalog REST APIs: read-only metadata (databases, tables, schemas) and DDL (create/drop databases and tables)
• [FIP-49][gateway] Add schema-aware batch writes with ordered outcomes and admission limits
• [FIP-49][gateway] Add batched primary-key and prefix lookups with admission limits

Native connectivity line (config-gated, parallel to the capability line)

• [FIP-49][gateway] Add the native Fluss backend: service-identity connectivity, resilience, and the metadata cache (the testing backend stays the default until wiring lands)
• [FIP-49][gateway] Propagate HTTP principals to Fluss with bounded act-as connection pools
• [FIP-49][gateway] Wire native writes and lookups and upgrade the dockerized end-to-end suite to full business flows

Delivery

• [FIP-49][gateway] Add the Gateway container image and the release pipeline (depends only on the runtime foundation)
• [FIP-49][gateway] Wrap up: complete the Prometheus metric families with a checklist regression (per-capability metrics land with their own PRs), add user documentation and the executed end-to-end quickstart

Willingness to contribute

• I'm willing to submit a PR!

[beryllw]

@leonardBang @loserwang1024 cc