## Summary

A table that sets `required-capability.<name>=true` admits only clients
that declare `<name>` in the `X-OpenHouse-Client-Capabilities` request
header. The tables service checks before loading table metadata and
before committing metadata or snapshots. A client that lacks a required
capability gets a 422. Only the Spark catalog declares
`column-defaults`, so `required-capability.column-defaults=true`, #797's
opt-in, admits only Spark. The Java catalog loads column defaults into
the schema but declares nothing, because engines built on it, such as
Trino, return NULL instead of the defaults. #797's override header,
`X-OpenHouse-Dangerously-Bypass-Column-Default-Policy: true`, skips the
`column-defaults` check and no other capability. A catalog sends it
through #833's `header.` catalog properties.

On #681, below #794, which counts a snapshot commit's client as
default-aware when it declares `column-defaults`, so #794 can merge only
after this PR.

Replaces #844, which replaced #774. Moving the gate below #794 put every
#844 commit into #794's branch, so GitHub marked #844 as merged into
that branch; nothing reached `main`, and #844 had no submitted reviews.
#774 went the same way when the gate moved below #841, and it superseded
#772, reopened from a `linkedin/openhouse` branch because stacks cannot
include fork branches; neither had reviews or comments.

## Before merging

Merging changes nothing for tables that require no capability. A table
with `required-capability.column-defaults=true` rejects every client
that does not declare `column-defaults`, and no released client declares
it until a Spark runtime with this PR ships. On 2026-10-05 no opted-in
table was known.

So the rollout check moves from merging to opting a table in: before
setting the property on a table, every client that loads or writes it
must use OpenHouse's Spark catalog with this PR, or set the override.
After deploy, the tables service logs each gated request (`Admitting` or
`Rejecting <table> requiring [...]` with the request's `X-Client-Name`,
User-Agent and declared capabilities), which shows who would be
rejected.

## Changes

- [x] Client-facing API Changes
- [x] New Features
- [x] Documentation
- [x] Tests

- `ClientCapabilityGate` with `checkLoad` (table load) and `checkWrite`
(table create/update and snapshots PUT). A write is checked against both
the stored and the proposed properties, so removing the requirement in a
write cannot bypass the gate. Table search is not gated, including
`fields=tableLocation`.
- `X-OpenHouse-Client-Capabilities` is a comma-separated,
case-insensitive list; repeated headers add up.
`CatalogRequestContext.clientCapabilities()` reads it.
- No server configuration: the requirement is a table property.
`cluster.read-bridge.column-default.minimum-client-version` is removed.
- A rejection throws `ClientCapabilityMissingException`, a
`CodedApiException` that maps to HTTP 422:

  ```
CLIENT_CAPABILITY_MISSING: db.tbl requires the column-defaults
capability, which this client does not declare (X-Client-Name: trino,
User-Agent: openhouse-java-client/4.2.100,
X-OpenHouse-Client-Capabilities: (none)). Use a client that declares
column-defaults in X-OpenHouse-Client-Capabilities.
  ```
- Java client: `OpenHouseCatalog.capabilities()` returns the
capabilities a catalog declares, none by default. The Spark
`OpenHouseCatalog` (Spark 3.1 and 3.5) returns `column-defaults`. A
configured `header.X-OpenHouse-Client-Capabilities` adds capabilities
and cannot remove the catalog's.
- The mock tables application and the lock-policy service tests provide
a gate; no table there requires a capability.
- `SETUP.md` documents the header, the property and the override's
scope.

## Testing Done

- [x] Added new tests for the changes made.
- [x] Updated existing tests to reflect the changes made.

Switched from a client-version gate to declared capabilities on
2026-10-22 (`d0904e48`). With JDK 17, `:services:tables:test` (951
tests, including `ClientCapabilityGateTest` (5) and
`ReadBridgeColumnDefaultE2ETest` (11)) and `:services:common:test` (33)
pass, `SmokeTest` (33) passes, the Spark 3.1 and 3.5 itest suites pass,
including `ColumnDefaultSparkITest`, and `spotlessCheck` is clean.

Updated on 2026-10-26 (`ca2d0fbd0`): only the Spark catalog declares
`column-defaults`; the Java catalog declares nothing, so Trino and other
engines built on it are refused. With JDK 17, `SmokeTest` passes on
Iceberg 1.2 and 1.5, `ColumnDefaultSparkITest` passes on Spark 3.1 and
3.5 (5 each), and `spotlessApply` made no changes beyond this commit.

<details>
<summary>Earlier updates, when this PR gated on the client
version</summary>

Restacked on #790's typed failures and service-layer conversion
(`40a08022`), with JDK 17: `ColumnDefaultSparkITest` with the gate
configured passes on Spark 3.1 and Spark 3.5 (3 tests each),
`:services:tables:test` (925 tests) and `:services:common:test` (33
tests) pass, and `spotlessCheck` is clean. The read-bridge end-to-end
test's stored-default and declared-default PUTs send the gate's
Spark-client headers.

Restacked on #831's header override (`a4ec7494`): the same suites pass
with the same counts. In a Spark 3.1 session against the local test
server, a catalog without the header gets the 422 on load and #797's 400
on opt-out, both naming
`header.X-OpenHouse-Dangerously-Bypass-Column-Default-Policy=true`; a
catalog with that property loads, inserts into and opts out of the
table.

Restacked on #831's split into #833 (client `header.` properties) and
#831 (server handling): `SmokeTest` (26 + 26 tests, Iceberg 1.2 and 1.5)
and `ColumnDefaultSparkITest` (3 + 3, Spark 3.1 and 3.5) rerun and pass,
and `spotlessCheck` is clean. The server code did not change, and Gradle
reports `:services:tables:test`, `:services:common:test` and
`:iceberg:openhouse:internalcatalog:test` up to date with their earlier
passing runs.

Rebased on `main` after #790 merged, with #831's services reading the
override header: `:services:tables:test` (941 tests) passes,
`ColumnDefaultSparkITest` passes on Spark 3.1 and Spark 3.5 (5 tests
each; the Parquet backfill case is skipped), and `spotlessCheck` is
clean. Gradle restored `:services:common:test` (33),
`:iceberg:openhouse:internalcatalog:test` (312) and `SmokeTest` (26 +
26) from its build cache because their inputs did not change.

Restacked on #837 and #831's `CatalogRequestContext` design: the gate
reads the override header name from `TablesMvcConstants` (`cd3c0ee5`).
With JDK 17, `:services:tables:test` (942 tests) passes,
`ColumnDefaultSparkITest` passes on Spark 3.1 and Spark 3.5 (5 tests
each; the Parquet backfill case is skipped), and `spotlessCheck` is
clean.

The client identity moved from a dedicated catalog in
`ColumnDefaultSparkITest` into `TestSparkSessionUtil`, so this PR no
longer changes the itest (`11bd8914`). With JDK 17, the Spark 3.1
`catalogTest` suite and `ColumnDefaultSparkITest` on Spark 3.5 pass (5
run + 1 skipped each for `ColumnDefaultSparkITest`).

The gate throws `ColumnDefaultClientRejectedException` instead of the
generic `UnprocessableEntityException` (`a836d79f`); the status and
error body are unchanged. `ReadBridgeColumnDefaultE2ETest` (14 tests)
passes.

Search and no-op table updates are no longer gated, and an unset minimum
gets its own 422 message (`1ba660a5`). With JDK 17,
`:services:tables:test` (942 tests) passes; the e2e test now asserts
that a `fields=tableLocation` search succeeds for an incompatible
client.

Restacked on #797's property-only switch and toggle-override removal,
and on #841, which moves the column-default checks into `doCommit`
(`ed0073f1`). The read-bridge e2e cases follow #797's rewrites and
removals, the lock-service test fixtures mock the one-argument
`prepare`, and `ApiConfig` imports `CatalogRequestContext` itself now
that #841's strip-protection bean no longer does. With JDK 17,
`:services:tables:test` (921 tests) passes, `ColumnDefaultSparkITest`
passes on Spark 3.1 and Spark 3.5 (5 run, 1 skipped each), and
`spotlessCheck` is clean.

Restacked on the commit policy that #831 now adds (`5941d7a8`). #831 now
defines `CatalogRequestContext` in
`com.linkedin.openhouse.tables.config` instead of the internal catalog,
so the gate, `ApiConfig`, the mock tables application and the
lock-service tests import it from there, and `ApiConfig` defines the
gate next to the `ColumnDefaultCommitPolicy` bean. With JDK 17,
`:services:tables:test` (942 tests) passes, `ColumnDefaultSparkITest`
passes on Spark 3.1 and Spark 3.5 (5 run, 1 skipped each), and
`spotlessCheck` is clean.

Restacked on 2026-10-16 after review of #833 and #831 (`dced459c`), with
no change here: #831 now holds the opt-in lock and a typed commit-policy
contract, and #833 ignores, with a warning, `header.` properties it
cannot send instead of failing initialization. With JDK 17,
`:services:tables:test` (942 tests) and `SmokeTest` (27 + 27, Iceberg
1.2 and 1.5) pass, `ColumnDefaultSparkITest` passes on Spark 3.1 and
Spark 3.5 (5 run, 1 skipped each), and `spotlessCheck` is clean.

Restacked again on 2026-10-16 (`f8a4980c`), with no change here: #831
was split to the catalog hook alone, and the opt-in lock,
`CatalogRequestContext` and the override header moved up to #797. With
JDK 17, `:services:tables:test` (942 tests) and `SmokeTest` (28 + 28,
Iceberg 1.2 and 1.5) pass, `ColumnDefaultSparkITest` passes on Spark 3.1
and Spark 3.5 (5 run, 1 skipped each), and `spotlessCheck` is clean.

Restacked again on 2026-10-16 (`93f1065a`): #797 injects
`CatalogRequestContext` directly and no longer has
`CatalogRequestContext.NONE`, so `ApiConfig` injects the context into
`ColumnDefaultClientGate` directly, the mock tables application injects
the request context it scans, and the lock-service tests build a no-op
context inline. With JDK 17, `:services:tables:test` (947 tests) and
`SmokeTest` (28 + 28, Iceberg 1.2 and 1.5) pass,
`ColumnDefaultSparkITest` passes on Spark 3.1 and Spark 3.5 (5 run, 1
skipped each), and `spotlessCheck` is clean.

Moved below #841 on 2026-10-16 (`1d65be41`), now on #794 (`5fb39f1f`).
The only changes are the signatures from below: the lock-service tests
mock #794's two-argument `prepare`, and the gate still takes
`CatalogRequestContext`. With JDK 17, `:services:tables:test` (946
tests) and `SmokeTest` (28 + 28, Iceberg 1.2 and 1.5) pass,
`ColumnDefaultSparkITest` passes on Spark 3.1 and Spark 3.5 (5 run, 1
skipped each), and `spotlessCheck` is clean.

Restacked on 2026-10-16 onto #794 (`51a6569f`), no content changes.
Updated after review (`461dd074`): the gate reads the client name and
User-Agent through `CatalogRequestContext` instead of casting, the
admission check is `admitsClient()`, and SETUP.md keeps #797's opt-in
paragraph. The gate e2e (11) and strip protection (22) tests pass.

Updated after review on 2026-10-16 (`7ada6ed2`): the gate reads
`X-Client-Name` and `User-Agent` through `CatalogRequestContext`, like
the bypass, instead of casting `RequestContextHolder`'s attributes, and
`COLUMN_DEFAULT_POLICY_BYPASS_HINT` is a format string. With JDK 17,
`:services:tables:test` (945 tests) passes and `spotlessCheck` is clean.

Moved below #794 on 2026-10-17 (`dd2302b2`), now on #681. #794's bypass
hint constant moves here with the first commit that uses it, and two
test imports move with the commits that use them; every other commit is
unchanged. With JDK 17, the read-bridge, column-default and lock-service
tests pass (gate e2e 10, strip protection 22, commit policy 18), Spark
3.1's `catalogTest` (51) and `statementTest` (60) pass, and
`spotlessCheck` is clean.

Merged #681 with #797's typed opt-in rejection on 2026-10-21
(`7c49f8ec`), no change here. With JDK 17, the full `:services:tables`
(951 tests), `:iceberg:openhouse:internalcatalog` (298) and
`:services:common` (33) suites pass at the top of the stack.

Updated after review on 2026-10-21 (`10b19acf`, merged in as
`6f2e1dcc`): the gate compares the release parts `RELEASE` matches
instead of using Spring Data's `Version`, so a release ranks below its
hotfixes (`4.1.324` < `4.1.324.0`); `check` is split into `checkLoad`
and `checkWrite`; the 422 names `X-Client-Name` and `User-Agent`; and
the e2e test admits `0.5.101`, `0.10.0` and `0.5.100.1`. Updates that
change nothing are gated too: `updateNeeded` is true for every committed
table, so SETUP.md no longer says otherwise. With JDK 17,
`:services:tables:test` (947 tests) passes on `10b19acf`, and after the
merge `ColumnDefaultClientGateTest`, `ReadBridgeColumnDefaultE2ETest`
(11), `ColumnDefaultCommitPolicyTest` (18) and the lock-service tests
pass and `spotlessCheck` is clean.

#Updated on 2026-10-21: an unset minimum is a server error (500
`TableConfigUnavailableException`) instead of a 422 that blames the
client, and an invalid minimum fails startup (`28e3f760`). Merged #831's
pluggable pre-commit checks (`e7566269`); the only conflict was
documentation in `CatalogRequestContext`. Verified at the top of the
stack (#794, below).

Updated on 2026-10-21 (`d4424a4d`): the property is renamed
`required-capability.column-defaults` (#797). The 422 follows the
template above: it lists the supported client instead of spelling out
the override. `checkWrite` takes the database and table ids, since a
create has no stored table, and `COLUMN_DEFAULT_POLICY_BYPASS_HINT` is
removed. `ColumnDefaultClientGateTest` (5),
`ReadBridgeColumnDefaultE2ETest` (11, asserting the full 422 message),
`LockEvaluationServiceTest` (25), `SystemOnlyLockPolicyServiceTest` (18)
and `RepositoryTest` (66) pass.

</details>

# Additional Information

- Declared capabilities are self-reported: this is compatibility
admission, not attestation or a security boundary.
- OpenHouse's maintenance jobs call the tables API directly. On an
opted-in table they are rejected unless their client declares
`column-defaults` or sets the override.
- JARs without #833's `header.` catalog properties cannot send the
override. Cached metadata and direct storage access are outside this
gate.

### Stack

Stack #849, bottom to top:

1. [#790](https://github.com/linkedin/openhouse/pull/790): fail loudly
when column defaults cannot be applied (merged)
2. [#833](https://github.com/linkedin/openhouse/pull/833): send
`header.` catalog properties as request headers (merged)
3. [#831](https://github.com/linkedin/openhouse/pull/831): add pluggable
pre-commit checks to `doCommit` (merged)
4. [#837](https://github.com/linkedin/openhouse/pull/837): throw a
catalog exception from `doCommit`'s table property rules (merged)
5. [#797](https://github.com/linkedin/openhouse/pull/797): lock the
column-default opt-in behind the override header, and make it the only
switch
6. [#681](https://github.com/linkedin/openhouse/pull/681): Spark catalog
itest (overlay)
7. **This PR**: gate tables on capabilities the client declares
8. [#794](https://github.com/linkedin/openhouse/pull/794): check column
defaults in `doCommit`, and require a default-aware client for every
snapshot commit (merges last)

<!-- kkf9m -->