Per-user warehouses are feature-complete behind a flag; proposing to turn it on

[mengw15]

Per-user warehouses (#6870) now work end to end behind storage.warehouse.enabled, which has been off since the first commit. Each user owns their own warehouses — as many as they need — instead of everything sharing one. Everything has landed in main: the storage foundation and REST-catalog work, the warehouse REST endpoints and Lakekeeper client, the dashboard tab (#8005), the on-canvas picker (#8551), and the backend requirement that every execution name a warehouse (#8586).

I would like to propose turning the flag on, and to hear objections before anyone does.

Demo

Screen.Recording.2026-09-21.at.2.28.21.PM.mov

1. The Warehouses dashboard tab: creating a warehouse, the row it produces, and the details dialog its name opens.
2. The warehouse picker on the canvas, next to the computing-unit selector, choosing where the next run writes.
3. With no warehouse left, the Run button becomes "Create Warehouse" and leads straight to the create dialog; once a warehouse exists, Run comes back.
4. A run started from the form view with no warehouse selected is refused with a message, rather than quietly writing to shared storage.
5. A normal execution, writing into the selected warehouse.

What changes for users when it is on

• A Warehouses tab appears in the dashboard: create, inspect and delete your own warehouses.
• A warehouse picker appears on the canvas, choosing where the next run writes.
• Every execution must name a warehouse. Until you have one, Run becomes "Create Warehouse"; runs started elsewhere (form view, run-up-to-operator, replay, and agent-driven runs) are refused with a message rather than silently writing to shared storage.
• Nothing is provisioned automatically — each user creates their own.

What it requires from a deployment

The switch only takes effect with the REST catalog: StorageConfig keeps the feature off unless the Iceberg catalog type is rest (Lakekeeper), because warehouses are Lakekeeper entities. Deployments on another catalog type are unaffected by the flag entirely.

What it does not change

• Results written before the flip stay readable. Their storage URIs carry no warehouse segment and continue to resolve to the shared default warehouse.
• The switch is reversible, and fails loudly rather than silently. Turned back off, a result that lives in a per-user warehouse is refused with an explicit message ("this result is stored in warehouse X; per-user warehouses are disabled in this deployment") instead of surfacing as "table not found", which would be indistinguishable from data loss. Nothing is deleted; re-enabling restores access.
• Where the data lives. Each warehouse is its own Lakekeeper catalog with its own key prefix inside the deployment's existing storage. Isolation is at the catalog level, not at the bucket level.

The ask

Speak up if enabling this would break something on a deployment you care about — particularly anything affecting existing users, or if the "every execution needs a warehouse" requirement looks too strict as a Phase 0 default. Absent objections, I will turn it on.

[kunwp1]

I really like how the demo explicitly prompts users to create a warehouse if one doesn't exist yet. But I want to make sure this feature is thoroughly tested. Are we confident enabling it by default won't introduce any unexpected side effects?

[mengw15]

Two things.

The prompt-to-create flow mirrors the computing-unit path deliberately — the same shape as "Connect" when no unit exists. Building it surfaced a real bug on the computing-unit side, where that button could not actually be clicked; #8590 fixed it, so both behave the same way now.

On risk. The resolver is unit-tested in both flag states, the ordering rule is pinned, and the tab, picker and agent chain have suites at both ends; I also ran the whole feature by hand against a local deployment with the flag on, live Lakekeeper and MinIO — that is the recording. The PR that flips the flag runs the whole suite with the feature on, so anything the tests can catch shows up before it merges.

My preference is to turn it on in main and raise issues for whatever surfaces. The feature is additive and reversible — nothing is migrated, results written before the flip stay readable, and turning it back off refuses explicitly rather than silently returning "table not found". main is where this should surface; releases are cut from release branches, so they stay stable regardless.