Skip to content

fix(local-dev): refresh the Lakekeeper warehouse S3 endpoint when the host IP changes #7358

Description

@mengw15

Feature Summary

bin/local-dev.sh stores the host laptop's LAN IP as the MinIO (S3) endpoint inside the Lakekeeper warehouse — it is the one address reachable both from containers and from Texera services running natively on the host.

flowchart LR
  subgraph L["Laptop — host IP 192.168.1.5"]
    TX["Texera services<br/>(native on host)"]
    subgraph D["Docker"]
      LK["Lakekeeper"]
      MO["MinIO :9000"]
    end
  end
  TX -->|"create table (REST catalog)"| LK
  LK -->|"S3 write → 192.168.1.5:9000"| MO
Loading

The IP is written once, at warehouse registration. Move the laptop to another network and the stored endpoint is stale:

flowchart LR
  subgraph L2["Laptop — host IP is now 10.0.0.7"]
    TX2["Texera services"]
    subgraph D2["Docker"]
      LK2["Lakekeeper<br/>storage profile still says<br/>192.168.1.5:9000"]
      MO2["MinIO :9000"]
    end
  end
  TX2 -->|"create table"| LK2
  LK2 -.->|"S3 write to 192.168.1.5:9000<br/>times out"| OLD["old address —<br/>unreachable"]
  LK2 -->|"RESTException: Unable to process"| TX2
Loading

Result: every workflow execution fails at startup (RESTException: Unable to process: with an empty body, thrown from createTable). Reads of existing tables still work — that metadata is served from Lakekeeper's own Postgres, never touching MinIO — which makes the failure look unrelated to networking.

Reproduce: bin/local-dev.sh up on network A → move the machine to network B → run any workflow.

Proposed Solution or Design

On every up, compare the host's current IP with the endpoint stored in the warehouse's storage profile (GET /management/v1/warehouse/{id}). If they differ, refresh it with one management-API call (POST /management/v1/warehouse/{id}/storage, reusing the existing profile with the new endpoint and the MinIO credentials compose already provisions).

Small change — about a dozen lines in bin/local-dev/main.sh; the only user-visible difference is one log line. Today the recovery is a manual curl against the management API, or recreating the warehouse.

Affected Area

Deployment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions