Skip to content

feat(table): add Iceberg catalog and administration commands - #365

Merged
overtrue merged 2 commits into
rustfs:mainfrom
GatewayJ:feat/s3-tables-commands
Sep 7, 2026
Merged

feat(table): add Iceberg catalog and administration commands#365
overtrue merged 2 commits into
rustfs:mainfrom
GatewayJ:feat/s3-tables-commands

Conversation

@GatewayJ

@GatewayJ GatewayJ commented Sep 6, 2026

Copy link
Copy Markdown
Member

Summary

Add rc table and rc admin table so operators can use the RustFS Iceberg catalog without constructing signed REST requests manually. Cover warehouse enablement, namespaces, table create/register/list/load/rename/drop, metadata, snapshots, refs, views, commits, maintenance, recovery, external catalog bridges, and migration.

Use the existing alias credentials and TLS configuration with SigV4 signing. Namespace listing accepts either a warehouse or a parent namespace, preserving the encoded parent across every page. Lists paginate automatically with bounded responses; JSON output uses the additive table_catalog v3 family and removes credential bundles. Credential filtering is scoped to protocol credential bundles and configuration keys, preserving business properties, schema fields and ref names. Administrative mutations require explicit confirmation where applicable, and writes are never automatically retried.

Concurrency and compatibility

Standard commits require explicit Iceberg requirements. Pointer commits require the expected version and metadata location and reject ignored updates. Ref creation with an absent-ref condition uses a standard commit requirement; view replacement requires the current metadata location. Drop preserves data files. Unsupported server operations are not emulated.

The transport targets the RustFS catalog contract at 22bff27aee9cfcb8054c9c2ad5b8b91a7376430a. Existing command output contracts remain unchanged. Usage examples and advanced request formats are documented in docs/usage/table-catalog.md.

Validation

  • cargo fmt --all --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace: 2,569 passed, 0 failed, 2 ignored.
  • cargo test --workspace catalog and cargo test -p rc-core --test catalog: passed after the final fixes, including real CLI subprocess tests against a local HTTP test server.
  • cargo test -p rustfs-cli --features golden --test golden
  • git diff --check

HTTP tests cover signing, namespace encoding, parent-scoped pagination, conflicts, credential redaction with business-data preservation, output envelopes, and exit codes. No end-to-end validation against a running RustFS deployment has been performed.

Related issues

N/A. This adds CLI coverage for the existing server catalog APIs.

@GatewayJ

GatewayJ commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

@codex

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 6, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-06T15:01:31.925827Z cb4fd7b Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cb4fd7b72b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +756 to +758
let client = rc_core::AliasManager::new()
.and_then(|aliases| aliases.get(&prepared.request.target.alias))
.and_then(|alias| rc_s3::AdminClient::new(&alias));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Move catalog client construction out of the CLI layer

Constructing rc_s3::AdminClient here makes the new CLI command depend directly on the S3 layer, even though execute_with_api already accepts the core TableCatalogApi abstraction. Move concrete client construction to a composition layer and inject the trait implementation so this feature preserves the documented CLI-to-core dependency boundary.

AGENTS.md reference: AGENTS.md:L310-L314

Useful? React with 👍 / 👎.

Comment thread crates/cli/src/commands/table/mod.rs Outdated
Comment on lines +439 to +440
TableCommands::Namespace(command) => match command {
NamespaceCommands::List(a) => list(Op::NamespaceList, a, Kind::Warehouse),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Support selecting a parent when listing namespaces

When a warehouse contains nested namespaces such as sales.eu, this arm requires an alias/warehouse target and the request model never carries a parent namespace. The Iceberg list-namespaces operation lists children of a selected parent, so a root listing can return sales but there is no CLI invocation capable of discovering sales.eu; accept a namespace parent and send it as the endpoint's parent query parameter.

Useful? React with 👍 / 👎.

@GatewayJ
GatewayJ requested review from cxymds and overtrue September 6, 2026 15:06
@overtrue
overtrue merged commit 03a824d into rustfs:main Sep 7, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants