Skip to content

[python] Add branch CRUD REST implementation to RESTCatalog#7747

Merged
JingsongLi merged 1 commit intoapache:masterfrom
TheR1sing3un:py-branch-rest-gaps
May 1, 2026
Merged

[python] Add branch CRUD REST implementation to RESTCatalog#7747
JingsongLi merged 1 commit intoapache:masterfrom
TheR1sing3un:py-branch-rest-gaps

Conversation

@TheR1sing3un
Copy link
Copy Markdown
Member

Purpose

pypaimon's Catalog ABC has stubs for create_branch / drop_branch / fast_forward / list_branches raising NotImplementedError, and is missing rename_branch entirely. The RESTCatalog never overrode any of them, so calling any branch operation on a real REST catalog raises NotImplementedError instead of issuing the REST call. Java defines all five (paimon-core/.../catalog/Catalog.java:843-912) with a complete REST implementation in paimon-core/.../rest/RESTCatalog.java:703-769. This PR closes that gap.

This is the sister PR of #7746 — Tag CRUD; same shape (abstract stub + RESTCatalog overrides + wire DTOs + URL builders + mock REST server route handlers + tests). FilesystemCatalog inherits the abstract NotImplementedError stubs; a Python-side BranchManager port is tracked separately.

Linked issue

n/a — closing an API gap surfaced while auditing pypaimon's Catalog against the Java contract.

Tests

  • pypaimon/tests/api/test_branch_dto_serde.py — 11 unit tests pinning the wire format to Java: exact JSON field names (branch / fromTag / toBranch / branches), URL templates (databases/{db}/tables/{tbl}/branches[/{branch}[/rename|/forward]]), and an explicit assertion that ListBranchesResponse is not paged (no nextPageToken).
  • pypaimon/tests/rest/rest_branch_test.py — 11 end-to-end tests against the in-process mock REST server mirroring Java RESTCatalogTest::testBranches (paimon-core/.../rest/RESTCatalogTest.java:2138-2191): create / list happy path, table-not-exist on create / list, duplicate create raises BranchAlreadyExistException, rename happy / collide / source-missing, drop happy / missing, fast-forward happy / missing. Plus 1 FilesystemCatalog NotImplementedError sanity test for the new rename_branch abstract stub.
  • Regression: pypaimon/tests/rest/rest_simple_test.py rest_function_test.py rest_catalog_test.py (52 tests) all pass unchanged.
  • All new/edited files pass flake8 --config=dev/cfg.ini.

API and format

Adds 1 new public method on Catalog: rename_branch(identifier, from_branch, to_branch). Defaults to NotImplementedError — non-breaking; existing 4 branch stubs are not modified.

Wire format additions:

Endpoint Method DTO
/v1/{prefix}/databases/{db}/tables/{tbl}/branches POST CreateBranchRequest { branch, fromTag? }
/v1/{prefix}/databases/{db}/tables/{tbl}/branches GET ListBranchesResponse { branches } (NOT paged)
/v1/{prefix}/databases/{db}/tables/{tbl}/branches/{branch} DELETE
/v1/{prefix}/databases/{db}/tables/{tbl}/branches/{branch}/rename POST RenameBranchRequest { toBranch }
/v1/{prefix}/databases/{db}/tables/{tbl}/branches/{branch}/forward POST ForwardBranchRequest {} (empty body)

These are wire-compatible with the Java REST server (paimon-api/.../rest/requests/CreateBranchRequest.java, RenameBranchRequest.java, ForwardBranchRequest.java, responses/ListBranchesResponse.java, RESTApi.java:945-1029, RESTCatalog.java:703-769).

No new typed exception classes are introduced (BranchNotExistException / BranchAlreadyExistException already exist on master).

Documentation

n/a — this is an API gap closure. Java docs cover the semantics; the Python port mirrors Java behavior verbatim.

Out of scope (separate PRs)

  • FilesystemCatalog branch implementation — needs a Python-side BranchManager port (corresponds to Java paimon-core/.../utils/BranchManager.java).
  • Paged list_branches — Java has no paged version, so neither do we. If Java adds listBranchesPaged in the future, mirror it then.
  • CLI branch commands — pypaimon CLI has no branch commands today.
  • Branch name validation rules (reserved keywords etc.) — Java's RESTCatalog.createBranch rejects with BadRequestException; we map that to IllegalArgumentError and forward the server message.
  • from_tag existence enforcement in the mock REST server — pypaimon does not yet have a Python-side TagManager port, so the mock accepts any non-empty from_tag. The real Java REST server validates against TagManager and returns 404+TAG when missing; our RESTCatalog.create_branch correctly maps that to TagNotExistException. The from_tag not-exists path will be covered once the Python TagManager port lands.

Strict-Java-alignment checklist

  • Wire DTO field names: CreateBranchRequest = {branch, fromTag}, RenameBranchRequest = {toBranch}, ForwardBranchRequest = {}, ListBranchesResponse = {branches}. Asserted by test_branch_dto_serde.py.
  • URL templates: exactly match Java ResourcePaths.java:240-276 char-for-char (databases/{db}/tables/{tbl}/branches[/{branch}[/rename|/forward]]). Asserted by test_resource_path_* cases.
  • No paged listing: ListBranchesResponse extends RESTResponse (NOT PagedResponse); rest_api.list_branches returns plain List[str]. test_response_is_not_paged asserts no next_page_token attribute and no nextPageToken substring in serialized output.
  • HTTP-to-exception mapping: each except block in the 5 RESTCatalog overrides carries an inline # RESTCatalog.java:<line> comment that points at the Java source it mirrors; reviewers can cross-check at a glance.
  • Method parameter naming: existing 4 methods keep branch_name / tag_name (master compat); the new rename_branch uses Java-style from_branch / to_branch since it has no prior history.

Generative AI disclosure

Implementation, tests, and PR description were drafted with the assistance of Claude (Anthropic). Claude was supplied with the Java reference files (Catalog.java, RESTCatalog.java, RESTApi.java, CreateBranchRequest.java, RenameBranchRequest.java, ForwardBranchRequest.java, ListBranchesResponse.java) and the existing pypaimon patterns (list_partitions_paged, create_function, RESTBaseTest, the Tag CRUD PR #7746); the design plan and code were reviewed and integrated by the author.

The pypaimon Catalog ABC has stubs for create_branch / drop_branch /
fast_forward / list_branches raising NotImplementedError but is
missing rename_branch entirely; the RESTCatalog never overrode any
of them, so calling any branch operation on a real REST catalog
raises NotImplementedError instead of doing the REST call. Java
defines all five in Catalog.java:843-912 and overrides them in
RESTCatalog.java:703-769. This commit closes that gap.

Sister PR of the recently submitted Tag CRUD PR — same shape:
abstract stub + RESTCatalog overrides + wire DTOs + URL builders +
mock REST server route handlers + tests. FilesystemCatalog inherits
NotImplementedError for now (a Python BranchManager port is tracked
separately).

Strict alignment with Java:
- CreateBranchRequest serializes only {branch, fromTag}; no extra
  client-side fields leak into the wire format. Mirrors Java
  CreateBranchRequest.java.
- RenameBranchRequest serializes {toBranch}.
- ForwardBranchRequest serializes the empty body {} per Java
  ForwardBranchRequest.java.
- ListBranchesResponse is NOT a paged response: Java's
  listBranches returns plain List<String>, no nextPageToken. A
  unit test locks this down (asserts no next_page_token attr and
  no nextPageToken substring in serialized output).
- URL templates: /v1/{prefix}/databases/{db}/tables/{tbl}/branches[/{branch}[/rename|/forward]]
  with RESTUtil.encode_string applied to the branch name.
- HTTP-to-exception mapping mirrors RESTCatalog.java:703-769
  line-for-line, with Java line-number references in code comments:
    create_branch  — :703-722
    drop_branch    — :724-732
    rename_branch  — :734-746
    fast_forward   — :748-758
    list_branches  — :760-769
  Mappings:
    404+TAG (create_branch)   -> TagNotExistException
    404 (other, create/list)  -> TableNotExistException
    404 (drop/rename/forward) -> BranchNotExistException
    409 (create_branch)       -> BranchAlreadyExistException(branch_name)
    409 (rename_branch)       -> BranchAlreadyExistException(to_branch)
    403                       -> TableNoPermissionException
    400 (create_branch)       -> IllegalArgumentError

Naming decision: existing 4 catalog methods preserve their Python
parameter names (branch_name / tag_name) for backward-compat with
the existing abstract stubs on master; the new rename_branch uses
Java-style short names from_branch / to_branch since it has no
prior history. Wire DTO field names are STRICTLY Java
(branch / fromTag / toBranch / branches).

Mock server: tag-store-equivalent set-of-branches keyed by full
table name; routes added to _handle_table_resource for the four
URL shapes (collection / single / rename / forward). The from_tag
existence check is a documented mock simplification — pypaimon
does not have a Python-side TagManager port yet, so the mock does
not enforce TAG existence on POST. This is harmless for branch
testing and is tracked with the future TagManager port.

Tests:
- pypaimon/tests/api/test_branch_dto_serde.py (11 cases): wire
  format + URL templates + assertion that ListBranchesResponse is
  not paged. Asserts exact Java JSON field names char-for-char.
- pypaimon/tests/rest/rest_branch_test.py: 11 end-to-end tests
  via mock REST server mirroring Java RESTCatalogTest.testBranches,
  plus 1 FilesystemCatalog NotImplementedError sanity test for the
  new rename_branch abstract stub. Validates table-not-exist on
  create/list, duplicate create, list returns created, rename
  happy path / collide / source-missing, drop happy path / missing,
  fast-forward happy path / missing.
- Regression: existing rest_simple_test.py / rest_function_test.py /
  rest_catalog_test.py (52 tests) all pass.
- All edited files pass flake8.
@JingsongLi
Copy link
Copy Markdown
Contributor

+1

@JingsongLi JingsongLi merged commit b4e54ad into apache:master May 1, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants