[python] Add branch CRUD REST implementation to RESTCatalog#7747
Merged
JingsongLi merged 1 commit intoapache:masterfrom May 1, 2026
Merged
[python] Add branch CRUD REST implementation to RESTCatalog#7747JingsongLi merged 1 commit intoapache:masterfrom
JingsongLi merged 1 commit intoapache:masterfrom
Conversation
The pypaimon Catalog ABC has stubs for create_branch / drop_branch /
fast_forward / list_branches raising NotImplementedError but is
missing rename_branch entirely; the RESTCatalog never overrode any
of them, so calling any branch operation on a real REST catalog
raises NotImplementedError instead of doing the REST call. Java
defines all five in Catalog.java:843-912 and overrides them in
RESTCatalog.java:703-769. This commit closes that gap.
Sister PR of the recently submitted Tag CRUD PR — same shape:
abstract stub + RESTCatalog overrides + wire DTOs + URL builders +
mock REST server route handlers + tests. FilesystemCatalog inherits
NotImplementedError for now (a Python BranchManager port is tracked
separately).
Strict alignment with Java:
- CreateBranchRequest serializes only {branch, fromTag}; no extra
client-side fields leak into the wire format. Mirrors Java
CreateBranchRequest.java.
- RenameBranchRequest serializes {toBranch}.
- ForwardBranchRequest serializes the empty body {} per Java
ForwardBranchRequest.java.
- ListBranchesResponse is NOT a paged response: Java's
listBranches returns plain List<String>, no nextPageToken. A
unit test locks this down (asserts no next_page_token attr and
no nextPageToken substring in serialized output).
- URL templates: /v1/{prefix}/databases/{db}/tables/{tbl}/branches[/{branch}[/rename|/forward]]
with RESTUtil.encode_string applied to the branch name.
- HTTP-to-exception mapping mirrors RESTCatalog.java:703-769
line-for-line, with Java line-number references in code comments:
create_branch — :703-722
drop_branch — :724-732
rename_branch — :734-746
fast_forward — :748-758
list_branches — :760-769
Mappings:
404+TAG (create_branch) -> TagNotExistException
404 (other, create/list) -> TableNotExistException
404 (drop/rename/forward) -> BranchNotExistException
409 (create_branch) -> BranchAlreadyExistException(branch_name)
409 (rename_branch) -> BranchAlreadyExistException(to_branch)
403 -> TableNoPermissionException
400 (create_branch) -> IllegalArgumentError
Naming decision: existing 4 catalog methods preserve their Python
parameter names (branch_name / tag_name) for backward-compat with
the existing abstract stubs on master; the new rename_branch uses
Java-style short names from_branch / to_branch since it has no
prior history. Wire DTO field names are STRICTLY Java
(branch / fromTag / toBranch / branches).
Mock server: tag-store-equivalent set-of-branches keyed by full
table name; routes added to _handle_table_resource for the four
URL shapes (collection / single / rename / forward). The from_tag
existence check is a documented mock simplification — pypaimon
does not have a Python-side TagManager port yet, so the mock does
not enforce TAG existence on POST. This is harmless for branch
testing and is tracked with the future TagManager port.
Tests:
- pypaimon/tests/api/test_branch_dto_serde.py (11 cases): wire
format + URL templates + assertion that ListBranchesResponse is
not paged. Asserts exact Java JSON field names char-for-char.
- pypaimon/tests/rest/rest_branch_test.py: 11 end-to-end tests
via mock REST server mirroring Java RESTCatalogTest.testBranches,
plus 1 FilesystemCatalog NotImplementedError sanity test for the
new rename_branch abstract stub. Validates table-not-exist on
create/list, duplicate create, list returns created, rename
happy path / collide / source-missing, drop happy path / missing,
fast-forward happy path / missing.
- Regression: existing rest_simple_test.py / rest_function_test.py /
rest_catalog_test.py (52 tests) all pass.
- All edited files pass flake8.
dab1b4d to
8ebb1f1
Compare
Contributor
|
+1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
pypaimon'sCatalogABC has stubs forcreate_branch/drop_branch/fast_forward/list_branchesraisingNotImplementedError, and is missingrename_branchentirely. TheRESTCatalognever overrode any of them, so calling any branch operation on a real REST catalog raisesNotImplementedErrorinstead of issuing the REST call. Java defines all five (paimon-core/.../catalog/Catalog.java:843-912) with a complete REST implementation inpaimon-core/.../rest/RESTCatalog.java:703-769. This PR closes that gap.This is the sister PR of #7746 — Tag CRUD; same shape (abstract stub + RESTCatalog overrides + wire DTOs + URL builders + mock REST server route handlers + tests).
FilesystemCataloginherits the abstractNotImplementedErrorstubs; a Python-sideBranchManagerport is tracked separately.Linked issue
n/a — closing an API gap surfaced while auditing pypaimon's
Catalogagainst the Java contract.Tests
pypaimon/tests/api/test_branch_dto_serde.py— 11 unit tests pinning the wire format to Java: exact JSON field names (branch/fromTag/toBranch/branches), URL templates (databases/{db}/tables/{tbl}/branches[/{branch}[/rename|/forward]]), and an explicit assertion thatListBranchesResponseis not paged (nonextPageToken).pypaimon/tests/rest/rest_branch_test.py— 11 end-to-end tests against the in-process mock REST server mirroring JavaRESTCatalogTest::testBranches(paimon-core/.../rest/RESTCatalogTest.java:2138-2191): create / list happy path, table-not-exist on create / list, duplicate create raisesBranchAlreadyExistException, rename happy / collide / source-missing, drop happy / missing, fast-forward happy / missing. Plus 1FilesystemCatalogNotImplementedErrorsanity test for the newrename_branchabstract stub.pypaimon/tests/rest/rest_simple_test.py rest_function_test.py rest_catalog_test.py(52 tests) all pass unchanged.flake8 --config=dev/cfg.ini.API and format
Adds 1 new public method on
Catalog:rename_branch(identifier, from_branch, to_branch). Defaults toNotImplementedError— non-breaking; existing 4 branch stubs are not modified.Wire format additions:
/v1/{prefix}/databases/{db}/tables/{tbl}/branchesCreateBranchRequest { branch, fromTag? }/v1/{prefix}/databases/{db}/tables/{tbl}/branchesListBranchesResponse { branches }(NOT paged)/v1/{prefix}/databases/{db}/tables/{tbl}/branches/{branch}/v1/{prefix}/databases/{db}/tables/{tbl}/branches/{branch}/renameRenameBranchRequest { toBranch }/v1/{prefix}/databases/{db}/tables/{tbl}/branches/{branch}/forwardForwardBranchRequest {}(empty body)These are wire-compatible with the Java REST server (
paimon-api/.../rest/requests/CreateBranchRequest.java,RenameBranchRequest.java,ForwardBranchRequest.java,responses/ListBranchesResponse.java,RESTApi.java:945-1029,RESTCatalog.java:703-769).No new typed exception classes are introduced (
BranchNotExistException/BranchAlreadyExistExceptionalready exist on master).Documentation
n/a — this is an API gap closure. Java docs cover the semantics; the Python port mirrors Java behavior verbatim.
Out of scope (separate PRs)
FilesystemCatalogbranch implementation — needs a Python-sideBranchManagerport (corresponds to Javapaimon-core/.../utils/BranchManager.java).list_branches— Java has no paged version, so neither do we. If Java addslistBranchesPagedin the future, mirror it then.RESTCatalog.createBranchrejects withBadRequestException; we map that toIllegalArgumentErrorand forward the server message.from_tagexistence enforcement in the mock REST server — pypaimon does not yet have a Python-sideTagManagerport, so the mock accepts any non-emptyfrom_tag. The real Java REST server validates againstTagManagerand returns 404+TAG when missing; ourRESTCatalog.create_branchcorrectly maps that toTagNotExistException. Thefrom_tagnot-exists path will be covered once the PythonTagManagerport lands.Strict-Java-alignment checklist
CreateBranchRequest={branch, fromTag},RenameBranchRequest={toBranch},ForwardBranchRequest={},ListBranchesResponse={branches}. Asserted bytest_branch_dto_serde.py.ResourcePaths.java:240-276char-for-char (databases/{db}/tables/{tbl}/branches[/{branch}[/rename|/forward]]). Asserted bytest_resource_path_*cases.ListBranchesResponseextendsRESTResponse(NOTPagedResponse);rest_api.list_branchesreturns plainList[str].test_response_is_not_pagedasserts nonext_page_tokenattribute and nonextPageTokensubstring in serialized output.exceptblock in the 5 RESTCatalog overrides carries an inline# RESTCatalog.java:<line>comment that points at the Java source it mirrors; reviewers can cross-check at a glance.branch_name/tag_name(master compat); the newrename_branchuses Java-stylefrom_branch/to_branchsince it has no prior history.Generative AI disclosure
Implementation, tests, and PR description were drafted with the assistance of Claude (Anthropic). Claude was supplied with the Java reference files (
Catalog.java,RESTCatalog.java,RESTApi.java,CreateBranchRequest.java,RenameBranchRequest.java,ForwardBranchRequest.java,ListBranchesResponse.java) and the existing pypaimon patterns (list_partitions_paged,create_function,RESTBaseTest, the Tag CRUD PR #7746); the design plan and code were reviewed and integrated by the author.