Skip to content

feat(catalog): add ListSnapshots() to public Catalog API#244

Merged
lxy-9602 merged 2 commits into
alibaba:mainfrom
mrdrivingduck:feat/catalog-list-snapshots
Apr 23, 2026
Merged

feat(catalog): add ListSnapshots() to public Catalog API#244
lxy-9602 merged 2 commits into
alibaba:mainfrom
mrdrivingduck:feat/catalog-list-snapshots

Conversation

@mrdrivingduck
Copy link
Copy Markdown
Contributor

Purpose

Add ListSnapshots() to the abstract Catalog interface and implement it in FileSystemCatalog. The method returns a vector of SnapshotInfo, a new plain-data struct that exposes user-facing snapshot metadata.

The primary use case is time travel: callers list snapshots to discover available versions, then pass a chosen snapshot_id to the read API.

API and Format

Yes — adds a new pure-virtual method ListSnapshots() to include/paimon/catalog/catalog.h and a new public header include/paimon/snapshot/snapshot_info.h. No storage format or protocol change.

Generative AI tooling

Generated-by: Claude Opus 4.7

Copilot AI review requested due to automatic review settings April 21, 2026 09:50
@mrdrivingduck mrdrivingduck force-pushed the feat/catalog-list-snapshots branch from 709a922 to 75a42c3 Compare April 21, 2026 09:51
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a new public Catalog API to list table snapshots (for time-travel discovery) by introducing a SnapshotInfo metadata struct and implementing ListSnapshots() for FileSystemCatalog.

Changes:

  • Added Catalog::ListSnapshots() pure-virtual API returning std::vector<SnapshotInfo>.
  • Introduced public SnapshotInfo + CommitKind types and string conversion helper.
  • Implemented snapshot listing in FileSystemCatalog via SnapshotManager.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/paimon/core/snapshot_info.cpp Implements CommitKindToString() for the new public enum.
include/paimon/snapshot/snapshot_info.h Adds public SnapshotInfo and CommitKind definitions.
include/paimon/catalog/catalog.h Extends the public Catalog interface with ListSnapshots().
src/paimon/core/catalog/file_system_catalog.h Declares FileSystemCatalog::ListSnapshots() override.
src/paimon/core/catalog/file_system_catalog.cpp Implements snapshot listing and commit-kind conversion.
src/paimon/CMakeLists.txt Adds new compilation unit for snapshot info support.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/paimon/core/catalog/file_system_catalog.cpp Outdated
Comment thread src/paimon/core/catalog/file_system_catalog.cpp Outdated
Comment thread src/paimon/core/snapshot_info.cpp Outdated
@mrdrivingduck mrdrivingduck force-pushed the feat/catalog-list-snapshots branch 2 times, most recently from a3a08da to 4714c2a Compare April 21, 2026 10:15
@mrdrivingduck mrdrivingduck force-pushed the feat/catalog-list-snapshots branch 2 times, most recently from d6eda73 to 93282ed Compare April 21, 2026 10:40
@mrdrivingduck
Copy link
Copy Markdown
Contributor Author

Planning to support time travel in duckdb-paimon, so as a prerequisite, we need to know all the snapshots we currently have. Export it from paimon-cpp.

Comment thread include/paimon/snapshot/snapshot_info.h Outdated
Comment thread src/paimon/core/catalog/file_system_catalog.cpp Outdated
Comment thread src/paimon/core/catalog/file_system_catalog.cpp
Comment thread src/paimon/core/catalog/file_system_catalog.cpp
@mrdrivingduck mrdrivingduck force-pushed the feat/catalog-list-snapshots branch 2 times, most recently from d417ceb to de7ec9f Compare April 21, 2026 15:46
@mrdrivingduck mrdrivingduck requested a review from lxy-9602 April 21, 2026 15:48
Comment thread test/inte/list_snapshots_inte_test.cpp Outdated
Comment thread test/inte/list_snapshots_inte_test.cpp Outdated
Comment thread test/inte/list_snapshots_inte_test.cpp Outdated
@lxy-9602 lxy-9602 requested a review from lucasfang April 22, 2026 01:08
@mrdrivingduck mrdrivingduck force-pushed the feat/catalog-list-snapshots branch from b201101 to 3d13f66 Compare April 22, 2026 02:30
Introduce a plain-data SnapshotInfo struct and a scoped CommitKind enum
in a new public header include/paimon/snapshot/snapshot_info.h. Both
types are fully exported via PAIMON_EXPORT and carry no dependency on
internal Jsonizable or RapidJSON headers, keeping the public API surface
clean. The struct exposes only user-facing fields: snapshot_id,
schema_id, commit_user, commit_kind, time_millis, total_record_count,
delta_record_count, and watermark. Internal manifest list paths and
commit_identifier are intentionally omitted.

Extend the abstract Catalog interface with a new pure-virtual method
ListSnapshots(), which returns all snapshots of a table ordered by
snapshot id ascending. FileSystemCatalog delegates to the existing
SnapshotManager::GetAllSnapshots() and converts each internal Snapshot
to a SnapshotInfo through a file-local ConvertCommitKind helper, leaving
all internal code untouched.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
@mrdrivingduck mrdrivingduck force-pushed the feat/catalog-list-snapshots branch from 3d13f66 to 0012782 Compare April 22, 2026 02:32
@mrdrivingduck mrdrivingduck requested a review from lxy-9602 April 22, 2026 02:36
Copy link
Copy Markdown
Collaborator

@lxy-9602 lxy-9602 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lucasfang
Copy link
Copy Markdown
Collaborator

+1

@lxy-9602 lxy-9602 merged commit 1ba3589 into alibaba:main Apr 23, 2026
9 checks passed
@mrdrivingduck mrdrivingduck deleted the feat/catalog-list-snapshots branch April 23, 2026 02:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants