Skip to content

[core][spark] Add REST management procedures - #9410

Merged
JingsongLi merged 2 commits into
apache:masterfrom
JingsongLi:codex/spark-rest-management
Aug 26, 2026
Merged

[core][spark] Add REST management procedures#9410
JingsongLi merged 2 commits into
apache:masterfrom
JingsongLi:codex/spark-rest-management

Conversation

@JingsongLi

Copy link
Copy Markdown
Contributor

Purpose

Complete the REST management stack after #9393, #9400, and #9406 by exposing the merged permission and policy contracts through Spark SQL procedures.

This is the final PR from the original management API proof of concept.

Changes

  • Harden managed row-filter and column-mask decoding and include all mask input columns in the physical read projection.
  • Extend the test REST catalog server with permission and policy storage, stable resource identifiers, pagination, authorization checks, and table lifecycle handling.
  • Add Spark procedures for granting, listing, and revoking permissions.
  • Add Spark procedures for creating, listing, and dropping data policies. Policy creation remains strict and reports duplicate policies instead of replacing them.
  • Support ARRAY literals in procedure arguments and document the full CALL syntax with examples.

Tests

  • mvn -pl paimon-core -am -Pfast-build -DwildcardSuites=none -Dtest=TableQueryAuthResultTest -DfailIfNoTests=false test
  • mvn -pl paimon-core -am -Pfast-build -DwildcardSuites=none -Dtest=PolicyKeyTest,RESTPermissionStoreTest -DfailIfNoTests=false test
  • mvn -pl paimon-spark/paimon-spark-ut -am -Pfast-build -Pspark3 -DfailIfNoTests=false -DwildcardSuites=org.apache.paimon.spark.procedure.PermissionProcedureTest -Dtest=none test
  • mvn -pl paimon-spark/paimon-spark-ut -am -Pfast-build -Pspark3 -DfailIfNoTests=false -DwildcardSuites=org.apache.paimon.spark.sql.CatalogQualifiedCreateTableLikeTest -Dtest=none test
  • yarn --cwd docs test:rest-openapi
  • Non-fast Maven validation passed Spotless, Checkstyle, and the targeted core tests.

@JingsongLi
JingsongLi force-pushed the codex/spark-rest-management branch from 8130c16 to 9bb8229 Compare August 26, 2026 13:23
@JingsongLi
JingsongLi marked this pull request as ready for review August 26, 2026 13:24

@leaves12138 leaves12138 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Spark management procedures are consistent with the REST permission and policy contracts, including resource validation, pagination, policy identity, and ARRAY argument handling. The targeted Spark 3 procedure suite passes locally. Looks good to me.

@JingsongLi
JingsongLi merged commit 8de456d into apache:master Aug 26, 2026
12 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants