Skip to content

OpenAPI, Core: Add VariantType to the REST catalog spec#17256

Open
nssalian wants to merge 3 commits into
apache:mainfrom
nssalian:rest-api-spec-variant
Open

OpenAPI, Core: Add VariantType to the REST catalog spec#17256
nssalian wants to merge 3 commits into
apache:mainfrom
nssalian:rest-api-spec-variant

Conversation

@nssalian

@nssalian nssalian commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Rationale for the change

Adds the variant type to the REST catalog OpenAPI spec so that variant columns are representable in table schemas exchanged over the REST protocol.

Builds on #14242. That change modeled variant as an object ({"type": "variant"}); this change instead models it as a bare string const: "variant" to match how SchemaParser serializes it on the wire (SchemaParser writes the type as the string "variant")

Changes

  • open-api/rest-catalog-open-api.yaml Added a VariantType schema
  • open-api/rest-catalog-open-api.py - regenerated via make generate

Testing

  • cd open-api && make install && make lint && make generate && git diff --exit-code - spec validates and the generated .py is up to date.
  • Verified against the regenerated pydantic module: Type.model_validate("variant") resolves to VariantType, and other type strings (int, string, long,decimal(10,2), fixed[16]) still resolve to PrimitiveType.

Co-authored-by: Amogh Jahagirdar <amoghj@apache.org>
@nssalian
nssalian marked this pull request as ready for review July 16, 2026 21:05

@laskoviymishka laskoviymishka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really like that this lands the spec change, the CatalogTests round-trip, and the RCK coverage together. That feels like the right shape for adding a new type to the REST contract, and the bare-string "variant" encoding matches SchemaParser and Appendix C.

I’d still hold this before merge on two points.

The main one is code generation. The not: {const: "variant"} constraint on PrimitiveType is correct and keeps the oneOf unambiguous in JSON Schema, but datamodel-codegen drops not:. As a result, the generated PrimitiveType remains RootModel[str] and still accepts "variant".

Since the Type union is evaluated left-to-right and PrimitiveType comes first, Type.model_validate("variant") resolves to PrimitiveType, not VariantType. So the schema has the right invariant, but the generated client does not. I’d either put VariantType before PrimitiveType in the oneOf and regenerate, or add a validator that rejects "variant" from PrimitiveType.

The other blocker is the RCK behavior. RESTCompatibilityKitCatalogTests.supportsVariant() should read an RCK_SUPPORTS_VARIANT property, defaulting to false, rather than returning true unconditionally. That lets catalogs without v3 or variant support opt out.

Since this changes the REST spec, it should also get the usual dev-list note. Amogh raised the same point on #14242. Please link the thread here once it is posted so the change remains traceable.

The smaller comments are inline: add a format-version assertion to the round-trip test, cover a nested variant, and confirm whether the generated .py file came from make generate rather than being edited manually. The class placement made me wonder.

Separately, neither iceberg-rust nor PyIceberg has a VariantType in its own type system yet. Tracking issues for round-trip support in those implementations seem like the right follow-up, rather than expanding this PR further.

Once the codegen and RCK points are addressed, I’m happy to take another pass.

Comment thread core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java Outdated
Comment thread core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java Outdated
Comment thread core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java Outdated
Comment thread open-api/rest-catalog-open-api.py
Comment thread open-api/rest-catalog-open-api.py Outdated
@nssalian

Copy link
Copy Markdown
Collaborator Author

Thanks for the review @laskoviymishka. I'll address the comments and send a note to dev@ as well.

Comment thread core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java Outdated
@nssalian

nssalian commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator Author

Tracking issues for round-trip support in those implementations seem like the right follow-up, rather than expanding this PR further.

Once the spec change is complete, I'll do the follow ups as well (RCK, etc) and open the tracking for each impl. Currently, this PR has been scoped down to the REST API spec alone. Thanks @laskoviymishka .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants