Skip to content

[#12651] improvement(core): add OCC for topic writes - #12680

Merged
jerryshao merged 2 commits into
apache:mainfrom
yuqi1129:feat/12651-occ-topic
Aug 31, 2026
Merged

[#12651] improvement(core): add OCC for topic writes#12680
jerryshao merged 2 commits into
apache:mainfrom
yuqi1129:feat/12651-occ-topic

Conversation

@yuqi1129

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

  • Use topic_meta.current_version as the OCC token for topic alters and direct deletes.
  • Run the root topic CAS and dependent metadata cleanup in one transaction.
  • Classify stale writes as either OptimisticLockException or NoSuchEntityException.
  • Lock the parent schema during topic creation.
  • Preserve monotonic versions and persisted topic IDs during overwrite across H2, MySQL, and PostgreSQL.
  • Invalidate the entity cache after overwrite because the database may preserve existing identity.
  • Add SQL-provider, service, converter, and cache tests for the OCC behavior.

Why are the changes needed?

Concurrent topic writes could overwrite newer metadata, and a stale delete could remove relationships belonging to a newer topic. Overwrite could also reset the OCC version or cache an entity whose ID differed from the row retained by the database.

Fix: #12651

Does this PR introduce any user-facing change?

Yes. Concurrent stale topic writes now fail with an optimistic-lock conflict, while writes against a topic that was deleted or renamed report that the topic no longer exists.

No public API or property key is changed.

How was this patch tested?

  • ./gradlew --no-daemon :core:spotlessApply
  • SKIP_DOCKER_TESTS=true ./gradlew --no-daemon :core:check -PskipITs -PskipDockerTests=true
  • Targeted Topic service, SQL-provider, PO converter, and relational entity-store tests

Copilot AI lite review requested due to automatic review settings August 27, 2026 13:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown

Code Coverage Report

Overall Project 68.55% +0.23% 🟢
Files changed 87.03% 🟢

Module Coverage
aliyun 19.74% 🔴
api 52.09% 🟢
authorization-common 85.96% 🟢
authorization-ranger 4.38% 🔴
aws 53.54% 🟢
azure 32.1% 🔴
catalog-common 9.9% 🔴
catalog-fileset 80.3% 🟢
catalog-glue 69.24% 🟢
catalog-hive 82.96% 🟢
catalog-jdbc-common 45.69% 🟢
catalog-jdbc-doris 82.44% 🟢
catalog-jdbc-mysql 79.33% 🟢
catalog-jdbc-postgresql 83.39% 🟢
catalog-jdbc-starrocks 79.16% 🟢
catalog-kafka 76.99% 🟢
catalog-lakehouse-generic 60.55% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 85.86% 🟢
catalog-lakehouse-paimon 84.26% 🟢
catalog-model 77.99% 🟢
cli 44.48% 🟢
client-java 77.44% 🟢
common 55.5% 🟢
core 83.62% +0.14% 🟢
filesystem-hadoop3 76.45% 🟢
flink 0.0% 🔴
flink-common 50.29% 🟢
flink-runtime 0.0% 🔴
gcp 32.2% 🔴
hadoop-auth 68.0% 🟢
hadoop-common 17.84% 🔴
hive-metastore-common 53.4% 🟢
iceberg-aliyun-bundle 0.0% 🔴
iceberg-common 64.75% 🟢
iceberg-rest-server 75.96% 🟢
idp-basic 85.98% 🟢
integration-test-common 0.0% 🔴
jobs 62.92% 🟢
lance-common 32.63% 🔴
lance-rest-server 65.46% 🟢
lineage 53.02% 🟢
optimizer 83.24% 🟢
optimizer-api 21.95% 🔴
server 88.15% 🟢
server-common 80.5% 🟢
spark 28.57% 🔴
spark-common 48.92% 🟢
tencent 69.84% 🟢
trino-connector 47.44% 🟢
Files
Module File Coverage
core TopicMetaPostgreSQLProvider.java 100.0% 🟢
POConverters.java 88.31% 🟢
TopicMetaSQLProviderFactory.java 87.1% 🟢
TopicMetaBaseSQLProvider.java 81.82% 🟢
TopicMetaService.java 77.34% 🟢
TopicMetaMapper.java 0.0% 🔴

@yuqi1129
yuqi1129 force-pushed the feat/12651-occ-topic branch from d6e32f8 to d9ddce9 Compare August 30, 2026 12:40
@yuqi1129
yuqi1129 requested a review from jerryshao August 31, 2026 02:44
@jerryshao
jerryshao merged commit 8f41d92 into apache:main Aug 31, 2026
41 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Subtask] Implement version-CAS OCC for topic

3 participants