Skip to content

Use table PKs for topic keys; add client reconnect#229

Merged
jamals86 merged 1 commit into
mainfrom
029-datafusion-modernization
Apr 24, 2026
Merged

Use table PKs for topic keys; add client reconnect#229
jamals86 merged 1 commit into
mainfrom
029-datafusion-modernization

Conversation

@jamals86
Copy link
Copy Markdown
Member

Derive topic message keys from table primary keys and improve client reconnection handling. Add TopicPrimaryKeyLookup trait and SchemaRegistryTopicPrimaryKeyLookup, wire an optional primary-key lookup into TopicPublisherService, and use PK columns to extract stable message keys and determine partitioning. Refactor payload handling to cache a JSON map, add extract_primary_key and hash_key helpers, and adjust publishing logic and tests to ensure same-PK rows land in the same partition. On the client side, expose an explicit connect() in the TS SDK, add browser/WASM reconnection scheduling and resume logic (resubscribe on reconnect), update WASM client ping defaults, and add runtime/coverage tests plus a browser resume e2e page. Misc: update related imports/exports and bump Tailwind-related entries in UI package-lock.json.

Summary

Describe the change in 1-3 sentences.

Related

  • Closes #
  • Related docs/specs:

What Changed

  • [ X] Bug fix
  • Feature
  • Refactor
  • Docs
  • Tests
  • Breaking change

Validation

List the commands or checks you ran.

# Example

Checklist

  • I kept the change scoped to the problem being solved.
  • I added or updated tests when behavior changed.
  • I updated docs when the public behavior, workflow, or architecture changed.
  • If this touches auth, SQL, APIs, storage, or transport, I reviewed the security guidance in SECURITY.md and docs/security/security-checklist.md.

Notes

Add rollout notes, screenshots, or follow-up work if needed.

Derive topic message keys from table primary keys and improve client reconnection handling. Add TopicPrimaryKeyLookup trait and SchemaRegistryTopicPrimaryKeyLookup, wire an optional primary-key lookup into TopicPublisherService, and use PK columns to extract stable message keys and determine partitioning. Refactor payload handling to cache a JSON map, add extract_primary_key and hash_key helpers, and adjust publishing logic and tests to ensure same-PK rows land in the same partition. On the client side, expose an explicit connect() in the TS SDK, add browser/WASM reconnection scheduling and resume logic (resubscribe on reconnect), update WASM client ping defaults, and add runtime/coverage tests plus a browser resume e2e page. Misc: update related imports/exports and bump Tailwind-related entries in UI package-lock.json.
@jamals86 jamals86 merged commit 33d3797 into main Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant