Repository navigation
Replies: 1 comment 1 reply
|
Spark has Delta Lake also has identity columns starting in version 3.3. They generate unique Which Spark/Delta versions are you using, and do you mean Spark Structured Streaming or a managed streaming table? Also, should replaying the same source event preserve its ID? That affects whether a generated ID or a source event key fits your use case. |
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
While we have use for uuid in this case for generating unique ids for every row uuids become more of processing instead is there anything like hash64 which is natively into deltalake which can help us achieve uniqueness while ingestion into delta lake
All reactions