Client: Java ID generator helper #1347
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The ULID spec notes that IDs within the same millisecond dont need to be ordered: https://github.com/ulid/spec?tab=readme-ov-file#sorting
Overflow should happen relative to the 80-bit random generated at the timestamp rather than relative to a separate counter at the timestamp which is applied to the random. This provides sort order guarantee using the recommended approach from the spec: https://github.com/ulid/spec?tab=readme-ov-file#monotonicity
batiati
reviewed
Dec 14, 2023
| } | ||
|
|
||
| var ulidB = new DecodedULID(UInt128.ULID()); | ||
| ulidB.assertMonotonicSince(ulidA); |
There was a problem hiding this comment.
Can we also use UInt128.asBigInteger to compare both ULIDs?
This will assert that the binary representation is really what we expect at Zig's side (a monotonically increasing u128 little-endian).
var bigIntA = UInt128.asBigInteger(...);
var bigIntB = UInt128.asBigInteger(...);
assertTrue(bigIntB.compareTo(bigIntA) > 0); There was a problem hiding this comment.
Sort of: asBigInteger interprets the bytes as 2 little-endian u64 values of a u128. ULID however stores timestamp as {u32, u16} and random as u80, both big-endian. We can instead interpret the bytes directly via new BigInteger(ULID()) since it assumes big-endian. This also means Zig side might not see it as monotonically increasing little-endian.
There was a problem hiding this comment.
ULID has been renamed to ID and it stores the individual fields in little-endian which allows use of asBigInteger.
batiati
reviewed
Dec 14, 2023
For setups which download both `zig/zig` and `zig/zig.exe` for convenience (i.e. windows WSL), this ensures the Java CI uses the correct one instead of relying on shell to append `.exe` (which doesn't happen since non `.exe` version exists).
matklad
force-pushed
the
king/java-ulid
branch
from
922cae4
to
1ac5e48
Compare
ULID spec requires storing the values as big endian which means, when interpreted by a replica on the Zig side, the `u128` ID is no longer monotonically incrementing. This resorts in unique but effectively randomly-ordered IDs which doesn't take advantage of data/lookup optimizations in the storage engine. Instead, the Java client now exposes a generic `ID()` helper (instead of specifically `ULID`) which allows us to return u128 bytes in little-endian format while still using the core ideas around uniqueness & monotonicity from ULID underneath.
kprotty
changed the title
Switches to using the 48 LSB of timestamp instead of 16 LSB and 32 MSB. Also updates the data-modeling documentation to give note about storing ULID fields in little endian.
batiati
requested changes
Jan 20, 2024
Standardize upper-case hex literals and refer to ULID spec as `ID()`'s inspiration.
batiati
approved these changes
Jan 22, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Creates a static helper function in the java client for generating thread-safe, unique, and monotonic
u128IDs to use for Accounts and Transfers.