proposal: crypto/rand: add UUIDv4 and UUIDv7

[seankhliao]

Proposal Details

// UUIDv4 generates a RFC 9562 Version 4 UUID
// using only random bytes in the format: 
// xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.
func UUIDv4() string

// UUIDv7 generates a RFC 9562 Version 7 UUID
// using the current time and random bytes in the format:
// xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.
func UUIDv7() string

As an alternative to #62026,
we propose the addition of 2 new helpers in crypto/rand
similar to the Text helper added in #67057.

We scope the standard library's support to only include generation of UUIDs,
and treat them as generally opaque identifiers.
The functions return a string, sidestepping any questions about marshaling.
The important comparisons are naturally available: ==, !=, strings.Compare (for sorting).
Users needing advanced functionality such as parsing, alternative marshaling, or otherwise inspecting UUIDs
can continue to use github.com/google/uuid (104k imports).

Why v4?

UUDv4 is purely random and should result in an even distribution of IDs.
It is also the most common type in use today.

• baseline (all uuid types) 487k: https://github.com/search?q=language%3AGo%20%2Fuuid%5C.New%2F&type=code
• generating v4 (uuid.New, uuid.NewString, uuid.NewRandom, uuid.NewUUID) 401k: https://github.com/search?q=language%3AGo+%2Fuuid%5C.New%28String%7CRandom%7CUUID%29%3F%5C%28%2F&type=code

Why v7?

While less popular by code metrics as v4,
it is still over twice as popular as all other type combined.
The time / sortability of these allow for more efficient use with some database implementations,
as seen by postgres implementing only v4 and v7: https://www.postgresql.org/docs/current/functions-uuid.html

• generating v7 (uuid.NewV7) 5.6k: https://github.com/search?q=language%3AGo+%2Fuuid%5C.NewV7%5C%28%2F&type=code
• all other uuids 2.4k: https://github.com/search?q=language%3AGo+%2Fuuid%5C.New%28V6%7CMD5%7CSHA1%7CHash%7CDCE.*%29%5C%28%2F&type=code

cc @FiloSottile

[gabyhelp]

Related Issues

• proposal: crypto/uuid: add API to generate and parse UUID #62026
• proposal: add uuid generator to stdlib #23789 (closed)
• proposal: math/rand or crypto/rand: add random strings generators #53447 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[sergeyprokhorenko]

Reference implementation is the most advanced official implementation for PostgreSQL, offering superior functionality.

See the article: Postgres UUIDv7 + Per-Backend Monotonicity

The PostgreSQL algorithm provides additional guarantees compared to RFC 9562. In fact, it combines Method 3 with Method 1. The entire timestamp can also function as a counter when more than about 4 identifiers per microsecond are generated. Therefore, the monotonicity of generated UUIDs is ensured within the same backend (a single process).

Monotonically increasing identifiers are generated even if the system clock jumps backward, if access to the system clock is unavailable, or if UUIDs are generated at a very high frequency, due to the internal timestamp functioning as a counter to maintain order.

The uuidv7() function can accept an optional offset parameter of type interval which is added to the internal timestamp. If the offset parameter results in a timestamp overflow or a negative timestamp, an adjusted timestamp value is automatically used. The timestamp behaves like a ring buffer: when the maximum value is exceeded, it wraps around to the minimum value. Similarly, if the absolute value of the negative offset exceeds the time elapsed since 00:00:00 UTC on January 1, 1970, the timestamp wraps around to the maximum value.

In Percona Server for MySQL the UUID_V7() function can have either no argument or a one integer argument: the argument is the number of milliseconds to adjust the timestamp forward or backward (negative values). Timestamp shift allows to hide the true date of record creation, prevents lock contention during parallel generation of UUIDs in several processes, and also ensures monotonicity while generation on remote clients

Using UUID as a 16-byte binary array provides a compact and efficient alternative to proposed string formats for handling UUIDs. See https://pkg.go.dev/github.com/google/uuid#UUID

RFC 9562 recommends using binary UUIDs in databases.

The authors of RFC 9562 are currently opting for a preffered compact text encoding of UUIDs instead of the traditional hexadecimal encoding. RFC 9562 will be updated accordingly.

[seankhliao]

As stated in the proposal, we will treat them as opaque identifiers, so the exact timestamp used is not guaranteed.
Introspection / using them as counters is out of scope, as is any alternate encoding that will take years to be adopted across the wider ecosystem.

[sergeyprokhorenko]

The idea to add functions generating UUIDs in the standard library is right.

Exact timestamps do not contradict the requirement for opaque identifiers. They are needed for monotonicity but extracting timestamps from UUIDs is unnecessary. Monotonicity can and should be guaranteed within one process.

I do not suggest extracting timestamps for counters. Rather, timestamps can be used as counters at generation time in critical situations.

Also, I do not propose alternative encoding now but binary format, like PostgreSQL's native UUID type and github.com/google/uuid's binary representation.

This approach keeps UUIDs opaque externally but uses timestamps internally for monotonic UUIDv7 generation, matching PostgreSQL's 128-bit binary UUID storage and format.

[seankhliao]

I don't think we should promise monotonic, only that it is clustered around the time it was generated.

[earthboundkid]

It's a shame it's always hex. The alternative is to add a UUID type that has a hex string method, base64 method, and raw bytes, but then one wonders why it's not just a whole package with parsing and so on.

[apparentlymart]

While I certainly agree that UUIDv4 is in wide use by existing callers of other UUID implementations, there are presumably not yet any callers of rand.UUIDv4 because it doesn't exist yet, and anyone who wants to switch to these functions will need to modify their existing callers to use them.

Given that, is there any downside to them adopting UUIDv7 at the same time as making that change? (This is not a rhetorical question: I'm genuinely curious why someone might prefer UUIDv4 over UUIDv7, other than inertia.)

The framing here suggests that these are intended only for generating opaque unique strings that are not to be further dissected, in which case I wouldn't expect that it matters a great deal whether those unique strings were generated using the UUIDv4 scheme the UUIDv7 scheme.

Could be sufficient to implement only rand.UUIDv7 and thus have only that one new function to maintain under the compatibility promises?

[apparentlymart]

I feel in two minds about only providing the canonical hex string encoding and not a binary representation.

On one hand, it's annoying that there has been historical disagreement about how to represent a UUID as a binary value: big endian in individual subfields, little endian in individual subfields, or as a single uint128 value. Using the standardized string representation neatly sidesteps that.

On the other hand, RFC 9562 does specify a single canonical binary representation: big endian in individual subfields. It also (as mentioned in an earlier comment) recommends using that representation for storage in databases. It would be unfortunate if folks started trying to parse the string representation into a binary representation for storage purposes.

Back on the first hand, though: database technologies that use string-based command syntaxes like SQL tend to accept the canonical string representation and translate it into a binary representation of their choice, so it might end up being pretty rare that someone needs to do that conversion in application code vs. just letting their database engine do it. The application would only need to deal with the binary format if interacting with a database whose protocol requires it; is there such a database, and does it follow the canonical binary format from RFC 9562 if so?

I think overall I find myself leaning back towards the original proposal of only returning strings as a relatively simple API for generating unique strings of a predictable length, and having folks use third-party libraries if they have more specific needs.