Package rid provides a performant, goroutine-safe generator of short k-sortable unique IDs suitable for use where inter-process ID generation coordination is not required.
Using a non-standard character set (fewer vowels), IDs Base-32 encode as a
16-character URL-friendly, case-insensitive representation like
dfp7qt0v2pwt0v2x.
An ID is a:
- 4-byte timestamp value representing seconds since the Unix epoch, plus a
- 6-byte random value; see the Random Source discussion.
Built-in (de)serialization simplifies interacting with SQL databases and JSON.
cmd/rid provides the rid utility to generate or inspect IDs. Thanks to
internal/fastrand introduced in Go 1.19 and made the default math/rand source in Go
1.20, ID generation starts fast and scales well as cores are added. De-serialization
has also been optimized. See Package Benchmarks.
Why rid instead of alternatives?
- At 10 bytes binary, 16 bytes Base32 encoded, rid.IDs are case-insensitive and short, yet with 48 bits of uniqueness per second, are unique enough for many use cases.
- IDs have a random component rather than a potentially guessable monotonic counter found in some libraries.
Acknowledgement: This package borrows heavily from rs/xid (https://github.com/rs/xid), a zero-configuration globally-unique high-performance ID generator that leverages ideas from MongoDB (https://docs.mongodb.com/manual/reference/method/ObjectId/).
id := rid.New()
fmt.Printf("%s\n", id.String())
// Output: dfp7qt97menfv8ll
id2, err := rid.FromString("dfp7qt97menfv8ll")
if err != nil {
fmt.Println(err)
}
fmt.Printf("%s %d %v\n", id2.Time(), id2.Random(), id2.Bytes())
// Output: 2022-12-28 09:24:57 -0800 PST 43582827111027 [99 172 123 233 39 163 106 237 162 115]Package rid also provides the rid tool for id generation and inspection.
$ rid
dfpb18y8dg90hc74
$ rid -c 2
dfp9l9cgs05blztq
dfp9l9d80yxdf804
# produce 4 and inspect
$ rid `rid -c 4`
dfp9lmz9ksw87w48 ts:1672255955 rnd:256798116540552 2022-12-28 11:32:35 -0800 PST ID{ 0x63, 0xac, 0x99, 0xd3, 0xe9, 0x8e, 0x78, 0x83, 0xf0, 0x88 }
dfp9lmxefym2ht2f ts:1672255955 rnd:190729433933902 2022-12-28 11:32:35 -0800 PST ID{ 0x63, 0xac, 0x99, 0xd3, 0xad, 0x77, 0xa8, 0x28, 0x68, 0x4e }
dfp9lmt5zjy7km9n ts:1672255955 rnd: 76951796109621 2022-12-28 11:32:35 -0800 PST ID{ 0x63, 0xac, 0x99, 0xd3, 0x45, 0xfc, 0xbc, 0x78, 0xd1, 0x35 }
dfp9lmxt5sms80m7 ts:1672255955 rnd:204708502569607 2022-12-28 11:32:35 -0800 PST ID{ 0x63, 0xac, 0x99, 0xd3, 0xba, 0x2e, 0x69, 0x94, 0x2, 0x87 }
Since cryptographically secure IDs are not an objective for this package, other
approaches could be considered. With Go 1.19, rid utilized an internal runtime
fastrand64, providing single and multi-core performance benefits. Go
1.20 exposed fastrand64 via the stdlib. As of rid v1.1.6, the package depends
on Go 1.22 math/rand/v2, which provides Uint64N().
You may also enjoy reading:
- Fast thread-safe randomness in Go.
- For more information on fastrand (wyrand) see: https://github.com/wangyi-fudan/wyhash
To satisfy whether rid.IDs are unique enough for your use case, run eval/uniqcheck/main.go with various values for number of go routines and iterations, or, at the command line, produce IDs and use OS utilities to check:
rid -c 2000000 | sort | uniq -d
// None output
- 2023-03-02 v1.1.6: Package depends on math/rand/v2 and now requires Go 1.22+.
- 2023-01-23 Replaced the stdlib Base32 encoding/decoding with an unrolled version for decoding performance.
- 2022-12-28 The "10byte" branch was merged to master; the "15byte-historical" branch will be left dormant.
Contributions are welcome.
Comparison table generated by eval/compare/main.go:
| Package | BLen | ELen | K-Sort | Encoded ID and Next | Method | Components |
|---|---|---|---|---|---|---|
| solutionroute/rid | 10 | 16 | true | dqjllq1sr0lrb93kdqjllq40n8t6yx3r |
math/rand/v2 | 4 byte ts(sec) : 6 byte random |
| rs/xid | 12 | 20 | true | cnijjn34l33778tp3ingcnijjn34l33778tp3io0 |
counter | 4 byte ts(sec) : 2 byte mach ID : 2 byte pid : 3 byte monotonic counter |
| segmentio/ksuid | 20 | 27 | true | 2dCoMU7h8xaTlBhacYmys0CETTc2dCoMQwFBYWoEi9mue6Kv1esiwy |
math/rand | 4 byte ts(sec) : 16 byte random |
| google/uuid | 16 | 36 | false | c9c5e75e-1ec1-4718-99d9-024c84422e27e6387602-e980-44d2-a295-8c444d815d52 |
crypt/rand | v4: 16 bytes random with version & variant embedded |
| oklog/ulid | 16 | 26 | true | 01HR3PM0YWWF2WK24EVAFHSTR801HR3PM0YWY56DH7AC0GX6YC4K |
crypt/rand | 6 byte ts(ms) : 10 byte counter random init per ts(ms) |
| kjk/betterguid | 17 | 20 | true | -Ns6PVER-PgtrwP2IgwR-Ns6PVER-PgtrwP2IgwS |
counter | 8 byte ts(ms) : 9 byte counter random init per ts(ms) |
With only 48 bits of randomness per second, rid does not attempt to weigh
in as a globally unique ID generator. If that is your requirement, rs/xid is a solid
feature comparable alternative for such needs.
For a comparison of various Go-based unique ID solutions, see: https://blog.kowalczyk.info/article/JyRZ/generating-good-unique-ids-in-go.html
A benchmark suite for the above-noted packages can be found in
eval/bench/bench_test.go. All runs were done with scaling_governor set to performance:
echo "performance" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
$ go test -cpu 1,2,4,8,16,32 -test.benchmem -bench .
goos: linux
goarch: amd64
pkg: github.com/mwyvr/rid/eval/bench
cpu: Intel(R) Core(TM) i9-14900K
BenchmarkRid 39116605 28.59 ns/op 0 B/op 0 allocs/op
BenchmarkRid-2 70493946 16.31 ns/op 0 B/op 0 allocs/op
BenchmarkRid-4 82231498 14.26 ns/op 0 B/op 0 allocs/op
BenchmarkRid-8 75729330 14.88 ns/op 0 B/op 0 allocs/op
BenchmarkRid-16 59416164 19.69 ns/op 0 B/op 0 allocs/op
BenchmarkRid-32 68666065 18.63 ns/op 0 B/op 0 allocs/op
BenchmarkXid 44036608 27.24 ns/op 0 B/op 0 allocs/op
BenchmarkXid-2 39810664 27.07 ns/op 0 B/op 0 allocs/op
BenchmarkXid-4 36596661 30.20 ns/op 0 B/op 0 allocs/op
BenchmarkXid-8 36267940 32.22 ns/op 0 B/op 0 allocs/op
BenchmarkXid-16 35589727 31.99 ns/op 0 B/op 0 allocs/op
BenchmarkXid-32 56330694 26.68 ns/op 0 B/op 0 allocs/op
BenchmarkKsuid 5444830 220.1 ns/op 0 B/op 0 allocs/op
BenchmarkKsuid-2 4578231 249.0 ns/op 0 B/op 0 allocs/op
BenchmarkKsuid-4 4704010 254.0 ns/op 0 B/op 0 allocs/op
BenchmarkKsuid-8 4546252 251.8 ns/op 0 B/op 0 allocs/op
BenchmarkKsuid-16 4491115 259.2 ns/op 0 B/op 0 allocs/op
BenchmarkKsuid-32 4682730 265.2 ns/op 0 B/op 0 allocs/op
BenchmarkGoogleUuid 6017610 198.3 ns/op 16 B/op 1 allocs/op
BenchmarkGoogleUuid-2 9588854 117.8 ns/op 16 B/op 1 allocs/op
BenchmarkGoogleUuid-4 17966739 67.01 ns/op 16 B/op 1 allocs/op
BenchmarkGoogleUuid-8 26756277 38.35 ns/op 16 B/op 1 allocs/op
BenchmarkGoogleUuid-16 37318750 37.57 ns/op 16 B/op 1 allocs/op
BenchmarkGoogleUuid-32 39958574 29.55 ns/op 16 B/op 1 allocs/op
BenchmarkUlid 201711 5970 ns/op 5440 B/op 3 allocs/op
BenchmarkUlid-2 367596 3084 ns/op 5440 B/op 3 allocs/op
BenchmarkUlid-4 636337 1732 ns/op 5440 B/op 3 allocs/op
BenchmarkUlid-8 1000000 1084 ns/op 5440 B/op 3 allocs/op
BenchmarkUlid-16 1000000 1116 ns/op 5440 B/op 3 allocs/op
BenchmarkUlid-32 1000000 1140 ns/op 5440 B/op 3 allocs/op
BenchmarkBetterguid 22956550 50.49 ns/op 24 B/op 1 allocs/op
BenchmarkBetterguid-2 21297140 51.48 ns/op 24 B/op 1 allocs/op
BenchmarkBetterguid-4 15610149 67.59 ns/op 24 B/op 1 allocs/op
BenchmarkBetterguid-8 14513452 80.38 ns/op 24 B/op 1 allocs/op
BenchmarkBetterguid-16 11680210 105.1 ns/op 24 B/op 1 allocs/op
BenchmarkBetterguid-32 9156219 133.4 ns/op 24 B/op 1 allocs/op
PASS
ok github.com/mwyvr/rid/eval/bench 47.927s