Add ZeroTrie, an efficient string-to-int collection #2722

sffc · 2022-10-05T23:46:43Z

Docs:
http://icu4x-pr-artifacts.storage.googleapis.com/gha/37107e708359b7549553b91a2f5481e2fc757831/docs/zerotrie/index.html

Manishearth · 2022-10-06T04:25:33Z

Haven't reviewed but it's cool this can live in a separate crate. (In retrospect, might have made sense to do the same for ZeroMap)

sffc · 2022-10-06T04:43:16Z

Haven't reviewed but it's cool this can live in a separate crate. (In retrospect, might have made sense to do the same for ZeroMap)

I was trying to decide between separate crate, icu_collections, or zerovec, so I started in a separate crate to defer that decision.

sffc · 2022-10-07T17:52:42Z

Since this isn't reviewed yet, I'll make it draft and keep pushing things to the branch. I wrote up a builder implementation that I want to polish a bit more.

sffc · 2022-11-21T02:03:53Z

With a bigger data set:

get/subtags/AsciiTrie   time:   [3.0731 us 3.0743 us 3.0761 us]
get/subtags/AsciiTrie2  time:   [7.6229 us 7.6264 us 7.6301 us]
get/subtags/ZeroMap/usize
                        time:   [6.6433 us 6.6484 us 6.6544 us]
get/subtags/ZeroMap/u8  time:   [5.2245 us 5.2279 us 5.2321 us]
get/subtags/HashMap     time:   [2.1586 us 2.1623 us 2.1665 us]

AsciiTrie2 is modestly smaller in both code and data than AsciiTrie, but it performs more than 2x worse. I don't know exactly why, because they should have the same number of operations. AsciiTrie is probably more cache-friendly.

sffc · 2022-12-05T16:54:20Z

From discussion with @markusicu: try an AsciiTrie3 with binary instead of trinary.

sffc · 2022-12-17T22:58:01Z

AsciiTrie3 is not better:

get/subtags/AsciiTrie   time:   [3.1154 us 3.1258 us 3.1356 us]
get/subtags/AsciiTrie2  time:   [7.4181 us 7.4235 us 7.4300 us]
get/subtags/AsciiTrie3  time:   [8.0013 us 8.0066 us 8.0128 us]
get/subtags/ZeroMap/usize
                        time:   [6.7374 us 6.7466 us 6.7585 us]
get/subtags/ZeroMap/u8  time:   [5.2130 us 5.2145 us 5.2161 us]

Last thing I'll try is a different varint encoding that puts the length in the first byte (more like UTF-8).

sffc · 2022-12-17T23:37:22Z

My initial measurement has the new varint about 10% slower across the board (since all three implementations use it).

Edit: I verified that the varint implementation is behaving correctly, so the 10% slower measurements seem accurate.

sffc · 2022-12-19T12:16:37Z

Size comparison of the first_weekday_for_region.rs example (total code size of the main function with everything inlined) and the short_subtags data set in the PR:

Impl	Code Size	Data Size
AsciiTrie	0x19A	9204
AsciiTrie2	0x172	8424
AsciiTrie3	0x146	9289
ZeroHashMap	?	ca. 15173 B*

* Assumes strings stored in a VarZeroVec with 2 bytes of hash metadata per string

sffc · 2022-12-19T12:45:06Z

Current conclusions:

AsciiTrie is faster than either AsciiTrie2 or AsciiTrie3, and its performance is competitive with a hash map on small to medium sized data sets. Overall I lean toward AsciiTrie.
AsciiTrie2 is smaller in both code size (40 B) and data size (780 B) than AsciiTrie, and the data could likely be made even smaller with branch pruning, but its performance is not competitive with other data structures.
AsciiTrie3 is smaller in code size (44 B) than AsciiTrie2, but it loses the data size advantage over AsciiTrie. It performs slightly worse than AsciiTrie2. I see no compelling reason to choose AsciiTrie3.
All three are most likely smaller in data size than ZeroHashMap. I do not yet have speed or code size comparisons.
AsciiTrie2 and AsciiTrie3 have some properties that you don't get with AsciiTrie; I'm not sure if these properties are important:
1. A set of values is attainable from a simple linear walk forward over the buffer
2. Random indexing (and adjusting for varint) will land you in a valid point in the trie

WDYT @zbraniecki @robertbastian @markusicu @echeran? Here are my potential paths forward on this PR:

Pick one of the implementations (most likely AsciiTrie) and make it into something reviewable
Pick multiple implementations and make them into something reviewable
Close this PR as a fun thought experiment but nothing we want to actually use in ICU4X
Do more benchmarks before proceeding. If you recommend this, which benchmarks would you like to see?
1. ZeroHashMap speed and code size
2. BytesTrie, which I would port to Rust, on all metrics
Explore another trie implementation (AsciiTrie4). What would you like to see that I haven't tried yet?

robertbastian · 2022-12-20T17:32:26Z

With the direction we're going in #2885 we need key structures (sorted array, perfect hash, ascii trie) that are simple maps from &str to usize. Can you benchmark this against binary search on &[&str] or VarZeroVec<str>? We don't need a ZeroMap for this.

sffc · 2022-12-20T18:15:56Z

The ZeroMap benchmark is basically a binary search over strings, but I can add another one that does a string slice and VZV directly. The usize value in those benches would be understood to be the index of the element.

sffc · 2023-01-16T01:30:16Z

I'm planning to revisit this PR once ZeroHashMap (#2579) is merged so that I can add it to the benchmarks.

sffc · 2023-02-24T04:41:04Z

I added the ZeroHashMap benches.

Small bench:

Benchmarking get/basic/AsciiTrie: Collecting 100 samples in estimated 5.0001 s (48M ite                                                                                       get/basic/AsciiTrie     time:   [104.34 ns 104.70 ns 105.11 ns]
                        change: [-11.477% -11.201% -10.945%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 25 outliers among 100 measurements (25.00%)
  7 (7.00%) low severe
  7 (7.00%) low mild
  6 (6.00%) high mild
  5 (5.00%) high severe
Benchmarking get/basic/AsciiTrie2: Collecting 100 samples in estimated 5.0002 s (33M it                                                                                       get/basic/AsciiTrie2    time:   [149.02 ns 149.53 ns 150.09 ns]
                        change: [-0.1683% +0.1942% +0.5685%] (p = 0.31 > 0.05)
                        No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe
Benchmarking get/basic/AsciiTrie3: Collecting 100 samples in estimated 5.0000 s (34M it                                                                                       get/basic/AsciiTrie3    time:   [146.90 ns 147.27 ns 147.62 ns]
                        change: [-0.4641% -0.1158% +0.2224%] (p = 0.52 > 0.05)
                        No change in performance detected.
Found 18 outliers among 100 measurements (18.00%)
  15 (15.00%) low mild
  1 (1.00%) high mild
  2 (2.00%) high severe
Benchmarking get/basic/ZeroMap/usize: Collecting 100 samples in estimated 5.0005 s (44M                                                                                       get/basic/ZeroMap/usize time:   [113.27 ns 113.57 ns 113.86 ns]
                        change: [-0.8415% -0.5492% -0.2367%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 22 outliers among 100 measurements (22.00%)
  7 (7.00%) low severe
  10 (10.00%) low mild
  2 (2.00%) high mild
  3 (3.00%) high severe
Benchmarking get/basic/ZeroMap/u8: Collecting 100 samples in estimated 5.0003 s (52M it                                                                                       get/basic/ZeroMap/u8    time:   [95.369 ns 95.625 ns 95.889 ns]
                        change: [-1.2355% -0.5995% -0.0770%] (p = 0.03 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
Benchmarking get/basic/HashMap: Collecting 100 samples in estimated 5.0001 s (59M itera                                                                                       get/basic/HashMap       time:   [85.354 ns 86.072 ns 87.147 ns]
                        change: [+0.3076% +7.4054% +18.870%] (p = 0.08 > 0.05)
                        No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe
Benchmarking get/basic/ZeroHashMap/usize: Collecting 100 samples in estimated 5.0002 s                                                                                        get/basic/ZeroHashMap/usize
                        time:   [111.27 ns 111.55 ns 111.82 ns]
                        change: [+2.3400% +2.6571% +2.9320%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 21 outliers among 100 measurements (21.00%)
  1 (1.00%) low severe
  16 (16.00%) low mild
  4 (4.00%) high mild
Benchmarking get/basic/ZeroHashMap/u8: Collecting 100 samples in estimated 5.0001 s (56                                                                                       get/basic/ZeroHashMap/u8
                        time:   [88.323 ns 89.146 ns 90.685 ns]
                        change: [+5.9094% +6.8107% +7.8053%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

Medium bench:

Benchmarking get/subtags/AsciiTrie: Collecting 100 samples in estimated 5.0152 s (1.3M                                                                                        get/subtags/AsciiTrie   time:   [3.6910 µs 3.7000 µs 3.7086 µs]
                        change: [-1.5978% -1.2175% -0.8624%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 27 outliers among 100 measurements (27.00%)
  10 (10.00%) low severe
  12 (12.00%) low mild
  3 (3.00%) high mild
  2 (2.00%) high severe
Benchmarking get/subtags/AsciiTrie2: Collecting 100 samples in estimated 5.0406 s (465k                                                                                       get/subtags/AsciiTrie2  time:   [10.795 µs 10.834 µs 10.872 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
Benchmarking get/subtags/AsciiTrie3: Collecting 100 samples in estimated 5.0089 s (460k                                                                                       get/subtags/AsciiTrie3  time:   [10.911 µs 10.947 µs 10.983 µs]
Found 24 outliers among 100 measurements (24.00%)
  4 (4.00%) low severe
  11 (11.00%) low mild
  7 (7.00%) high mild
  2 (2.00%) high severe
Benchmarking get/subtags/ZeroMap/usize: Collecting 100 samples in estimated 5.0142 s (7                                                                                       get/subtags/ZeroMap/usize
                        time:   [6.8100 µs 6.8416 µs 6.8770 µs]
Found 14 outliers among 100 measurements (14.00%)
  6 (6.00%) low mild
  7 (7.00%) high mild
  1 (1.00%) high severe
Benchmarking get/subtags/ZeroMap/u8: Collecting 100 samples in estimated 5.0191 s (954k                                                                                       get/subtags/ZeroMap/u8  time:   [5.2495 µs 5.2654 µs 5.2813 µs]
Found 25 outliers among 100 measurements (25.00%)
  20 (20.00%) low mild
  1 (1.00%) high mild
  4 (4.00%) high severe
Benchmarking get/subtags/HashMap: Collecting 100 samples in estimated 5.0097 s (2.2M it                                                                                       get/subtags/HashMap     time:   [2.3302 µs 2.3380 µs 2.3457 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
Benchmarking get/subtags/ZeroHashMap/usize: Collecting 100 samples in estimated 5.0110                                                                                        get/subtags/ZeroHashMap/usize
                        time:   [3.2209 µs 3.2286 µs 3.2360 µs]
Found 28 outliers among 100 measurements (28.00%)
  17 (17.00%) low severe
  4 (4.00%) low mild
  1 (1.00%) high mild
  6 (6.00%) high severe
Benchmarking get/subtags/ZeroHashMap/u8: Collecting 100 samples in estimated 5.0032 s (                                                                                       get/subtags/ZeroHashMap/u8
                        time:   [2.5595 µs 2.5658 µs 2.5717 µs]
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) low mild
  1 (1.00%) high mild

Large bench (new):

Benchmarking get/subtags_full/AsciiTrie: Collecting 100 samples in estimated 5.2934 s (                                                                                       get/subtags_full/AsciiTrie
                        time:   [81.655 µs 81.892 µs 82.127 µs]
Found 12 outliers among 100 measurements (12.00%)
  9 (9.00%) low mild
  2 (2.00%) high mild
  1 (1.00%) high severe
Benchmarking get/subtags_full/AsciiTrie2: Collecting 100 samples in estimated 5.4004 s                                                                                        get/subtags_full/AsciiTrie2
                        time:   [177.85 µs 178.43 µs 179.05 µs]
Found 12 outliers among 100 measurements (12.00%)
  10 (10.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe
Benchmarking get/subtags_full/AsciiTrie3: Collecting 100 samples in estimated 5.0660 s                                                                                        get/subtags_full/AsciiTrie3
                        time:   [167.08 µs 167.55 µs 168.03 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
Benchmarking get/subtags_full/ZeroMap/usize: Collecting 100 samples in estimated 5.0982                                                                                       get/subtags_full/ZeroMap/usize
                        time:   [126.04 µs 126.37 µs 126.70 µs]
Found 19 outliers among 100 measurements (19.00%)
  8 (8.00%) low severe
  7 (7.00%) low mild
  2 (2.00%) high mild
  2 (2.00%) high severe
Benchmarking get/subtags_full/ZeroMap/u8: Collecting 100 samples in estimated 5.0934 s                                                                                        get/subtags_full/ZeroMap/u8
                        time:   [112.43 µs 112.99 µs 113.66 µs]
Found 26 outliers among 100 measurements (26.00%)
  17 (17.00%) low mild
  2 (2.00%) high mild
  7 (7.00%) high severe
Benchmarking get/subtags_full/HashMap: Collecting 100 samples in estimated 5.0237 s (21                                                                                       get/subtags_full/HashMap
                        time:   [23.646 µs 23.729 µs 23.812 µs]
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) low mild
  1 (1.00%) high mild
Benchmarking get/subtags_full/ZeroHashMap/usize: Collecting 100 samples in estimated 5.                                                                                       get/subtags_full/ZeroHashMap/usize
                        time:   [33.772 µs 34.098 µs 34.575 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
Benchmarking get/subtags_full/ZeroHashMap/u8: Collecting 100 samples in estimated 5.122                                                                                       get/subtags_full/ZeroHashMap/u8
                        time:   [25.991 µs 26.054 µs 26.113 µs]
Found 24 outliers among 100 measurements (24.00%)
  13 (13.00%) low severe
  4 (4.00%) low mild
  4 (4.00%) high mild
  3 (3.00%) high severe

Summary:

Implementation	Small Data*	Small Perf	Medium Data	Medium Perf	Large Data	Large Perf
AsciiTrie	40 B	104 ns	1077 B	3700 ns	9204 B	81892 ns
AsciiTrie2	32 B	149 ns	1017 B	10834 ns	8424 B	178430 ns
AsciiTrie3	33 B	147 ns	1110 B	10947 ns	9289 B	167550 ns
ZeroMap<usize>	75 B	113 ns	1331 B	6841 ns	15182 B	126370 ns
ZeroMap<u8>	65 B	95 ns	1330 B	5265 ns	13304 B	112990 ns
HashMap	N/A	86 ns	N/A	2338 ns	N/A	23729 ns
ZeroHashMap<usize>	148 B	111 ns	2837 B	3228 ns	30200 B	34098 ns
ZeroHashMap<u8>	138 B	89 ns	2836 B	2571 ns	28322 B	26054 ns

* The small data and small perf are both small, but they are not on the same data set

Small = 7 elements
Medium = 188 elements
Large = 1877 elements

sffc · 2023-02-24T05:39:56Z

sffc · 2023-02-24T05:44:36Z

sffc · 2023-02-24T05:46:59Z

In code, the displacements are (u32, u32). We decided that we could most likely go less than that, right? That would account for ZeroHashMap's data size being so much greater than the rest.

sffc · 2023-06-28T11:16:04Z

Docs: http://icu4x-pr-artifacts.storage.googleapis.com/gha/37107e708359b7549553b91a2f5481e2fc757831/docs/zerotrie/index.html

experimental/zerotrie/Cargo.toml

Manishearth · 2023-06-28T14:40:39Z

experimental/zerotrie/src/byte_phf/mod.rs

+#[cfg(feature = "alloc")]
+pub use cached_owned::PerfectByteHashMapCacheOwned;
+
+use ref_cast::RefCast;


issue: imo we should do this manually and not include the extra dep

(we should also consider making this a utility exported by zerovec_derive since zerovec situations need it a lot

experimental/zerotrie/examples/byteph.rs

experimental/zerotrie/src/lib.rs

Manishearth · 2023-06-28T14:47:07Z

experimental/zerotrie/src/zerotrie.rs

+pub struct ZeroTrie<S>(pub(crate) ZeroTrieInner<S>);
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum ZeroTrieInner<S> {


suggestion: such data structure enums are often called flavors, if you're looking for a better name for this (ZeroTrieFlavor)

experimental/zerotrie/src/zerotrie.rs

Manishearth · 2023-06-28T14:59:42Z

experimental/zerotrie/src/zerotrie.rs

+    ExtendedCapacity(ZeroTrieExtendedCapacity<S>),
+}
+
+/// A data structure that compactly maps from ASCII strings to integers.


issue: docs pertaining to the specific variant (for all three)

Can you clarify? The docs for this one say "ASCII strings to integers" which is correct. I added a link back to ZeroTrie docs.

(specifically about the format of each and the general format)

I think the client-facing docs are okay, and the internal files now have fairly extensive docs about the internal difference between these types. Let me know where there might still lack clarity.

experimental/zerotrie/src/zerotrie.rs

sffc · 2023-07-16T22:15:59Z

I wrote improved docs for just about everything now, public and private; you can view them either in source or rendered nicely with:

cargo doc --document-private-items --all-features --no-deps --open

I think I fixed or responded to all the feedbacks so far.

sffc · 2023-07-16T22:34:44Z

If it helps, here's what I consider to be the main building blocks:

byte_phf is a standalone module that computes a PHF given the constraint that there are at most 256 classes. It has its own methods for doing so, loosely inspired by ZeroHashMap.
reader now contains docs on the layout of the ZeroTrie.
builder::konst and builder::nonconst are designed to be fairly close clones of each other, but one works in const mode and the other does not. The actual builder algorithm loop might be a bit hard to follow and I don't know how best to explain it in commentary, but I'm happy to go over it on the whiteboard some day.
The rest of the code in builder are helpers and utilities that have minimal insightful value.
varint is a little interesting. I discussed it at some length with Manish and now it has better docs.
The serde impls are also a little interesting. Take note at how I distinguish a ZeroTrie from a ZeroTrieAsciiOnly and a ZeroTriePerfectHash.

robertbastian · 2023-07-17T13:04:33Z

I'll defer to Manish for the detailed logic reviews

sffc · 2023-07-17T13:56:56Z

I went ahead and did my best to write docs for how the builder works.

Manishearth

Reviewed followup commits 1d55557 .. 1ccb26a

Manishearth · 2023-07-25T17:40:03Z

experimental/zerotrie/src/builder/mod.rs

+//!
+//! When a node is prepended, it is shown in **boldface**.
+//!
+//! 1. Initialize the builder by setting `i=3`, `j=4`, `prefix_len=5` (the last string),


issue: i don't think i and j have been sufficiently explained for this to be obvious, could you explain them more above?

Any suggestions on how I can be more clear than

//! - `prefix_len` indicates the byte index we are currently processing. //! - `i` and `j` bracket a window of strings in the input that share the same prefix.

It is extremely unclear as to what "bracket a window of strings" means.

Examples scoped to explaining what they are may help. For example, here you can explicitly say "the only string sharing the same prefix here is adhgk, so i and j bracket it and you get 3..4

Manishearth · 2023-07-25T18:14:05Z

experimental/zerotrie/src/reader.rs

+pub fn get_bsearch_only(mut trie: &[u8], mut ascii: &[u8]) -> Option<usize> {
+    loop {
+        let (b, x, i, search);
+        (b, trie) = trie.split_first()?;


Mostly from a correctness standpoint: this code is somewhat stateful when it comes to whether the current cursor in the trie is itself a valid trie, and it seems like a useful distinction to maintain in code. But not blocking, like i said.

Manishearth

I have not carefully reviewed the builder aside from the docs. I hope to get to it this week, but I am comfortable having this land as-is.

srl295 · 2023-08-21T16:51:48Z

a new license file, checker broke when #3875 merged

- feature added in #3875, other license files in #2722 - when #2722 was merged, #3875 broke

sffc requested a review from a team as a code owner October 5, 2022 23:46

sffc marked this pull request as draft October 7, 2022 17:52

sffc mentioned this pull request Nov 10, 2022

Add get_indexed_range to LiteMap #2730

Closed

sffc added the needs-approval One or more stakeholders need to approve proposal label Dec 19, 2022

sffc removed the needs-approval One or more stakeholders need to approve proposal label Jan 19, 2023

sffc added the discuss-priority Discuss at the next ICU4X meeting label Feb 24, 2023

sffc removed the discuss-priority Discuss at the next ICU4X meeting label Mar 16, 2023

sffc changed the title ~~Experimental AsciiTrie implementation~~ Add ZeroTrie, an efficient string-to-int collection Jun 27, 2023

sffc marked this pull request as ready for review June 28, 2023 11:15

sffc requested a review from Manishearth as a code owner June 28, 2023 11:15

sffc mentioned this pull request Jun 28, 2023

Add IANA/BCP47 time zone name mappings #3499

Closed

Manishearth reviewed Jun 28, 2023

View reviewed changes

sffc added 7 commits July 16, 2023 17:43

f2 docs

2395d30

More docs on builder utilities

9ab08a6

Some more builder docs

7847652

More code review comments

1a7ac1a

Rob feedback

4973911

Merge branch 'main' into asciitrie

8df4756

Fix criterion dependency

1ccb26a

sffc requested review from Manishearth and robertbastian July 16, 2023 22:13

fmt & tidy

c687cd8

robertbastian removed their request for review July 17, 2023 13:04

Write docs for the builder and simplify it slightly.

3b415bd

Manishearth reviewed Jul 25, 2023

View reviewed changes

Manishearth approved these changes Jul 25, 2023

View reviewed changes

robertbastian closed this Aug 8, 2023

robertbastian reopened this Aug 8, 2023

sffc merged commit ca9f30c into unicode-org:main Aug 18, 2023
46 of 48 checks passed

sffc deleted the asciitrie branch August 18, 2023 19:58

sffc mentioned this pull request Aug 18, 2023

ZeroTrie follow-ups #3894

Open

4 tasks

srl295 added a commit that referenced this pull request Aug 21, 2023

chore(license): license fix for experimental/zerotrie

1a74399

- feature added in #3875, other license files in #2722 - when #2722 was merged, #3875 broke

srl295 mentioned this pull request Aug 21, 2023

chore(license): license fix for experimental/zerotrie #3903

Merged

sffc pushed a commit that referenced this pull request Aug 21, 2023

chore(license): license fix for experimental/zerotrie (#3903)

b48ae76

- feature added in #3875, other license files in #2722 - when #2722 was merged, #3875 broke

robertbastian mentioned this pull request Sep 11, 2023

ZeroTrie: change map_store to pub convert_store #4022

Merged

Manishearth mentioned this pull request Sep 21, 2023

1.3 utils release list #4066

Closed

13 tasks

sffc mentioned this pull request Oct 24, 2023

Speed up resource lookup and fallback at runtime #2699

Open

Add ZeroTrie, an efficient string-to-int collection #2722

Add ZeroTrie, an efficient string-to-int collection #2722

Conversation

sffc commented Oct 5, 2022 • edited

Manishearth commented Oct 6, 2022

sffc commented Oct 6, 2022

sffc commented Oct 7, 2022

sffc commented Nov 21, 2022

sffc commented Dec 5, 2022

sffc commented Dec 17, 2022

sffc commented Dec 17, 2022 • edited

sffc commented Dec 19, 2022 • edited

sffc commented Dec 19, 2022 • edited

robertbastian commented Dec 20, 2022

sffc commented Dec 20, 2022

sffc commented Jan 16, 2023

sffc commented Feb 24, 2023 • edited

sffc commented Feb 24, 2023

sffc commented Feb 24, 2023

sffc commented Feb 24, 2023

sffc commented Jun 28, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sffc commented Jul 16, 2023

sffc commented Jul 16, 2023

robertbastian commented Jul 17, 2023

sffc commented Jul 17, 2023

Manishearth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Manishearth left a comment

Choose a reason for hiding this comment

srl295 commented Aug 21, 2023

sffc commented Oct 5, 2022 •

edited

sffc commented Dec 17, 2022 •

edited

sffc commented Dec 19, 2022 •

edited

sffc commented Dec 19, 2022 •

edited

sffc commented Feb 24, 2023 •

edited