Read after write

[dholms]

This is a basic implementation of read stick on the pds which gives us read-after-write semantics for some operations.

The basic approach here is:

• add a optional logical clock to each commit in the repo
• alongside each record, notate the logical clock at which it was added/updated
• in appview responses, set a header (atproto-clock) which includes it's last seen logical clock for the requesting user
• after each proxied request to the appview, check which of the user's records, if any, have not made it into the appview's indexes
• if a relevant record has not made it in yet, add that data into the view locally

Importantly, while this includes application-specific logic, it does not include any application-specific indexes and may be done generically off of the record table.

The application of this read-sticky approach is not exhaustive. However, this applies it to some of the most common and impactful occurrences. Namely:

• profiles in getProfile
• profiles in getProfiles
• posts in getPostThread
• posts in getTimeline
• posts and profiles in getAuthorFeed

Note: this currently builds a PDS url for any local images that get added in, however after landing #1248, we can deterministically build appview image urls

[bnewbold]

I was a bit worried we had specified ourself in to a corner, but thankfully:

Adding optional fields to commit and MST node objects may or may not result in a repo format version change.

@ericvolp12 had the idea to use rev or revision as the field name instead of clock. I'm fine with clock after having used it a bunch, but we might be in a bubble and rev might be clearer and be back-pressure against folks assuming the TID value is a safe, trusted, general-purpose timestamp for the commit (eg, if they aren't familiar with Lamport clocks).

[devinivy]

🤯 a couple thoughts/questions, but you nailed it.

[devinivy]

Spidey senses tingled a little seeing a loop over feed that also mutates feed, but this should be fine since it causes a break.

[dholms]

cleaned it up a bit with a findIndex instead 👌