Clock Synchronisation

[DavidBuchanan314]

There are a lot of systems out there with inaccurate clock timestamps, either due to incremental drift over time (on the order of seconds or minutes), or by multiple hours, due to misconfigured timezones.

The net result of this is that many clients are inadvertently setting inaccurate createdAt timestamps in their records, among other headaches (e.g. bluesky-social/social-app#1064).

A native client can use NTP or similar, but a web client doesn't have that luxury.

Vague idea towards a solution: some sort of "NTP over websocket" protocol, that a client could use to accurately sync with the clock of a PDS.

[bnewbold]

Philosophically, my gut reaction is that rolling our own NTP-over-websocket is not a great idea. While cloud/VPS PDS instances are more likely to be synchronized than, eg, an old laptop or mobile phone, the blast radius of a poorly-synchronized PDS (eg, raspberry pi in a hot closet?) could be larger. We have a bunch of timestamps flying around the system (with more to come), and my intuition is that we need to ensure everything works reasonably with a couple minutes of drift.

Having a mechanism to detect clock drift could be helpful; maybe every response from the PDS could include a timestamp? Eg, httpbin.org sets the Date HTTP header with second precision in all GET responses. There is a tlsdate tool floating around which uses TLS connection metadata to a trusted HTTPS endpoint (could be PDS) to bootstrap a local clock. I guess the later isn't accessible to a web client.

[snarfed]

I get the issue of drift across PDSes in a heterogeneous, broad network. (And even BGSes and AppViews I guess, but probably to a lesser degree in practice.) I expect Mastodon has some experience reconciling that with timeline rendering, notification handling, etc. It'd be interesting to see what they've done.

I'm curious though, what was the background behind having clients set timestamps like createdAt instead of PDSes? That probably expands the amount and impact of drift a ton. PDSes are generally signing records with signing keys, unlike more client-centric protocols like eg Nostr, so I assume the motivation wasn't sigs...?

[bnewbold]

createdAt is part of the record, and posts are created using createRecord, which is lexicon/schema/application agnostic. We would have needed to create some kind of special carve-out to allow the PDS to reach in and modify the record before creation. We do actually do a bit of that today (I think $type gets auto-populated if it wasn't included, but that is in a fixed/known location in records), but try to avoid it to maintain clean abstraction layers.

In a lot of contexts, we just ignore things like createdAt, because they can't really be trusted from either of clients or PDS instances. The AppView keeps an indexedAt timestamp, and that should be coherent in that an AppView (or feedgen) is the party doing feed construction coherently. There are open questions around how to handle cold-start backfill and pauses/catchups in firehose indexing (populate indexedAt with createdAt?).

[snarfed]

Ah! Right, createdAt is indeed inside the record and lexicon-specific. Avoiding reaching inside makes sense.

It does still beg the question a bit of why to have client-provided timestamps at all. Here, maybe there wasn't a clear place to put protocol-level metadata like that, outside but alongside records. Implicitly demoting them like you say makes sense, esp given indexedAt available to AppView. Might eventually be worth formalizing that a bit? Not urgent, obviously.

[snarfed]

Realized one odd quirk here just now. If posts and similar records didn't have createdAt, and/or if clients didn't populate it in good faith, that would mean people couldn't post the same text more than once, since the record contents and therefore the CID would be the same, right? Funny if so.

[DavidBuchanan314]

You can still post the exact same record more than once, as long as each one has a unique MST rkey (and if they're TIDs, they will)

[snarfed]

Ah right, TIDs, true, thanks. Never mind. 😆