Stable ID generator for off-protocol services

[43081j]

The stuff in #2584 basically summarises as "we need a way to identify non-atproto documents".

Given that the authority in such a situation is the service (e.g. GitHub), this means we need a function which basically computes a stable ID given a URL on said service.

This is necessary because we need a stable way to reference github repos, packages, and anything else from a service which isn't on atproto.

Implementation

in the case of GitHub, each repo already has an ID which doesn't change even if it gets transferred. So it'd look like this:

async function githubStableID(url: URL): Promise<string | null> {
  // do a bunch of stuff to check if this is a repo...
  if (!isRepo) {
    return null; // can't compute IDs for non-repos yet
  }
  const repoDoc = await somehowGetRepoFromGitHub(repoName);
  return repoDoc.id;
}

over time, we'd end up with a function like this for each "integration". For example, the npm one may just be a sync piece of logic that returns {name}@{version}.

we should make a function (here or in a new published library others can share) which essentially exports:

export function generateId(url: URL): Promise<string | null> {
  switch (somehowGetType(url)) {
    case 'github': return githubStableId(url);
    // ...
  }
}

[fatfingers23]

In the context of "what should the stable id saved to the atproto record look like":

So the subject/id should ideally not matter much. It's just a background thing, it shouldn't really matter to us what it really is other than it's unique and easy to work with.

The big constraint is it needs to be "url like" for constellation so https:// or madeup://. It won't be at:// since there will be a different property on those records that is the strongRef of an at:// uri to the record resource if there is one. For example if a user claims that package and wants to add a list of maintainers for it

The case for using a url like https://api.github.com/repositories/71995937
pros:

• Points to a real tangible resource
• Clearly belongs to the off protocol org since it's their url

cons:

• The url schema could change. doesn't really change what we use it for. just lose pro number 1 if it does.
• Can be a bit long in the tooth. In this instance we are calling that exact endpoint when we need to, but working with urls and parsing out info can be a bit awkward sometimes as we saw with the npmx.dev links for current lexicons.

The case for a made up protocol like pataksid://
pros:

• we control the schema. If we want to do a protocol for each off protocol resource we can or we can do it as part of the schema like pataksid://github/123. Also won't ever change
• It's shorter

cons:

• It's made up and should we be making our own scheme
• it doesn't really point to anything, I mean i guess it does if we decide it does. But back to con 1, it's a made up thing

[patak-cat]

Thanks for writing this down @fatfingers23! I'm leaning towards a custom protocol, and we could use the same for all centralized platforms for the library we talked about. So it is consistent and everybody knows what these are. The library could have a function to convert these into a URL, and that will always work as we can update the library if the URL scheme ever changes. So we can still get the benefits of a valid URL scheme by using the lib in the middle.

It could make things easier to work with in a standard way, like:

ct://{platform}/{resource-type}/{stable-id}

With something like ct:// as centralized transport. For the {platform}, @alexdln was suggesting we don't use domains for things like github.com, but instead that we have a table like github.com -> github, so it would be something like ct://github/repository/{id} and ct://npm/package/@vitejs/plugin-react (in case we don't find an id in the npm)