Planned Changes to DID Documents (August 2023)

[bnewbold]

We're planning to roll out some changes to DID Documents as they are used in atproto, in particular the verificationMethod syntax for public cryptographic keys. This change will impact:

• Any accounts using did:web with atproto: you'll need to update the well-known DID document at some point during the transition period
• Any services, clients, or developers working directly with DID Documents in atproto: you'll need to support both formats during a transition that will start soon-ish

If you're only working with identity (DIDs and handles) via the PDS API endpoints or the repo event stream, and not verifying identities locally via direct DID resolution, you shouldn't need to do anything.

No internal changes to did:plc operation logs or atproto repositories is required.

Timeline

• soon: publish release of Typescript @atproto/identity library which supports both DID document formats
• give folks a few days (TBD) to make any client-side changes
• switch over did:plc directory to new format, first in sandbox then in production
• transition period, during which we encourage anybody with a legacy-style did:web document to update that doc
• after some transition time (TBD), remove support for the legacy format in our libraries, the specification, and our production services

The Changes

The two changes to verificationMethod objects are to:

• include the full DID identifier followed by an #atproto fragment in the id field (instead of just the fragment). This is only required for verificationMethod, not for service objects, per the DID Core specification.
• switch to the Multikey type, instead of per-key-type, and use the correct publicKeyMultibase encoding for that type (still base58btc multibase; but include multicodec to indicate key type; and use "compressed" bytes for P-256 and K-256 keys)

It is good practice to support full DID identifiers in the service id field as well, though we will continue to produce the terser fragment-only format for now (as allowed by the specification). Eg, check if the id field ends with #atproto_pds instead of if it equals #atproto_pds.

The current format for publicKeyMultibase uses un-compressed bytes (which is why the strings are so long), and do not include a multicodec type indicator, relying on the type field to disambiguate. This is the only place in the entire atproto protocol that this encoding is used, and it is confusingly similar to the new format, so removing it should remove a major footgun for developers. The new encoding matches that of did:key encoding (that is, the part following the did:key: prefix).

The atproto specifications haven't been updated yet, but you can preview the changes here:

• specs: update DID document and crypto key encoding atproto-website#178
• https://bnewbold-did-doc-iteration.atproto-website.pages.dev/specs/did#did-documents
• https://bnewbold-did-doc-iteration.atproto-website.pages.dev/specs/cryptography#public-key-encoding

Examples

Here is the current DID document for the @atproto.com account:

{
  "@context": [
    "https://www.w3.org/ns/did/v1",
    "https://w3id.org/security/suites/secp256k1-2019/v1"
  ],
  "id": "did:plc:ewvi7nxzyoun6zhxrhs64oiz",
  "alsoKnownAs": [
    "at://atproto.com"
  ],
  "verificationMethod": [
    {
      "id": "#atproto",
      "type": "EcdsaSecp256k1VerificationKey2019",
      "controller": "did:plc:ewvi7nxzyoun6zhxrhs64oiz",
      "publicKeyMultibase": "zQYEBzXeuTM9UR3rfvNag6L3RNAs5pQZyYPsomTsgQhsxLdEgCrPTLgFna8yqCnxPpNT7DBk6Ym3dgPKNu86vt9GR"
    }
  ],
  "service": [
    {
      "id": "#atproto_pds",
      "type": "AtprotoPersonalDataServer",
      "serviceEndpoint": "https://bsky.social"
    }
  ]
}

and here is what it would look like after these changes:

  "@context": [
    "https://www.w3.org/ns/did/v1",
    "https://w3id.org/security/multikey/v1",
    "https://w3id.org/security/suites/secp256k1-2019/v1"
  ],
  "id": "did:plc:ewvi7nxzyoun6zhxrhs64oiz",
  "alsoKnownAs": [
    "at://atproto.com"
  ],
  "verificationMethod": [
    {
      "id": "did:plc:ewvi7nxzyoun6zhxrhs64oiz#atproto",
      "type": "Multikey",
      "controller": "did:plc:ewvi7nxzyoun6zhxrhs64oiz",
      "publicKeyMultibase": "zQ3shXjHeiBuRCKmM36cuYnm7YEMzhGnCmCyW92sRJ9pribSF"
    }
  ],
  "service": [
    {
      "id": "#atproto_pds",
      "type": "AtprotoPersonalDataServer",
      "serviceEndpoint": "https://bsky.social"
    }
  ]
}

[snarfed]

Hell yes! thank god, glad to hear it. The old uncompressed raw bytes encoding for publicKeyMultibase was (arguably) underspecified and hard to generate. I was able to do it with pycryptodome, but that doesn't support low-S sigs, so I had to switch to pyca/cryptography, and I was struggling just this morning to figure out how to get raw (non-DER etc) public key bytes out of it and use them to generate publicKeyMultibase. I didn't find a way, so I'm very glad I don't have to worry about it now.

[bnewbold]

We are going to take our time with this roll out to ensure it isn't as disruptive as the recent repo v3.

A week or so before making any large change in prod we will:

• announce a timeline here and in dev comms channels, with notes on expected impact and how to upgrade
• published compatible versions of packages in Typescript and golang
• publish written specs
• deploy everything to sandbox for folks to test against
• reach out to ecosystem devs and projects who might be impacted

Towards that final point, going to tag a couple folks as an early heads up if they aren't aware of this change (not trying to promote/exclude anybody; and sorry if this is spammy): @rudyfraser @redsolver @MarshalX @mozzius @jcsalterego @videah @myConsciousness @DavidBuchanan314 @jessejanderson @Amazingca @MattStypa

[bnewbold]

We are ready to move ahead with this change. Plan is to deploy this today in sandbox, then next next week in production (plc.directory).

[bnewbold]

As some more context and details:

• backwards compatibility is possible and expected: clients can support old and new syntax at the same time. there is no change to underlying PLC operation data (aka, that which gets hashed and signed)
• compatible atproto libraries for Typescript and golang were released more than a week ago, and services like the PDS and feedgenerator template have been updated to use them. atproto.com specs have been updated.
• this change will impact code or services doing DID identity parsing and verification. in theory this could be any/all services (feed generators, client, etc). In practice, many clients rely on the PDS for identity processing.
• if the PLC roll-out in any environment causes painful disruption, we can roll back the deployment safely

[bnewbold]

This change was deployed in prod today.

[mackuba]

Do I need to update anything in the did:web: document for the feed generator service if it looks like this (basically cloned from the JavaScript template repo) https://blue.mackuba.eu/.well-known/did.json ?

[dholms]

Getting a Forbidden when I try to load that^^

But nope you should be good 👍

Altho if you verify auth tokens for your feed generator, you'll need to ensure that you're using the most recent version of the @atproto/identity package - which has been updating in main of the feed-generator repo

[mackuba]

Hmm it should be working 🤨

[bnewbold]

did:web docs don't need to be updated with any urgency: clients should continue to support both syntaxes (syntax? syntaxae? you know what I mean). But in the medium-term it would be good to get them updated so we can fully drop that confusing old syntax.

I would wait until the PLC cut-over lands and dust settles, otherwise your single account might have trouble.

We should be able to run through and enumerate all did:web docs we've seen, check them, and poke folks to update, before any deprecation.

[dead10ck]

Hi, I seem to be affected by this. My DID is did:web:dead10ck.dev. I got a push notification from the app that said I was tagged in a post related to this change, but when I try to log in, I just see an error message about the jwt signature not matching the issuer.

I was able to find this page, but it's unclear to me how to fix my public key. The docs say that the format has been changed to Multikey and that it needs to use the compressed representation, but I have no clue how to perform this encoding.

I tried just fixing the format of the DID to include the handle in front of the #atproto fragment and fix the type field to see if it would work with the same previous public key format, but now the app's error banner says there's an internal server error.

I don't really understand why any valid multibase encoding wouldn't work, as it's my impression that that's kind of the whole point of multibase, but it also seems to be such a niche format that I can't find much in the way of tooling to perform the encoding for me.

This could have been communicated better. Not everyone is going to follow the GitHub page for breaking changes like this. I'm not a prospective API developer, I'm just a user who wanted to set up did:web.

[yamarten]

The real problem is that your repository has moved to a new PDS, not a key format. Please check #1832 for details.
You need to update not only publicKeyMultibase but also serviceEndpoint, and both new values are listed in #1874.

[bnewbold]

As @yamarten notes, the core issue is the migration of PDS location and new signing key, not the format. The legacy format is still supported for now, though updating to the new format at the same time would be a good idea.

The previous multibase encoding was actually incorrect and off-spec, which is why we made the change. Sorry about the disruption!

It is true that use of did:web is not as smooth an experience as did:plc right now, because the PDS can not pro-actively perform updates on behalf of accounts. We will try to do better to make this less disruptive in the future, and hear you about the lack of tooling to help with this.

[dead10ck]

Ahh, thanks a lot for the pointer @yamarten. @bnewbold I totally get making breaking changes, and I completely understand that by making a did:web the onus of fixing breaking changes is up to me; I made that choice and that's not a problem at all. But I should not find out that this breaking change was happening by suddenly being unable to log into my account and having to go looking in GitHub to figure out if it was intentional or not.