New account types, starting with ActivityPub

[snarfed]

Right now, you can set up your (indie)web site with BF and federate it into the fediverse. We should also support the other direction, ie sign up with your fediverse (ActivityPub) account and federate it into the (indie)web.

Big project! It's the obvious next step toward making BF truly multi-protocol, though, and avoids conflating the multi-protocol work with the work to add a new protocol.

• User data model
  • ... Or do we always need a User since we'll need keys for sending response activities, eg Accept?
  • are web external users special? Since they have their own domain that we don't control?
  • should we use per-protocol User subclasses? Pros, cons?
  • If we don't, how do we avoid User key id collisions across protocols, eg if more than one uses DNS domain as identity?
  • create external Users on demand, when they interact with us!
  • migrate User.actor_as2 to Object #547
• Object data model
  • generalize domains to users or something similar?
• Follower data model
  • generalize key from [from] [to] URLs and @-@s to protocol-independent. maybe just integer id and User key properties for from and to?
  • migrate last_follow to Object?
• UI
  • signup: is there any? or do we just create a new User the first time an AP user interacts with us. If so, what's that logic? Create one if the web site/receiving user doesn't have a User yet?
  • user pages
    • including URLs. Maybe go the Bridgy route and use /[protocol]/[handle], eg /web/snarfed.org?
    • add these new URL prefixes to robots.txt
    • allow URL path to be different from key id. Eg for AP users, key id is actor id URL, but ideally user page path would be @-@
• new per-protocol subdomains to let us infer desired destination user protocol for requests
  • update protocol translation tables
  • DNS records, LE certs, hooked up to GAE
  • infer protocol from subdomain in Protocol.receive etc
  • move dest protocol label in /convert/ etc from URL path to subdomain
  • move protocol label in /[domain] actor endpoint from URL path to subdomain
• how to map incoming targets to protocols? #548
• Webfinger for external users (sites)
  • fetch external h-cards on demand
  • move into activitypub.py?
  • generalize from Web to Protocol
  • support OGP, Twitter cards, meta tags, etc for Web users #551
• /r/... redirects for external users
  • home pages
  • posts
  • make them discoverable when they're plugged into fediverse (eg Mastodon) search. (they currently serve AS2 via conneg, and the default HTML response includes a rel-alternate link with type="application/activity+json", so I dunno what's missing.)
  • move into web.py?
• serve AP users and posts as HTML
  • how do these HTTP serving endpoints interact with protocol abstraction?
  • does redirect.py need to be split up across protocols?
• AP => wm logic, data conversions, tests
• finish unifying webmention receiving logic into Protocol.receive #529
• Atom/RSS feeds as alternative to Webmention trigger #550

Questions:

• How do we represent external (ie not signed up) sites?
  • Do we generate Users, user pages in the UI, etc for them? Probably not?
  • What are they in AS2? Eg if https://example.com/ is signed up, their AS2 actor id is https://fed.brid.gy/example.com . If they're not signed up, is it the same? Or is it https://fed.brid.gy/r/https://example.com ? Something else? Can't use the former if we want to support http sites. I assume we do?
• Maybe add a new User.signed_up property, to distinguish explicit users from implicit/external?
• Will we ever need a User for receiving side indirect/external users?

Decisions:

• Exernal sites' fediverse handles use fed.brid.gy as their instance, eg @example.com@fed.brid.gy. This one is basically decided for us since they won't be serving the .well-known redirects themselves.
• External sites use /r/ for their actor ids. I decided against auto-generating Users for them, I think the distinction vs users who have explicitly signed up is probably best, and useful.
• ...turns out we probably need full Users for external users too, since we need keys for them for AP and at least some other protocols like Nostr.
• Extended the current Protocol subclasses to be User subclasses too, like Bridgy classic.
  * No User.signed_up property to distinguish explicit users from implicit/external. We want to let people use the bridge without explicitly signing up.
• Added a new User.direct property to distinguish users who have explicit signed up or otherwise interacted with the bridge from "receiving" side users who haven't.domains
• Migrate both User.actor_as2 and Follower.last_follow to Object to share the fetching, conversion, etc logic.

[snarfed]

Hit a milestone here, AP users can discover and see arbitrary web sites in Mastodon via BF!

[snarfed]

🤩

[snarfed]

Running this right now to migrate Followers to the new schema with from, to, follow:

with appengine_config.ndb_client.context(), app.test_request_context(base_url='https://fed.brid.gy/'):
  for f in Follower.query():
    if not f.src or f.dest:
      f.dest, f.src = f.key.id().split()
    f.from_ = (ActivityPub if util.is_web(f.src) else Web).get_or_create(
      f.src, actor_as2=as1.get_object(f.last_follow or {}, 'actor')).key
    f.to = (ActivityPub if util.is_web(f.dest) else Web).get_or_create(
      f.dest, actor_as2=as1.get_object(f.last_follow or {})).key
    if f.last_follow:
      id = f.last_follow.get('id')
      if id:
        f.follow = Object.get_or_insert(id, as2=f.last_follow).key
    f.put()

[snarfed]

Basically done. UI is still TBD, filed #661 for that.