Basic implementation of "client to server" ActivityPub protocol

[yvolk]

Note on current state (as of 2020-05-24)

Pleroma

Minimal "client to server" #ActivityPub implementation is available. It is tested at https://queer.hacktivis.me/AndStatus (that site has the latest Pleroma builds... some requests work at other Pleroma instances also!) and can:

• authenticate using OAuth 2.0, read the Actor's profile...
• read timelines, including notes with attached images;
• post notes, including attached images.

Latest #AndStatus version is here: #456

All critical Pleroma problems fixed as I test developer's builds at https://queer.hacktivis.me/ , so "Basic ActivityPub C2S implementation" could be considered completed.

Mastodon

Mastodon (not AndStatus!) doesn't give you enough features to switch client connection to ActivityPub simply because Home timeline and posting via ActivityPub connection is not supported yet (corresponding pages give 404 Not found error).
But you can see your authenticated Actor's own posts on Mastodon servers and navigate to other Actors and their posts via them (you can see profiles of other Actors and see their posts...). You can even play with Mastodon as an "ActivityPub" type Social network in AndStatus today (I implemented a hack to log into Mastodon... not ActivityPub way unfortunately...)! But prepare to see empty Home timeline (due to above mentioned deficiency) and to be read-only there for now. Switch to "Sent" timeline and see your Actor's posts, see other actors and their posts similarly...
These new Mastodon improvements make me very optimistic about the future of ActivityPub in the client apps across all server platforms.

Description

As defined at https://www.w3.org/TR/activitypub/ , ActivityPub provides two layers:

• A server to server federation protocol (so decentralized websites can share information)
• A client to server protocol (so users, including real-world users, bots, and other automated processes, can communicate with ActivityPub using their accounts on servers, from a phone or desktop or web application or whatever)

The "ActivityPub test suite" (worked in 2018-2019... see on ActivityPub testing here) site contained a test suite that checked basic features that are required for a client app to implement #ActivityPub.

Below is a list of corresponding steps/features (for the start, copied from that site), which we are implementing in AndStatus, together with their implementation status:

AndStatus UI changes:

• Add "ActivityPub" type of Social network.

Client: Basic submission

• MUST Client discovers the URL of a user's outbox from their profile
• MUST Client submits activity by sending an HTTP post request to the outbox URL with the Content-Type of application/ld+json; profile="https://www.w3.org/ns/activitystreams"
• MUST Client submission request body is either a single Activity or a single non-Activity Object
• Client submission request is authenticated with the credentials of the user to whom the outbox belongs (this could be using an OAuth 2.0 bearer token or something else)
  Currently we reused OAuth 2.0 implementation that works in AndStatus for Mastodon.

Client: Required properties

• MUST Client provides the object property when submitting the following activity types to an outbox: Create, Update, Delete, Follow, Add, Remove, Like, Block, Undo.
• MUST Client provides the target property when submitting the following activity types to an outbox: Add, Remove.

Client: Add targets on reply
Reply to a post with multiple recipients.

• SHOULD Before submitting a new activity or object, Client infers appropriate target audience by recursively looking at certain properties (e.g. inReplyTo, See Section 7), and adds these targets to the new submission's audience. (The client suggests audience targeting based on participants in the referenced thread)
• SHOULD The client also limits depth of recursion used to gather targeting.

Client: Accept header on object retrieval
Trigger the client to retrieve some remote object.

• MUST When retrieving objects, Client specifies an Accept header with the application/ld+json; profile="https://www.w3.org/ns/activitystreams" media type.

[ned14]

Due to pressing need due to imminent end of Google Plus, I have been knocking together the world's simplest client-to-server ActivityPub service which exclusively works on static HTML website generators such as Jekyll and Hugo. No databases, no subscriptions, no followers, no complexity. All I want is that an Android app can post new blog posts, possibly with photos and/or location, to my static HTML website, like I currently can do with Google Plus.

What I have so far can be found at https://github.com/ned14/static-website-activitypub. It is still missing the inbox and outbox implementations (coming soon). Can I request that your checkbox list above can be broken into a simpler "initial support"?

What I would need:

• Client discovers the URL of a user's outbox from their profile via .well-know/webfinger?resource=acct:email@domain
• Client submits activity by sending an HTTP post request to the outbox URL with the Content-Type of application/ld+json; profile="https://www.w3.org/ns/activitystreams"
• Client submission request body is a single Create Activity of Post or Article object, with embedded images/videos/location appended as attachments.
• Client submission request is authenticated with the credentials of the user to whom the outbox belongs using HTTP signatures (Mastodon, Pleroma etc all seem to use HTTP signatures for ActivityPub, not OAuth)
• Client implements the following activity types when POSTing to an outbox: Create, Update, Delete, Remove. I don't need the others.

[yvolk]

@ned I see that I need to start with at least "Add "ActivityPub" type of Social network" and adding "application/ld+json" to a couple of places as mentioned above. This could be done quickly and will allow to add other features as "compatibility improvements".
Please note that AndStatus supports pump.io for years, and pump.io API is quite close to what ActivityPub is...

[ned14]

Thanks for the tip. Reading through https://github.com/pump-io/pump.io/blob/master/API.md, as far as I can tell these are the main significant differences between pump.io and W3C ActivityPub:

• Use of OAuth instead of HTTP signatures to authenticate. In fairness. the W3C spec leaves it open ended, but any of the W3C ActivityPub implementations I've curled so far use the latter.
• pump.io appears to hardcode the REST endpoints, and with slightly different naming. The W3C spec leaves ./well-known/webfinger complete flexibility to specify the endpoints to use.
• MIME types appear to be more specific in the W3C spec. Moreover, if you don't specify the specific W3C MIME types in the Accept: header, pleroma and Mastodon appear to try 301 redirecting you to elsewhere.

Otherwise, they do indeed look quite similar. The major blocker is the OAuth support really. I'm a bit loathe to implement it, because it's fundamentally broken, where HTTP signatures are not. But if it gets me up and running with andStatus quicker, I guess I can live with it.

[yvolk]

@ned14 Regarding restpoints. In pump.io, they are not hardcoded. Although they are usually the same, and this may lead to the hardcoded implementations in clients.
See the sample pump.io response with endpoints explicitly set: https://github.com/andstatus/andstatus/blob/master/app/src/androidTest/res/raw/pumpio_actor_lists_person.json

Recently I fixed hardcoded endpoints for pump.io in AndStatus (exactly for ActivityPub migration), see https://github.com/andstatus/andstatus/blob/master/app/src/main/java/org/andstatus/app/net/social/ActorEndpoints.java

[yvolk]

@ned14 I think that now you can start testing/working at a server-side of "client to server" ActivityPub protocol. AndStatus client is just updated to allow creation of a Social Network of ActivityPub type.
See more details and a link to the prebuilt alpha version of AndStatus in the top post of this thread.

As I wrote there, I reused OAuth 2.0 implementation that works in AndStatus for Mastodon (with bearer token...). Everything else - from existing Pump.io implementation. Plus added "application/ld+json..." as required.
If you need additional changes to the Client registration + User authentication/authorization part, feel free to change existing code. Btw

• Client Registration is implemented in subclasses of org.andstatus.app.net.http.HttpConnection#registerClient (called from org.andstatus.app.account.AccountSettingsActivity.OAuthRegisterClientTask ) - one implementation works in Mastodon, another - in Pump.io...
• User authentication/authorisation - in org.andstatus.app.account.AccountSettingsActivity.OAuthAcquireRequestTokenTask and org.andstatus.app.account.AccountSettingsActivity.OAuthAcquireAccessTokenTask

In order to move forward, past authorisation step, I need an account on a server that successfully authenticates/authorises a User...

[yvolk]

A couple of years ago we did the same OAuth implementation exercise with @Gargron for Mastodon, please see this post #419 (comment) and below.
In short, an example/the simplest client application is needed (with one empty activity, e.g. from an Android template...), using any libraries, that can, using hard-coded username and password, connect to your server and request Home timeline for the User.
So we will know that:

• Client registration works
• Authentication works
• At least one timeline request works (no need to parse it)
  I then integrate this into AndStatus, so the next steps will be easier.
  ?!

[autogestion]

@ned14 Mastodon, Pleroma and others uses http signatures only when interacting server-to-server. For interacting with clients Mastodon uses custom API, not related to ActivityPub. Pleroma just cloned it from Mastodon, and other servers uses custom APIs. OAuth is recommended way to authorize the client https://www.w3.org/TR/activitypub/#authorization -> https://www.w3.org/wiki/SocialCG/ActivityPub/Authentication_Authorization#Client_to_Server

When test suite on https://test.activitypub.rocks/ is trying to obtain a token (when testing client-to-server interaction with server), it follows next steps

1. hit .well-know/webfinger?resource=acct:email@domain and get link to user profile
2. fetch user profile and get from its "endpoints" list oauthTokenEndpoint url or oauthAuthorizationEndpoint url,
   it's described in spec https://www.w3.org/TR/activitypub/#actor-objects
3. and then follows oauth procedure

[ned14]

Bah, looks like I'll have to implement OAuth 2.0 so. It looks complex and hard to get right. But I'll give it a go. Thanks for the feedback, you saved me work!

[autogestion]

@ned14 You can try copypasting from this repo https://github.com/autogestion/pubgate
all that steps are already implemented there) except oauthAuthorizationEndpoint. But I think token endpoint is enough

[yvolk]

@autogestion @ned14 Please note that in both Pump.io and Mastodon client-to-server OAuth implementations the Client application starts with Dynamic client registration (implemented differently though...). Something like https://tools.ietf.org/html/rfc7591#section-1.3 (see Dynamic Registration as part of https://openid.net/connect/ also)
Only after successful Client Registration, User Identification and Authorisation starts...

I think we must have Client Registration in our new solutions, because even existing social networks implemented this (formally optional) step years ago. Even Twitter has Client Registration, although it's not dynamic: application developer has to register his Client application manually at Twitter's developer's site.
As we are building bricks for a federated network with lots of servers, the Client registration should be dynamic in order to avoid a need for manual steps.

If you want a server code to be compatible with different client applications, not only with the one that You (or we) created, please make it at least look like generic OAuth 2.0 server solution (i.e. having similar public API / process implemented).
For example, at least for beginning, you can implement "Dynamic Client Registration" as a simple static stub that returns the same "client identifier" and "client secret" (and maybe other server metadata...) to all clients and doesn't persist any client registration data. This should be easy.
?!

Dynamic registration should result in at least two ids, returned from a server: "client identifier" and "client secret". They are referred to e.g. as "YOUR_API_KEY" and "YOUR_API_SECRET" in the ScribeJava client library docs: https://github.com/scribejava/scribejava
BTW, there is a PR to add OpenID Connect support to the ScribeJava scribejava/scribejava#646

And another note on dynamic registration metadata (returned by a server) for @autogestion and his Pubgate (as defined in RFC 7591) :
"grant_types" - Array of OAuth 2.0 grant type strings that the client can use at the token endpoint... If omitted, the default behavior is that the client will use only the "authorization_code" Grant Type.
This means that for the "password" grant type this metadata field should be provided explicitly.
I found another interesting reference about the "password" grant type (in an answer to https://softwareengineering.stackexchange.com/questions/297373/oauth2-ropc-vs-basic-auth-for-public-rest-apis ): "OAuth2 is much more than Resource Owner Password Credentials which, according to the specification, exists for "legacy or migration reasons", is considered "higher risk than other grant types" and the specification explicitly states that the clients and authorization servers "SHOULD minimize use of this grant type and utilize other grant types whenever possible"."

[kaniini]

pleroma.site (as well as any other Pleroma install) has ActivityPub C2S available through OAuth. You will need to use the Mastodon API to obtain client secrets for OAuth.