Update Account Following REST API to Return All Users

[mlcohen]

Pitch

If you use any Mastodon app to view who someone follows, you can only see those who belong to the same Mastodon instance as you. For instance, I have an account on mas.to and therefore I can only see those who also belong to mas.to. If, however, you go to a Mastodon web app you can view all user regardless of what server they belong to. This appears to be a restriction placed on the Mastodon REST API. Specially the /api/v1/accounts/:id/following endpoint.

Suggestion: Update the account following REST API to return all accounts that are being followed; not just those accounts that the requestor belongs to.

Please note that I have reached out to 3rd-party Mastodon developers (such as Tapbot) about this and they have noted that the issue is indeed with the REST API.

Motivation

The ability to explore who someone follows whether via a mobile app or the Mastodon web app would go a long way towards helping improve discoverability. The easier it is for people on Mastodon to find others, the better the overall experience. Since many people regularly use a Mastodon mobile app, such as the official Mastodon iOS app or a 3rd-party app like Tapbot's Ivory, it would be beneficial for these apps to let users easily view who someone follows without the need to navigate over to a specific Mastodon web server instance.

[trwnh]

This appears to be a restriction placed on the Mastodon REST API.

it's not a restriction. the API returns all the data it has. the problem is that the data it has is only what it knows

duplicate of #19880

[mlcohen]

the API returns all the data it has. the problem is that the data it has is only what it knows

That seems rather... odd.

If the client app can't show who someone follows outside of the server you belong to but you can see the followers if you go directly to that user's Mastodon server, wouldn't that imply that the given server I navigated to would in fact have the data? In which case, couldn't that data be provided to the client?

For instance, using a client app like Ivory, if I look at who @tapbots@tapbots.social follows, I don't see any followers. When I navigate to https://tapbots.social/@tapbots/following I can see all the accounts @tapbot follows. Granted all the accounts belong to tapbots.social but the point here is that there is a Mastodon server which does have the data.

The data might belong to an instance that is different from the one I belong to but that data exists... somewhere. A client should be able to easily enough reach out to get the following data even if it is a separate request.

This seems like a missed opportunity to really help improve discoverability. Especially for new people that come to Mastodon.

duplicate of #19880

@trwnh Thanks for the reference. Will close this ticket.

[trwnh]

you can see the followers if you go directly to that user's Mastodon server,

this is discussed further in the other issue but basically

• the other server doesn't have to be running mastodon
• the other server might not publicize the data
• the other server might lie about the data

in particular, that last point is most important. on the backend, mastodon only shows what it knows and can trust. right now, mastodon blindly trusts the total number, so it's possible to see cases where an account claims 0 followers, negative followers, billions of followers, and so on (and this has happened before).

[mlcohen]

@trwnh Got it. Interesting.

the other server doesn't have to be running mastodon

I see. Perhaps this is an opportunity to establish the concept of who someone follows down at the protocol level (ActivityPub) or, maybe, establish a common REST API standard that any server that wants to be part of the fediverse can adhere to.

the other server might not publicize the data

That's fine. Seems like that can be part of the server's REST API to properly advertise that it does not want to publicize the data. The client can use the information to present a more meaningful notice to the user.

the other server might lie about the data

Sure but given that why should clients like Ivory or the official Mastodon mobile app even bother to show how many people someone follows? Since the following count is shown, to users their assumption is that the number is truthful even if it may not be. Since the count is shown then might as well show the individual users. People joining Mastodon, many (most?) of them being non-tech savvy will be confused why they are not able to see all the users someone follows. Trying to explain different instances to such folks just makes their eyes glaze over wondering what's the point.

Since the problem appears to come down to trust, maybe the solution is to establish a network or trust or zones of trust among servers (Mastodon or ActivityPub/fediverse compliant servers). For instance, the server mastodon.social would have a high zone of trust. That server could vouch for other instances (like mas.to). For servers within a given zone of trust, a client can then use that information to declare that the following count is accurate and the individual users returned from the server can be trusted (or at least those accounts belonging to trusted servers).

[trwnh]

there's already the concept of a followers collection in ActivityPub but this depends on the server. it's not a requirement and it comes with no guarantees. there's no way to verify the information. i can't speak as to why mastodon trusts the number but not the contents (insofar as the contents are actually public). personally, i'd not bother with showing the numbers, but i'm not gargron. (i'd also not bother with publishing the followers at all, as i believe that should be private information, but i digress...)

as far as actually trusting the items contained within a follower collection, there are more issues to consider, such as having to resolve more profiles in general, potentially recursing through those, if you want to have that information ahead-of-time instead of on-demand, and so on. i don't know if it necessarily makes sense to have a network of trust in this specific application, but it could work; it'd just require admins to have to make more decisions about all of the servers they federate with. in any case, it's not an entirely new idea, as it has been brought up in similar "trustworthiness" cases such as with link preview information. but at least you can verify link previews for yourself...

[mlcohen]

personally, i'd not bother with showing the numbers, but i'm not gargron

This really comes down to a matter of principal on where the line of trust and privacy is at. Kind of like with the vibrant discussion about whether or not to add quote tweets/posts functionality to Mastodon.

Visibility around following count and the seeing the individuals who are being followed is here albeit hampered. Popular client apps (include the Mastodon web server) display it in one form or another. So it's not something that's going to be taken away. (Well, technically it could but that's going backwards IMO). I'm not sure about the broader fediverse ecosystem, but for Mastodon itself, it'd be better to look at a reasonable solution to make following data accessible, trusted (within reason), and built around privacy.

there are more issues to consider, such as having to resolve more profiles in general, potentially recursing through those

Seems like that's something apps can deal with themselves. I mean, that's essentially what Mastodon web app is doing right now when you go look at who someone follows on the web app, unless I'm misinterpreting what you mean by a profile.

As for recursing, not sure I understand why it would be required.

Let's say I have account on mastodon.social (say, xyz@mastodon.social) and I follow four accounts: alice@mastodon.social, bob@mas.to, catherine@hachyderm.io and dan@macaw.social. Assuming the server would return information for the accounts being followed (and not just accounts restricted to mastodon.social), a client would just need to iterate over the accounts in the paginated response and fetch information from the respective server. Saying all that, if I look at the people I'm following on the web app, Mastodon shows all the accounts across instances, so it must have some amount of account information on hand. I can't image the server doing any kind of recursion to display those followers.

i don't know if it necessarily makes sense to have a network of trust in this specific application, but it could work; it'd just require admins to have to make more decisions about all of the servers they federate with

Yeah, there is that. My quick take would be to at least provide the option. Admins can then decide if the burden of managing trust is worth it. If not, fine, don't bother. But for those who do want to handle trust they can.

[trwnh]

it's not something that's going to be taken away. (Well, technically it could but that's going backwards IMO

i don't think it's "backwards" unless you think seeing counts and numbers and metrics is a core feature of social media. and, well, that may be the case if you consider social media as a status game. but if you want to have good conversations then it shouldn't matter how many followers the other person has. the fact that the counts can be faked or otherwise claimed to be anything you want, well, that just exacerbates the issue.

for Mastodon itself, it'd be better to look at a reasonable solution to make following data accessible, trusted (within reason), and built around privacy.

i could see a use-case for building upon the "featured accounts" which used to be shown on the public profile pages, back when there were separate public profile pages. you could have people opting in to recommending other accounts to check out or follow. but moreover, i just think that having the expectation of knowing someone's entire network is not feasible or even a good idea. you might be able to achieve it on a centralized network like Twitter, but then there are sites like Tumblr that hide follower counts and only show it to yourself. and then you consider the comparison to email (since ActivityPub is basically email for websites), where people don't generally publish their entire address books as that's considered private information. so the way that mastodon works right now (only showing you locally verifiable data) is a bit of a compromise that doesn't really please anyone -- the people who want privacy don't like it because it's unnecessary, and the people who want information don't like it because it's not complete.

As for recursing, not sure I understand why it would be required.

say you load a profile's followers. you get 20 accounts. not all of those accounts may be known by your server. so your server fetches any new accounts it discovers this way. then if it wants to prefetch their followers, it might again discover new accounts in the process. at minimum you will end up resolving more accounts that no one follows, and at maximum you may end up spidering the entire network if you're not careful. also consider that the followers collection is not the only thing that you might want to fetch or prefetch for any given profile -- you may be interested in their avatar, their header image, their bio, their pinned posts, replies to their pinned posts, and so on. that's all more data to process and keep around for indeterminate time periods.

[mlcohen]

but if you want to have good conversations then it shouldn't matter how many followers the other person has

Ah, okay. I think we need to reset. I'm only referring to see who some follows. Regarding seeing the people who follow someone, yeah, I certainly agree with you. Your point about follower count can indeed lead to potential toxic environment as is evidenced on The Bird Site. Those who trumpet follower count for clout with intent of misuse, abuse, vanity, etc is something I also would like to keep away on Mastodon (and the fediverse in general). With that being said, the follower count is there -- for better or worse.

Regarding recursion, yeah, that makes sense WRT to followers. Appreciate the details.

Just focusing on who someone follows, let's take a simple, concrete example. I'll use tapbots@tapbots.social account.

When I make a request against the public accounts/:id/following endpoint, I get back the following:

$ curl https://tapbots.social/api/v1/accounts/109331967957147464/following | jq '.'

[
  {
    "id": "...",
    "username": "todd",
    ...
  },
  {
    "id": "...",
    "username": "mark",
    ...
  },
  {
    "id": "...",
    "username": "paul",
    ...
  }
]

So the data is there (again only focusing on following, not followers). Everything a client app would need to show each account's basic, relevant information is available in the response (username, note, avatar, etc). Putting aside trust, there doesn't appear to be anything limiting a client from obtaining data about who someone follows.

[trwnh]

that makes sense WRT to followers

it applies to following lists too -- the point is that you will be likely discovering new accounts in the process and therefore have to be careful with how many child requests you spawn

Just focusing on who someone follows, let's take a simple, concrete example [...] Everything a client app would need to show each account's basic, relevant information is available in the response (username, note, avatar, etc). Putting aside trust, there doesn't appear to be anything limiting a client from obtaining data about who someone follows.

everything a client app would need to show basic information as presented by a remote server. yes, you could show a username, display name, avatar. but there's two big issues with this:

• you can't do anything against that account unless you resolve it on your local server
• the remote server might not be running mastodon or serving the mastodon api endpoint

let's say that tapbots.social was running misskey instead of mastodon. then the /api/v1/accounts/:account_id/following would not exist.

alternatively, consider how you discover that tapbots@tapbots.social is account 109331967957147464 on tapbots.social. it wasn't straightforward, was it? you probably had to use the search api, or otherwise you extracted some status_id from its uri and tried to load that status in the api for the sole purpose of discovering the account_id.

the standardized way to do this would be to fetch the actor and see if they have a following collection, then fetch that and process its items or orderedItems (or if it was paginated, start paginating from the first or last page). but this isn't something that any mastodon api client should be expected to do! this is something that ideally the server would do on behalf of the client. at minimum, the mastodon client would end up implementing parts of activitypub parsing in order to extract any relevant activitypub id properties and then feed those into the mastodon search api with the resolve parameter. and this is assuming that the activitypub representations are available publicly without any sort of authentication or authorization, which isn't always the case -- you will likely need HTTP Signatures that you will not be able to perform because your private keys are being managed by your mastodon server.

[mlcohen]

The initial approach I used was via the Mastodon REST API. Started with /api/v1/accounts/lookup?acct=tapbots to get the account ID 109331967957147464. Not too bad. But, yeah, there is a problem with this approach: it's specific to Mastodon. Other type of fediverse application can of course choose to implement the Mastodon REST API but most don't. (Friendica does).

the standardized way to do this would be to fetch the actor and see if they have a following collection

Right. And that is something that Mastodon does expose via WebFinger:

$ curl https://tapbots.social/.well-known/webfinger\?resource\=tapbots@tapbots.social -H "accept: application/activity+json" | jq '.'

{
  "subject": "acct:tapbots@tapbots.social",
  "aliases": [
    "https://tapbots.social/@tapbots",
    "https://tapbots.social/users/tapbots"
  ],
  "links": [
    {
      "rel": "http://webfinger.net/rel/profile-page",
      "type": "text/html",
      "href": "https://tapbots.social/@tapbots"
    },
    {
      "rel": "self",
      "type": "application/activity+json",
      "href": "https://tapbots.social/users/tapbots"
    },
    {
      "rel": "http://ostatus.org/schema/1.0/subscribe",
      "template": "https://tapbots.social/authorize_interaction?uri={uri}"
    }
  ]
}

Then from there you can get the ActivityPub actor/account object:

$ curl https://tapbots.social/users/tapbots -H "accept: application/activity+json" | jq '.'

{
  "@context": [...],
  "id": "https://tapbots.social/users/tapbots",
  "type": "Person",
  "following": "https://tapbots.social/users/tapbots/following",
  "followers": "https://tapbots.social/users/tapbots/followers",
  "inbox": "https://tapbots.social/users/tapbots/inbox",
  "outbox": "https://tapbots.social/users/tapbots/outbox",
  "featured": "https://tapbots.social/users/tapbots/collections/featured",
  "featuredTags": "https://tapbots.social/users/tapbots/collections/tags",
  "preferredUsername": "tapbots",
  "name": "Tapbots",
  "summary": "<p>We create delightful apps for iOS and the Mac.</p>",
  "url": "https://tapbots.social/@tapbots",
  "manuallyApprovesFollowers": false,
  "discoverable": true,
  "published": "2022-11-12T00:00:00Z",
  "devices": "https://tapbots.social/users/tapbots/collections/devices",
  "publicKey": { ... },
  "tag": [],
  "attachment": [],
  "endpoints": {
    "sharedInbox": "https://tapbots.social/inbox"
  },
  ...
}

And as you noted, page over the following collection. Of course this is still specific to Mastodon (the exposed WebFinger endpoint). If there is some account that belongs outside of Mastodon but part still part of the fediverse, the client wouldn't be able to tell and therefore unsure how to get the user ActivityPub account object. There's nothing saying an account is a friendica account or a misskey account. If there was information indicating that an account belonged to, say, misskey, then the client could use it to either filter the account out of make it clear that it sites outside Mastodon. Can then just use the profile URL to link to the account.

this is assuming that the activitypub representations are available publicly without any sort of authentication or authorization, which isn't always the case

Yeah, there is that to consider. If authz/authn is required, the client could just fallback to linking out to the account's given profile.

[mlcohen]

Of course this is still specific to Mastodon (the exposed WebFinger endpoint)

Correction: Any ActivityPub-compliant service that wants to be interoperable with Mastodon must use WebFinger. From the Mastodon docs:

Because Mastodon heavily relies on mentions for addressing other profiles, WebFinger is required for fully interoperating with Mastodon... Searching for any objects or profiles from an ActivityPub implementation without WebFinger will fail because the author cannot be converted to a user in the local database.

For a client, it can get all the users an account follows via WebFinger. No need to use a Mastodon specific REST API.