Missing replies

[alexandru]

Hi all,

I'm self-hosting my own Mastodon instance, just for me.

The big problem I have is ... missing replies. This seems to manifest especially on boosted posts of people I don't follow, although replies to posts of people I follow may have missing comments too.

Boosts and favorite counts are also missing too, seeing only a count from people I follow. This doesn't bother me as much as missing replies. The problem with missing replies is that it's hard engage in a conversation like this.

Maybe this is how Mastodon works. I'd like to know if there's something that I can do (and possibly avoid connecting to a relay, since that seems to be costly).

[alexandru]

To add detail ... here's is a toot on its original instance (I hope linking to such content is OK):
https://wandering.shop/@aleen/109545360209489228

Here's that same toot synced on my own instance:
https://social.alexn.org/@aleen@wandering.shop/109545360249033153

Note I am not following the author, this was a boosted toot from someone else that I follow. Is this by design?

Note, I found out about the sidekiq/queues endpoint, below is a screenshot of it. I don't think it's having issues (except for a couple of timeouts):

[melroyvandenberg]

Now it seems (after a while) syncs the replies as well??

[ClearlyClaire]

Hi! This is expected behavior, as Mastodon doesn't ensure it gets all replies to remote posts.

What it does is try to fetch self-replies as well as the parent of a post whenever it is discovered. It also forwards replies to followers in most cases, but replies are never forwarded to non-followers. Doing otherwise would involve somehow subscribing to every post you may have seen, and would massively increase the amount of traffic across the fediverse.

[bensyverson]

@ClearlyClaire I understand why you wouldn't want to proactively subscribe to replies for every incoming message, but could it be done lazily when users view an individual post?

For example, imagine I'm @me@small.instance and I click on a post from @other@example.com:
https://small.instance/@other@example.com/1099

My expectation as a new user is that my instance will now fetch replies to this post from example.com, even if it takes some time for them to populate. It seems super counterintuitive that my instance just sits there showing zero replies.

[nanos]

Just thought I’d highlight here, that there are now two tools to deal with this issue:

FediFetcher is a python script you can either run as a cron job, or a GitHub action (full disclosure: I’m the author)

Alternatively, there is combine.social which does essentially the same, but as a service.

[lgrn]

just FYI, the second option combine.social now returns 404 and a self-signed cert

[nanos]

Yeah, sadly it disappeared a while ago.

If you use Mastodon exclusively in the browser, Substitoot might be an alternative, but otherwise unfortunately I think FediFetcher is the only remaining option.

[ludovicchabant]

I have the OP's problem too, and I want to bring up the fact that there are two things at play here: how post federation works, and how the user experience should be. The fact that not all posts are brought to my instance is fine -- I don't need to see everything in my feed, and I don't need my instance's database to blow up. However, what annoys me a lot is the user experience, which can be solved IMHO.

When I click on a post, it doesn't go and fetch extra information from the post's original instance. I understand the pros/cons of keeping requests "local" (to one's own instance) vs hitting other servers, but when I click on the post, I expect to see as much as possible. Instead of fetching extra information from the original instance, the web UI shows only the incomplete information from my instance. I need to go look at the original post on its own instance and then play a game of back and forth to interact with the replies I didn't see (hitting the original instance anyway, multiple times!) This is the sort of stuff that mobile apps might eventually do for me, giving me a better user experience than the default Mastodon web app. I think fetching extra information should become a standard approach to Mastodon clients.

[eobet]

I'm on a small instance, and I just compared the popular posts on Mastodon.social with what I see on my instance:

Literally 10x the amounts of boosts, replies and likes on .social compared to what I see, even when I paste the exact url of the post into my instance (doesn't matter if it's on the web or using an app).

Why do we on smaller instances have to miss out? Perhaps Mastodon isn't for smaller instances and we should instead switch to Misskey, Pleroma or Firefish (can't remember which one exactly but at least one of them had "remote fetch" or whatever they called it to actually sync up all answers, boosts and likes)...

[Kdecherf]

Sadly, Pleroma doesn't support remote fetch of all replies to a post neither