New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add federation relay support #7998

Merged
merged 4 commits into from Jul 13, 2018

Conversation

Projects
None yet
8 participants
@Gargron
Member

Gargron commented Jul 11, 2018

Fix #7399

What? Federation relays are intermediaries between Mastodon servers. A Mastodon server can subscribe to a relay and publish all public posts to it, receiving all public posts it receives from others in turn.

Why? New & smaller servers struggle with content. New users who sign up on such servers find themselves in a desolate place and don't stick around. It takes users following other users to fill up the federated timeline... Or so it was before federation relays. Removes the need for follow-bots

Both participation in a relay as well as the choice of a relay are completely optional.

image

TODO:

  • Backend
  • Admin settings

The source code for the relay server is: https://source.joinmastodon.org/mastodon/pub-relay

private
def follow_activity(activity_id)
{

This comment has been minimized.

@puckipedia

puckipedia Jul 11, 2018

Collaborator

the Follow activity does not contain the actor, it seems? It seems the actor is implied at the relay side based on who signed the message... I'd prefer just implicitly mentioning it in the Follow..

This comment has been minimized.

@nightpool

nightpool Jul 12, 2018

Collaborator

+1

This comment has been minimized.

@nightpool

nightpool Jul 12, 2018

Collaborator

I assume you meant "explicitly" in the last sentence?

This comment has been minimized.

@akihikodaki

akihikodaki Jul 12, 2018

Collaborator

some_local_account is also confusing. It is already present in ReportService, but we are going to have two activities (Follow for this case and Flag for ReportService). I think it is a good time to have account-independent Service if Mastodon is going to have more and more such activities.

@aveao

This comment has been minimized.

aveao commented Jul 12, 2018

Doesn't this damage the point of federation by adding a single point of failure (the relay)? It is optional, but still, I feel like this shouldn't be a part of mastodon.

Everyone can host one themselves, sure, but it still means that other instances would need to agree to relay to and from the self-hosted relay, in turn making it useless to most people, especially people of smaller instances (which is the exact group of people that this feature is aiming to help).

@nightpool

This comment has been minimized.

Collaborator

nightpool commented Jul 12, 2018

@aveao nah, admins can add multiple relays and there's nothing saying anyone needs to use a relay. It's just a nicety to prevent people from resorting to followbots.

there's nothing here that is a "single point of failure", since it's only a cosmetic upgrade. but even with respect to the relay feature there's no single point of failure because it's easy to use multiple relays

@aveao

This comment has been minimized.

aveao commented Jul 12, 2018

@nightpool I just realized that this is probably the wrong place to write this on (issue would be better), so I'll probably take it there, but I think that you missed the point of my message (it wasn't about being able to do something, it's about how it ends up being in practice).

@Gargron

This comment has been minimized.

Member

Gargron commented Jul 12, 2018

@aveao

  • Using the relay is optional. If you want to keep the status quo of organic connections, you can. If the relay goes down, it's not catastrophic, you're just back at square one.
  • You can choose any relay you want. Yes it's a centralized point, but that's the only way it can fulfill it's purpose. Nobody says the relays couldn't use p2p tech between each other, potentially...
  • Although I am not decided on how I will design the admin UI for this yet, the code itself is built to support working with multiple relays at the same time, so you could have redundancy.
@akihikodaki

This comment has been minimized.

Collaborator

akihikodaki commented Jul 12, 2018

(Thanks unarist for telling me about this pull request.)

You say:

Relay.create!(inbox_url: 'https://relay.joinmastodon.org/inbox').enable!

But is it a temporary means, or you are planning to keep doing that even though you may wrap it with Web UI? It is very strange that it activates relay while Follow activity is actually performed. Follow is for subscription, not for publishing.

There are two options to solve the problem:

  1. Just use toot:Relay instead of Follow
  2. Divide relays into relay_subscribers and relay_publishers, and perform Accept for relay_subscribers and Follow for relay_publishers.

I'm for 2 because it is closer to the semantics of follow-bots. I accept them because it is not harmful for us, but when it comes the current implementation of relay, accepting a relay also means I send Follow activity to the instance, subscribing the statuses on the instance. I have to consider various things in this case: what kind of toots does it have? Is it reliable? Isn't it going to be a spamming instance?
The easiest solution for me would be accepting follow-bots and rejecting relays. I suspect it is also true for other people [not confirmed].

@nightpool

This comment has been minimized.

Collaborator

nightpool commented Jul 12, 2018

@akihikodaki i don't understand your suggestion. Are you saying relays should follow us as well as us following them?

@nightpool

This comment has been minimized.

Collaborator

nightpool commented Jul 12, 2018

I don't think most admins care about whether they're sending follows or accepts or whatever, this is purely a federation question.

@akihikodaki

This comment has been minimized.

Collaborator

akihikodaki commented Jul 12, 2018

@nightpool
I realized #7998 (comment) is too long and hard to understand while reading :/ In short:
The current relay implementation is always bi-directional. But it is:

  • a technically wrong use of Follow activity
  • not an exact alternative of follow-bots, which is uni-directional.
@nightpool

This comment has been minimized.

Collaborator

nightpool commented Jul 12, 2018

@akihikodaki

https://www.w3.org/TR/activitypub/#shared-inbox-delivery

Additionally, if an object is addressed to the Public special collection, a server MAY deliver that object to all known sharedInbox endpoints on the network.

We're delivering to a subset here, but I think it's still within the spirit of the MAY

@akihikodaki

This comment has been minimized.

Collaborator

akihikodaki commented Jul 12, 2018

@nightpool

We're delivering to a subset here, but I think it's still within the spirit of the MAY

It is not sharedInbox. It is inbox.

Even if it applies to this case, it is still MAY so we cannot assume that happens. For example, if someone want to relay with your instance and sends Follow activity, you have to reply it with Accept or Reject. But how do you determine that? You may know it is bi-directional but you should not assume that. If we want this kind of assumption, a new type like toot:Relay is more appropriate, as I suggested.

@Gargron

This comment has been minimized.

Member

Gargron commented Jul 12, 2018

You can send to the relay without subscribing. I don't see anything that would prevent that. It's completely up to the implementation of the relay server how to handle that, and it's entirely separate from the Follow activity. Follow activity is only for subscribing. And indeed in my code the two are united, at the same time as you subscribe, we "enable" it on our end to send stuff to it too.

@Gargron

This comment has been minimized.

Member

Gargron commented Jul 12, 2018

The relay only has one inbox, so inbox/sharedInbox doesn't make a difference there. The relay code will discard any message that does not contain the public collection in its primary audience, so accidentally sending a DM to the relay actor should not be problematic.

The relay code on its side uses some_local_account's inbox instead of sharedInbox, and i understand that could be a problem with non-Mastodon subscribers, but the relay code is very immature and provisional in general right now so I'll just fix that, it's not a big problem.

I plan to replace some_local_account with a system user in the future but see no reason to bind that to this PR.

@adbelle

Nice.

Gargron added some commits Jul 12, 2018

@ykzts

ykzts approved these changes Jul 12, 2018

@Gargron Gargron merged commit e55dce3 into master Jul 13, 2018

11 checks passed

ci/circleci: build Your tests passed on CircleCI!
Details
ci/circleci: check-i18n Your tests passed on CircleCI!
Details
ci/circleci: install Your tests passed on CircleCI!
Details
ci/circleci: install-ruby2.3 Your tests passed on CircleCI!
Details
ci/circleci: install-ruby2.4 Your tests passed on CircleCI!
Details
ci/circleci: install-ruby2.5 Your tests passed on CircleCI!
Details
ci/circleci: test-ruby2.3 Your tests passed on CircleCI!
Details
ci/circleci: test-ruby2.4 Your tests passed on CircleCI!
Details
ci/circleci: test-ruby2.5 Your tests passed on CircleCI!
Details
ci/circleci: test-webui Your tests passed on CircleCI!
Details
codeclimate All good!
Details

@Gargron Gargron deleted the feature-relays branch Jul 13, 2018

@akihikodaki

This comment has been minimized.

Collaborator

akihikodaki commented Jul 13, 2018

And indeed in my code the two are united, at the same time as you subscribe, we "enable" it on our end to send stuff to it too.

I am worrying the case when Follow activity is rejected. If rejection causes disabling publishing, it is an unexpected side effect. If rejection does not disable publishing, the state of subscription and publishing will be inconsistent.

@annando

This comment has been minimized.

annando commented Jul 13, 2018

Is there some documentation how to get content from the relay and how to get the relay to distribute your content? Although Friendica doesn't support AP right now, I would like to use the relay when this is done.

@Gargron

This comment has been minimized.

Member

Gargron commented Jul 13, 2018

Gargron added a commit that referenced this pull request Jul 16, 2018

Fix ActivityPub::UpdateDistributionWorker regression
Regression from #7998 let to profile updates not sending

Gargron added a commit that referenced this pull request Jul 16, 2018

Fix ActivityPub::UpdateDistributionWorker regression (#8039)
Regression from #7998 let to profile updates not sending

byronhulcher added a commit to byronhulcher/mastodon that referenced this pull request Aug 18, 2018

Add federation relay support (tootsuite#7998)
* Add federation relay support

* Add admin UI for managing relays

* Include actor on relay-related activities

* Fix i18n

byronhulcher added a commit to byronhulcher/mastodon that referenced this pull request Aug 18, 2018

kedamaDQ added a commit to kedamaDQ/mastodon that referenced this pull request Aug 23, 2018

@bitboxer bitboxer referenced this pull request Aug 26, 2018

Closed

Federation relay? #1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment