New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to ActivityPub for federation #104

Closed
tcitworld opened this Issue Oct 21, 2017 · 15 comments

Comments

Projects
None yet
5 participants
@tcitworld
Collaborator

tcitworld commented Oct 21, 2017

ActivityPub is a W3C standard made for federation of decentralized servers. At first look, it's not really intended for what PeerTube tries to achieve, but it may be a rather good protocol to improve user to user interactions on different servers (such as comments).

@Chocobozzz

This comment has been minimized.

Show comment
Hide comment
@Chocobozzz

Chocobozzz Oct 23, 2017

Owner

It's a very interesting protocol but yes, not suitable for PeerTube because we only have very simple server to server interactions. For the comments I don't really know if we will federate them (it's a lot of work and not the main goal of PeerTube).

However there are some key concepts that I think we will use (followers/following, send messages only to followers...). I'll create a dedicated issue with the new federation architecture in some days/weeks.

Owner

Chocobozzz commented Oct 23, 2017

It's a very interesting protocol but yes, not suitable for PeerTube because we only have very simple server to server interactions. For the comments I don't really know if we will federate them (it's a lot of work and not the main goal of PeerTube).

However there are some key concepts that I think we will use (followers/following, send messages only to followers...). I'll create a dedicated issue with the new federation architecture in some days/weeks.

@Chocobozzz Chocobozzz closed this Oct 23, 2017

@Chocobozzz

This comment has been minimized.

Show comment
Hide comment
@Chocobozzz

Chocobozzz Nov 2, 2017

Owner

After a discussion with @Gargron I'm reopening this issue. We should not have a private federation protocol.

Owner

Chocobozzz commented Nov 2, 2017

After a discussion with @Gargron I'm reopening this issue. We should not have a private federation protocol.

@Chocobozzz Chocobozzz reopened this Nov 2, 2017

@Chocobozzz Chocobozzz self-assigned this Nov 2, 2017

@Chocobozzz Chocobozzz changed the title from Consider ActivityPub for federation to Move to ActivityPub for federation Nov 8, 2017

@Chocobozzz

This comment has been minimized.

Show comment
Hide comment
@Chocobozzz
Owner

Chocobozzz commented Nov 15, 2017

Working on activitypub branch

@Chocobozzz

This comment has been minimized.

Show comment
Hide comment
@Chocobozzz

Chocobozzz Nov 28, 2017

Owner

Merged in develop!

For now, only Server-Server communication is implemented. Of course, the implementation is far from perfect and it misses some features (Block, Reject...) that I'll add later with dedicated issues (I'll create an "ActivityPub" label).
I discovered JSON-LD, ActivityStream and ActivityPub three weeks ago so there may be some inconsistencies. Of course they will be fixed progressively :)

PeerTube Case

  • It is the administrators who choose which other servers they follow. Because they need to choose which kind of videos they want to display on their instance. -> We have servers that follow other servers
  • There is no interface for users to follow other actors
  • Users can have 1..n video channels. A video channel contains 0..n videos
  • A video can be deleted, updated, viewed, liked or disliked

Implementation

  • A PeerTube instance is an Application actor
  • Each user on a PeerTube server is an actor
  • Each video channel on a PeerTube server is a Group actor attributedTo a user actor
  • Administrators have an interface to make their instance following other PeerTube instance
  • Accept is automatically sent (we cannot choose which server can follow us)
  • When a user uploads a video:
    • We send a Create Activity with the video channel as target to the user's followers
    • The video channel Announce this Create Activity to notify its followers
    • The server Announce this Create Activity to notify its followers (other PeerTube instances)
  • A video has likes, shares and dislikes (not in the spec) collections
  • When a user updates their video, the server sends an Update Activity to actors involved in the object (it's the same for Delete/Like/Dislike/Undo activities)
  • We use the View Activity Stream object to notify of a view event. The actor is the PeerTube instance because a video could be viewed by an anonymous user
  • Actor outbox contains all their public activities
  • On Accept, we fetch the target outbox to save previous activities
  • We use jsonld-signatures module to sign activities with RsaSignature2017
  • We use http-signature module to sign HTTP requests
  • We use Webfinger to fetch remote account (when we don't have their URL)
  • Activities interfaces can be viewed in shared/models/activitypub directory
    • We use the Video ActivityStream object to describe videos but we added some attributes (category, licence...). URL array contains video file URL, magnets URI, thumbnail URL...
  • The language attribute of a video is the audio language, not the content language
  • A video is linked with a video channel with the attributedTo attribute

To conclude, PeerTube implements in a classic way the ActivityPub protocol. The only specificity is that the instance is an actor too and it automatically Announce all Create activities of its users. A PeerTube user cannot follow other users (there is no interface for this), but a Mastodon user could follow a specific PeerTube user, or a specific video channel of a user or directly the instance for example.

@cwebber Do not hesitate to comment/email/IRC me if you see potential issues with your future work on MediaGoblin!

@Gargron I hope in the future Mastodon and video platforms will be able to communicate. I imagine something like:

  • Mastodon receives a Create/Add video activity
  • Mastodon creates a status with an embed of the video
  • Replies to this status are added to the video platforms as comments

This way, we have for example 4 different plateforms (Diaspora, PeerTube, Mastodon, MediaGoblin) that unify the video source and the comments! Better than Facebook/Twitter/YouTube no? :)

But there are plenty of other possibilities, very excited by the ActivityPub future :)

I'll let this issue opened a few days for discussion.

Owner

Chocobozzz commented Nov 28, 2017

Merged in develop!

For now, only Server-Server communication is implemented. Of course, the implementation is far from perfect and it misses some features (Block, Reject...) that I'll add later with dedicated issues (I'll create an "ActivityPub" label).
I discovered JSON-LD, ActivityStream and ActivityPub three weeks ago so there may be some inconsistencies. Of course they will be fixed progressively :)

PeerTube Case

  • It is the administrators who choose which other servers they follow. Because they need to choose which kind of videos they want to display on their instance. -> We have servers that follow other servers
  • There is no interface for users to follow other actors
  • Users can have 1..n video channels. A video channel contains 0..n videos
  • A video can be deleted, updated, viewed, liked or disliked

Implementation

  • A PeerTube instance is an Application actor
  • Each user on a PeerTube server is an actor
  • Each video channel on a PeerTube server is a Group actor attributedTo a user actor
  • Administrators have an interface to make their instance following other PeerTube instance
  • Accept is automatically sent (we cannot choose which server can follow us)
  • When a user uploads a video:
    • We send a Create Activity with the video channel as target to the user's followers
    • The video channel Announce this Create Activity to notify its followers
    • The server Announce this Create Activity to notify its followers (other PeerTube instances)
  • A video has likes, shares and dislikes (not in the spec) collections
  • When a user updates their video, the server sends an Update Activity to actors involved in the object (it's the same for Delete/Like/Dislike/Undo activities)
  • We use the View Activity Stream object to notify of a view event. The actor is the PeerTube instance because a video could be viewed by an anonymous user
  • Actor outbox contains all their public activities
  • On Accept, we fetch the target outbox to save previous activities
  • We use jsonld-signatures module to sign activities with RsaSignature2017
  • We use http-signature module to sign HTTP requests
  • We use Webfinger to fetch remote account (when we don't have their URL)
  • Activities interfaces can be viewed in shared/models/activitypub directory
    • We use the Video ActivityStream object to describe videos but we added some attributes (category, licence...). URL array contains video file URL, magnets URI, thumbnail URL...
  • The language attribute of a video is the audio language, not the content language
  • A video is linked with a video channel with the attributedTo attribute

To conclude, PeerTube implements in a classic way the ActivityPub protocol. The only specificity is that the instance is an actor too and it automatically Announce all Create activities of its users. A PeerTube user cannot follow other users (there is no interface for this), but a Mastodon user could follow a specific PeerTube user, or a specific video channel of a user or directly the instance for example.

@cwebber Do not hesitate to comment/email/IRC me if you see potential issues with your future work on MediaGoblin!

@Gargron I hope in the future Mastodon and video platforms will be able to communicate. I imagine something like:

  • Mastodon receives a Create/Add video activity
  • Mastodon creates a status with an embed of the video
  • Replies to this status are added to the video platforms as comments

This way, we have for example 4 different plateforms (Diaspora, PeerTube, Mastodon, MediaGoblin) that unify the video source and the comments! Better than Facebook/Twitter/YouTube no? :)

But there are plenty of other possibilities, very excited by the ActivityPub future :)

I'll let this issue opened a few days for discussion.

@Gargron

This comment has been minimized.

Show comment
Hide comment
@Gargron

Gargron Nov 28, 2017

  • Mastodon receive a Create/Add video activity
  • Mastodon creates a status with an embed of the video
  • Replies to this status are added to the video platforms as comments

@Chocobozzz Have you considered publishing a Create/Note activity alongside the Create/Video one? The contents could just be title of the video and link to its page. I could implement this sort of fallback on Mastodon's side, if you like, I am not sure which side should do this, but feels like you'll get more potential compatibility out of it?

Another small thing for better Mastodon integrations: Mastodon can look up OEmbed/OpenGraph data about a URL included in a status message. There is an OpenGraph tag that you'll find useful (which unfortunately carries a branded name): twitter:player - it's supposed to contain the URL of an embedded video player. For example, Twitch Clips uses that, and Mastodon displays those clips in the client just like other link previews. So you could have the PeerTube video player there. I'd rather avoid fetching PeerTube videos using our caching/camo-esque method because videos can get pretty big and it would strain the system, and this would be a good middle way.

Gargron commented Nov 28, 2017

  • Mastodon receive a Create/Add video activity
  • Mastodon creates a status with an embed of the video
  • Replies to this status are added to the video platforms as comments

@Chocobozzz Have you considered publishing a Create/Note activity alongside the Create/Video one? The contents could just be title of the video and link to its page. I could implement this sort of fallback on Mastodon's side, if you like, I am not sure which side should do this, but feels like you'll get more potential compatibility out of it?

Another small thing for better Mastodon integrations: Mastodon can look up OEmbed/OpenGraph data about a URL included in a status message. There is an OpenGraph tag that you'll find useful (which unfortunately carries a branded name): twitter:player - it's supposed to contain the URL of an embedded video player. For example, Twitch Clips uses that, and Mastodon displays those clips in the client just like other link previews. So you could have the PeerTube video player there. I'd rather avoid fetching PeerTube videos using our caching/camo-esque method because videos can get pretty big and it would strain the system, and this would be a good middle way.

@Chocobozzz

This comment has been minimized.

Show comment
Hide comment
@Chocobozzz

Chocobozzz Nov 28, 2017

Owner

Have you considered publishing a Create/Note activity alongside the Create/Video one? The contents could just be title of the video and link to its page. I could implement this sort of fallback on Mastodon's side, if you like, I am not sure which side should do this, but feels like you'll get more potential compatibility out of it?

Sure I can do this.

Another small thing for better Mastodon integrations: Mastodon can look up OEmbed/OpenGraph data about a URL included in a status message. There is an OpenGraph tag that you'll find useful (which unfortunately carries a branded name): twitter:player - it's supposed to contain the URL of an embedded video player. For example, Twitch Clips uses that, and Mastodon displays those clips in the client just like other link previews. So you could have the PeerTube video player there. I'd rather avoid fetching PeerTube videos using our caching/camo-esque method because videos can get pretty big and it would strain the system, and this would be a good middle way.

We already have the twitter:player tag, but it seems we have an error on Mastodon:

Refused to frame 'https://peertube.cpy.re/videos/embed/d2a5ec78-5f85-4090-8ec5-dc1102e022ea' because it violates the following Content Security Policy directive: "frame-src 'none'".

Example URL : https://mastodon.tetaneutral.net/@FF255/98935617310552785

Of course I did not want you to fetch the videos, just display the embed tag. But sending a Note should be a good solution! @cwebber, what do you think about that?

Owner

Chocobozzz commented Nov 28, 2017

Have you considered publishing a Create/Note activity alongside the Create/Video one? The contents could just be title of the video and link to its page. I could implement this sort of fallback on Mastodon's side, if you like, I am not sure which side should do this, but feels like you'll get more potential compatibility out of it?

Sure I can do this.

Another small thing for better Mastodon integrations: Mastodon can look up OEmbed/OpenGraph data about a URL included in a status message. There is an OpenGraph tag that you'll find useful (which unfortunately carries a branded name): twitter:player - it's supposed to contain the URL of an embedded video player. For example, Twitch Clips uses that, and Mastodon displays those clips in the client just like other link previews. So you could have the PeerTube video player there. I'd rather avoid fetching PeerTube videos using our caching/camo-esque method because videos can get pretty big and it would strain the system, and this would be a good middle way.

We already have the twitter:player tag, but it seems we have an error on Mastodon:

Refused to frame 'https://peertube.cpy.re/videos/embed/d2a5ec78-5f85-4090-8ec5-dc1102e022ea' because it violates the following Content Security Policy directive: "frame-src 'none'".

Example URL : https://mastodon.tetaneutral.net/@FF255/98935617310552785

Of course I did not want you to fetch the videos, just display the embed tag. But sending a Note should be a good solution! @cwebber, what do you think about that?

@Gargron

This comment has been minimized.

Show comment
Hide comment
@Gargron

Gargron Nov 28, 2017

Refused to frame 'https://peertube.cpy.re/videos/embed/d2a5ec78-5f85-4090-8ec5-dc1102e022ea' because it violates the following Content Security Policy directive: "frame-src 'none'".

Seems like that instance has badly configured CSP headers. Our documentation does not recommend setting them! It wouldn't just be your player broken then, any embed would be broken on that instance.

Gargron commented Nov 28, 2017

Refused to frame 'https://peertube.cpy.re/videos/embed/d2a5ec78-5f85-4090-8ec5-dc1102e022ea' because it violates the following Content Security Policy directive: "frame-src 'none'".

Seems like that instance has badly configured CSP headers. Our documentation does not recommend setting them! It wouldn't just be your player broken then, any embed would be broken on that instance.

@Chocobozzz

This comment has been minimized.

Show comment
Hide comment
@Chocobozzz

Chocobozzz Nov 28, 2017

Owner

My bad, thanks.

It seems to work well with another instance.

Owner

Chocobozzz commented Nov 28, 2017

My bad, thanks.

It seems to work well with another instance.

@cwebber

This comment has been minimized.

Show comment
Hide comment
@cwebber

cwebber Nov 28, 2017

Have you considered publishing a Create/Note activity alongside the Create/Video one? The contents could just be title of the video and link to its page. I could implement this sort of fallback on Mastodon's side, if you like, I am not sure which side should do this, but feels like you'll get more potential compatibility out of it?

Sure I can do this.

While this technically works and may be the fastest path to interop with Mastodon, I wonder if it wouldn't be better for Mastodon to be able to display the Create/Video. Reason being is that in Pubstrate, Kroeg, and other applications which do understand Video as an independent object, now you'll see the same entry show up twice in your feed.

@Gargron how hard would it be to get Mastodon to start to understand Create/Video (and Image too?)

cwebber commented Nov 28, 2017

Have you considered publishing a Create/Note activity alongside the Create/Video one? The contents could just be title of the video and link to its page. I could implement this sort of fallback on Mastodon's side, if you like, I am not sure which side should do this, but feels like you'll get more potential compatibility out of it?

Sure I can do this.

While this technically works and may be the fastest path to interop with Mastodon, I wonder if it wouldn't be better for Mastodon to be able to display the Create/Video. Reason being is that in Pubstrate, Kroeg, and other applications which do understand Video as an independent object, now you'll see the same entry show up twice in your feed.

@Gargron how hard would it be to get Mastodon to start to understand Create/Video (and Image too?)

@Gargron

This comment has been minimized.

Show comment
Hide comment
@Gargron

Gargron Nov 29, 2017

@cwebber Not hard, I think. We'll just create a text post with title + link. Natively supporting video or image attachments less likely though. Especially video. If you think this is better than implementing text posts in PeerTube, sure. I can see your reasoning.

Gargron commented Nov 29, 2017

@cwebber Not hard, I think. We'll just create a text post with title + link. Natively supporting video or image attachments less likely though. Especially video. If you think this is better than implementing text posts in PeerTube, sure. I can see your reasoning.

@Chocobozzz

This comment has been minimized.

Show comment
Hide comment
@Chocobozzz

Chocobozzz Nov 29, 2017

Owner

Thanks @cwebber for the feedback.

@Gargron I'll fix the issues you pointed out in your MR. I just want to add that a comment system in PeerTube is not implemented yet, and it might take some weeks before I develop this feature. I'll try to be quick :)

Owner

Chocobozzz commented Nov 29, 2017

Thanks @cwebber for the feedback.

@Gargron I'll fix the issues you pointed out in your MR. I just want to add that a comment system in PeerTube is not implemented yet, and it might take some weeks before I develop this feature. I'll try to be quick :)

@Chocobozzz

This comment has been minimized.

Show comment
Hide comment
@Chocobozzz

Chocobozzz Dec 19, 2017

Owner

After some feedbacks (thanks @puckipedia), I removed the video channel collection. A video channel is now a classic actor, that announces its videos. We just send classic video Create activities.

I updated my comment that described the implemention.

I made some tests with Mastodon, and everything seems to work (thanks @Gargron!):

  • Send the video and embed it in Mastodon
  • Favourite it on Mastodon and send it to PeerTube (the opposite does not work, it seems that Mastodon does not handle remote likes?)
  • Share it on Mastodon and send it to PeerTube

Now I'll work on comments federation and improve the current implementation for a greater robustness. I think I can close this issue.

Thanks all :)

Owner

Chocobozzz commented Dec 19, 2017

After some feedbacks (thanks @puckipedia), I removed the video channel collection. A video channel is now a classic actor, that announces its videos. We just send classic video Create activities.

I updated my comment that described the implemention.

I made some tests with Mastodon, and everything seems to work (thanks @Gargron!):

  • Send the video and embed it in Mastodon
  • Favourite it on Mastodon and send it to PeerTube (the opposite does not work, it seems that Mastodon does not handle remote likes?)
  • Share it on Mastodon and send it to PeerTube

Now I'll work on comments federation and improve the current implementation for a greater robustness. I think I can close this issue.

Thanks all :)

@Chocobozzz Chocobozzz closed this Dec 19, 2017

@Gargron

This comment has been minimized.

Show comment
Hide comment
@Gargron

Gargron Dec 20, 2017

it seems that Mastodon does not handle remote likes?

I assure you that it does. Can you describe to me how you tried to do it?

Gargron commented Dec 20, 2017

it seems that Mastodon does not handle remote likes?

I assure you that it does. Can you describe to me how you tried to do it?

@Chocobozzz

This comment has been minimized.

Show comment
Hide comment
@Chocobozzz

Chocobozzz Dec 21, 2017

Owner
  • PeerTube instance creates a video and sends it to a Mastodon instance
  • Mastodon instance creates a status with the video URL
  • PeerTube user likes the video
  • PeerTube instance sends a Like activity to the Mastodon instance
  • Mastodon instance does not proceed the Like activity

From what I saw, Mastodon seems to reject the Like because the status account is not local: https://github.com/tootsuite/mastodon/blob/master/app/lib/activitypub/activity/like.rb#L7

But maybe I'm wrong.

Owner

Chocobozzz commented Dec 21, 2017

  • PeerTube instance creates a video and sends it to a Mastodon instance
  • Mastodon instance creates a status with the video URL
  • PeerTube user likes the video
  • PeerTube instance sends a Like activity to the Mastodon instance
  • Mastodon instance does not proceed the Like activity

From what I saw, Mastodon seems to reject the Like because the status account is not local: https://github.com/tootsuite/mastodon/blob/master/app/lib/activitypub/activity/like.rb#L7

But maybe I'm wrong.

@poVoq

This comment has been minimized.

Show comment
Hide comment
@poVoq

poVoq Jan 24, 2018

Just wanted to point out that Nextcloud also uses ActivityPub for federation and might be a really interesting target for allowing people to seed videos or something like that.

poVoq commented Jan 24, 2018

Just wanted to point out that Nextcloud also uses ActivityPub for federation and might be a really interesting target for allowing people to seed videos or something like that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment