Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Podcast RSS feeds #5487

Merged
merged 92 commits into from
May 22, 2023
Merged

Add Podcast RSS feeds #5487

merged 92 commits into from
May 22, 2023

Conversation

agates
Copy link
Contributor

@agates agates commented Dec 28, 2022

Description

The primary purpose of this PR is to add a new podcast Feed format that generates "Podcast" RSS feeds using The Podcast Namespace and to make the feeds easily consumable by standard podcast players and podcast indexes.

The secondary purpose is to add support for live streams in RSS as well as additional infrastructure to make live streams more usable in podcast applications. It does so by clearing the cache of the Podcast RSS feed whenever the live stream goes live or ends, which ensures any client will have up-to-date information for the live stream.

It also adds the following action hook:

  • action:live.video.state.updated - Gets fired when a live stream starts or ends. Makes it possible for plugins to handle live stream notifications, which I will be using for a podping.cloud plugin.

And the following three filter hooks:

  • filter:feed.podcast.channel.create-custom-tags.result - Returns a list of CustomTag objects for plugins to add custom elements to the Podcast RSS channel.
  • filter:feed.podcast.video.create-custom-tags.result - Returns a list of CustomTag objects for plugins to add custom elements to Podcast RSS items or liveItems.
  • filter:feed.podcast.rss.create-custom-xmlns.result - Returns a list of CustomXMLNS objects for plugins to add custom XMLNS definitions to the Podcast RSS feed.

Lastly, an isEmailPublic boolean field is added to the UserModel to allow the user to opt-in to showing their email address in the <podcast:locked> tag. This allows the user to "claim" their feed in various podcast services to prove that it is theirs. I put the toggle button for this above the "Change your email" section of the user profile. It very clearly needs work (see screenshot).

Related issues

Podcasting 2.0 RSS support
Publish livestreams in RSS feed

Has this been tested?

  • 👍 yes, I added tests to the test suite
  • 💭 no, because this PR is a draft and still needs work
  • 🙅 no, because this PR does not update server code
  • 🙋 no, because I need help

Screenshots

screenshot of draft is email public toggle

Other notes

Support for Apple Podcast is not a purpose of this PR as one would need to do additional testing on an Apple device, but adding the extra itunes tags to the feed generator wouldn't be too difficult. However, Apple requires very large images (3000x3000, I think).

Additionally, I made the decision not to include fragmented MP4s in the feeds because they offer an extremely poor user experience where some players wont play them correctly or only play the first segment. If a webtorrent version of a video exists, it will default to the webtorrent MP4. If only HLS exists, it defaults to the m3u8 file.

Lastly, this PR supports only channel level podcast feeds as that is the generally accepted standard use case for podcasting. I don't think it would be difficult to extend to account- or subscription-level but as that is not a primary goal I haven't included it in testing.

Feed dependency

I'm making this PR with the dependency on my fork of @peertube/feed, pfeed-podcast, because my PR there is still open. This can and should be changed once that is updated.

agates added 30 commits March 21, 2021 18:29
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.

I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.

I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.

I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
Remove video/stream titles, add optional height attribute to podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.

I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
Remove video/stream titles, add optional height attribute to podcast RSS
@Agorise
Copy link

Agorise commented May 3, 2023

We'll implement every bit of this into hive-tube, this is so exciting! Great work Alecks and Chocobozzz!! :)

@agates
Copy link
Contributor Author

agates commented May 4, 2023

The PR Add support for feed generation using the Podcast Namespace is also a dependency.

@agates agates requested a review from Chocobozzz May 4, 2023 02:40
Copy link
Owner

@Chocobozzz Chocobozzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

I merged the develop branch in this PR. I also updated the feed dependency.

},
nsfw,
// Prevent podcast feeds from listing videos in other instances
// helps prevent duplicates when they are indexed -- only the author should control them
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure to understand why we would not want to generate a feed of podcast from remote instances?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Essentially this boils down to the concept of ownership, duplicate feeds, and the user experience in the end user application.

Feeds get user-submitted and often automatically crawled for inclusion into podcast indexes such as https://podcastindex.org. When multiple feeds of one podcast get submitted there comes into a question of who owns it and which one is "correct".

For the same reasons I don't allow additional URL parameters because all of the different parameters get included as duplicate.

That said, I'm actually not sure how relevant this is on the channel level. Querying by the video channel ID will only include the local videos anyway, right?

return flatten(this.VideoStreamingPlaylists.map(p => videoFilesModelToFormattedJSON(this, p.VideoFiles, { includeMagnet })))
}

getFormattedVideoFilesWithStreamingPlaylistsJSON (includeMagnet = true): VideoFile[] {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure to understand the difference with the previous method?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to separate out getFormattedVideoFilesJSON to avoid deduplicating videos after the query. We need to get all video files and this seemed like the best approach. Otherwise I'd just be duplicating the logic of videoFilesModelToFormattedJSON.

I believe you had suggested to use getMaxQualityFile but that only gets the highest quality and not all available files.

@Chocobozzz
Copy link
Owner

Thanks, I'll update this PR to integrate your changes in develop. Then you'll be able to create other PRs if there things to fix/improve.

@Chocobozzz
Copy link
Owner

Chocobozzz commented May 17, 2023

Do we consider the channel mandatory? Or can we generate a podcast feed of local videos of the instance?

Found the answer, it's a yes :)

@Agorise
Copy link

Agorise commented May 21, 2023

Down to just 1 failed check now, that's awesome! I so hope we can see all this work in v5.2 :)

@Chocobozzz Chocobozzz merged commit cb0eda5 into Chocobozzz:develop May 22, 2023
@Chocobozzz
Copy link
Owner

Thanks @agates!

@agates agates deleted the feature/podcast branch May 27, 2023 03:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants