Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Self-hosted extractors: Mastodon, PeerTube and Misskey (with haruhi-dl merge) #1791

Open
wants to merge 37 commits into
base: master
Choose a base branch
from

Conversation

Lesmiscore
Copy link
Contributor

@Lesmiscore Lesmiscore commented Nov 25, 2021

Please follow the guide below

  • You will be asked some questions, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your pull request (like that [x])
  • Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check one of the following options:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

Both are intentionally checked because some code from this PR includes the code from haruhi-dl and #1505.

What is the purpose of your pull request?

  • Bug fix
  • Improvement
  • New extractor
  • New feature

Description of your pull request and other information

This PR adds support for 3 self-hosted services: Mastodon, PeerTube, Misskey
Also, this replaces GabIE with MastodonIE.

supersedes #1505, #4609
cc @selfisekai

TOC

  • Extractors
    • additional extractor for some instances (only Misskey)
  • Instance lists
  • Script to update instance list above
  • some other things

TODO

  • where to go: To download videos from Gab Social or Truth Social, use --ies mastodon.*,generic.
  • handling of currently failing tests (because it lacks instance list that are assumed to exist in the tree)

@selfisekai
Copy link
Contributor

I think the probe should first try fetching the instance's NodeInfo, which is a standard used in all of these services (and more of fediverse, see the standard's webpage; excluding Gab, for unknown to me reason, but it's not really a selfhosted service and it's always on the instance list) and allows to just see the software name and supported features

@pukkandan pukkandan added enhancement New feature or request site-request Request to support a new website labels Nov 27, 2021
Lesmiscore added a commit to ytdl-patched/ytdl-patched that referenced this pull request Nov 30, 2021
@Lesmiscore Lesmiscore changed the title Self-hosted extractors: Mastodon, PeerTube, Misskey (with haruhi-dl merge) Self-hosted extractors: Mastodon, PeerTube and Misskey (with haruhi-dl merge) Dec 12, 2021
@pukkandan pukkandan force-pushed the master branch 2 times, most recently from a63ff77 to b14d523 Compare May 18, 2022 03:35
pukkandan added a commit that referenced this pull request Aug 24, 2022
Deprecates `--force-generic-extractor`

Closes #3234, Closes #2044

Related: #4307, #1791
@Lesmiscore Lesmiscore force-pushed the the-melancholy-of-haruhi-suzumiya branch from 86459e2 to 5834fb0 Compare August 24, 2022 04:10
@Lesmiscore Lesmiscore marked this pull request as draft August 24, 2022 13:06
@Lesmiscore
Copy link
Contributor Author

Fallback instance lists will now have required ones for passing tests

@Lesmiscore Lesmiscore marked this pull request as ready for review August 29, 2022 14:17
@Lesmiscore Lesmiscore linked an issue Aug 29, 2022 that may be closed by this pull request
10 tasks
Lesmiscore added a commit to ytdl-patched/ytdl-patched that referenced this pull request Aug 31, 2022
@pukkandan pukkandan force-pushed the master branch 2 times, most recently from ee280c7 to 7aeda6c Compare May 24, 2023 18:09
@coletdjnz coletdjnz added the stale-pr PR that has been pending fixes for a long time label Sep 9, 2023
@Chocobozzz
Copy link
Contributor

Any news on this PR? :)

PeerTube relies heavily on yt-dlp to import remote channels/videos/playlists. It would be nice if the software could automatically detect a peertube, as we are limited for now by the URLs list written directly in the code.

@selfisekai
Copy link
Contributor

ugh, afaik the PR author is in jail, lmao

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request site-request Request to support a new website stale-pr PR that has been pending fixes for a long time
Projects
Status: Self hosted extraction
Development

Successfully merging this pull request may close these issues.

[Site support request] PeerTube instance greatview.video
5 participants