Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Vbox7,DASH] Fix and improve extraction with DASH manifests using range and base URL #32710

Merged
merged 6 commits into from
Feb 2, 2024

Conversation

dirkf
Copy link
Contributor

@dirkf dirkf commented Jan 27, 2024

Boilerplate (own/yt-dl/yt-dlp code, bug fix/improvement) ## Please follow the guide below
  • You will be asked some questions, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your pull request (like that [x])
  • Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

  • I am the original author of this code and I am willing to release it under Unlicense, except for code adapted from PRs to this project and from yt-dlp, where this or the below has already been asserted
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

  • Bug fix
  • Improvement
  • New extractor
  • New feature
---

Description of your pull request and other information

This PR has three parts.

  1. Back-port DASH subtitle extraction from yt-dlp, included to avoid redundant changes when implementing subtitle extraction from manifest URLs or when porting these changes downstream.
  2. Support range-based DASH manifests, based on defunct PR support byte range for dash mpd #30279 from former GH user kikuyan (thx, fixes [common] _parse_mpd_formats doesn't handle range-based segment lists #13784, fixes Video from MPEG-DASH stream much larger than same video from HLS stream #30235), adapted to
  • match the above
  • use new APIs
  • properly resolve BaseURLs.
  1. Fix and improve extraction for site vbox7.com, based on defunct PR [extractor/vbox7] improve extraction #29680 from former GH user kikuyan (thx, fixes Vbox7.com broken #14685, fixes cant download from vbox7 #26063, fixes can't extract video from vbox7 #27575, resolves Urgent - Vbox7 support needs to be fixed, so that archivists can save nearly all its' videos before they're permanently lost on Feb 22 #32701). The site normally offers range-based DASH, although combined AV mp4 formats are also available.

* adapted from ytdl-org#30279
* thx former GH user kikuyan
Specs:
* ISO/IEC 23009-1:2012 section 5.6
* RFC 3986 section 5.
* DASH extraction no longer fails with new range support
* but always find combined formats if available
* suppress ineffective XFF geo-bypass (causes time-outs)
* adapted from ytdl-org#29680
* thx former GH user kikuyan
* based on PR ytdl-org#29680
* added hack to force invoking `transform_source`
* fixes ytdl-org#26218
@stdakov
Copy link

stdakov commented Feb 2, 2024

How can I manually install it and test it?

@dirkf
Copy link
Contributor Author

dirkf commented Feb 2, 2024

Supposing that you have a supported Python installation, git clone the source repo; then go into the youtube-dl directory and use python -m youtube_dl as the youtube-dl command (where python is what you type to run Python).

Otherwise you can manually replace the changed files in an installation from this repo's master or from the latest nightly build. You may need to become admin to do that: if so, also run youtube-dl --version as admin to ensure that cached byte-code files are updated (or the system will ignore the new sources in favour of the previous compiled versions).

@stdakov
Copy link

stdakov commented Feb 2, 2024

Supposing that you have a supported Python installation, git clone the source repo; then go into the youtube-dl directory and use python -m youtube_dl as the youtube-dl command (where python is what you type to run Python).

Otherwise you can manually replace the changed files in an installation from this repo's master or from the latest nightly build. You may need to become admin to do that: if so, also run youtube-dl --version as admin to ensure that cached byte-code files are updated (or the system will ignore the new sources in favour of the previous compiled versions).

it works for downloading videos and m4a
Thank you!

@dirkf
Copy link
Contributor Author

dirkf commented Feb 2, 2024

There are some extra features in the yt-dlp PR that can be added, but this can probably be merged as-is for now.

@dirkf dirkf merged commit 4416f82 into ytdl-org:master Feb 2, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment