Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[niconico] Support encrypted official videos #24093

Open
tsukumijima opened this issue Feb 19, 2020 · 2 comments
Open

[niconico] Support encrypted official videos #24093

tsukumijima opened this issue Feb 19, 2020 · 2 comments

Comments

@tsukumijima
Copy link

@tsukumijima tsukumijima commented Feb 19, 2020

Checklist

  • I'm reporting a site feature request
  • I've verified that I'm running youtube-dl version 2020.02.16
  • I've searched the bugtracker for similar site feature requests including closed ones

Description

I did a pull requests (#23824) before, but I'll post it to the issue just in case.

niconico (Nico Nico Douga) is not known outside of Japan, but is the largest video sharing site in Japan.

Problem

niconico began using HLS to encrypt the distribution of some official videos, such as anime, on March 5, 2019.
As a result, some official videos can no longer be downloaded with the previously used Extractor.
(Example of video that can be downloaded as before: https://www.nicovideo.jp/watch/sm32868846)
(Example of video that cannot be downloaded due to HLS encryption: https://www.nicovideo.jp/watch/so36189921)

Correction and resolution

The pull requests I mentioned earlier include an improvement that allows you to download niconico's encrypted HLS videos.
Specifically, it has been modified to make a request to the (most recently published) Ping API (otherwise the download will fail).
In addition, the encrypted HLS video has been modified to send different parameters to the API for acquiring the viewing session than the HTTP video.

On niconico, if you do not send a "heartbeat" every 60 seconds, the download session will be closed and a 403 error will occur.
Short videos can complete the download before the session closes, but long videos do not.
You can "resume" videos that support HTTP, but you can't resume videos that are HLS only.
Therefore, it is necessary to implement a "heartbeat".

The pull requests also include a change that allows you to send a "heartbeat" once every 60 seconds during the download, and to download long videos to the end.
I understand that although some pull requests implementing "heartbeat" have been sent in the past, such as #14582, they have not been merged even now doing.
The code for "heartbeat" is based on archiif/youtube-dl@8499534.
I tried it on various video sites, but adding this feature will not hinder other sites from downloading.
Also, since February 2018, it has been improved to play videos without logging in, so you do not need an account to download.

lastly

As a side change, we made changes to get high quality thumbnails and original titles.
In addition, support for "heartbeat" has made it easier to download normal niconico HTTP videos (especially long videos) as a side effect, eliminating the need to "resume".

I strongly hope that this change will be incorporated into the project.
Can you merge this (#23824) Pull requests?

Thank you.

(Sentence may be a bit strange because I am using translation)

@southrop
Copy link

@southrop southrop commented Feb 26, 2020

Would like to see this merged as well. Perhaps worth mentioning that archiif@8499534 is actually also a PR here as #18230

@tsukumijima
Copy link
Author

@tsukumijima tsukumijima commented Feb 28, 2020

Would like to see this merged as well. Perhaps worth mentioning that archiif/youtube-dl@8499534 is actually also a PR here as #18230

It is OK to Merge # 18230 before my pull request. However, it has been left for over a year, I don't think it will be merged into the project in the future ...
I thought # 18230 would never be merged anymore, so I incorporated the content of the # 18230 pull request.
I'm not very used to pull requests, but if I get in the way when merging, should I close this pull request once?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.