Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conflict between --dump-pages and --dump-json #5787

Closed
Alavar opened this issue May 23, 2015 · 8 comments
Closed

Conflict between --dump-pages and --dump-json #5787

Alavar opened this issue May 23, 2015 · 8 comments

Comments

@Alavar
Copy link

@Alavar Alavar commented May 23, 2015

I'm trying to both dump the page source and the JSON in one single command.
Example: youtube-dl --dump-pages --dump-json URL

I am however only getting the output of --dump-json, but not the output of --dump-pages.

There are currently two solutions that I am able to think of:

  1. Run two commands; one getting the result of --dump-json and the other --dump-pages. The large downside to this solution is that all requests are going to be doubled as the sources would have to be grabbed twice.

  2. Using --write-pages works perfectly together with --dump-json, but the downside to this option is that the pages are saved to a folder which requires a script that goes through the outputted files and opens them. The other part which makes things really difficult is the ability to distinguish the order of the pages with --write-pages as some sites (YouTube for example) require additional requests (manifests).

Since the files all have the same creation time, it's hard to know which file was requested first which would require a regex making the issue even more complicated.

@jaimeMF
Copy link
Collaborator

@jaimeMF jaimeMF commented May 23, 2015

--dump-pages is only intended to debug problems, we won't make it work with --dump-json. So you have to use --write-pages.

@jaimeMF jaimeMF closed this May 23, 2015
@Alavar
Copy link
Author

@Alavar Alavar commented May 23, 2015

--dump-pages is only intended to debug problems, we won't make it work with --dump-json. So you have to use --write-pages.

How will users know which page was downloaded first?
It would be helpful if they were numbered

For example:
1 - http_yt_test.dmp
2 - http_yt_manifest.dmp

@jaimeMF
Copy link
Collaborator

@jaimeMF jaimeMF commented May 23, 2015

You can't, but since I gues you are not interested on the DASH manifest you can just look for the *watch?v=* pages.

@Alavar
Copy link
Author

@Alavar Alavar commented May 23, 2015

What about other sites?

@jaimeMF
Copy link
Collaborator

@jaimeMF jaimeMF commented May 23, 2015

You'll have to analyse case by case, you can't just use the same approach for all sites because the downloaded pages vary a lot (they can be: html, json, xml, javascript or some custom text format). Before you start doing it, you should really consider if you need to do it and if some of the info you need could be directly extracted by youtube-dl.

@Alavar
Copy link
Author

@Alavar Alavar commented May 23, 2015

If the JSON dump provided a simple value which showed if the signature was encrypted from the start then the problem would be solved. Something as simple as "encryptedSignature": 1 would do the trick.

The reason I want this is because I want to log the amount of videos that have encrypted signature and in some cases also restrict them from being downloaded.

@jaimeMF
Copy link
Collaborator

@jaimeMF jaimeMF commented May 23, 2015

If the JSON dump provided a simple value which showed if the signature was encrypted from the start then the problem would be solved. Something as simple as "encryptedSignature": 1 would do the trick.

Open a new issue, as I said in #5781 (comment)

@Alavar
Copy link
Author

@Alavar Alavar commented May 23, 2015

I didn't see your reply.
I will create a new issue, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.