-j and --write-description now incompatible with each other?

[joeschmoe40]

I have a script that runs youtube-dl like this:

youtube-dl.py -j --write-description -o tmptmp "$url" | something ...

The objective of the above line is to write the description to the file "tmptmp.description" as well as to write the "json" data to the standard output (it is piped into some further program(s) that analyze the json data).

This was working correctly as recently as October 23rd (2014), but got broken some time after that (in particular, it is broken in the 10/27/2014 version). Now, I get no "tmptmp.description" file. Note that no error message is generated; it just silently ignores it. If I remove "-j" form the command line, then it works as before. So, it seems like it is a "one or the other, but not both" situation.

I would like to see the previous behavior restored. I have work-around-ed it in my script, but I'd prefer it work the way it used to.

Note: This is all on Mac OSX, using the Unix command line.

[jaimeMF]

That was an intentional change (see #3036). Since the help message for --dump-json says simulate, quiet but print JSON information we don't write anything to disk (see the --simulate option).
Note that you can access the description from the json data itself, so you can write it to the tmptmp.description file if you want.

[joeschmoe40]

OK. Yes, it is easy enough to workaround, using the description in the json data.

As for correctness, I'm sure that's in the in the eye of the beholder; given that I wrote my script based on the existing behavior, I liked the old behavior. I take it, then, that the new behavior is intended to be permanent, right? So, I can permanently remove the parts of the script that assume the old behavior?

Also, it seems like it should produce at least a warning - to the effect that the combination of options is incompatible - that you're not going to get any output even though you specified --output.

Edit: Also note: The old behavior was desirable in at least one respect, which is that the output in the file was cleaner, better formatted, etc than the data in the json dump. I think it was broken up into separate lines and also some escape sequences get expanded - i.e., the json dump contains things like "\n". Because I don't have an old version to test, I can't easily verify this at the moment, but it would be desirable (at least for me) if there were a way to get the old behavior (maybe some speciel switch that says "Yes, I really do want some output along with my json dump".

[joeschmoe40]

Actually, after a bit more poking around, I figured out this command line, which gets you pretty close:

youtube-dl.py --skip-download --write-info-json --write-descrption -o tmptmp "$url"

This gets you a json file and a description file, which is good, except for one little problem.

The file created by the --write-info-json option is almost, but not quite, the same as the output you get using "-j". For one thing, there is no _filename option in the --write-info-json file. Not that this in particular is a show-stopper, but it makes you wonder. Why is it slightly different? Just like with the description text - where what you get in the output of "-j" is subtly different from what you get from --write-description. It all makes one wonder…

[jaimeMF]

Sorry for the delay. In the next version the _filename field will always exist, thanks for reporting it.

About the \n, they are perfectly fine in json, I don't know which programming language are you using for processing the output, but it will probably have json libraries. For shell scripts I recommend using jq.

About the description differences, for the test video http://www.youtube.com/watch?v=BaW_jenozKc, they look identical to me:

$ youtube-dl test:youtube -j | jq .description -r
test chars:  "'/\ä↭𝕐
test URL: https://github.com/rg3/youtube-dl/issues/1892

This is a test video for youtube-dl.

For more information, contact phihag@phihag.de .

$ cat youtube-dl\ test\ video\ \'\'_ä↭𝕐-BaW_jenozKc.info.json | jq .description -r
test chars:  "'/\ä↭𝕐
test URL: https://github.com/rg3/youtube-dl/issues/1892

This is a test video for youtube-dl.

For more information, contact phihag@phihag.de .

$ cat youtube-dl\ test\ video\ \'\'_ä↭𝕐-BaW_jenozKc.mp4.description
test chars:  "'/\ä↭𝕐
test URL: https://github.com/rg3/youtube-dl/issues/1892

This is a test video for youtube-dl.

For more information, contact phihag@phihag.de .

I think the only difference is that the .description file has a newline at the end, but I thinks that's expected, since most files have it.

I'm closing the issue, but feel free to open a new one if you think the description in the json dump has any problem or with any other issue you find.

[phihag]

I believe this issue to be fixed in youtube-dl version 2015.01.30.1 and newer. See our FAQ if you need help updating.

[joeschmoe40]

[quote]I believe this issue to be fixed in youtube-dl version 2015.01.30.1 and newer. See our FAQ if you need help updating.[/quote]

Two comments:

1. When you say "I believe this issue to be fixed", to which issue do you specifically refer? There were several issues raised in this thread (I should know, I started most of them)

2. I recently "upgraded" to version 2015.01.30.2, and found a major breakage (not related to this thread specifically). Whenever I view a certain kind of video (fetched via youtube-dl), VLC plays it as if it was only 10 seconds long. Backing up to an earlier version of youtube-dl fixes the problem, so I know that's where the problem is.

[jaimeMF]

1. The issue of _filename not always being present. If you want to have warnings for conflicting options, please open a new issue.
2. As explained in the BUG reporting guide, open a new issue with the output of youtube-dl --verbose URL, otherwise we can't really do anything.

[joeschmoe40]

The issue of _filename not always being present. If you want to have warnings for conflicting options, please open a new issue.

OK - thanks for the clarification. No worries there.

As explained in the BUG reporting guide, open a new issue with the output of youtube-dl --verbose URL, otherwise we can't really do anything.

No worries. I'm sure other people will notice this same problem and file more formal reports on it.