Return codes/ Exit statuses -- are they documented ?

[esantoro]

Hello there!
I use youtube-dl regularily on my laptop and i've seen it's also used in shk3/edx-downloader (a tool to download videso from edx online university), which I forked tonight and that i started hacking.

Now: one of the bugs i noticed is that it invokes youtube-dl via os.system, but if something bad happens it just lets the download fails and won't re-download the video.

I was then wondering if there are some particular exit codes to check for.
Or can i just assume that anything different from zero is a bad return value ?

Thanks in advance,
Emanuele Santoro

[yasoob]

Hey you can invoke youtube-dl in your python scripts easily by first importing youtube-dl and logging:

import youtube_dl
import logging

Then you have to make these classes:

class NoneFile(object):
    '''
    A file-like object that does nothing
    '''
    def write(self,msg):
        pass
    def flush(self,*args,**kaargs):
        pass
    def isatty(self):
        return False

class ScreenFile(NoneFile):
    def write(self,msg):
        logging.debug(msg)

class SimpleFileDownloader(youtube_dl.FileDownloader):
    def __init__(self,*args,**kargs):
        super(SimpleFileDownloader,self).__init__(*args,**kargs)
        self._screen_file=ScreenFile()
        self._ies = youtube_dl.gen_extractors()
        for ie in self._ies:
            ie.set_downloader(self)

and then to extract the information for a specific video you can just use:

fd = SimpleFileDownloader({'outtmpl':'%(title)s'})
results = fd.extract_info(url, download = False)

I dont know why shk3/edx-downloader uses os.system I guess its creator didnt know that there is a more pythonic way to use youtube-dl in python scripts. If you have any furthur questions then feel free to post them here. I will try to help you as much as i can 👍

[yasoob]

Keep this in mind that the code which i wrote above just gives you the information of a video like :

>>> print results
{'extractor': u'dailymotion', '_type': 'compat_list', 'entries': [{'upload_date': None, 'playlist': None, 'title': u'Zindagi Gulzar Hai by Hum Tv - Episode 16 - Part 1/3', 'url': u'http://www.dailymotion.com/cdn/H264-512x384/video/xy8ddj.mp4?auth=1371110654-d4aeca72844fad8df507872ec579c19e', 'playlist_index': None, 'ext': 'mp4', 'extractor': u'dailymotion', 'uploader': u'pkdramas2013', 'id': 'xy8ddj'}]

It wont download the video for you ! If you want it to download the video then you have to use:

fd = SimpleFileDownloader({'outtmpl':'%(title)s'})
results = fd.extract_info(url, download = True)

If youtube-dl fails to extract the info then it will give you an error and you wont have any value in results variable. @esantoro Any further questions ?

[alphapapa]

May I appropriate this issue to request that youtube-dl use exit codes to indicate success or failure? I am running youtube-dl from a script, and because youtube-dl doesn't return anything but 0, I have to do things like redirect its output to a log and grep it to see if it failed, instead of simply:

if ! youtube-dl; then
  echo Oops
fi

[AMCerasoli]

I'm also interested in this feature (exit codes).

[jschwalbe]

+1 (if not already implemented)

[weedy]

No they are not. #852
If everything goes well you get 0 most of the time. Everything else is non-zero (1 I think I haven't looked in months).

[esantoro]

It might be cool to have some particular return codes at least for some macro-category of errors.

Assuming that 0 is success and everything else is failure, you might want to report when some situations are "recoverable" (the specifically requested format is unavailable? I might want to try and get whatever the best available format is).

It's been three years now, and random people still keep coming at this issue. I think it may be worth implementing this feature.

[yan12125]

On Linux exit codes can be 0~255. I guess there will eventually be more than 256 cases if we start to define exit codes. Another issue of exit codes is that those numbers are meaningless. I suggest using the Python API instead of relying on process exit codes for automatic jobs. Anyway there should be more exception classes than ExtractorError. A request can be found at #10308.

[esantoro]

I said

particular return codes at least for some macro-category of errors

@yan12125 said:

I guess there will eventually be more than 256 cases if we start to define exit codes.

This is why we should have some error codes for macro-categories of errors (network errors, API errors, IO errors etc), and some error codes for very common failures (video id unavailable, requested format unavailable, etc). And a command line flag for detailed failure description.

This way we can have macro- and micro- error reporting.

[yan12125]

Sorry, I didn't read the full thread.

Defining macro categories is a good idea. How about HTTP-like status codes? For example 150 indicate network errors, 51100 indicate I/O errors, and so on (I'm not sure what's "API errors" mean, could you explain more?)

[esantoro]

@yan12125 said:

I'm not sure what's "API errors" mean

API might be interpreted as:

• "you correctly reached the youtube API endpoint but supplied a wrong/insufficient/vague set of arguments" (examples: you asked for a video and supplied a video-id but there is no such video with that id, you asked for 1280x720 mp4 format, but there only is 480x360 mp4 format)

or

• "you called the youtube-dl script with wrong/insufficient command-line options"

The idea here is to let users know why youtube-dl failed, so that they can act consequently.

[kidol]

I think the long term goal should be that the extractors simply return predefined error codes with optional error messages. These error codes can then be used as exit codes as well.

1-100 = Reserved for network errors, usage errors etc. (everything not extractor related)

101 = Video not found
102 = Video deleted
103 = Country restricted
104 = Copyright claim
105 = Requires login (private video or requires subscription)

[yan12125]

101 is already implemented for "--max-downloads exceeded". Sounds like it's some "API error" proposed by @esantoro. It shouldn't be changed, so 10x can be reserved for API errors.

[esantoro]

@yan12125 , @kidol we are definitely movin in the right direction here!

[hz]

It's been six years now, and random people still keep coming at this issue. I think it may be worth implementing this feature.

[esantoro]

Yeah I'm honestly sorry I've been bothering the developers for the last six years with this, I really didn't mean to.
I still think it would be worth implementing, though.

[rag-hav]

+1 from me too.

[toppits]

would like to add my vote as well.

[esantoro]

Lol almost seven years and and people are still asking for this.

[Jokertion]

+1 That's useful for user.