Add ability to log errors to a file

[ntcarver0]

Checklist

• I'm reporting a feature request
• I've verified that I'm running youtube-dl version 2020.07.28
• I've searched the bugtracker for similar feature requests including closed ones

Description

Could a feature be added so that errors encountered while downloading (failed downloads, ETC.) are logged to a file for perusal later? This would greatly help, as I have a script that downloads a large number of URLs and it's difficult to find if any failed.

[0l-l0]

Could you a bit elaborate on what kind of script you use? I mean, what language the script is written in, how you invoke youtube-dl etc.

[ntcarver0]

@0l-l0, the script is a Windows batch script simply out of convenience because it was the quickest way for me to do what I needed to do. The script looks like this:

@echo off
youtube-dl -i --no-continue --add-metadata --write-description -o "%%(extractor)s/%%(uploader)s/%%(uploader)s - %%(upload_date)s - %%(title)s.%%(ext)s" -a links.txt

[0l-l0]

How about I/O redirections? Don't they do the trick?
E.g.

@echo off
youtube-dl -i --no-continue --add-metadata --write-description ^
-o "%%(extractor)s/%%(uploader)s/%%(uploader)s - %%(upload_date)s - %%(title)s.%%(ext)s" ^
-a links.txt >your_log_file.log 2>&1

[ntcarver0]

That could work so long as the terminal output is logged and also printed to the terminal for monitoring.

[0l-l0]

That's where embedded youtube-dl can help. I sketched this little python script based on your settings and the youtube-dl manual. In this case the log will be written to the file youtube_dl.log as well as to the terminal.

import logging
import youtube_dl


class TeeLogger(object):
    def debug(self, msg):
        print(msg)
        logging.debug(msg)

    def warning(self, msg):
        print(msg)
        logging.warning(msg)

    def error(self, msg):
        print(msg)
        logging.error(msg)

def get_url_list(filename):
    batch_file = open(filename, 'r')

    url_list = []
    for line in batch_file:
        l = line.strip()
        if l and not l.startswith(('#', ';', ']')):
            url_list.append(l)

    return url_list

ydl_opts = {
    'addmetadata': True,
    'continuedl': False,
    'ignoreerrors': True,
    'logger': TeeLogger(),
    'outtmpl': '%(extractor)s/%(uploader)s/%(uploader)s - %(upload_date)s - %(title)s.%(ext)s',
    'writedescription': True,
}
with youtube_dl.YoutubeDL(ydl_opts) as ydl:
    logging.basicConfig(filename='youtube_dl.log', level=logging.NOTSET, format='')

    try:
        ydl.download(get_url_list('links.txt'))
    except:
        logging.exception('')
        raise

[0l-l0]

An even shorter one without reinventing the wheel:

import logging
import youtube_dl


class TeeLogger(object):
    def debug(self, msg):
        print(msg)
        logging.debug(msg)

    def warning(self, msg):
        print(msg)
        logging.warning(msg)

    def error(self, msg):
        print(msg)
        logging.error(msg)

ydl_opts = {
    'addmetadata': True,
    'continuedl': False,
    'ignoreerrors': True,
    'logger': TeeLogger(),
    'outtmpl': '%(extractor)s/%(uploader)s/%(uploader)s - %(upload_date)s - %(title)s.%(ext)s',
    'writedescription': True,
}
with youtube_dl.YoutubeDL(ydl_opts) as ydl:
    logging.basicConfig(filename='youtube_dl.log', level=logging.NOTSET, format='')

    try:
        batch_file = open('links.txt', 'r')
        ydl.download(youtube_dl.read_batch_urls(batch_file))
    except:
        logging.exception('')
        raise

[0l-l0]

Of course, if you don't wanna use Python for this but you have PowerShell installed, you can use the Tee-Object utility.

[ntcarver0]

@0l-l0, thanks for the scripts. It helps a lot.