YoutubeDL - How to get a status object after download has completed

[MichaelJAndy]

I'm trying basically to get information out of what seems to be a status object that's hitting the hook in Youtube-DL, and then I'm trying to save it to the db. I've got a 'song' object with attributes such as "filename" I'm trying to save once the download is complete, and maybe even continually update the database with progress.

There's three ways I can think of to do this but I've not been able to get them to work

• Send the my_hook function a db and song object and then save it all in there once status == finished. Problem is I'm unable to pass additional parameters to the hook unless I'm missing something
• Get the my_hook function to return d and then save that, problem is that I don't think I can access that it would return to (youtube-dl source)
• Get ydl.download([song.url]) to return a status object that I can process, I don't think it does this though
• I don't want to do this, but I can output a .json file and get it from there, or just simply guess the name of the file given that I'm dictating it initially 👎

def my_hook(d):
    if d['status'] == 'finished':
        file_tuple = os.path.split(os.path.abspath(d['filename']))
        print("Done downloading {}".format(file_tuple[1]))
    if d['status'] == 'downloading':
        print(d['filename'], d['_percent_str'], d['_eta_str'])

class MyLogger(object):
    def debug(self, msg):
        pass

    def warning(self, msg):
        pass

    def error(self, msg):
        print(msg)


class Downloader(object):
    def get_opts(self):
        ydl_opts = {
            'format': 'bestaudio/best',
            'outtmpl': os.path.join(app.config['VIDEOS_FOLDER'], '%(id)s.%(ext)s'),
            'logger': MyLogger(),
            'progress_hooks': [my_hook],
        }
        return ydl_opts

    def download(self, song):
        ydl = youtube_dl.YoutubeDL(self.get_opts())
        ydl.download([song.url])

[dstftw]

I don't see any problem with first approach. When download is finished my_hook is fired with finished status and you can put here any logic either writing to db right here or putting it to dbmanager queue or whatever you have. You can do anything in my_hook since it's your code.

[MichaelJAndy]

@dstftw I don't see how to pass the hook the relevant variables it needs in order to update the db or even get the row ID in order to do so.

I've tried altering this def my_hook(d): to be more like this def my_hook(d, song, db): but that causes problems because other parts of the youtube-dl code only expect it to be taking the status object.

Not only that, but then the line 'progress_hooks': [my_hook], expects me to detail out what my_hook should be expecting, but that also fails also.

Can you show an example where I can pass parameters to my_hook?

[dstftw]

You don't need to pass them to hook. You should resolve song, db and so on in your code.

def handle_finished(d):
    song = get_song_by_filename(d['filename'])
    db = get_db()
    db.write(song)

def my_hook(d):
    if d['status'] == 'finished':
        file_tuple = os.path.split(os.path.abspath(d['filename']))
        print("Done downloading {}".format(file_tuple[1]))
        handle_finished(d)
    if d['status'] == 'downloading':
        print(d['filename'], d['_percent_str'], d['_eta_str'])

[MichaelJAndy]

Yes, sorry I absolutely agree that I can create a new db object from the hook but the issue is that the 'filename' is not unique so I actually need to get the originating unique song.id known to my program and not youtube-dl - the only way I see that to be possible is to somehow pass it from def download(self, song): to the hook

[dstftw]

filename must be unique otherwise how do you expect it to behave when downloading two videos in the same file? id in output template provides well enough unicity. However if you want guaranteed unicity just generate a GUID and add it to output template.

[MichaelJAndy]

how do you expect it to behave when downloading two videos in the same file?

I didn't explain that well, the song file is not unique within the "song" table, it's possible for multiple entries in that table to be pointing at the same file. It's actually more like a "song request" that has a "song file" entry but all in one table.

Anyway thinking about it from the POV of your comments this may point to problems in the model itself, because of the one to many nature of "song requests" to "song files" I should break it down, this will make the file name unique in the "song file" entry, and I can fetch by youtubeID.

Thanks for your help!

[binksdata]

Hi Michael,

Would you be willing to share your code how to add downloaded descriptions to the database? Did you use MySQL?

Thanks

[MichaelJAndy]

Hi, the database I was using doesn't matter - I was using http://www.sqlalchemy.org/ which would allow me to connect the code to any supported database. It's usually quickest to simply create and connect to a SQLite DB for dev, then test on the target prod type DB, then deploy.

Have a read up on how sqlalchemy works, you simply define a model, objects with attributes represent tables and columns, you can then populate that model into a connected DB.

The code it out in the open if you want to take a look:
https://github.com/MichaelJAndy/flask-wallboard-music

[ubaidseth]

Here's what I used:

       with youtube_dl.YoutubeDL(ydl_opts) as ydl:
            meta = ydl.extract_info(link, download = False)
            print(meta.uploader)
            # and then I started it again (the YT process) to download the link, now that I know what the filename is going to be.
             ydl.download([link])

Hope that helps