Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do I rename previously-downloaded files with nothing but the youtube video ID? #5692

Closed
6 of 8 tasks
cxr6548 opened this issue Dec 2, 2022 · 16 comments
Closed
6 of 8 tasks
Labels
question Question

Comments

@cxr6548
Copy link

cxr6548 commented Dec 2, 2022

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

  • I'm asking a question and not reporting a bug or requesting a feature
  • I've looked through the README
  • I've verified that I'm running yt-dlp version 2022.11.11 (update instructions) or later (specify commit)
  • I've searched the bugtracker for similar questions including closed ones. DO NOT post duplicates
  • I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood

I have already checked #3301 but it references a JSON file I don't have.

I do have a downloaded.txt in addition to each file having the ID in its filename.

How can I use yt-dlp and/or other tools to rename every previously-downloaded video? In particular I'm interested in adding the upload date to every filename.

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

No response

@cxr6548 cxr6548 added the question Question label Dec 2, 2022
@jab416171
Copy link

I'm doing this right now to move videos into folders named after the channel, what I'm doing is this, where u is the folder/channel name and url is the channel url:

yt-dlp -s "$url" -P "home:youtube/$u" > "youtube/$u"/videos
for i in $(grep -E "youtube.*Downloading android" "youtube/$u/videos" | awk '{print $2}' | tr -d ":" | sort | uniq ); do mv -iv *"$i"* "$u"; done

The key here is the -s and not passing in --download-archive

@bashonly
Copy link
Member

bashonly commented Dec 3, 2022

Here's a solution using bash and yt-dlp (and no other utils).

  • It assumes you want to have the youtube ID in the final filename.
  • If yt-dlp extraction fails, the script will abort renaming that video and add that archive entry to retry_archive.txt in your current working directory.
  • It will rename ALL files in your download_path whose filenames contain archived youtube IDs (not just videos) to the new output template, but attempt to retain their original extension.
  • A potential unwanted side effect may be that *.info.json files are renamed to only *.json -- though it sounds like you don't have info JSON files for any of the videos you want renamed
  • IMPORTANT: The script also assumes you don't have multiple videos with the same youtube ID and same file extension (e.g. abc12345efg-720p.mp4 and abc12345efg-1080p.mp4). If you do, running this script would result in all but one of those versions being overwritten.

Edit the commented lines as necessary:

#!/usr/bin/env bash

outtmpl="%(upload_date)s-%(id)s-%(title)s.%(ext)s"  # adjust to your desired output template
archive="/path/to/archive/downloaded.txt"  # replace with actual archive file path
download_path="/path/to/downloads/folder"  # replace with actual directory path

pattern='^youtube(:\w+)?\s([A-Za-z0-9_-]{10}[AEIMQUYcgkosw048])$'
downloads=("${download_path}/"*)
while read -r input; do
    [[ "${input}" =~ ${pattern} ]] || continue
    input_id="${BASH_REMATCH[2]}"
    filename=$(yt-dlp --no-download-archive -o "${outtmpl}" --get-filename -- "${input_id}")
    [[ "${filename}" =~ ${input_id} ]] || { echo "${input}" >> "retry_archive.txt"; continue; }
    for download in "${downloads[@]}"; do
        [[ "${download}" =~ ${input_id} && -f "${download}" ]] || continue
        final_path="${download_path}/${filename%.*}.${download/#*.}"
        [[ "${download}" == "${final_path}" ]] && continue
        mv -- "${download}" "${final_path}" || break 2
    done
done < "${archive}"

@54m4d
Copy link

54m4d commented Dec 4, 2022

Any idea how to approach this on Windows?

@dirkf
Copy link
Contributor

dirkf commented Dec 4, 2022

@Grub4K
Copy link
Member

Grub4K commented Dec 4, 2022

Python version using yt_dlp API
import shutil
from pathlib import Path

from yt_dlp import YoutubeDL

outtmpl = "%(upload_date)s-%(id)s-%(title)s.%(ext)s"
archive_path = Path("P:/ath/to/archive.txt")
download_path = Path("P:/ath/to/download/folder/")
result_path = Path("P:/ath/to/result/folder/")

with archive_path.open("r") as file, YoutubeDL({"outtmpl": outtmpl}) as ydl:
    for line in file:
        line = line.strip()
        if not line:
            continue

        ie_name, _, video_id = line.partition(" ")
        if not ie_name.startswith("youtube"):
            continue

        candidates = list(download_path.rglob(f"*{video_id}*"))
        if not candidates:
            print(f"{video_id}: ERROR: No candidate to rename found, skipping")
            continue
        elif len(candidates) > 1:
            print(f"{video_id}: ERROR: Multiple candidate to rename found, skipping")
            continue

        info_dict = ydl.extract_info(video_id, download=False)
        result_name = ydl.prepare_filename(info_dict, warn=False)

        candidate = candidates[0]
        previous_name = candidate.relative_to(download_path)
        destination = (result_path/result_name).resolve()
        if destination.exists():
            print(f'{video_id}: ERROR: "{destination}" already exists, skipping')
            continue

        print(f'{video_id}: Renaming "{previous_name}" => "{result_name}"')
        destination.parent.mkdir(parents=True, exist_ok=True)
        shutil.move(candidate, destination)

If you don't have Python installed you can try this code, though Batch is a bit jank so might not work 100%. It might break on certain characters, and allows some code injections through %%C and %%F

Batch version using yt-dlp.exe
@echo off
setlocal EnableDelayedExpansion

:: Make sure to double `%` in batch
set "outtmpl=%%(upload_date)s-%%(id)s-%%(title)s.%%(ext)s"
set "archivePath=P:\ath\to\archive.txt"
set "downloadPath=P:\ath\to\download\folder\"

pushd "!downloadPath!"
for /F "usebackq tokens=1,2 delims= " %%A in ("!archivePath!") do (
    set "ieName=%%A"
    if "!ieName:~0,7!"=="youtube" for /F "delims=" %%F in (
        'yt-dlp --no-download-archive -o "!outtmpl!" --get-filename -- "%%B"'
    ) do (
        set "foundName="
        for /R %%C in ("*%%B*") do (
            if defined foundName (
                echo(ERROR: ID "%%B" exists multiple times, skipping.
                set "foundName=."
            ) else (
                set "foundName=%%C"
                set "foundPath=%%~dpC"
            )
        )
        if defined foundName if not "!foundName!"=="." (
            echo(Renaming "!foundName!" to "%%F"
            mkdir "!foundPath!"
            move "!foundName!" "!foundPath!\%%F"
        )
    )
)
popd

@54m4d
Copy link

54m4d commented Dec 4, 2022

My Linux abilities are very limited.

Can i rather put this batch file in a folder where the files that need to be renamed are, edit it, and run it so it will rename all the files in this specific folder?

But i also need to know what to edit to make it work properly..

@Grub4K
Copy link
Member

Grub4K commented Dec 4, 2022

The top three lines set lines are your configuration, adjust them to your need.

:: Desired output template
set "outtmpl=%%(upload_date)s-%%(id)s-%%(title)s.%%(ext)s"
:: Path to your download archive
set "archivePath=P:\ath\to\archive.txt"
:: Path to the folder containing downloaded files
set "downloadPath=P:\ath\to\download\folder\"

Please be careful when using Batch as it might break easily. prefer the Python or Bash versions if possible.

@54m4d
Copy link

54m4d commented Dec 4, 2022

Thanks!

Can i eliminate archive since i only have a folder with the video files and haven't used archive?

:: Path to your download archive
set "archivePath=P:\ath\to\archive.txt"

@bashonly
Copy link
Member

bashonly commented Dec 4, 2022

Can i eliminate archive since i only have a folder with the video files and haven't used archive?

No. None of these scripts will work without a download archive file. Furthermore, any videos without entries in your archive would not be renamed.

@54m4d
Copy link

54m4d commented Dec 4, 2022

Can i eliminate archive since i only have a folder with the video files and haven't used archive?

No. None of these scripts will work without a download archive file. Furthermore, any videos without entries in your archive would not be renamed.

Even if i don't want to use the ID in the file name?

Let's just say: "%%(upload_date)s-%%(title)s.%%(ext)s"

@bashonly
Copy link
Member

bashonly commented Dec 4, 2022

Even if i don't want to use the ID in the file name?

All of the above scripts rely on the ID values from the archive to find the files that are to be renamed. There's no point in trying to use them without an archive.

The bash solution states that it will only work if you intend to have the ID in the final filename. IMO it's not a good idea to strip the ID from the filename anyways, as that would make it very difficult for you if you wanted to rename them again using metadata values.

For a solution that would work without a download archive, all of the youtube videos you want renamed would need to have been originally named with the same output template, or at least very similar output templates (e.g. the youtube ID is always at the end of the filename), and the script would need to be written to handle that specific naming scheme.

@cxr6548
Copy link
Author

cxr6548 commented Dec 5, 2022

bashonly's script Here's a solution using bash and yt-dlp (and no other utils).
 * It assumes you want to have the youtube ID in the final filename.

 * If yt-dlp extraction fails, the script will abort renaming that video and add that archive entry to `retry_archive.txt` in your current working directory.

 * It will rename ALL files in your `download_path` whose filenames contain archived youtube IDs (not just videos) to the new output template, but attempt to retain their original extension.

 * A potential unwanted side effect may be that `*.info.json` files are renamed to only `*.json` -- though it sounds like you don't have info JSON files for any of the videos you want renamed

 * **IMPORTANT**: The script also assumes you don't have multiple videos with the same youtube ID **and** same file extension (e.g. `abc12345efg-720p.mp4` and `abc12345efg-1080p.mp4`). If you do, running this script would result in all but one of those versions being overwritten.

Edit the commented lines as necessary:

#!/usr/bin/env bash

outtmpl="%(upload_date)s-%(id)s-%(title)s.%(ext)s"  # adjust to your desired output template
archive="/path/to/archive/downloaded.txt"  # replace with actual archive file path
download_path="/path/to/downloads/folder"  # replace with actual directory path

pattern='^youtube(:\w+)?\s([A-Za-z0-9_-]{10}[AEIMQUYcgkosw048])$'
downloads=("${download_path}/"*)
while read -r input; do
    [[ "${input}" =~ ${pattern} ]] || continue
    input_id="${BASH_REMATCH[2]}"
    filename=$(yt-dlp -o "${outtmpl}" --get-filename -- "${input_id}")
    [[ "${filename}" =~ ${input_id} ]] || { echo "${input}" >> "retry_archive.txt"; continue; }
    for download in "${downloads[@]}"; do
        [[ "${download}" =~ ${input_id} && -f "${download}" ]] || continue
        final_path="${download_path}/${filename%.*}.${download/#*.}"
        [[ "${download}" == "${final_path}" ]] && continue
        mv -- "${download}" "${final_path}" || break 2
    done
done < "${archive}"

This is exactly what I had in mind. Should I close the issue since my problem is solved or leave it open for the benefit of windows/mac users?

@cxr6548
Copy link
Author

cxr6548 commented Dec 5, 2022

It's worth mentioning that the bash script silently failed to change the filenames when I had these options in my ~/.config/yt-dlp/config file

#--cookies /home/me/.config/yt-dlp/cookies.txt
#--download-archive downloaded.txt
#-o "%(upload_date)s-%(title).96s-[%(id)s].%(ext)s"

I'm pretty sure only the output line caused trouble since retry_archive.txt was populated appropiately

@bashonly
Copy link
Member

bashonly commented Dec 5, 2022

It's worth mentioning that the bash script silently failed to change the filenames when I had these options in my ~/.config/yt-dlp/config file

The --download-archive arg in your config was what caused the trouble. This was an oversight on my part, I should've added --no-download-archive to the yt-dlp command line in the script. I've edited it in my comment above.

@pukkandan
Copy link
Member

While the question was technically out of scope, this is a frequently asked question - so I have pinned it. If anyone has better/alternate scripts for automating this, feel free to continue posting them here

@rileywbaker
Copy link

I wrote a bash script to rename Twitch VODs previously downloaded by yt-dlp and use a new output template. It doesn't require an archive.txt, instead it gets all its information from the existing files.

It's cluttered with features specific to my use-case and it assumes a plex-friendly media library layout but it might be a helpful starting place for someone. It also does timezone conversion from Unix epoch timestamp to localized ISO-8601, which I needed for something else.

https://gist.github.com/rileywbaker/d58dec09e43027d26b95d2a219953a54

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Question
Projects
None yet
Development

No branches or pull requests

8 participants