No metadata was returned by youtube-dl

[KandyMan1990]

I've just attempted to set up TubeSync on my Synology NAS using docker and it is failing to download anything. I've looked into the logs and came across this:

2023-07-18 14:33:32,863 [tubesync/DEBUG] [youtube:tab] Extracting URL: https://www.youtube.com/channel/UCDrqiuwNRbEahL1UEB0hkKQ/videos
2023-07-18 14:33:32,863 [tubesync/DEBUG] [youtube:tab] UCDrqiuwNRbEahL1UEB0hkKQ/videos: Downloading webpage
2023-07-18 14:33:52,882 [tubesync/WARNING] [youtube:tab] <urlopen error [Errno -3] Temporary failure in name resolution>. Retrying (1/3)...
2023-07-18 14:33:52,882 [tubesync/DEBUG] [youtube:tab] UCDrqiuwNRbEahL1UEB0hkKQ/videos: Downloading webpage
2023-07-18 14:34:12,898 [tubesync/WARNING] [youtube:tab] <urlopen error [Errno -3] Temporary failure in name resolution>. Retrying (2/3)...
2023-07-18 14:34:12,898 [tubesync/DEBUG] [youtube:tab] UCDrqiuwNRbEahL1UEB0hkKQ/videos: Downloading webpage
2023-07-18 14:34:32,916 [tubesync/WARNING] [youtube:tab] <urlopen error [Errno -3] Temporary failure in name resolution>. Retrying (3/3)...
2023-07-18 14:34:32,916 [tubesync/DEBUG] [youtube:tab] UCDrqiuwNRbEahL1UEB0hkKQ/videos: Downloading webpage
2023-07-18 14:34:52,933 [tubesync/WARNING] [youtube:tab] Unable to download webpage: <urlopen error [Errno -3] Temporary failure in name resolution> (caused by URLError(gaierror(-3, 'Temporary failure in name resolution'))). Giving up after 3 retries
2023-07-18 14:34:52,934 [tubesync/DEBUG] [youtube:tab] UCDrqiuwNRbEahL1UEB0hkKQ/videos: Downloading API parameters API JSON
2023-07-18 14:35:12,947 [tubesync/WARNING] [youtube:tab] <urlopen error [Errno -3] Temporary failure in name resolution>. Retrying (1/3)...
2023-07-18 14:35:12,948 [tubesync/DEBUG] [youtube:tab] UCDrqiuwNRbEahL1UEB0hkKQ/videos: Downloading API parameters API JSON
2023-07-18 14:35:32,969 [tubesync/WARNING] [youtube:tab] <urlopen error [Errno -3] Temporary failure in name resolution>. Retrying (2/3)...
2023-07-18 14:35:32,970 [tubesync/DEBUG] [youtube:tab] UCDrqiuwNRbEahL1UEB0hkKQ/videos: Downloading API parameters API JSON
2023-07-18 14:35:52,988 [tubesync/WARNING] [youtube:tab] <urlopen error [Errno -3] Temporary failure in name resolution>. Retrying (3/3)...
2023-07-18 14:35:52,989 [tubesync/DEBUG] [youtube:tab] UCDrqiuwNRbEahL1UEB0hkKQ/videos: Downloading API parameters API JSON
2023-07-18 14:36:13,014 [tubesync/ERROR] ERROR: [youtube:tab] UCDrqiuwNRbEahL1UEB0hkKQ: Unable to download API page: <urlopen error [Errno -3] Temporary failure in name resolution> (caused by URLError(gaierror(-3, 'Temporary failure in name resolution')))
Rescheduling Index media from source "beyblade"
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/background_task/tasks.py", line 43, in bg_runner
func(*args, **kwargs)
File "/app/sync/tasks.py", line 159, in index_source_task
videos = source.index_media()
^^^^^^^^^^^^^^^^^^^^
File "/app/sync/models.py", line 517, in index_media
response = indexer(self.index_url)
^^^^^^^^^^^^^^^^^^^^^^^
File "/app/sync/youtube.py", line 60, in get_media_info
raise YouTubeError(f'Failed to extract_info for "{url}": No metadata was '
sync.youtube.YouTubeError: Failed to extract_info for "https://www.youtube.com/channel/UCDrqiuwNRbEahL1UEB0hkKQ/videos": No metadata was returned by youtube-dl, check for error messages in the logs above. This task will be retried later with an exponential backoff.

I'm not sure if this is an error in the latest build or on error on my end. If anyone could help me I'd really appreciate it, I have a feeling it's probably a set up issue. I've tried restarting docker and the issue persists.

[meeb]

The actual error here buried in your traceback is "Temporary failure in name resolution". That would imply that tubesync can't resolve the host "www.youtube.com". Something would appear to be broken with your DNS resolver in your containers. The channel URL looks fine to me otherwise.

[KandyMan1990]

I've figured it out, just needed to allow the docker network through the Synology firewall, thanks.

Unrelated but I'm seeing the container is using over 5.5gb of ram for downloading 522 videos. Is that normal? Before it had even started downloading them, just indexing and thumbnails took it up to that much usage, I've seen it as high as 5.75gb. I had to shut down a bunch of other containers to free up some ram as the Synology only has 8gb total. Even though a bunch of videos are downloaded it's still staying around 5.6gb, should it not be coming back down as the downloads finish?

[meeb]

That does seem like way too much RAM allocated. Some other people have reported some memory issues previously but I've never been able to replicate it to find the cause. Can you see what in the container is using RAM?

[KandyMan1990]

I restarted the container after that large download was complete as the system was maxed out. I currently have 6 playlists being watched, all of which finished downloading last night, and currently it is sat at 1.87gb ram usage

[KandyMan1990]

If you want to try and replicate it, you can see in my original post the channel I was attempting to download, maybe that's a good test case?

[meeb]

The issue is the current (jank) worker system. On my system it releases RAM after downloads regardless of what channels I add. It seems in some deployments it doesn't and the RAM usage continues to increase over time. Generally as I was going to replace the workers at some point I didn't spend that much time looking into. Your process list looks OK, even if the gunicorn workers do seem to be allocating way more RAM than I would expect.

[mavrik132]

I am having a similar problem.

When I add a channel ID for some channels, I get the same "No metadata was returned"... error. The traceback shows "[tubesync/ERROR] ERROR: [youtube:tab] @bigclivedotcom: Unable to download API page: HTTP Error 404: Not Found (caused by <HTTPError 404: Not Found>); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U"

The interesting part is I get a 404 if I access the URL displayed by TubeSync manually in chrome, but if I remove the "c/" from the URL, chrome will load the page.

[alexander-henkes]

I am having a similar problem.

When I add a channel ID for some channels, I get the same "No metadata was returned"... error. The traceback shows "[tubesync/ERROR] ERROR: [youtube:tab] @bigclivedotcom: Unable to download API page: HTTP Error 404: Not Found (caused by <HTTPError 404: Not Found>); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U"

The interesting part is I get a 404 if I access the URL displayed by TubeSync manually in chrome, but if I remove the "c/" from the URL, chrome will load the page.

On which device do you use Tubesync? If you have used docker compose, a new network will be created, which may need to be enabled in the firewall. Here is the example on a Synology NAS.

Alternatively, you can also integrate Tubesync into the bridge network. Then usually no firewall approval is necessary. Add "network_mode: bridge" to your compose.yaml.

[meeb]

@alexander-henkes thanks for the input, however a 404 error isn't likely to be a network issue. The 404 error is returned by YouTube so @mavrik132 's container can connect out and has networking.

@mavrik132 likely you are experiencing an issue with some channels not being channels, discussed here: #389 #299 etc.

[meeb]

This looks resolved, thanks @tcely !