Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google search youtube video links #7398

Open
9 of 11 tasks
TheHardew opened this issue Jun 23, 2023 · 3 comments
Open
9 of 11 tasks

Google search youtube video links #7398

TheHardew opened this issue Jun 23, 2023 · 3 comments
Labels
site-bug Issue with a specific website site-enhancement Feature request for some website

Comments

@TheHardew
Copy link

TheHardew commented Jun 23, 2023

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Region

Poland

Example URLs

https://www.google.com/search?q=proton%20vs%20windows%20fps#fpstate=ive&vld=cid:178fa594,vid:_EwD4hv7ZR8

Provide a description that is worded well enough to be understood

When searching google, it sometimes shows a video section. For example at: https://www.google.com/search?q=proton%20vs%20windows%20fps

In this video section, the title of the video links to youtube: https://www.youtube.com/watch?v=_EwD4hv7ZR8
But the thumbnail (somtimes) links to: https://www.google.com/search?q=proton%20vs%20windows%20fps#fpstate=ive&vld=cid:178fa594,vid:_EwD4hv7ZR8

It happens with youtube, not with vimeo, no idea about other providers
It seems to happen in qutebrowser but not in firefox
If the video section does not show up by itself, you can press on the video tab
The link might also include some tracking data, but the vid: part still exsits in such a case

It would be nice if yt-dlp could handle such links properly, downloading the youtube video

Edit: After I've logged in to google in Firefox it also started happening

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

~ $ yt-dlp -vU 'https://www.google.com/search?q=proton%20vs%20windows%20fps#fpstate=ive&vld=cid:178fa594,vid:_EwD4hv7ZR8'
[debug] Command-line config: ['-vU', 'https://www.google.com/search?q=proton%20vs%20windows%20fps#fpstate=ive&vld=cid:178fa594,vid:_EwD4hv7ZR8']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.06.22 [812cdfa06]
[debug] Python 3.11.3 (CPython x86_64 64bit) - Linux-6.3.3-zen1-1.1-zen-x86_64-with-glibc2.37 (OpenSSL 3.1.1 30 May 2023, glibc 2.37)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0
[debug] Optional libraries: Cryptodome-3.12.0, brotlicffi-1.0.9.2, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1851 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.06.22, Current version: stable@2023.06.22
yt-dlp is up to date (stable@2023.06.22)
[generic] Extracting URL: https://www.google.com/search?q=proton%20vs%20windows%20fps#fpstate=ive&vld=cid:178fa594,vid:_EwD4hv7ZR8
[generic] search?q=proton vs windows fps#fpstate=ive&vld=cid:178fa594,vid:_EwD4hv7ZR8: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] search?q=proton vs windows fps#fpstate=ive&vld=cid:178fa594,vid:_EwD4hv7ZR8: Extracting information
[debug] Looking for embeds
[redirect] Following redirect to https://www.google.com/search?q=proton+vs+windows+fps&gbv=1&sei=JoCVZNiKB4anqtsP0YSrgAw
[generic] Extracting URL: https://www.google.com/search?q=proton+vs+windows+fps&gbv=1&sei=JoCVZNiKB4anqtsP0YSrgAw
[generic] search?q=proton+vs+windows+fps&gbv=1&sei=JoCVZNiKB4anqtsP0YSrgAw: Downloading webpage
[redirect] Following redirect to https://consent.google.com/ml?continue=https://www.google.com/search%3Fq%3Dproton%2Bvs%2Bwindows%2Bfps%26gbv%3D1%26sei%3DJoCVZNiKB4anqtsP0YSrgAw&gl=PL&m=0&pc=srp&uxe=none&hl=pl&src=1
[generic] Extracting URL: https://consent.google.com/ml?continue=https://www.google.com/search%3Fq%3Dproton%2Bvs%2Bwindows%2Bfps%26gbv%3D1%26sei%3DJoCVZNiKB4anqtsP0YSrgAw&gl=PL&m=0&pc=srp&uxe=none&hl=pl&src=1
[generic] search?q=proton+vs+windows+fps&gbv=1&sei=JoCVZNiKB4anqtsP0YSrgAw&gl=PL&m=0&pc=srp&uxe=none&hl=pl&src=1: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] search?q=proton+vs+windows+fps&gbv=1&sei=JoCVZNiKB4anqtsP0YSrgAw&gl=PL&m=0&pc=srp&uxe=none&hl=pl&src=1: Extracting information
[debug] Looking for embeds
ERROR: Unsupported URL: https://consent.google.com/ml?continue=https://www.google.com/search%3Fq%3Dproton%2Bvs%2Bwindows%2Bfps%26gbv%3D1%26sei%3DJoCVZNiKB4anqtsP0YSrgAw&gl=PL&m=0&pc=srp&uxe=none&hl=pl&src=1
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1555, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1631, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 708, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/generic.py", line 2568, in _real_extract
    raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL: https://consent.google.com/ml?continue=https://www.google.com/search%3Fq%3Dproton%2Bvs%2Bwindows%2Bfps%26gbv%3D1%26sei%3DJoCVZNiKB4anqtsP0YSrgAw&gl=PL&m=0&pc=srp&uxe=none&hl=pl&src=1
@TheHardew TheHardew added site-request Request to support a new website triage Untriaged issue labels Jun 23, 2023
@pukkandan pukkandan added site-enhancement Feature request for some website site-bug Issue with a specific website and removed site-request Request to support a new website triage Untriaged issue labels Jun 23, 2023
@pukkandan
Copy link
Member

While the URL is unsupported, gvsearch: is supposed to be, but it looks like that is broken as well.

❯ yt-dlp "gvsearch1:proton windows fps" -v
[debug] Command-line config: ['--ignore-config', 'gvsearch1:proton windows fps', '-v']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.06.22 [812cdfa06] (source)
[debug] Lazy loading extractors is disabled
[debug] Git HEAD: 928e9d420
[debug] Python 3.11.0 (CPython AMD64 64bit) - Windows-10-10.0.22000-SP0 (OpenSSL 1.1.1q  5 Jul 2022)
[debug] exe versions: ffmpeg n5.1.2-8-g5746987bad-20221204 (fdk,setts), ffprobe N-109332-g45ab5307a6-20221201, phantomjs 2.1.1
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1851 extractors
[video.google:search] Extracting URL: gvsearch1:proton windows fps
[download] Downloading playlist: proton windows fps
[video.google:search] gvsearch:proton windows fps: Downloading result page 1
[video.google:search] Playlist proton windows fps: Downloading 1 items of 1
[download] Downloading item 1 of 1
[generic] Extracting URL: #fpstate=ive&amp;vld=cid:bbb27f26,vid:_EwD4hv7ZR8
ERROR: [generic] None: '#fpstate=ive&amp;vld=cid:bbb27f26,vid:_EwD4hv7ZR8' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:#fpstate=ive&amp;vld=cid:bbb27f26,vid:_EwD4hv7ZR8" ) to search YouTube
  File "D:\Programs\Source\yt-dlp\yt-dlp\yt_dlp\extractor\common.py", line 710, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Programs\Source\yt-dlp\yt-dlp\yt_dlp\extractor\generic.py", line 2403, in _real_extract
    raise ExtractorError(

[download] Finished downloading playlist: proton windows fps

@TheHardew
Copy link
Author

TheHardew commented Jun 23, 2023

Sorry, I searched for google in the supported sites and did not notice video.google:search way down the list

@pukkandan
Copy link
Member

No worries. It's just site bug than a site request. While we separate the templates, I don't actually expect users to be always able to tell the difference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
site-bug Issue with a specific website site-enhancement Feature request for some website
Projects
None yet
Development

No branches or pull requests

2 participants