Youtube Search does not work when cookies are provided

[peet1993]

Checklist

• I'm reporting a broken site support issue
• I've verified that I'm running youtube-dl version 2020.07.28
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar bug reports including closed ones
• I've read bugs section in FAQ

Verbose log

./youtube-dl-executable -v -- "ytsearch10:broken dreams"
[debug] System config: [u'--cookies=/home/<username>/cookies.txt', u'--force-ipv4']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--', u'ytsearch10:broken dreams']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.07.28
[debug] Python version 2.7.16 (CPython) - Linux-4.19.0-9-amd64-x86_64-with-debian-10.4
[debug] exe versions: ffmpeg 4.1.6-1, ffprobe 4.1.6-1
[debug] Proxy map: {}
[youtube:search] query "broken dreams": Downloading page 1
ERROR: An extractor error has occurred. (caused by KeyError(1,)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "./youtube-dl-executable/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "./youtube-dl-executable/youtube_dl/extractor/common.py", line 3005, in _real_extract
    return self._get_n_results(query, n)
  File "./youtube-dl-executable/youtube_dl/extractor/youtube.py", line 3183, in _get_n_results
    html_content = data[1]['body']['content']
KeyError: 1
Traceback (most recent call last):
  File "./youtube-dl-executable/youtube_dl/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "./youtube-dl-executable/youtube_dl/extractor/common.py", line 543, in extract
    raise ExtractorError('An extractor error has occurred.', cause=e)
ExtractorError: An extractor error has occurred. (caused by KeyError(1,)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

The Youtube Search currently does not work when there are session cookies for YouTube provided. Without a provided cookies file, the search works correctly. I implemented a proof-of-concept workaround that fixes the issue, but that just removes the cookies from the downloader during search and re-adds them afterwards: peet1993/youtube-dl@56817ed

The crash happens because with a valid login session, YouTube answers the search query with {"reload":"now"} instead of the search result as expected by youtube-dl. I'm not sure why that happens. However, it is reproducible with curl:

curl -v -b Desktop/cookies.txt 'https://www.youtube.com/results?search_query=test&spf=navigate&disable_polymer=true'
*   Trying 2607:f8b0:4002:c09::be:443...
* Connected to www.youtube.com (2607:f8b0:4002:c09::be) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=*.google.com
*  start date: Jul  7 08:04:38 2020 GMT
*  expire date: Sep 29 08:04:38 2020 GMT
*  subjectAltName: host "www.youtube.com" matched cert's "*.youtube.com"
*  issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55cb713d19f0)
> GET /results?search_query=test&spf=navigate&disable_polymer=true HTTP/2
> Host: www.youtube.com
> user-agent: curl/7.71.1
> accept: */*
> cookie: <other cookies>; SIDCC=AJi4Q<truncrated>Zqi; <other cookies>
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200 
< x-frame-options: SAMEORIGIN
< content-type: application/json; charset=UTF-8
< cache-control: no-cache
< expires: Tue, 27 Apr 1971 19:44:06 GMT
< x-content-type-options: nosniff
< accept-ch: DPR
< accept-ch-lifetime: 2592000
< strict-transport-security: max-age=31536000
< content-disposition: attachment
< date: Sun, 02 Aug 2020 16:27:42 GMT
< server: YouTube Frontend Proxy
< x-xss-protection: 0
* Replaced cookie SIDCC="AJi4Q<truncrated>GGSM" for domain youtube.com, path /, expire 1627921662
< set-cookie: SIDCC=AJi4Q<truncrated>GGSM; expires=Mon, 02-Aug-2021 16:27:42 GMT; path=/; domain=.youtube.com; priority=high
< alt-svc: h3-29=":443"; ma=2592000,h3-27=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
< 
* Connection #0 to host www.youtube.com left intact
{"reload":"now"}%

The issue was previously reported in #26204, but OP closed it after I added my workaround as reference.

[peet1993]

It would be nice to at least have a reference to what this is a duplicate of so I can follow it.