Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CNBCVideo error #8378

Closed
10 of 11 tasks
jacobrosenfeld opened this issue Oct 18, 2023 · 2 comments · Fixed by #8741
Closed
10 of 11 tasks

CNBCVideo error #8378

jacobrosenfeld opened this issue Oct 18, 2023 · 2 comments · Fixed by #8741
Labels
site-bug Issue with a specific website

Comments

@jacobrosenfeld
Copy link

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Region

USA

Provide a description that is worded well enough to be understood

CNBCVideo not working

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['https://www.cnbc.com/video/2023/10/18/the-palestinians-are-not-hamas-says-yeshiva-university-president-rabbi-ari-berman.html', '-vU']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.10.13 [b634ba742] (pip)
[debug] Python 3.12.0 (CPython arm64 64bit) - macOS-14.0-arm64-arm-64bit (OpenSSL 3.1.3 19 Sep 2023)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.43.2, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1890 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.10.13, Current version: stable@2023.10.13
yt-dlp is up to date (stable@2023.10.13)
[CNBCVideo] Extracting URL: https://www.cnbc.com/video/2023/10/18/the-palestinians-are-not-hamas-says-yeshiva-university-president-rabbi-ari-berman.html
[CNBCVideo] the-palestinians-are-not-hamas-says-yeshiva-university-president-rabbi-ari-berman: Downloading JSON metadata
[CNBC] Extracting URL: http://video.cnbc.com/gallery/?video=7000319414
[ThePlatform] Extracting URL: http://link.theplatform.com/s/gZWlPC/media/guid/2408950221/7000319414?mbr=true&manifest=m3u#__youtubedl_smuggle=%7B%22force_smil_url%22%3A+true%7D
[ThePlatform] 7000319414: Downloading SMIL data
ERROR: [ThePlatform] 7000319414: Unable to download XML: HTTP Error 404: Not Found (caused by <HTTPError 404: Not Found>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 715, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/extractor/theplatform.py", line 303, in _real_extract
    formats, subtitles = self._extract_theplatform_smil(smil_url, video_id)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/extractor/theplatform.py", line 36, in _extract_theplatform_smil
    meta = self._download_xml(
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 1069, in download_content
    res = getattr(self, download_handle.__name__)(url_or_request, video_id, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 1033, in download_handle
    res = self._download_webpage_handle(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/extractor/adobepass.py", line 1366, in _download_webpage_handle
    return super(AdobePassIE, self)._download_webpage_handle(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 903, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 860, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/networking/_urllib.py", line 410, in _send
    res = opener.open(urllib_req, timeout=float(request.extensions.get('timeout') or self.timeout))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 521, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 630, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 559, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 492, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 639, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/YoutubeDL.py", line 4051, in urlopen
    return self._request_director.send(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/networking/common.py", line 114, in send
    response = handler.send(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/networking/_helper.py", line 204, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/networking/common.py", line 325, in send
    return self._send(request)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/networking/_urllib.py", line 415, in _send
    raise HTTPError(UrllibResponseAdapter(e.fp), redirect_loop='redirect error' in str(e)) from e
yt_dlp.networking.exceptions.HTTPError: HTTP Error 404: Not Found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 847, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.10.13/libexec/lib/python3.12/site-packages/yt_dlp/YoutubeDL.py", line 4070, in urlopen
    raise _CompatHTTPError(e) from e
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 404: Not Found
@jacobrosenfeld jacobrosenfeld added site-bug Issue with a specific website triage Untriaged issue labels Oct 18, 2023
@bashonly
Copy link
Member

Related: #5871

But this seems like a different issue

@Noor-5
Copy link
Contributor

Noor-5 commented Nov 28, 2023

My group and I are in a Software Engineering class in college. We will implement an extractor specifically for CNBC to tackle this issue.

gonzalezjo pushed a commit to gonzalezjo/yt-dlp that referenced this issue Dec 10, 2023
Fixes yt-dlp#8378 by rewriting the CNBC video extractor.

- Removes `CNBCIE`, which is an extractor for a CNBC video platform that no longer exists.
- Updates CNBCVideoIE
    - Adds tests
    - Supports author, channel, formats, ID (of course), upload date, URL (of course), title (of course), description, duration, timestamp, and thumbnails.

Squashed commit of the following:

commit c4c6a3e
Author: J. Gonzalez <gonzalezjo@users.noreply.github.comm>
Date:   Sat Dec 9 20:12:51 2023 -0500

    Final cleanups

commit 98837e5
Author: J. Gonzalez <gonzalezjo@users.noreply.github.comm>
Date:   Sat Dec 9 20:06:01 2023 -0500

    Rebase 'Further cleanups + better extraction'

    Cool.

commit 210a0e5
Author: zhijinwuu <zhijinw@andrew.cmu.edu>
Date:   Sat Dec 9 19:48:58 2023 -0500

    new tests

commit 91fcdfb
Author: zhijinwuu <zhijinw@andrew.cmu.edu>
Date:   Sat Dec 9 19:41:45 2023 -0500

    added a test

commit 28a0100
Author: ruiminggu <ruimingg@andrew.cmu.edu>
Date:   Sat Dec 9 19:11:34 2023 -0500

    fixed test

commit 37c4b72
Author: ruiminggu <ruimingg@andrew.cmu.edu>
Date:   Sat Dec 9 19:11:17 2023 -0500

    fixed test

commit 6f42a60
Author: J. Gonzalez <gonzalezjo@users.noreply.github.comm>
Date:   Sat Dec 9 18:47:17 2023 -0500

    Remove CNBCIE

    Seems to not exist anymore.

commit d9dd742
Author: J. Gonzalez <gonzalezjo@users.noreply.github.comm>
Date:   Sat Dec 9 18:44:11 2023 -0500

    Code cleanups

commit 72e9d82
Merge: b272679 b5260a1
Author: J. Gonzalez <gonzalezjo@users.noreply.github.comm>
Date:   Sat Dec 9 18:36:30 2023 -0500

    Merge branch 'rachel-cnbc' into cnbc

commit b272679
Merge: 1823eb9 188191a
Author: J. Gonzalez <gonzalezjo@users.noreply.github.comm>
Date:   Sat Dec 9 18:35:33 2023 -0500

    Merge branch 'noor-cnbc' into cnbc

commit 188191a
Author: J. Gonzalez <gonzalezjo@users.noreply.github.comm>
Date:   Sat Dec 9 18:34:11 2023 -0500

    Simplify noor-code

commit 7c43a9d
Author: J. Gonzalez <gonzalezjo@users.noreply.github.comm>
Date:   Sat Dec 9 17:37:00 2023 -0500

    Mostly fix tests

    Nice.

commit db204a5
Author: Noor Mostafa <noormostafa@noors-air-3.wifi.local.cmu.edu>
Date:   Sat Dec 9 17:18:57 2023 -0500

    test case for CNBC video

commit b5260a1
Author: zhijinwuu <zhijinw@andrew.cmu.edu>
Date:   Sat Dec 9 17:17:04 2023 -0500

    format upload_date

commit 1823eb9
Author: ruiminggu <ruimingg@andrew.cmu.edu>
Date:   Sat Dec 9 15:55:04 2023 -0500

    modify for properly capitalize the channel

commit c164f09
Author: zhijinwuu <zhijinw@andrew.cmu.edu>
Date:   Sat Dec 9 15:38:40 2023 -0500

    add the case where author is NA

commit 3bee446
Author: zhijinwuu <zhijinw@andrew.cmu.edu>
Date:   Sat Dec 9 15:26:03 2023 -0500

    upload_date, video_status, author

commit 668503b
Author: zhijinwuu <zhijinw@andrew.cmu.edu>
Date:   Sat Dec 9 15:25:08 2023 -0500

    original

commit fb8f173
Author: zhijinwuu <zhijinw@andrew.cmu.edu>
Date:   Sat Dec 9 15:09:44 2023 -0500

    upload_date, video_status, and author

commit b57090b
Author: Noor Mostafa <noormostafa@noors-air-3.wifi.local.cmu.edu>
Date:   Fri Dec 8 16:24:36 2023 -0500

    metadata using builtin functions

commit a3b0068
Author: ruiminggu <ruimingg@andrew.cmu.edu>
Date:   Fri Dec 8 14:59:57 2023 -0500

    channel and duration

commit 09ccfd7
Author: Noor Mostafa <noormostafa@noors-air-3.wifi.local.cmu.edu>
Date:   Thu Dec 7 17:16:17 2023 -0500

    Added metadata for thumbnail, timestamp, and description

commit fcb554e
Author: Noor Mostafa <noormostafa@noors-air-3.wifi.local.cmu.edu>
Date:   Thu Dec 7 14:51:10 2023 -0500

    thumbnail metadata

commit 2101192
Author: J. Gonzalez <gonzalezjo@users.noreply.github.comm>
Date:   Thu Dec 7 12:45:36 2023 -0500

    Basic video extraction supported for CNBC.

    Does the absolute barest minimum.
@pukkandan pukkandan removed the triage Untriaged issue label Dec 10, 2023
seproDev added a commit that referenced this issue Feb 23, 2024
Closes #5871, Closes #8378
Authored by: gonzalezjo, Noor-5, zhijinwuu, ruiminggu, seproDev

Co-authored-by: Noor Mostafa <93787875+Noor-5@users.noreply.github.com>
Co-authored-by: zhijinwuu <zhijinw@andrew.cmu.edu>
Co-authored-by: ruiminggu <ruimingg@andrew.cmu.edu>
Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
aalsuwaidi pushed a commit to aalsuwaidi/yt-dlp that referenced this issue Apr 21, 2024
Closes yt-dlp#5871, Closes yt-dlp#8378
Authored by: gonzalezjo, Noor-5, zhijinwuu, ruiminggu, seproDev

Co-authored-by: Noor Mostafa <93787875+Noor-5@users.noreply.github.com>
Co-authored-by: zhijinwuu <zhijinw@andrew.cmu.edu>
Co-authored-by: ruiminggu <ruimingg@andrew.cmu.edu>
Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
site-bug Issue with a specific website
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants