Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tvp] "Video requires payment and log-in, but log-in is not implemented" #6987

Closed
11 tasks done
Displacedpie opened this issue May 4, 2023 · 10 comments · Fixed by #6989
Closed
11 tasks done

[tvp] "Video requires payment and log-in, but log-in is not implemented" #6987

Displacedpie opened this issue May 4, 2023 · 10 comments · Fixed by #6989
Labels
site-bug Issue with a specific website

Comments

@Displacedpie
Copy link

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Region

Canada

Provide a description that is worded well enough to be understood

I've been downloading episodes from website: Vod.tvp.pl / https://vod.tvp.pl/seriale,18/polowanie-na-cmy-odcinki,390116

I've been able to get every episode with no issues what so ever, with the exception of Episode #7. https://vod.tvp.pl/seriale,18/polowanie-na-cmy-odcinki,390116/odcinek-7,S01E07,398869

[tvp:vod] Extracting URL: https://vod.tvp.pl/seriale,18/polowanie-na-cmy-odcinki,390116/odcinek-7,S01E07,398869
[tvp:vod] 398869: Downloading JSON metadata
[tvp:embed] Extracting URL: tvp:69230468
[tvp:embed] 69230468: Downloading webpage
ERROR: [tvp:embed] 69230468: Video requires payment and log-in, but log-in is not implemented; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U

(Same error with -vU)

The video works in browser, I've created an account and tried --cookies-from-browser chrome with no luck.

I've updated twice hoping for a resolution - video has been available since April 30th, 2023.

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://vod.tvp.pl/seriale,18/polowanie-na-cmy-odcinki,390116/odcinek-7,S01E07,398869']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@2023.05.03.204705 [2f07c4c1d] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19044-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: none
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.2
[debug] Proxy map: {}
[debug] Loaded 1813 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Available version: nightly@2023.05.03.204705, Current version: nightly@2023.05.03.204705
Current Build Hash: b73b1cb4c64dcbd61a084ba9910dca9837cedb7b450c75605ec5d197cdc8ad01
yt-dlp is up to date (nightly@2023.05.03.204705)
[tvp:vod] Extracting URL: https://vod.tvp.pl/seriale,18/polowanie-na-cmy-odcinki,390116/odcinek-7,S01E07,398869
[tvp:vod] 398869: Downloading JSON metadata
[tvp:embed] Extracting URL: tvp:69230468
[tvp:embed] 69230468: Downloading webpage
ERROR: [tvp:embed] 69230468: Video requires payment and log-in, but log-in is not implemented; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 694, in extract
  File "yt_dlp\extractor\tvp.py", line 392, in _real_extract
@Displacedpie Displacedpie added site-bug Issue with a specific website triage Untriaged issue labels May 4, 2023
@dirkf
Copy link
Contributor

dirkf commented May 4, 2023

The extractor finds the TVP id 69230468 from a metadata API URL (the returned data includes a lot of other and tries this URL: https://www.tvp.pl/sess/TVPlayer2/api.php?id=69230468&@method=getTvpConfig&@callback=sasin70 (callback is picked randomly from a list). Fetching that URL returns a page with JSON like this

[{'context': 'id',
  'data': None,
  'desc': 'Obiekt wymaga płatności',
  'type': 'Error'}]

The browser JS is using some different method, or passing additional headers, which might become apparent from a network trace using the browser's development tools.

@Displacedpie
Copy link
Author

Thank you for the reply!

Could I ask for advice on how to run a network trace?

I'm pretty new to all of this.

@dirkf
Copy link
Contributor

dirkf commented May 4, 2023

Actually, the answer is this: https://vod.tvp.pl/api/products/398869/videos/playlist?platform=BROWSER&videoType=MOVIE

JSON playlist result
{
  'sources': {
    'HLS': [
      {
        'src': 'https://rsdt-krk201-4.tvp.pl/token/video/vod/69230468/20230504/785384984/X2P8b7AC2hD-POBG5j1lWx35b4kPhUN76BrY8Sx2N6hXDVBtsw6o8GFB1VSqmw5aDiZutk97EpJd64s-0KV8ZM7s59QW7UQvuwJQVNIv84e9mTIiuLMsfDRW0bM3EigHpQLnSybEg7TeQPaG_-uSuPPlpVcGlC5o8r5cNu3lG_Q/video.ism/video-fmp4.m3u8'
      }
    ],
    'DASH': [
      {
        'src': 'https://rsdt-krk201-4.tvp.pl/token/video/vod/69230468/20230504/785384984/X2P8b7AC2hD-POBG5j1lWx35b4kPhUN76BrY8Sx2N6hXDVBtsw6o8GFB1VSqmw5aDiZutk97EpJd64s-0KV8ZM7s59QW7UQvuwJQVNIv84e9mTIiuLMsfDRW0bM3EigHpQLnSybEg7TeQPaG_-uSuPPlpVcGlC5o8r5cNu3lG_Q/video.ism/video.mpd'
      }
    ]
  },
  'stripe': {
    'id': 120545,
    'url': 'https://klatki.tvp.pl/rbig69230468/stripe_{index}.jpg',
    'interval': 2,
    'count': 8,
    'width': 640,
    'height': 360
  },
  'externalVideoSource': true
}

According to ytdl-org/youtube-dl#32028, there is a new scheme at TVP; your problem show seems only to be accessible through that scheme. Instead of passing the external ID (69230468) it now uses the ID 398869 from the URL. The videoType=MOVIE is required: I have no idea if other values exist.

Once the yt-dl PR is merged (or ready), it'll come here.

@selfisekai
Copy link
Contributor

I'm also surprised to see that it just plays for me in NL now, so I guess they've loosened the restrictions

@bashonly bashonly changed the title Trouble with only one Episode on site. [tvp] "Video requires payment and log-in, but log-in is not implemented" May 4, 2023
@bashonly bashonly removed the triage Untriaged issue label May 4, 2023
selfisekai added a commit to selfisekai/yt-dlp that referenced this issue May 4, 2023
@gamer191
Copy link
Collaborator

gamer191 commented May 4, 2023

Could I ask for advice on how to run a network trace?

This issue seems to have already been solved, but if you want to learn reverse engineering, I’d recommend playing with your browser’s dev tools (ctrl-shift-i). The network tab is the most useful, since it lets you see what network requests the webpage is making (in other words, it lets you see where the website is getting the video from)

@october262
Copy link

this video - https://vod.tvp.pl/seriale,18/polowanie-na-cmy-odcinki,390116/odcinek-7,S01E07,398869 can be downloaded & played without payment.
i used brave browser & the browser add-on called "the stream detector"
set the stream detector to ffmpeg, now choose the option to watch with ads,
copy the mpd & download.

@dirkf
Copy link
Contributor

dirkf commented May 5, 2023

That's what OP said. The problem is that the API URL used in the existing extractor returns that error: solved in PR #6989.

pukkandan pushed a commit that referenced this issue May 6, 2023
Authored by: selfisekai
Closes #6987
@Gwynnbleid1
Copy link

Gwynnbleid1 commented Jun 13, 2023

I tried to check if this fix would let me download a movie (https://vod.tvp.pl/filmy-fabularne,136/masz-talent,386831) from the same site.
I have updated my binary to nigtly and got:

.\yt-dlp.exe "https://vod.tvp.pl/filmy-fabularne,136/masz-talent,386831"
[tvp:vod] Extracting URL: https://vod.tvp.pl/filmy-fabularne,136/masz-talent,386831
[tvp:vod] 386831: Downloading JSON metadata
ERROR: '_io.BytesIO' object has no attribute 'status'

I got the same result with nightly@2023.05.06.001010.

With nightly@2023.05.05.055923 I received:
.\yt-dlp.exe "https://vod.tvp.pl/filmy-fabularne,136/masz-talent,386831"
[tvp:vod] Extracting URL: https://vod.tvp.pl/filmy-fabularne,136/masz-talent,386831
[tvp:vod] 386831: Downloading JSON metadata
[tvp:embed] Extracting URL: tvp:33745643
[tvp:embed] 33745643: Downloading webpage
ERROR: [tvp:embed] 33745643: Video requires payment and log-in, but log-in is not implemented; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U

Here is result of -vU command:

.\yt-dlp.exe -vU "https://vod.tvp.pl/filmy-fabularne,136/masz-talent,386831"
[debug] Command-line config: ['-vU', 'https://vod.tvp.pl/filmy-fabularne,136/masz-talent,386831']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@2023.05.05.055923 [ddae337] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k 25 Mar 2021)
[debug] exe versions: ffmpeg 2023-01-04-git-4a80db5fc2-full_build-www.gyan.dev (setts), ffprobe 2023-01-04-git-4a80db5fc2-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.2
[debug] Proxy map: {}
[debug] Loaded 1813 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Available version: nightly@2023.06.13.032334, Current version: nightly@2023.05.05.055923
Current Build Hash: 2cab78434aaa749ed9a8a2bb87585fd233a33a8940f521efcc26a4b2c5e55d0f
[debug] Downloading _update_spec from https://github.com/yt-dlp/yt-dlp-nightly-builds/releases/latest/download/_update_spec
Updating to nightly@2023.06.13.032334 ...
[debug] Downloading yt-dlp.exe from https://github.com/yt-dlp/yt-dlp-nightly-builds/releases/latest/download/yt-dlp.exe
[debug] Downloading SHA2-256SUMS from https://github.com/yt-dlp/yt-dlp-nightly-builds/releases/latest/download/SHA2-256SUMS
Updated yt-dlp to nightly@2023.06.13.032334
[debug] Restarting: "C:\Users\Matys\Downloads\ffmpeg-2023-01-04-git-4a80db5fc2-full_build\bin\yt-dlp.exe" -vU "https://vod.tvp.pl/filmy-fabularne,136/masz-talent,386831"
[debug] Command-line config: ['-vU', 'https://vod.tvp.pl/filmy-fabularne,136/masz-talent,386831']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@2023.06.13.032334 [cab94a0] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k 25 Mar 2021)
[debug] exe versions: ffmpeg 2023-01-04-git-4a80db5fc2-full_build-www.gyan.dev (setts), ffprobe 2023-01-04-git-4a80db5fc2-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1843 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Available version: nightly@2023.06.13.032334, Current version: nightly@2023.06.13.032334
Current Build Hash: 1e2a38e9ddb7199bf1fc977c322e9fbffd4daca6eb77e05b840fdb619306c623
yt-dlp is up to date (nightly@2023.06.13.032334)
[tvp:vod] Extracting URL: https://vod.tvp.pl/filmy-fabularne,136/masz-talent,386831
[tvp:vod] 386831: Downloading JSON metadata
ERROR: '_io.BytesIO' object has no attribute 'status'
Traceback (most recent call last):
File "yt_dlp\YoutubeDL.py", line 1552, in wrapper
File "yt_dlp\YoutubeDL.py", line 1628, in __extract_info
File "yt_dlp\extractor\common.py", line 704, in extract
File "yt_dlp\extractor\tvp.py", line 568, in _real_extract
File "yt_dlp\extractor\tvp.py", line 491, in _call_api
File "tempfile.py", line 468, in getattr
AttributeError: '_io.BytesIO' object has no attribute 'status'

Am I doing something wrong or should I post a new issue releated to this ?

@dirkf
Copy link
Contributor

dirkf commented Jun 16, 2023

In this fragment from the updated extractor code (l.485-), the status attribute isn't supported in Py < 3.9:

    def _call_api(self, resource, video_id, query={}, **kwargs):
        is_valid = lambda x: 200 <= x < 300
        document, urlh = self._download_json_handle(
            f'{self._API_BASE_URL}/{resource}', video_id,
            query={'lang': 'pl', 'platform': 'BROWSER', **query},
            expected_status=lambda x: is_valid(x) or 400 <= x < 500, **kwargs)
        if is_valid(urlh.status):
            return document
        raise ExtractorError(f'Woronicza said: {document.get("code")} (HTTP {urlh.status})')

For the claimed 3.7+ support, something like this is needed in a compat module

import urllib.response
try:
    urllib.response.addinfourl.status
except AttributeError:
    urllib.response.addinfourl.status = property(lambda self: self.getcode())

@coletdjnz
Copy link
Member

coletdjnz commented Jun 16, 2023

IMO All extractors that are incorrectly using .status need to be updated to use getcode() for now. Alternatively, in the YoutubeDL layer (in urlopen), we could inject the property as a temporary measure.

No point adding compat support in the backend - we're merging a rewrite very soon which adds our own response interface.

aalsuwaidi pushed a commit to aalsuwaidi/yt-dlp that referenced this issue Apr 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
site-bug Issue with a specific website
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants