Issue with syfy when using comcast sso

[joshtish]

Please follow the guide below

• You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
• Put an x into all the boxes [ ] relevant to your issue (like this: [x])
• Use the Preview tab to see what your issue will actually look like

---

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2019.04.17. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

• I've verified and I assure that I'm running youtube-dl 2019.04.17

Before submitting an issue make sure you have:

• At least skimmed through the README, most notably the FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones
• Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

---

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

---

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'https://www.syfy.com/themagicians/videos/413-no-better-to-be-safe-than-sorry', u'--ap-mso=Comcast_SSO', u'--ap-username=PRIVATE', u'--verbose']
Type TV provider account password and press [Return]: 
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.04.17
[debug] Python version 2.7.10 (CPython) - Darwin-18.5.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3, rtmpdump 2.4
[debug] Proxy map: {}
[Syfy] 413-no-better-to-be-safe-than-sorry: Downloading webpage
[Syfy] 3940836: Downloading Provider Redirect Page
ERROR: Unable to extract post url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 529, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/syfy.py", line 47, in _real_extract
    url, video_id, 'syfy', resource)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/adobepass.py", line 1448, in _extract_mvpd_auth
    self._DOWNLOADING_LOGIN_PAGE)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/adobepass.py", line 1360, in post_form
    post_url = self._html_search_regex(r'<form[^>]+action=(["\'])(?P<url>.+?)\1', form_page, 'post url', group='url')
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1013, in _html_search_regex
    res = self._search_regex(pattern, string, name, default, fatal, flags, group)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1004, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract post url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description of your issue, suggested solution and other information

Attempting to use youtube-dl on syfy using my comcast credentials results in a "Unable to extract post url" bug. It asked me to file a bug here, so I'm filing a bug here.

[ealgase]

As far as I can tell the Comcast_SSO doesn't work at all (didn't work for me on Simpsons World).

[ealgase]

This also is related/duplicate of #11257, #11538, and #16630.

[joshtish]

Could well be a dupe of that last one for sure - the "Automatically signing you in" is the page I get when browsing from my home network - and it might be hitting that and failing out in the same way as #16630.

[joshtish]

Just to follow-up for people finding this in the future. I tested on an external connection where I don't get automatically signed-in, and things work as one would expect. So, oddly enough... if you want to use your comcast credentials, you can't be in your own home, or if you are home, you must use VPN.

[tv21]

For those running from source code or compiling your own, the quick fix for Comcast Xfinity is to use this version of adobepass.py:

https://raw.githubusercontent.com/bhamboss/youtube-dl/patch-1/youtube_dl/extractor/adobepass.py

That is from https://github.com/bhamboss/youtube-dl/blob/patch-1/youtube_dl/extractor/adobepass.py and I can confirm this works with Xfinity's new login if you are coming in from an Xfinity IP address.

Just to be clear, Xfinity authenticates differently if you are coming in from one of their IP addresses vs. if you are coming in from the IP of some other provider. I tested the above patch coming from an Xfinity IP. I did NOT test it from another provider's IP.

If I run diff -u on the original and patched files I get this:

$ diff -u adobepass.py.original adobepass.py.patched
--- adobepass.py.original	2018-10-17 01:22:56.000000000 -0400
+++ adobepass.py.patched	2019-04-23 08:53:36.000000000 -0400
@@ -1433,6 +1433,14 @@
                             provider_redirect_page, 'oauth redirect')
                         self._download_webpage(
                             oauth_redirect_url, video_id, 'Confirming auto login')
+                    elif 'automatically signed in with' in provider_redirect_page:
+                        # Seems like comcast is rolling up new way of automatically signing customers
+                        oauth_redirect_url = self._html_search_regex(
+                            r'continue:\s*"(https://oauth.xfinity.com/oauth/authorize\?.+)"',
+                            provider_redirect_page, 'oauth redirect (signed)')
+                        # Just need to process the request. No useful data comes back
+                        self._download_webpage(
+                            oauth_redirect_url, video_id, 'Confirming auto login')
                     else:
                         if '<form name="signin"' in provider_redirect_page:
                             provider_login_page_res = provider_redirect_page_res