Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Syfy User Agent #9087

Closed
eremini opened this issue Apr 5, 2016 · 9 comments
Closed

Syfy User Agent #9087

eremini opened this issue Apr 5, 2016 · 9 comments
Labels
broken-IE problem with existing site extraction geo-restricted

Comments

@eremini
Copy link

eremini commented Apr 5, 2016

Hi,
Seems like Syfy needs a different user agent set.

youtube-dl http://www.syfy.com/12monkeys/videos/201-year-of-the-monkey
[generic] 201-year-of-the-monkey: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 201-year-of-the-monkey: Downloading webpage
[generic] 201-year-of-the-monkey: Extracting information
[redirect] Following redirect to http://www.syfy.com/sites/syfy/themes/syfy/pages/lte-ie8.html
[generic] lte-ie8: Requesting header
WARNING: Falling back on generic information extractor.
[generic] lte-ie8: Downloading webpage
[generic] lte-ie8: Extracting information
ERROR: Unsupported URL: http://www.syfy.com/sites/syfy/themes/syfy/pages/lte-ie8.html

I tried this way
youtube-dl --user-agent "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36" http://www.syfy.com/12monkeys/videos/201-year-of-the-monkey

But didn't seem to work

Regards

@yan12125
Copy link
Collaborator

yan12125 commented Apr 6, 2016

I got:

This content is not available in your location.

What's your location?

@siddht4
Copy link

siddht4 commented Apr 6, 2016

i actually found that the video is restricted as its hosted by digital distribution vod.

video url http://player.theplatform.com/p/HNK2IC/syfy20141201/select/media/guid/2304990266/3010068?form=html&playerType=full-episode&autoPlay=true#playerurl=http%3A%2F%2Fwww.syfy.com%2F12monkeys%2Fvideos%2F201-year-of-the-monkey

mobile url http: //player.theplatform.com/p/HNK2IC/dd_syfy_vod_noauth/embed/select/media/guid/2304990266/3010068?form=html&playerType=full-episode&autoPlay=true&FWsiteSection=syfy_tve_embed_vod&FWsiteSectionMobile=syfy_tve_embed_vod_hh&FWsiteSectionTablet=syfy_tve_embed_vod_tab#playerurl=http%3A%2F%2Fwww.syfy.com%2F12monkeys%2Fvideos%2F201-year-of-the-monkey

@siddht4
Copy link

siddht4 commented Apr 6, 2016

@siddht4
Copy link

siddht4 commented Apr 6, 2016

the video is not available in my location, using proxy and it downloads

@siddht4
Copy link

siddht4 commented Apr 6, 2016

can anyone parse this

tp:plugin1="type=tracking|url=http://sb.scorecardresearch.com/c2/plugins/streamsense_plugin_theplatform.swf|labelmapping=ns_st_cl%3Dclip.length%2Cns_st_pl%3Dplaylist.title%2Cns_st_pr%3Dplaylist.feed%2Cns_st_ep%3Dplaylist.title%2Ctp_aid%3Dplaylist.accountID%2Ctp_st_cat%3Dcontent.baseclip.categories%2Ctp_st_mp%3Dplaylist.player%2Ctp_st_guid%3Dcontent.baseclip.guid|sendErrors=true|c2=9924155|priority=1|persistentlabels=c1%3D2%2Cns_site%3Dtpf%2Ctp_cid%3DNBCU-MPAT%2Ctp_type%3Dproduction|timinglabel=tp_st_ps|sendload=true"
tp:plugin2="type=tracking|url=http://sb.scorecardresearch.com/c2/plugins/streamsense_plugin_theplatform.js|labelmapping=ns_st_cl%3Dclip.length%2Cns_st_pl%3Dplaylist.title%2Cns_st_pr%3Dplaylist.feed%2Cns_st_ep%3Dplaylist.title%2Ctp_aid%3Dplaylist.accountID%2Ctp_st_cat%3Dcontent.baseclip.categories%2Ctp_st_mp%3Dplaylist.player%2Ctp_st_guid%3Dcontent.baseclip.guid|sendErrors=true|c2=9924155|priority=1|persistentlabels=c1%3D2%2Cns_site%3Dtpf%2Ctp_cid%3DNBCU-MPAT%2Ctp_type%3Dproduction|timinglabel=tp_st_ps|sendload=true"
tp:plugin3="type=metadataComponent|url=http://pdk.theplatform.com/5.6.6/pdk/swf/ratings.swf|reminder=20|enableAgeVerification=false|priority=1|path=http%3A%2F%2Ftve_static-syfy.nbcuni.com%2Fprod%2FPDK%2Fimg%2Fratings%2F|showRatings=true|delay=10"
tp:plugin4="type=metadataComponent|url=http://pdk.theplatform.com/5.6.6/pdk/js/plugins/ratings.js|reminder=20|enableAgeVerification=false|priority=1|path=http%3A%2F%2Ftve_static-syfy.nbcuni.com%2Fprod%2FPDK%2Fimg%2Fratings%2F|showRatings=true|delay=10"
tp:plugin5="type=adcomponent|url=http://pdk.theplatform.com/5.6.6/pdk/swf/freewheel.swf|siteSectionId=syfy_vod_hub|playerProfileHTML5=nbcu_live_jsam|pemURLsSeparator=%2C|isLive=false|customVideoAssetIdField=externalAdvertiserId|networkId=169843|pemURLs=http%3A%2F%2Fadm.fwmrm.net%2Fp%2Fnbcu_live%2FAnalyticsExtension.swf|bannerScriptName=tpExtCompanionAdScriptCallback|autoplay=true|exitFullscreenOnPause=false|priority=1|adManagerUrl=http%3A%2F%2Fadm.fwmrm.net%2Fp%2Fnbcu_live%2FAdManager.%7Bext%7D|playerProfile=nbcu_live_as3|serverUrl=http%3A%2F%2F29773.v.fwmrm.net"
tp:plugin6="type=adcomponent|url=http://pdk.theplatform.com/5.6.6/pdk/js/plugins/freewheel.js|siteSectionId=syfy_vod_hub|playerProfileHTML5=nbcu_live_jsam|pemURLsSeparator=%2C|isLive=false|customVideoAssetIdField=externalAdvertiserId|networkId=169843|pemURLs=http%3A%2F%2Fadm.fwmrm.net%2Fp%2Fnbcu_live%2FAnalyticsExtension.swf|bannerScriptName=tpExtCompanionAdScriptCallback|autoplay=true|exitFullscreenOnPause=false|priority=1|adManagerUrl=http%3A%2F%2Fadm.fwmrm.net%2Fp%2Fnbcu_live%2FAdManager.%7Bext%7D|playerProfile=nbcu_live_as3|serverUrl=http%3A%2F%2F29773.v.fwmrm.net"
tp:plugin7="type=tracking|url=http://tve-static-syfy.nbcuni.com/prod/PDK/plugin/ggtp400-tmp.js|enablecustomparse=true|DPR=true|priority=1|lfappend=true|sfcode=us|apid=PD037F3CE-5055-48FC-ABA4-87A1B7E27F32|apn=Syfy.com+Player"
tp:plugin8="type=overlay|url=http://tve-static-syfy.nbcuni.com/prod/PDK/js/UIPlugin.js|priority=1"
tp:plugin9="type=content|url=http://pdk.theplatform.com/5.6.6/pdk/swf/authentication.swf|priority=1"
tp:plugin10="type=content|url=http://pdk.theplatform.com/5.6.6/pdk/js/plugins/authentication.js|priority=1"
tp:plugin11="type=tracking|url=http://sb.scorecardresearch.com/c2/plugins/streamsense_plugin_theplatform.swf|labelmapping=c3%3D%22syfy%22%2Cc4%3D%22*null%22%2Cc6%3Dcontent.baseclip.categories|c2=3000060|priority=1"
tp:plugin12="type=tracking|url=http://sb.scorecardresearch.com/c2/plugins/streamsense_plugin_theplatform.js|labelmapping=c3%3D%22syfy%22%2Cc4%3D%22*null%22%2Cc6%3Dcontent.baseclip.categories|c2=3000060|priority=1"
tp:plugin13="type=content|url=http://pdk.theplatform.com/5.6.6/pdk/swf/akamaiHD.swf|fallback=switch%3Dhttp|bufferProfile=livestable|priority=1|manifest=true|videoLayer=akamaihd"
tp:plugin14="type=content|url=http://pdk.theplatform.com/5.6.6/pdk/js/plugins/akamaiHD.js|fallback=switch%3Dhttp|bufferProfile=livestable|priority=1|manifest=true|videoLayer=akamaihd"

@yan12125 yan12125 added the broken-IE problem with existing site extraction label Apr 6, 2016
@yan12125
Copy link
Collaborator

yan12125 commented Apr 6, 2016

@siddht1 I got the video from the first ThePlatform URL via a US proxy, but the mobile URL

http://player.theplatform.com/p/HNK2IC/dd_syfy_vod_noauth/embed/select/media/guid/2304990266/3010068?form=html&playerType=full-episode&autoPlay=true&FWsiteSection=syfy_tve_embed_vod&FWsiteSectionMobile=syfy_tve_embed_vod_hh&FWsiteSectionTablet=syfy_tve_embed_vod_tab#playerurl=http%3A%2F%2Fwww.syfy.com%2F12monkeys%2Fvideos%2F201-year-of-the-monkey

says "This content is not available from this domain.". Is it working for you?
And what's the purpose of codes in the last comment?

@siddht4
Copy link

siddht4 commented Apr 7, 2016

@yan12125 the URL you specified is also not available , but works via Norway,UK,Sweden proxy.

the codes were actually leading to the same video, you can either understand it as load sharing or vod broadcast mirroring, but its just that i can`t parse the full link either which i found from the html source of http://player.theplatform.com/p/HNK2IC/syfy20141201/select/media/guid/2304990266/3010068
, the video is hosted here . If you view the source code from line no 135 to 148 , that’s the part where all the video mirrors are stored

@remitamine
Copy link
Collaborator

If you view the source code from line no 135 to 148 , that’s the part where all the video mirrors are stored.

this codes are not related to the video urls.
for the urls can be extracted from release url.

...
    <link rel="alternate" href="http://link.theplatform.com/s/HNK2IC/media/Ay3MRfJG7e8b?player=Syfy.com%20Player&policy=2713542" type="application/smil+xml" />
...
                tp:releaseUrl="http://link.theplatform.com/s/HNK2IC/media/Ay3MRfJG7e8b?player=Syfy.com%20Player&amp;policy=2713542"
...

the release url accepts query params that affect the urls returned in the platform smil(manifest, switch, assetTypes...) and it depends on the account configuration.
you can see ThePlatform extractor and extractor that use it for examples of these params(nationalgeographic, aenetworks, cbs, nbc...).
i posted a patch that should fix the problem but i can't test it(#3820 (comment)).

@siddht4
Copy link

siddht4 commented Apr 7, 2016

please let me try idm(internet download manager , i m sure it will capture it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
broken-IE problem with existing site extraction geo-restricted
Projects
None yet
Development

No branches or pull requests

4 participants