Skip to content

/ youtube-dl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cdnapisec.kaltura.com #25757

Open
maxst33l opened this issue Jun 22, 2020 · 10 comments
Open

cdnapisec.kaltura.com #25757

maxst33l opened this issue Jun 22, 2020 · 10 comments

Comments

@maxst33l
Copy link

@maxst33l maxst33l commented Jun 22, 2020 •
edited

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dl version 2020.06.16.1
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

PASTE VERBOSE LOG HERE

youtube-dl -v kaltura:1307701:1_ph1xslf9
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'kaltura:1307701:1_ph1xslf9']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2020.06.16.1
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg git-2020-06-17-0b3bd00
[debug] Proxy map: {}
[Kaltura] 1_ph1xslf9: Downloading video info JSON
[Kaltura] 1_ph1xslf9: Checking mp4-1373 URL
[Kaltura] 1_ph1xslf9: mp4-1373 URL is invalid, skipping
[Kaltura] 1_ph1xslf9: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 404: Not Found
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on 'http://cdnapi.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_ph1xslf9/format/url/protocol/http/flavorId/1_hlcfxgv2'
ERROR: unable to download video data: HTTP Error 404: Not Found
Traceback (most recent call last):
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\YoutubeDL.py", line 1926, in process_info
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\YoutubeDL.py", line 1865, in dl
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\downloader\common.py", line 366, in download
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\downloader\http.py", line 341, in real_download
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\downloader\http.py", line 109, in establish_connection
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\YoutubeDL.py", line 2238, in urlopen
File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default
urllib.error.HTTPError: HTTP Error 404: Not Found

Description

WRITE DESCRIPTION HERE

I am in the same location as this video so it shouldn't be an issue. I can't seem to get a working formula. I've searched all other issues (And even commented there) but decided this wasn't quite the same, so I'm opening this one. Hopefully it's something simple.

I'm trying to get the video from here (I am in this geo-location):
https://curio.ca/en/video/the-great-transformation-1746/

It is a kaltura video.

The watch link has this bit of information in the web source:

<script src="https://cdnapisec.kaltura.com/p/1307701/sp/130770100/embedIframeJs/uiconf_id/28723181/partner_id/1307701"></script>
<script src="/static/sites/curio/legacy/custom/js/sku-v2.js"></script>
<script type="text/javascript">
    jQuery(document).ready(function() {
        var auto_play = true;
        Player.init('1746', '28723181', '1_ph1xslf9',
                    '',
                    '',
                    'MjNjYmNjN2JhZGU5ZGFkYjJhNTRiMWU5ZTBlZjM4YWQ4MTNjMWY5MnwxMzA3NzAxOzEzMDc3MDE7MTU5MjkyMzI4MTswOzQwMTg4O2hlcmljay5tb25pekByYWRpby1jYW5hZGEuY2E7c3ZpZXc6MV9waDF4c2xmOQ==',
                    false,
                    'None',
                    false,
                    auto_play);
        $('#id_play_btn').click(function(){
            $('.kWidgetPlayBtn').click();
            $('.rc-freetrial-playAlert').hide();
        });
    });
</script>

And I was able to find the API call like youtube-dl does:

http://cdnapi.kaltura.com/api_v3/index.php?2%3AresponseProfile%3Atype=1&clientTag=kdp%3Av3.8.5&4%3Afilter%3AentryIdEqual=1_ph1xslf9&3%3Aks=%7B1%3Aresult%3Aks%7D&4%3Aks=%7B1%3Aresult%3Aks%7D&action=null&1%3AwidgetId=_28723181&4%3Aservice=caption_captionasset&2%3Aaction=get&3%3AentryId=1_ph1xslf9&1%3Aexpiry=86400&1%3Aaction=startWidgetSession&2%3Aks=%7B1%3Aresult%3Aks%7D&3%3Aservice=flavorAsset&3%3Aaction=getbyentryid&2%3AresponseProfile%3Afields=createdAt%2CdataUrl%2Cduration%2Cname%2Cplays%2CthumbnailUrl%2CuserId&4%3Aaction=list&1%3Aservice=session&service=multirequest&2%3AentryId=1_ph1xslf9&format=1&2%3Aservice=baseentry&apiVersion=3.1.5

but this did not 'work'
so I found a working one with "sec" in the URL:

https://cdnapisec.kaltura.com/api_v3/index.php?4%3Aks=%7B1%3Aresult%3Aks%7D&clientTag=kdp%3Av3.8.5&2%3Aks=%7B1%3Aresult%3Aks%7D&2%3AentryId=1_ph1xslf9&1%3Aexpiry=86400&4%3Aaction=list&2%3Aservice=baseentry&1%3Aaction=startWidgetSession&2%3AresponseProfile%3Atype=1&2%3Aaction=get&3%3AentryId=1_ph1xslf9&apiVersion=3.1.5&2%3AresponseProfile%3Afields=createdAt%2CdataUrl%2Cduration%2Cname%2Cplays%2CthumbnailUrl%2CuserId&3%3Aaction=getbyentryid&3%3Aks=%7B1%3Aresult%3Aks%7D&format=1&4%3Afilter%3AentryIdEqual=1_ph1xslf9&4%3Aservice=caption_captionasset&action=null&1%3Aservice=session&3%3Aservice=flavorAsset&1%3AwidgetId=_1307701&service=multirequest

output:

[{"partnerId":1307701,"ks":"djJ8MTMwNzcwMXzoS_CD6fv3K80BDY_Pck7znlH1abmKBJDKpVRqb53UndfG7eF0GGbHfWUhs0IyB56KCS2etxFk3IGFgrBt3V9JXympSTOWLNbyH6Yt8gge3A==","userId":0,"objectType":"KalturaStartWidgetSessionResponse"},{"dataUrl":"https://cdnapisec.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_ph1xslf9/format/url/protocol/https","plays":10473,"duration":6380,"name":"The Great Transformation","userId":"hericke.moniz@radio-canada.ca","type":1,"createdAt":1374782038,"thumbnailUrl":"https://cfvod.kaltura.com/p/1307701/sp/130770100/thumbnail/entry_id/1_ph1xslf9/version/100081","objectType":"KalturaMediaEntry"},[{"flavorParamsId":0,"width":718,"height":360,"bitrate":1373,"frameRate":29.97,"isOriginal":true,"isWeb":true,"containerFormat":"mp42","videoCodecId":"avc1","status":2,"isDefault":false,"id":"1_x22x1afq","entryId":"1_ph1xslf9","partnerId":1307701,"version":"1","size":1069547,"tags":"web","fileExt":"mp4","createdAt":1374782765,"updatedAt":1374782773,"description":"","objectType":"KalturaFlavorAsset"},{"flavorParamsId":487041,"width":480,"height":256,"bitrate":461,"frameRate":29.97,"isOriginal":false,"isWeb":true,"containerFormat":"isom","videoCodecId":"avc1","status":2,"isDefault":false,"id":"1_shhf71g1","entryId":"1_ph1xslf9","partnerId":1307701,"version":"1","size":359424,"tags":"mobile,web,mbr,iphone,iphonenew","fileExt":"mp4","createdAt":1374782773,"updatedAt":1374788038,"description":"","objectType":"KalturaFlavorAsset"},{"flavorParamsId":487051,"width":640,"height":360,"bitrate":660,"frameRate":29.97,"isOriginal":false,"isWeb":true,"containerFormat":"isom","videoCodecId":"avc1","status":2,"isDefault":false,"id":"1_nkq47xph","entryId":"1_ph1xslf9","partnerId":1307701,"version":"1","size":514048,"tags":"mobile,web,mbr,iphone,iphonenew","fileExt":"mp4","createdAt":1374782773,"updatedAt":1374827629,"description":"audio warnings: 2106,#Redundant bitrate.\n","objectType":"KalturaFlavorAsset"},{"flavorParamsId":487061,"width":640,"height":360,"bitrate":954,"frameRate":29.97,"isOriginal":false,"isWeb":true,"containerFormat":"isom","videoCodecId":"avc1","status":2,"isDefault":false,"id":"1_vgdbwbji","entryId":"1_ph1xslf9","partnerId":1307701,"version":"1","size":737280,"tags":"mobile,web,mbr,ipad,ipadnew","fileExt":"mp4","createdAt":1374782773,"updatedAt":1374825979,"description":"audio warnings: 2106,#Redundant bitrate.\n","objectType":"KalturaFlavorAsset"},{"flavorParamsId":487071,"width":640,"height":360,"bitrate":1560,"frameRate":29.97,"isOriginal":false,"isWeb":true,"containerFormat":"isom","videoCodecId":"avc1","status":2,"isDefault":false,"id":"1_hlcfxgv2","entryId":"1_ph1xslf9","partnerId":1307701,"version":"1","size":1216348,"tags":"mobile,web,mbr,ipad,ipadnew","fileExt":"mp4","createdAt":1374782773,"updatedAt":1374826281,"description":"","objectType":"KalturaFlavorAsset"},{"flavorParamsId":487091,"width":0,"height":0,"bitrate":0,"frameRate":0,"isOriginal":false,"isWeb":true,"status":4,"isDefault":false,"id":"1_e3sn5h0e","entryId":"1_ph1xslf9","partnerId":1307701,"size":0,"tags":"web,mbr","fileExt":"mp4","createdAt":1374782773,"updatedAt":1374782773,"description":"video warnings: 2107,1440,4000#The target flavor bitrate (1440) does not comply with the requested bitrate (4000).; 2106,#Redundant bitrate.\naudio warnings: 2106,#Redundant bitrate.\n","objectType":"KalturaFlavorAsset"},{"flavorParamsId":487081,"width":0,"height":0,"bitrate":0,"frameRate":0,"isOriginal":false,"isWeb":true,"status":4,"isDefault":false,"id":"1_nfotuudg","entryId":"1_ph1xslf9","partnerId":1307701,"size":0,"tags":"web,mbr","fileExt":"mp4","createdAt":1374782773,"updatedAt":1374782773,"description":"video warnings: 2107,1440,2500#The target flavor bitrate (1440) does not comply with the requested bitrate (2500).; 2106,#Redundant bitrate.\naudio warnings: 2106,#Redundant bitrate.\n","objectType":"KalturaFlavorAsset"}],{"objects":[],"totalCount":0,"objectType":"KalturaCaptionAssetListResponse"}]

I think the underlying issue may be that youtube-dl is using the non sec URL:
[debug] Invoking downloader on 'http://cdnapi.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_ph1xslf9/format/url/protocol/http/flavorId/1_hlcfxgv2'

instead of the sec URL "cdnapisec.kaltura.com" that you can see in my above outputs.

I have no idea what to do or how to fix from here.

I had a really hard time trying to find any information on the "kaltura:" URL parameter option as well.

Thanks in advance!
@stinkteeth
Copy link

@stinkteeth stinkteeth commented Jun 22, 2020

The cdnapisec domain requires a ks signature value. The kaltura extractor can sometimes get this value, but for your video it would need to be customized for this site specifically. The ks value for your video is within the following url (view-source):

https://curio.ca/en/player/1746/dialog/

With a fresh ks value, the following workarounds can be used for the time being (replace the ks value shown in example):

youtube-dl "https://cdnapisec.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_ph1xslf9/flavorIds/1_hlcfxgv2/format/http/protocol/https/a.mp4?referrer=aHR0cHM6Ly9jdXJpby5jYQ==&ks=MDE3MTIyMDAwYzM0MGMzMDZmZjFiZGFmZjdiMTg4ODIzZGU1OTc2M3wxMzA3NzAxOzEzMDc3MDE7MTU5MjkzMDcyODswOzQ1ODE1O2hlcmljay5tb25pekByYWRpby1jYW5hZGEuY2E7c3ZpZXc6MV9waDF4c2xmOQ=="

or:

youtube-dl "http://curio.radio-canada.ca/kaltura/content/r71v1/entry/data/253/787/1_ph1xslf9_1_hlcfxgv2_1.mp4"
@maxst33l
Copy link
Author

@maxst33l maxst33l commented Jun 22, 2020 •
edited

Woah! This is awesome, however, I'm not sure how you did this..?
In view-source:https://curio.ca/en/player/1746/dialog/ what is the ks value per say?

I think in my example above the ks is: "MjNjYmNjN2JhZGU5ZGFkYjJhNTRiMWU5ZTBlZjM4YWQ4MTNjMWY5MnwxMzA3NzAxOzEzMDc3MDE7MTU5MjkyMzI4MTswOzQwMTg4O2hlcmljay5tb25pekByYWRpby1jYW5hZGEuY2E7c3ZpZXc6MV9waDF4c2xmOQ=="

I'm not sure how you assembled it to get the big URL..

Lastly, how did you assemble it to get the final clean curio,radio-canada.ca URL??

Thanks again, in advance!!
(plugging away and editing this as I figure out things)
@maxst33l
Copy link
Author

@maxst33l maxst33l commented Jun 22, 2020 •
edited

ok I think I see how you got the clean url, it displays when I use -v:

$ youtube-dl -v "https://cdnapisec.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_ph1xslf9/flavorIds/1_hlcfxgv2/format/http/protoc
ol/https/a.mp4?referrer=aHR0cHM6Ly9jdXJpby5jYQ==&ks=MjNjYmNjN2JhZGU5ZGFkYjJhNTRiMWU5ZTBlZjM4YWQ4MTNjMWY5MnwxMzA3NzAxOzEzMDc3MDE7MTU5MjkyMzI4MTswOzQwMTg4O2hlcmljay
5tb25pekByYWRpby1jYW5hZGEuY2E7c3ZpZXc6MV9waDF4c2xmOQ=="
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://cdnapisec.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_ph1xslf9/flavorIds/1_hlcfxgv2/format/http/protocol/https/a.mp4?referrer=aHR0cHM6Ly9jdXJpby5jYQ==&ks=MjNjYmNjN2JhZGU5ZGFkYjJhNTRiMWU5ZTBlZjM4YWQ4MTNjMWY5MnwxMzA3NzAxOzEzMDc3MDE7MTU5MjkyMzI4MTswOzQwMTg4O2hlcmljay5tb25pekByYWRpby1jYW5hZGEuY2E7c3ZpZXc6MV9waDF4c2xmOQ==']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.06.16.1
[debug] Python version 3.5.2 (CPython) - Linux-4.4.0-18362-Microsoft-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: none
[debug] Proxy map: {}
[generic] a: Requesting header
WARNING: Falling back on generic information extractor.
[generic] a: Downloading webpage
[generic] a: Extracting information
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'http://curio.radio-canada.ca/kaltura/content/r71v1/entry/data/253/787/1_ph1xslf9_1_hlcfxgv2_1.mp4'
[f4m] Downloading f4m manifest

For me my system/python hangs here.. high memory usage and ends up dying:

Traceback (most recent call last):
File "/usr/local/bin/youtube-dl", line 11, in
sys.exit(main())
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/init.py", line 474, in main
_real_main(argv)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/init.py", line 464, in _real_main
retcode = ydl.download(all_urls)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/YoutubeDL.py", line 2019, in download
url, force_generic_extractor=self.params.get('force_generic_extractor', False))
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/YoutubeDL.py", line 808, in extract_info
return self.process_ie_result(ie_result, download, extra_info)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/YoutubeDL.py", line 863, in process_ie_result
return self.process_video_result(ie_result, download=download)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/YoutubeDL.py", line 1644, in process_video_result
self.process_info(new_info)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/YoutubeDL.py", line 1926, in process_info
success = dl(filename, info_dict)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/YoutubeDL.py", line 1865, in dl
return fd.download(name, info)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/downloader/common.py", line 366, in download
return self.real_download(filename, info_dict)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/downloader/f4m.py", line 331, in real_download
doc = compat_etree_fromstring(manifest)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/compat.py", line 2533, in compat_etree_fromstring
return etree.XML(text, parser=etree.XMLParser(target=_TreeBuilder()))
File "/usr/lib/python3.5/xml/etree/ElementTree.py", line 1333, in XML
parser.feed(text)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 0

and the file wont download. I have to use the new found short URL and it will download!

Now if I can only figure out how you assembled the original URL in the first place:
"https://cdnapisec.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_ph1xslf9/flavorIds/1_hlcfxgv2/format/http/protoc
ol/https/a.mp4?referrer=aHR0cHM6Ly9jdXJpby5jYQ==&ks=MjNjYmNjN2JhZGU5ZGFkYjJhNTRiMWU5ZTBlZjM4YWQ4MTNjMWY5MnwxMzA3NzAxOzEzMDc3MDE7MTU5MjkyMzI4MTswOzQwMTg4O2hlcmljay
5tb25pekByYWRpby1jYW5hZGEuY2E7c3ZpZXc6MV9waDF4c2xmOQ=="

I'll be in business... still trying here :)

I'm attempting to get another video in the series, and I can't seem to recreate this. :(

I'm sure, when you have a moment, if you can help me, that this thread will help others!
@stinkteeth
Copy link

@stinkteeth stinkteeth commented Jun 22, 2020

For the long url, change:
format/http/protocol/https/a.mp4
to:
format/url/protocol/https/a.mp4

And it will probably work directly.

I believe the playManifest url is within the mwEmbedFrame.php url, but I don't have the CA vpn connected now on this device in order to re-check.

But you can just use the stated playManifest url as a template, and replace the entryId and flavorId values (for different videos) as needed.
@maxst33l
Copy link
Author

@maxst33l maxst33l commented Jun 22, 2020 •
edited

You are correct, swapping html for url worked directly;
youtube-dl "https://cdnapisec.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_ph1xslf9/flavorIds/1_hlcfxgv2/format/url/protocol/https/a.mp4?referrer=aHR0cHM6Ly9jdXJpby5jYQ==&ks=MDE3MTIyMDAwYzM0MGMzMDZmZjFiZGFmZjdiMTg4ODIzZGU1OTc2M3wxMzA3NzAxOzEzMDc3MDE7MTU5MjkzMDcyODswOzQ1ODE1O2hlcmljay5tb25pekByYWRpby1jYW5hZGEuY2E7c3ZpZXc6MV9waDF4c2xmOQ=="
[generic] a: Requesting header
[redirect] Following redirect to http://curio.radio-canada.ca/kaltura/content/r71v1/entry/data/253/787/1_ph1xslf9_1_hlcfxgv2_1.mp4
[generic] 1_ph1xslf9_1_hlcfxgv2_1: Requesting header
[download] Resuming download at byte 67107840
[download] Destination: 1_ph1xslf9_1_hlcfxgv2_1-1_ph1xslf9_1_hlcfxgv2_1.mp4
[download] 100% of 1.16GiB in 00:32

I just tried updating the entryId and flavorId to get a different video:
https://curio.ca/en/video/when-the-world-began-1703/

  <div class="modal-dialog modal-lg" role="document">
    <div class="modal-content">
      <div class="modal-header">
        <button type="button" class="close" data-dismiss="modal" aria-label="Close"><span aria-hidden="true">×</span></button>
        <h2 class="modal-title" id="myModalLabel">
            
    When the World Began

        </h2>
      </div>
      <div class="modal-body fileinfo-modal-body">
          

<div class="rc-mediaFrame">
  <div id="kaltura_player_1703" class="player" itemprop="video" itemscope itemtype="http://schema.org/VideoObject" style="width:100%; height:524px;">
      <!-- Video Metadata -->
      <span itemprop="name" content="Canada: A People&#39;s History When the World Began"></span>
      <span itemprop="duration" content="6484000"></span>
      <span itemprop="thumbnailUrl" content="//media.curio.ca/filer_public_thumbnails/filer_public/22/94/2294588e-0aa5-4886-a23a-6cf114dbb607/1_dz56ag5q.jpg__428x248_q85_crop_subject_location-731%2C325_subsampling-2.jpg"></span>
  </div>
  <div id="k-chapterTimeline"></div>
  <div style="clear:both"></div>
</div>

<script src="https://cdnapisec.kaltura.com/p/1307701/sp/130770100/embedIframeJs/uiconf_id/28723181/partner_id/1307701"></script>
<script src="/static/sites/curio/legacy/custom/js/sku-v2.js"></script>
<script type="text/javascript">
    jQuery(document).ready(function() {
        var auto_play = true;
        Player.init('1703', '28723181', '1_dz56ag5q',
                    '',
                    '',
                    'Y2QyOGUxMWEyODRhYzE5MjgzMmViNzg3MThkMzlkMjIyOWU2MTNmY3wxMzA3NzAxOzEzMDc3MDE7MTU5Mjk0MDI4MzswOzUxMTU7aGVyaWNrLm1vbml6QHJhZGlvLWNhbmFkYS5jYTtzdmlldzoxX2R6NTZhZzVx',
                    false,
                    'None',
                    false,
                    auto_play);
        $('#id_play_btn').click(function(){
            $('.kWidgetPlayBtn').click();
            $('.rc-freetrial-playAlert').hide();
        });
    });
</script>

      </div>
    </div><!-- /.modal-content -->
  </div><!-- /.modal-dialog -->

But no go:

$ youtube-dl -v "https://cdnapisec.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_dz56ag5q/flavorIds/1_qldhzovx/format/url/protocol/https/a.mp4?referrer=aHR0cHM6Ly9jdXJpby5jYQ==&ks=MDE3MTIyMDAwYzM0MGMzMDZmZjFiZGFmZjdiMTg4ODIzZGU1OTc2M3wxMzA3NzAxOzEzMDc3MDE7MTU5MjkzMDcyODswOzQ1ODE1O2hlcmljay5tb25pekByYWRpby1jYW5hZGEuY2E7c3ZpZXc6MV9waDF4c2xmOQ=="
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://cdnapisec.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_dz56ag5q/flavorIds/1_qldhzovx/format/url/protocol/https/a.mp4?referrer=aHR0cHM6Ly9jdXJpby5jYQ==&ks=MDE3MTIyMDAwYzM0MGMzMDZmZjFiZGFmZjdiMTg4ODIzZGU1OTc2M3wxMzA3NzAxOzEzMDc3MDE7MTU5MjkzMDcyODswOzQ1ODE1O2hlcmljay5tb25pekByYWRpby1jYW5hZGEuY2E7c3ZpZXc6MV9waDF4c2xmOQ==']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.06.16.1
[debug] Python version 3.5.2 (CPython) - Linux-4.4.0-18362-Microsoft-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: none
[debug] Proxy map: {}
[generic] a: Requesting header
WARNING: Could not send HEAD request to https://cdnapisec.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_dz56ag5q/flavorIds/1_qldhzovx/format/url/protocol/https/a.mp4?referrer=aHR0cHM6Ly9jdXJpby5jYQ==&ks=MDE3MTIyMDAwYzM0MGMzMDZmZjFiZGFmZjdiMTg4ODIzZGU1OTc2M3wxMzA3NzAxOzEzMDc3MDE7MTU5MjkzMDcyODswOzQ1ODE1O2hlcmljay5tb25pekByYWRpby1jYW5hZGEuY2E7c3ZpZXc6MV9waDF4c2xmOQ==: HTTP Error 404: Not Found
[generic] a: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 404: Not Found (caused by <HTTPError 404: 'Not Found'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/common.py", line 627, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/YoutubeDL.py", line 2238, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python3.5/urllib/request.py", line 472, in open
response = meth(req, response)
File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.5/urllib/request.py", line 510, in error
return self._call_chain(*args)
File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
@stinkteeth
Copy link

@stinkteeth stinkteeth commented Jun 22, 2020

For a different video (or entryId), you'll need a new ks value that is for that entryId.

view-source:
https://curio.ca/en/player/1703/dialog/

So, the values that are unique to any video (or other) are the entryId, the flavorId, and the ks value.
@maxst33l
Copy link
Author

@maxst33l maxst33l commented Jun 22, 2020

I just updated the one I tried above with the ks from the source and it worked:

youtube-dl -v "https://cdnapisec.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_dz56ag5q/flavorIds/1_qldhzovx/format/url/protocol/https/a.mp4?referrer=aHR0cHM6Ly9jdXJpby5jYQ==&ks=Y2QyOGUxMWEyODRhYzE5MjgzMmViNzg3MThkMzlkMjIyOWU2MTNmY3wxMzA3NzAxOzEzMDc3MDE7MTU5Mjk0MDI4MzswOzUxMTU7aGVyaWNrLm1vbml6QHJhZGlvLWNhbmFkYS5jYTtzdmlldzoxX2R6NTZhZzVx"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://cdnapisec.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/1_dz56ag5q/flavorIds/1_qldhzovx/format/url/protocol/https/a.mp4?referrer=aHR0cHM6Ly9jdXJpby5jYQ==&ks=Y2QyOGUxMWEyODRhYzE5MjgzMmViNzg3MThkMzlkMjIyOWU2MTNmY3wxMzA3NzAxOzEzMDc3MDE7MTU5Mjk0MDI4MzswOzUxMTU7aGVyaWNrLm1vbml6QHJhZGlvLWNhbmFkYS5jYTtzdmlldzoxX2R6NTZhZzVx']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2020.06.16.1
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg git-2020-06-17-0b3bd00
[debug] Proxy map: {}
[generic] a: Requesting header
[redirect] Following redirect to https://cfvod.kaltura.com/scf/pd/p/1307701/sp/130770100/serveFlavor/entryId/1_dz56ag5q/v/1/flavorId/1_qldhzovx/name/a.mp4?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZnZvZC5rYWx0dXJhLmNvbS9zY2YvcGQvcC8xMzA3NzAxL3NwLzEzMDc3MDEwMC9zZXJ2ZUZsYXZvci9lbnRyeUlkLzFfZHo1NmFnNXEvdi8xKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTU5Mjk0NTY5OH19fV19&Signature=SRVa2HXLsm1i1TOgLaMPsY~RfogSnlUR1EiCbTyq-DH1JdrXhRV00TpjD-bDicFpV8iNCjdOdAR7ekxgvx8RafmRavyUQbxX45y8IG6Gn1rSIZJsKOvV7QnOsyULnk4tBCJHjE0kg4Z3ReBhxkG4IQcxRcj8L6-oHeZDrf2LTdmehs5io4ITqF-nw1RMLac-AT2ksGBXzix2NmsFnDOzBfLav8ntu-QL46cvxL5f7kx1B9v9lJphIzGohusOf-8XXz1Awn6g3Iq3UqglZcR2153MJScGUb8bH3ausplD1tEGbdsUbIi-wPu3u1-dzoQz7u0zGtAoXYOR61DRCBm6tA__&Key-Pair-Id=APKAJT6QIWSKVYK3V34A
[generic] a: Requesting header
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on 'https://cfvod.kaltura.com/scf/pd/p/1307701/sp/130770100/serveFlavor/entryId/1_dz56ag5q/v/1/flavorId/1_qldhzovx/name/a.mp4?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZnZvZC5rYWx0dXJhLmNvbS9zY2YvcGQvcC8xMzA3NzAxL3NwLzEzMDc3MDEwMC9zZXJ2ZUZsYXZvci9lbnRyeUlkLzFfZHo1NmFnNXEvdi8xKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTU5Mjk0NTY5OH19fV19&Signature=SRVa2HXLsm1i1TOgLaMPsY~RfogSnlUR1EiCbTyq-DH1JdrXhRV00TpjD-bDicFpV8iNCjdOdAR7ekxgvx8RafmRavyUQbxX45y8IG6Gn1rSIZJsKOvV7QnOsyULnk4tBCJHjE0kg4Z3ReBhxkG4IQcxRcj8L6-oHeZDrf2LTdmehs5io4ITqF-nw1RMLac-AT2ksGBXzix2NmsFnDOzBfLav8ntu-QL46cvxL5f7kx1B9v9lJphIzGohusOf-8XXz1Awn6g3Iq3UqglZcR2153MJScGUb8bH3ausplD1tEGbdsUbIi-wPu3u1-dzoQz7u0zGtAoXYOR61DRCBm6tA__&Key-Pair-Id=APKAJT6QIWSKVYK3V34A'
[download] Destination: a-a.mp4
[download] 74.2% of 1.18GiB at 7.22MiB/s ETA 00:43 [download] Got server HTTP error: Downloaded 941141680 bytes, expected 1267721055 bytes. Retrying (attempt 1 of 10)...
[download] Destination: a-a.mp4
[download] 100% of 1.18GiB in 02:58
@maxst33l
Copy link
Author

@maxst33l maxst33l commented Jun 22, 2020 •
edited

So the secret sauce here is:

  1. entryID and ks from the view-source
  2. the flavorId from the API call
youtube-dl -v "https://cdnapisec.kaltura.com/p/1307701/sp/130770100/playManifest/entryId/<ENTRYID>/flavorIds/<FLAVORID>/format/url/protocol/https/a.mp4?ks=<KS>"

Is there an easier way to get the flavorId's other than having to manually edit the API call?
(safe to assume not right now, since the all the manual effort?)
@stinkteeth
Copy link

@stinkteeth stinkteeth commented Jun 22, 2020

It all can be automated with a new extractor.

The flavorIds are in a few spots, but none easier.

The api, or inspection of the m3u8 url, are probably the easiest ways manually.
@maxst33l
Copy link
Author

@maxst33l maxst33l commented Jun 22, 2020

How does one get the m3u8 url to inspect?
(Once again, I really appreciate your input and help here, hopefully this helps others one day too!)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
@stinkteeth @maxst33l
You can’t perform that action at this time.