Viu OTT extractor error (KeyError) #26701

panda130 · 2020-09-25T05:02:19Z

Checklist

I'm reporting a broken site support
I've verified that I'm running youtube-dl version 2020.09.20
I've checked that all provided URLs are alive and playable in a browser
I've checked that all URLs and arguments with special characters are properly quoted or escaped
I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-i', '-v', 'https://www.viu.com/ott/sg/en-us/vod/233440/Hot-Stove-League']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.09.20
[debug] Python version 3.7.7 (CPython) - Darwin-16.7.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.1.4, ffprobe 4.1.4, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[viu:ott] 233440: Downloading video info
[viu:ott] 233440: Downloading stream info
ERROR: An extractor error has occurred. (caused by KeyError('data')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 532, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/viu.py", line 233, in _real_extract
    })['data']['stream']
KeyError: 'data'
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 532, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/viu.py", line 233, in _real_extract
    })['data']['stream']
KeyError: 'data'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 545, in extract
    raise ExtractorError('An extractor error has occurred.', cause=e)
youtube_dl.utils.ExtractorError: An extractor error has occurred. (caused by KeyError('data')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

As of today, the Viu OTT extractor is no longer working. I have tried multiple server locations.

This issue is occurring with videos that are geo-restricted to both Singapore and Thailand. They are still accessible via web browser using a proxy.

Singapore sample: https://www.viu.com/ott/sg/en-us/vod/233440/Hot-Stove-League
Thailand sample: https://www.viu.com/ott/th/en-us/vod/165839/A-Poem-A-Day

This is really bad because Viu is one of the main sites for Asian TV dramas that are not available anywhere else. Thank you for your help!

zackmark29 · 2020-09-25T18:54:46Z

@dstftw

I also tested on my C# Tool.
In PH, premium episode is now required premium account.

See this example:

FREE EPISODE:
https://d1k2us671qcoau.cloudfront.net/distribute_web_ph.php?ccs_product_id=2b90508fc68055b6be8489381d5dd3a1&language_flag_id=3

PREMIUM EPISODE:
https://d1k2us671qcoau.cloudfront.net/distribute_web_ph.php?ccs_product_id=353f52cfdf968f4cb195c9b7b29e4f69&language_flag_id=3&identity=8f36aeef79014287cbb770b75e688c4e1e1e8e20

panda130 · 2020-09-25T19:51:05Z

@zackmark29 Thank you for replying. I do not believe that either of the sample episodes I mentioned are premium episodes because you can watch them via web browser without a subscription.

Something seems to have changed to stop youtube-dl from even seeing the available video formats. Using "--list-formats" is not working.

So what you are saying is that even if youtube-dl could see the available video formats, the HD quality videos will no longer be available? The HD quality videos were working via youtube-dl a couple of days ago.

zackmark29 · 2020-09-25T19:52:47Z

@zackmark29 Thank you for replying. I do not believe that either of the sample episodes I mentioned are premium episodes because you can watch them via web browser without a subscription.

Something seems to have changed to stop youtube-dl from even seeing the available video formats. Using "--list-formats" is not working.

So what you are saying is that even if youtube-dl could see the available video formats, the HD quality videos will no longer be available? The HD quality videos were working via youtube-dl a couple of days ago.

No, in my case, if I don't have premium account, youtube-dl can't be able to check the available format as well. Because the cookie id is required.

I'll check now with SG vpn. I'll update you in a minute

zackmark29 · 2020-09-25T20:11:54Z

I'm able to get all resolution with you're given sg link

panda130 · 2020-09-25T20:24:55Z

@zackmark29 Thanks for following up. So it seems like all formats of the videos are still available. It's just that there is some syntax change on Viu's server side that the youtube-dl Viu extractor needs to compensate for.

zackmark29 · 2020-09-25T20:43:02Z

@zackmark29 Thanks for following up. So it seems like all formats of the videos are still available. It's just that there is some syntax change on Viu's server side that the youtube-dl Viu extractor needs to compensate for.

Yeah. If I just know Python I can edit it for you but I only know C#
@dstftw can do it for you

Ashish0804 · 2020-09-26T04:05:45Z

I'm able to get all resolution with you're given sg link

Is your C# tool available to public?

zackmark29 · 2020-09-26T05:16:32Z

I'm able to get all resolution with you're given sg link

Is your C# tool available to public?

No, this is just for personal use.

Ashish0804 · 2020-09-26T06:25:11Z

I'm able to get all resolution with you're given sg link

Is your C# tool available to public?

No, this is just for personal use.

can u share the extractor code? I can try and modify it for python

zackmark29 · 2020-09-26T13:46:11Z

I'm able to get all resolution with you're given sg link

Is your C# tool available to public?

No, this is just for personal use.

can u share the extractor code? I can try and modify it for python

I just tested again and I noticed this is the problem
Example url: https://www.viu.com/ott/sg/en-us/vod/233440/Hot-Stove-League

https://www.viu.com/ott/sg/index.php?r=vod/ajax-detail&platform_flag_label=web&area_id=2&language_flag_id=3&product_id=233440

to fix it should have &ut=2 at the end like this

https://www.viu.com/ott/sg/index.php?r=vod/ajax-detail&platform_flag_label=web&area_id=2&language_flag_id=3&product_id=233440&ut=2

And here's for the streams
https://d1k2us671qcoau.cloudfront.net/distribute_web_sg.php?ccs_product_id=9aceb6645dbdd1afa252581c2476af8f&language_flag_id=3

for some premium videos sometimes token is required to get the streams.

https://d1k2us671qcoau.cloudfront.net/distribute_web_sg.php?ccs_product_id=9aceb6645dbdd1afa252581c2476af8f&language_flag_id=3&identity=6398e7db08dab6cdc285cdf26656723c31888c2e

Let me know if you needed more.

panda130 · 2020-09-26T23:46:41Z

@zackmark29 Yay! Thank you so much! Based on your observation, I fixed the viu.py extractor by adding in the 'language_flag_id' and the 'ut' into the query. I am not a programmer, and this is the first time I have ever done anything in Python...

I have also submitted my first-ever pull request.

zackmark29 · 2020-09-27T01:21:08Z

@zackmark29 Yay! Thank you so much! Based on your observation, I fixed the viu.py extractor by adding in the 'language_flag_id' and the 'ut' into the query. I am not a programmer, and this is the first time I have ever done anything in Python...

I have also submitted my first-ever pull request.

Wow that's great! so is it work now?
I hope I also know how to modify python from youtube-dl extractor.

zackmark29 · 2020-09-27T01:23:14Z

But the ut i think it depends on the region. In my country (PH) it's ut=0 and for SG it's ut=2

panda130 · 2020-09-27T20:48:17Z

@zackmark29 I did some more testing. I actually removed the ut part of the query, and the extractor still works. So ut does not seem to be necessary.

There are a couple of remaining issues.

After I made the "language_flag_id" change to the query, the Viu extractor is only working for Singapore videos. For some reason, it does not work with Thailand videos. Can you find the correct video URL format for the Thailand videos? Sample URL: https://www.viu.com/ott/th/en-us/vod/165839/A-Poem-A-Day
The extractor is no longer working for Premium videos. It used to work for Premium videos.

zackmark29 · 2020-09-29T02:07:32Z

@zackmark29 I did some more testing. I actually removed the ut part of the query, and the extractor still works. So ut does not seem to be necessary.

There are a couple of remaining issues.

After I made the "language_flag_id" change to the query, the Viu extractor is only working for Singapore videos. For some reason, it does not work with Thailand videos. Can you find the correct video URL format for the Thailand videos? Sample URL: https://www.viu.com/ott/th/en-us/vod/165839/A-Poem-A-Day

The extractor is no longer working for Premium videos. It used to work for Premium videos.

In PH, ut is needed.
But I'll look into it later.

touchhyuk · 2020-09-29T10:59:54Z

how can i download from viu with viu.py? i did install python but i can't run viu python. please guide

panda130 · 2020-09-30T08:42:35Z

@touchhyuk I assume that you've already downloaded the youtube-dl master source code. You'll need to make the modifications to viu.py that are listed in my pull request.

Then you have to recompile youtube-dl.

In your bash terminal, set your current directory to the root folder of the youtube-dl master source code. For instance, if you downloaded the youtube-dl source code to your Downloads folder, you might have to set your current directory to "/Users/USERNAMEHERE/Downloads/youtube-dl-master" by typing "cd /Users/USERNAMEHERE/Downloads/youtube-dl-master" in your bash terminal.

Then enter the command "make" in your bash terminal. There will be an executable binary file created called "youtube-dl". Move that "youtube-dl" binary file to wherever your computer normally stores the youtube-dl binary file. Use the terminal command "which youtube-dl" if you need to find out where that is.

touchhyuk · 2020-09-30T12:04:03Z

@touchhyuk I assume that you've already downloaded the youtube-dl master source code. You'll need to make the modifications to viu.py that are listed in my pull request.

Then you have to recompile youtube-dl.

In your bash terminal, set your current directory to the root folder of the youtube-dl master source code. For instance, if you downloaded the youtube-dl source code to your Downloads folder, you might have to set your current directory to "/Users/USERNAMEHERE/Downloads/youtube-dl-master" by typing "cd /Users/USERNAMEHERE/Downloads/youtube-dl-master" in your bash terminal.

Then enter the command "make" in your bash terminal. There will be an executable binary file created called "youtube-dl". Move that "youtube-dl" binary file to wherever your computer normally stores the youtube-dl binary file. Use the terminal command "which youtube-dl" if you need to find out where that is.

i cant run make command
got this error

C:\Users\Administrator\Downloads\youtube-dl-master>make
'make' is not recognized as an internal or external command,
operable program or batch file.

panda130 · 2020-10-02T02:10:58Z

@touchhyuk I am sorry to hear that. I don't use Windows so I'm afraid I can't help with this particular problem. It seems to be a permissions issue. Please try asking on the Reddit or stack exchange page.

See this link:
https://stackoverflow.com/questions/55561387/how-to-rebuild-python-youtube-dl-from-the-source

lkho · 2020-10-02T10:12:39Z

@touchhyuk If you just want to run youtube-dl with python interpreter (which you have already downloaded), there is no need to recompile or rebuild a standalone exe. Just follow the developer instructions will work.

lkho · 2020-10-02T10:22:43Z

@panda130 @zackmark29 for your reference

In my country (HK), ut is also 0, area_id = 1, language_flag_id = 1 for zh-hk, 3 for en-us

touchhyuk · 2020-10-02T10:26:15Z

@touchhyuk If you just want to run youtube-dl with python interpreter (which you have already downloaded), there is no need to recompile or rebuild a standalone exe. Just follow the developer instructions will work.

i cant please giive your discord id or telegram id

touchhyuk · 2020-10-02T11:04:04Z

please someone fix youtube-dl.exe with viu.py and send here. i want downloaf from viu and tried a week but cant fix

zackmark29 · 2020-10-04T04:40:31Z

please someone fix youtube-dl.exe with viu.py and send here. i want downloaf from viu and tried a week but cant fix

You could try to follow my instruction from this
#26788

panda130 mentioned this issue Sep 26, 2020

[viu] Fix Viu extractor #26723

Closed

5 of 9 tasks complete

zackmark29 mentioned this issue Oct 3, 2020

[Extractor Error] Getting extractor error when downloading videos from VIU. #26788

Open

6 of 6 tasks complete

lkho linked a pull request that will close this issue Oct 15, 2020

[viu:ott] add support for premium account, add language_flag_id query param #26775

Open

6 of 6 tasks complete

ytdl-org / youtube-dl

Viu OTT extractor error (KeyError) #26701

Viu OTT extractor error (KeyError) #26701

panda130 commented Sep 25, 2020 •

edited

zackmark29 commented Sep 25, 2020 •

edited

panda130 commented Sep 25, 2020

zackmark29 commented Sep 25, 2020 •

edited

zackmark29 commented Sep 25, 2020

panda130 commented Sep 25, 2020

zackmark29 commented Sep 25, 2020 •

edited

Ashish0804 commented Sep 26, 2020

zackmark29 commented Sep 26, 2020

Ashish0804 commented Sep 26, 2020

zackmark29 commented Sep 26, 2020 •

edited

panda130 commented Sep 26, 2020

zackmark29 commented Sep 27, 2020

zackmark29 commented Sep 27, 2020 •

edited

panda130 commented Sep 27, 2020

zackmark29 commented Sep 29, 2020

touchhyuk commented Sep 29, 2020 •

edited

panda130 commented Sep 30, 2020

touchhyuk commented Sep 30, 2020

panda130 commented Oct 2, 2020

lkho commented Oct 2, 2020

lkho commented Oct 2, 2020

touchhyuk commented Oct 2, 2020

touchhyuk commented Oct 2, 2020

zackmark29 commented Oct 4, 2020

ytdl-org / youtube-dl

Join GitHub today

GitHub is where the world builds software

Viu OTT extractor error (KeyError) #26701

Viu OTT extractor error (KeyError) #26701

Comments

panda130 commented Sep 25, 2020 • edited

Checklist

Verbose log

Description

zackmark29 commented Sep 25, 2020 • edited

panda130 commented Sep 25, 2020

zackmark29 commented Sep 25, 2020 • edited

zackmark29 commented Sep 25, 2020

panda130 commented Sep 25, 2020

zackmark29 commented Sep 25, 2020 • edited

Ashish0804 commented Sep 26, 2020

zackmark29 commented Sep 26, 2020

Ashish0804 commented Sep 26, 2020

zackmark29 commented Sep 26, 2020 • edited

panda130 commented Sep 26, 2020

zackmark29 commented Sep 27, 2020

zackmark29 commented Sep 27, 2020 • edited

panda130 commented Sep 27, 2020

zackmark29 commented Sep 29, 2020

touchhyuk commented Sep 29, 2020 • edited

panda130 commented Sep 30, 2020

touchhyuk commented Sep 30, 2020

panda130 commented Oct 2, 2020

lkho commented Oct 2, 2020

lkho commented Oct 2, 2020

touchhyuk commented Oct 2, 2020

touchhyuk commented Oct 2, 2020

zackmark29 commented Oct 4, 2020

Essential cookies

Always active

Analytics cookies

panda130 commented Sep 25, 2020 •

edited

zackmark29 commented Sep 25, 2020 •

edited

zackmark29 commented Sep 25, 2020 •

edited

zackmark29 commented Sep 25, 2020 •

edited

zackmark29 commented Sep 26, 2020 •

edited

zackmark29 commented Sep 27, 2020 •

edited

touchhyuk commented Sep 29, 2020 •

edited