Windows 10: Response Code 400 #27

clayton-grey · 2021-05-20T04:03:03Z

I was having trouble with the executable on OSX (I recently migrated to a different computer, so the setup I had working before isn't handy anymore), so I tried the Windows version, which seemed to work more as described:

OS: Windows 10
Version: EXE

Here's the output:

hulusubs_dl --verbose

***Starting the script in Verbose Mode***

Enter Hulu URL : https://www.hulu.com/series/the-great-north-374cdbc2-3b08-4b81-a68f-a45a70665849
Reading Configuration File.
Whoops! Seems like I can't connect to website.
It's showing : <Response [400]>
Run this script with the --verbose argument and report the issue along with log file on Github.
Can't connect to website https://discover.hulu.com/content/v5/hubs/series/north-374cdbc2-3b08-4b81-a68f-a45a70665849?schema=1&limit=999&device_info=web:3.10.0&referralHost=production
Couldn't get series metadata. Exiting

The text was updated successfully, but these errors were encountered:

csisoap · 2021-05-28T14:30:38Z

Try https://www.hulu.com/series/great-north-374cdbc2-3b08-4b81-a68f-a45a70665849

ghost · 2021-05-31T03:27:15Z

@csisoap Thank you so much, I managed to download the subtitles of the TV series "Fear the Walking Dead" by renameing the link to

https://www.hulu.com/series/walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53

and used the new link and it worked (seems that the name works only if it has two words, "rick-and-morty" to work you need to rename it to "and-morty").

If the TV series has one word only like "futurama" the link will not work, adding "-" before the name will make the link works like "-futurama".

https://www.hulu.com/series/futurama-85bf4cc1-cd8b-4469-ad87-7289217a0b74

https://www.hulu.com/series/-futurama-85bf4cc1-cd8b-4469-ad87-7289217a0b74

Xonshiz · 2021-05-31T05:18:44Z

Woah, this is a weird thing that's happening. I'll add a bug tag to this and see if it's happening with all the series or just some handful of them.

ghost · 2021-05-31T05:29:39Z

@Xonshiz The script does not work with the links of the movies like

https://www.hulu.com/movie/terminator-dark-fate-3be9a47d-4a50-447a-980a-e5cae615f522

it works only when i replace "/movie/" and the name of the movie with "/watch/".

https://www.hulu.com/watch/3be9a47d-4a50-447a-980a-e5cae615f522

and the name of the subtitle file of the movie become "No Name Found - S01E01 [en Sub].srt"

Xonshiz · 2021-05-31T05:57:54Z

Yes, that "/watch" thing is a known "issue" or "thing" however you term it. It was me being lazy to not add one more entry of check after I had checked in a general code that worked for both. It's just an extra step haha. I'll patch it up in next fix. But, what I'm more curious about is that the script is working with just "2 words"... like that walking dead link. That needs to be looked into. I'll see if I can muster up some time to look into this.

Ah No Name found thing.... Didn't know about it. Seems like Hulu has changed some things again. I'll need to update this.

ghost · 2021-05-31T06:18:33Z

@Xonshiz It seems to be about the "-" more than the title has two words, like the TV series "Malcolm in the Middle" could work with only "-middle".

https://www.hulu.com/series/malcolm-in-the-middle-ca1ac46e-9883-4125-a6e8-97efce9a2bf5

https://www.hulu.com/series/-middle-ca1ac46e-9883-4125-a6e8-97efce9a2bf5

csisoap · 2021-05-31T06:27:46Z

@Xonshiz i think instead of grabbing the first two, try to grab five words from the last.

Xonshiz · 2021-05-31T06:48:21Z

See.. that's the thing... I don't remember putting any code that deletes/grabs any specific set of words. It should take all and send to HULU APIs. That's why I'll need to see what is happening, where and why.

clayton-grey · 2021-06-02T08:08:36Z

Thanks for the response. Your link was recognized, but I ran into a new issue:

Downloading Subtitle For The Great North - S1E1 [en Sub].vtt
Couldn't find vtt in Hulu

Which also doesn't produce a log, but it seems like it should work?

ghost · 2021-06-02T11:17:50Z

@clayton-grey You could download the subtitles of the TV show "The Great North" in "srt" format.

Xonshiz · 2021-06-02T12:18:22Z

That subtitle thing is another issue. Could you open a separate bug for that. Helps in checking what all things I need to fix.

Also, @Essam3111 good that you were helping, but, I'd suggest that you do not share an actual subtitle file over here. I think your comment was removed and I really don't want this repo getting nuked on some BS DMCA notice.

ghost · 2021-06-02T12:27:24Z

@Xonshiz I deleted the comment by myself for the same reason.

Xonshiz · 2021-06-02T12:34:25Z

Hhahah I see... good, thanks :D

ghost · 2021-06-02T18:12:45Z

@Xonshiz Just a simple observation, when I try to download the subtitles in "ttml" format or "smi" format for any TV show, the script download only the first episode, but when I go to the folder of the downloaded subtitles no subtitles are actually downloaded and the folder is empty, Hulu subtitles in "ttml" do not work correctly with Subtitle Edit, Subtitle Edit displays the time wrongly (nothing wrong with the subtitles themselves), and Hulu subtitles in "smi" format are encrypted.

Xonshiz · 2021-06-02T18:55:35Z

Ah, I see. Could you please open a new bug report for that and also mention some URLs where you can see this issue.

clayton-grey · 2021-06-02T23:49:06Z

@clayton-grey You could download the subtitles of the TV show "The Great North" in "srt" format.

I could, but I use VTT, so it saves me conversion steps, and it should be possible to get from their servers.

I'll open a new ticket for this.

ghost · 2021-06-03T00:47:38Z

@clayton-grey This is just a workaround, until the script got fixed you could use it for downloading the subtitles in "webvtt" format, in config file add 'webvtt'.

Hulusubs_dl.webvtt.zip

In utils.py and hulu_subs_dl.py I replaced 'vtt' with 'webvtt' (this solves the problem of the error message "Couldn't find vtt in Hulu")

In hulu.py I replaced

if extension == 'srt':

with

if not extension == 'srt':

In subtitle_processing.py I replaced

replacement = re.sub(r'(\d\d:\d\d:\d\d).(\d\d\d) --> (\d\d:\d\d:\d\d).(\d\d\d)(?:[ \-\w]+:[\w\%\d:]+)*\n',
                     r'\1,\2 --> \3,\4\n', file_contents)
replacement = re.sub(r'(\d\d:\d\d).(\d\d\d) --> (\d\d:\d\d).(\d\d\d)(?:[ \-\w]+:[\w\%\d:]+)*\n',
                     r'\1,\2 --> \3,\4\n', replacement)
replacement = re.sub(r'(\d\d).(\d\d\d) --> (\d\d).(\d\d\d)(?:[ \-\w]+:[\w\%\d:]+)*\n', r'\1,\2 --> \3,\4\n',
                     replacement)
replacement = re.sub(r'WEBVTT\n', '', replacement)
replacement = re.sub(r'Kind:[ \-\w]+\n', '', replacement)
replacement = re.sub(r'Language:[ \-\w]+\n', '', replacement)
replacement = re.sub(r'<c[.\w\d]*>', '', replacement)
replacement = re.sub(r'</c>', '', replacement)
replacement = re.sub(r'<\d\d:\d\d:\d\d.\d\d\d>', '', replacement)
replacement = re.sub(r'::[\-\w]+\([\-.\w\d]+\)[ ]*{[.,:;\(\) \-\w\d]+\n }\n', '', replacement)
replacement = re.sub(r'Style:\n##\n', '', replacement)

with

replacement = re.sub(r'(\d\d:\d\d:\d\d).(\d\d\d) --> (\d\d:\d\d:\d\d).(\d\d\d)(?:[ \-\w]+:[\w\%\d:]+)*\n',
                     r'\1.\2 --> \3.\4\n', file_contents)
#replacement = re.sub(r'(\d\d:\d\d).(\d\d\d) --> (\d\d:\d\d).(\d\d\d)(?:[ \-\w]+:[\w\%\d:]+)*\n',
#                     r'\1,\2 --> \3,\4\n', replacement)
#replacement = re.sub(r'(\d\d).(\d\d\d) --> (\d\d).(\d\d\d)(?:[ \-\w]+:[\w\%\d:]+)*\n', r'\1,\2 --> \3,\4\n',
#                     replacement)
#replacement = re.sub(r'WEBVTT\n', '', replacement)
#replacement = re.sub(r'Kind:[ \-\w]+\n', '', replacement)
#replacement = re.sub(r'Language:[ \-\w]+\n', '', replacement)
#replacement = re.sub(r'<c[.\w\d]*>', '', replacement)
#replacement = re.sub(r'</c>', '', replacement)
#replacement = re.sub(r'<\d\d:\d\d:\d\d.\d\d\d>', '', replacement)
#replacement = re.sub(r'::[\-\w]+\([\-.\w\d]+\)[ ]*{[.,:;\(\) \-\w\d]+\n }\n', '', replacement)
#replacement = re.sub(r'Style:\n##\n', '', replacement)

ghost · 2021-06-04T18:47:55Z

@Xonshiz This might help you in figuring out a solution for the problem of the non working links for the TV shows like

https://www.hulu.com/series/fear-the-walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53

and

https://www.hulu.com/series/futurama-85bf4cc1-cd8b-4469-ad87-7289217a0b74

when i compared between the "Hulusubs_dl_Error_Log" for the non working link for the TV show "Fear the walking dead"

https://www.hulu.com/series/fear-the-walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53

DEBUG: eab_id_matches: ['the', 'walking', 'dead', '6b075a0b', '3d97', '42a8', 'b2f1', 'c1d698d64d53']
DEBUG: initial eab_id: walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53
DEBUG: GET url: https://discover.hulu.com/content/v5/hubs/series/walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1&limit=999&device_info=web:3.10.0&referralHost=production
DEBUG: GET proxy: {'http': None, 'https': None}
DEBUG: Starting new HTTPS connection (1): discover.hulu.com:443
DEBUG: https://discover.hulu.com:443 "GET /content/v5/hubs/series/walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1&limit=999&device_info=web:3.10.0&referralHost=production HTTP/1.1" 400 277
DEBUG:
initial series_metadata:
None

and the working one for the TV show "Fear the walking dead"

https://www.hulu.com/series/walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53

DEBUG: eab_id_matches: ['dead', '6b075a0b', '3d97', '42a8', 'b2f1', 'c1d698d64d53']
DEBUG: initial eab_id: 6b075a0b-3d97-42a8-b2f1-c1d698d64d53
DEBUG: GET url: https://discover.hulu.com/content/v5/hubs/series/6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1&limit=999&device_info=web:3.10.0&referralHost=production
DEBUG: GET proxy: {'http': None, 'https': None}
DEBUG: Starting new HTTPS connection (1): discover.hulu.com:443
DEBUG: https://discover.hulu.com:443 "GET /content/v5/hubs/series/6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1&limit=999&device_info=web:3.10.0&referralHost=production HTTP/1.1" 200 38669
DEBUG:
initial series_metadata: {'_type': 'hub', 'id': '6b075a0b-3d97-42a8-b2f1-c1d698d64d53', 'href': 'https://discover.hulu.com/content/v5/hubs/series/6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1', 'name': 'Fear the Walking Dead', 'theme': 'hub_theme_entity_detail', 'artwork': {'title.treatment.horizontal': {'path':

I found that with the non working link for the TV show "Fear the walking dead" we get the wrong link (It has the words "walking-dead-" which they should not be in the link)

https://discover.hulu.com/content/v5/hubs/series/walking-dead-6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1&limit=999&device_info=web:3.10.0&referralHost=production

and in the working link for the TV show "Fear the walking dead" we get the correct link

https://discover.hulu.com/content/v5/hubs/series/6b075a0b-3d97-42a8-b2f1-c1d698d64d53?schema=1&limit=999&device_info=web:3.10.0&referralHost=production

and when i compared between the "Hulusubs_dl_Error_Log" for the non working link for the TV show "Futurama"

https://www.hulu.com/series/futurama-85bf4cc1-cd8b-4469-ad87-7289217a0b74

DEBUG: eab_id_matches: ['85bf4cc1', 'cd8b', '4469', 'ad87', '7289217a0b74']
DEBUG: initial eab_id: cd8b-4469-ad87-7289217a0b74
DEBUG: GET url: https://discover.hulu.com/content/v5/hubs/series/cd8b-4469-ad87-7289217a0b74?schema=1&limit=999&device_info=web:3.10.0&referralHost=production
DEBUG: GET proxy: {'http': None, 'https': None}
DEBUG: Starting new HTTPS connection (1): discover.hulu.com:443
DEBUG: https://discover.hulu.com:443 "GET /content/v5/hubs/series/cd8b-4469-ad87-7289217a0b74?schema=1&limit=999&device_info=web:3.10.0&referralHost=production HTTP/1.1" 400 259
DEBUG:
initial series_metadata:
None

and the working one for the TV show "Futurama"

https://www.hulu.com/series/-futurama-85bf4cc1-cd8b-4469-ad87-7289217a0b74

DEBUG: eab_id_matches: ['futurama', '85bf4cc1', 'cd8b', '4469', 'ad87', '7289217a0b74']
DEBUG: initial eab_id: 85bf4cc1-cd8b-4469-ad87-7289217a0b74
DEBUG: GET url: https://discover.hulu.com/content/v5/hubs/series/85bf4cc1-cd8b-4469-ad87-7289217a0b74?schema=1&limit=999&device_info=web:3.10.0&referralHost=production
DEBUG: GET proxy: {'http': None, 'https': None}
DEBUG: Starting new HTTPS connection (1): discover.hulu.com:443
DEBUG: https://discover.hulu.com:443 "GET /content/v5/hubs/series/85bf4cc1-cd8b-4469-ad87-7289217a0b74?schema=1&limit=999&device_info=web:3.10.0&referralHost=production HTTP/1.1" 200 16034
DEBUG:
initial series_metadata: {'_type': 'hub', 'id': '85bf4cc1-cd8b-4469-ad87-7289217a0b74', 'href': 'https://discover.hulu.com/content/v5/hubs/series/85bf4cc1-cd8b-4469-ad87-7289217a0b74?schema=1', 'name': 'Futurama', 'theme': 'hub_theme_entity_detail', 'artwork': {'title.treatment.horizontal': {'path':

I found that with the non working link for the TV show "Futurama" we get the wrong link (the links is missing "85bf4cc1-")

https://discover.hulu.com/content/v5/hubs/series/cd8b-4469-ad87-7289217a0b74?schema=1&limit=999&device_info=web:3.10.0&referralHost=production

and in the working link for the TV show "Futurama" we get the correct link

https://discover.hulu.com/content/v5/hubs/series/85bf4cc1-cd8b-4469-ad87-7289217a0b74?schema=1&limit=999&device_info=web:3.10.0&referralHost=production

ghost · 2021-06-05T00:43:30Z

@Xonshiz I found out why the link works only if it has two words, this regular expression removes one word

https://www.hulu.com/series/castle-rock-b11816c9-9e35-44f3-bf04-220b1d12f770

https://github.com/Xonshiz/Hulusubs_dl/blob/master/hulusubs_dl/hulu.py#L86

https://regex101.com/r/48hdPs/1

and this line removes another word

https://github.com/Xonshiz/Hulusubs_dl/blob/master/hulusubs_dl/hulu.py#L89

and we get the correct link

https://discover.hulu.com/content/v5/hubs/series/b11816c9-9e35-44f3-bf04-220b1d12f770?schema=1&limit=999&device_info=web:3.10.0&referralHost=production

this is the correct regular expression for selecting all the words

([0-9A-Za-z]+

https://regex101.com/r/ZogdBy/1

clayton-grey · 2021-06-05T05:45:21Z

Nice sleuthing.

Xonshiz · 2021-06-05T10:05:16Z

Holy crap... You've literally fixed it all. hahahaha Thank you so much @Essam3111 .

EDIT: Oh I think I just remembered why I used that regex. Didn't see if failing this badly :(

- Should close #27 - Added fix for downloading subtitles for movie urls.

ghost · 2021-06-05T12:44:04Z

@Xonshiz Thank you so much, I tested the dev branch and everything is working great until now, just one more problem still needs to be fixed, the movie subtitle file name is "No Name Found - S01E01 [en Sub]", here is the Hulusubs_dl_Error_Log for the movie

https://www.hulu.com/movie/terminator-dark-fate-3be9a47d-4a50-447a-980a-e5cae615f522

Hulusubs_dl_Error_Log.zip

Xonshiz · 2021-06-05T12:48:54Z

Yep, the thing is, I don't have the JSON that's returned. So, what I'm guessing is that for "movie", they have a different JSON getting back and for a show video, they have different JSON getting back. Thanks for sharing the verbose dump, this should have the JSON, maybe I can work my way out of that one.

Fixed #27 and support for movies

ghost · 2021-06-06T16:47:26Z

@Xonshiz Just a very minor problem with the name of the downloaded subtitle file for the movie, the number of the season added to the name of the subtitle file and a season folder created like.

C:\Hulusubs_dl\Terminator - Dark Fate\01\Terminator - Dark Fate - S01E01 [en Sub].srt

https://www.hulu.com/movie/terminator-dark-fate-3be9a47d-4a50-447a-980a-e5cae615f522

Xonshiz · 2021-06-06T17:02:55Z

Ah, got it. Could you open a separate bug for that? It's cased via different functionality and this issue serves purpose for 400 error.

Xonshiz added the bug label May 31, 2021

Xonshiz self-assigned this May 31, 2021

Xonshiz added a commit that referenced this issue Jun 5, 2021

Fix for #27 and support for "/movie/"

e3a3a08

- Should close #27 - Added fix for downloading subtitles for movie urls.

Xonshiz mentioned this issue Jun 5, 2021

Fixed #27 and support for movies #30

Merged

Xonshiz closed this as completed in #30 Jun 5, 2021

Xonshiz added a commit that referenced this issue Jun 5, 2021

Merge pull request #30 from Xonshiz/dev

0f2d9f7

Fixed #27 and support for movies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Windows 10: Response Code 400 #27

Windows 10: Response Code 400 #27

clayton-grey commented May 20, 2021

csisoap commented May 28, 2021

ghost commented May 31, 2021 •

edited by ghost

Xonshiz commented May 31, 2021

ghost commented May 31, 2021 •

edited by ghost

Xonshiz commented May 31, 2021

ghost commented May 31, 2021

csisoap commented May 31, 2021

Xonshiz commented May 31, 2021

clayton-grey commented Jun 2, 2021

ghost commented Jun 2, 2021

Xonshiz commented Jun 2, 2021

ghost commented Jun 2, 2021

Xonshiz commented Jun 2, 2021

ghost commented Jun 2, 2021 •

edited by ghost

Xonshiz commented Jun 2, 2021

clayton-grey commented Jun 2, 2021

ghost commented Jun 3, 2021 •

edited by ghost

ghost commented Jun 4, 2021

ghost commented Jun 5, 2021

clayton-grey commented Jun 5, 2021

Xonshiz commented Jun 5, 2021 •

edited

ghost commented Jun 5, 2021

Xonshiz commented Jun 5, 2021

ghost commented Jun 6, 2021

Xonshiz commented Jun 6, 2021

Windows 10: Response Code 400 #27

Windows 10: Response Code 400 #27

Comments

clayton-grey commented May 20, 2021

csisoap commented May 28, 2021

ghost commented May 31, 2021 • edited by ghost

Xonshiz commented May 31, 2021

ghost commented May 31, 2021 • edited by ghost

Xonshiz commented May 31, 2021

ghost commented May 31, 2021

csisoap commented May 31, 2021

Xonshiz commented May 31, 2021

clayton-grey commented Jun 2, 2021

ghost commented Jun 2, 2021

Xonshiz commented Jun 2, 2021

ghost commented Jun 2, 2021

Xonshiz commented Jun 2, 2021

ghost commented Jun 2, 2021 • edited by ghost

Xonshiz commented Jun 2, 2021

clayton-grey commented Jun 2, 2021

ghost commented Jun 3, 2021 • edited by ghost

ghost commented Jun 4, 2021

ghost commented Jun 5, 2021

clayton-grey commented Jun 5, 2021

Xonshiz commented Jun 5, 2021 • edited

ghost commented Jun 5, 2021

Xonshiz commented Jun 5, 2021

ghost commented Jun 6, 2021

Xonshiz commented Jun 6, 2021

ghost commented May 31, 2021 •

edited by ghost

ghost commented May 31, 2021 •

edited by ghost

ghost commented Jun 2, 2021 •

edited by ghost

ghost commented Jun 3, 2021 •

edited by ghost

Xonshiz commented Jun 5, 2021 •

edited