Animelab Extractor Downloads Mismatched Audio/Video By Default

[LunaGNUisance]

I'm pretty surprised that yt-dlp has an extractor that supports AnimeLab, of all things, but I've encountered an issue with it.

When using yt-dlp --cookies cookies.txt https://www.animelab.com/player/wonder-egg-priority-episode-1/subtitles, it tries to download bestaudio+bestvideo, which works successfully...however, it downloads the hardsubbed video (i.e. meant for Japanese audio) and pairs it up with the English audio - even when adding the "/subtitles" segment to the URL. I end up with an episode that has hardcoded English subtitles that are mismatched with the Japanese audio.

Why Animelab decided to hardcode the subtitles, when even Funimation doesn't, I have no idea...

In order to download the episode with matching video/audio, you first have to issue a yt-dlp -F https://www.animelab.com/player/wonder-egg-priority-episode-1/subtitles command and find the format IDs that match, then issue a yt-dlp -f 91268_nothardsubbed_en-5+184694_yeshardsubbed_ja-JP-audio-ENGLISH https://www.animelab.com/player/wonder-egg-priority-episode-1/subtitles command.

This isn't so bad in the grand scheme of things, but it does make the ability to download an entire series with one command (yt-dlp --cookies cookies.txt https://www.animelab.com/shows/wonder-egg-priority) effectively useless.

My use case is that I need the non-hardcoded subs video with Japanese audio, and I'll then insert Japanese subtitles into the video, however, if I was able to get either the subbed or dubbed version of an episode/series by default, that would be good enough.

[AnimeLab] Downloading login page
[AnimeLab] wonder-egg-priority-episode-2: Downloading URL https://www.animelab.com/player/wonder-egg-priority-episode-2/subtitles
[AnimeLab] 58821: Downloading m3u8 information
[AnimeLab] 58821: Downloading m3u8 information
[AnimeLab] 58821: Downloading MPD manifest
[AnimeLab] 58821: Downloading MPD manifest
[AnimeLab] wonder-egg-priority-episode-2: Downloading URL https://www.animelab.com/player/wonder-egg-priority-episode-2/dubbed
[AnimeLab] 58821: Downloading m3u8 information
[AnimeLab] 58821: Downloading m3u8 information
[AnimeLab] 58821: Downloading MPD manifest
[AnimeLab] 58821: Downloading MPD manifest
[info] 58821: Downloading 1 format(s): 185047_yeshardsubbed_ja-JP-5+192363_nothardsubbed_en-0
[download] Destination: Wonder Egg Priority - Episode 2 - The Terms of Friendship.f185047_yeshardsubbed_ja-JP-5.mp4
[download]   0.2% of 408.07MiB at  3.45MiB/s ETA 02:09

I'm using the latest release.

[Jules-A]

I'm a bit confused when it comes to subs but interested in using AnimeLab extractor in the future, is it possible to download dubs with English translations only on stuff like signposts, mobile phones ect. (what is that called? softsubs?). I currently don't have a subscription to AnimeLab so I can't test it.

[LunaGNUisance]

@Jules-A I would first like to note that, as far as I'm aware, Animelab is only accessible within Australia and New Zealand. It is essentially Funimation for those regions (they're owned by the same company - well, Crunchyroll is too, but Animelab and Funimation are more tightly integrated).

Hardsubs means that the subtitles are burned into the video; they are actually a part of the video's frames; the .mp4 file. Soft subtitles, on the other hand, are contained in subtitle files, like .srt, .ass, or .vtt files (commonly .srt), and they are then incorporated into the video. They can be turned on and off, which is very handy if you are, for example, trying to learn a language, or need clean footage for some sort of video production.

That's the non-technical explanation, anyway.

Put simply, the subtitles are a part of the JP-video version, and they can't be removed. In the English dub version, however, there are no subtitles at all, which is what I downloaded and paired up with the Japanese audio.

So, it is not possible to selectively turn off subtitles in the hardsub version for certain timecodes.

[LunaGNUisance]

To clarify, animelab does not have .srt files for English subtitles (or any other language subtitles for that matter) for the series I tested. You'd have to download the English dub episode and find an .srt file elsewhere. Kitsunekko is where I find my Japanese subtitles; it seems they have English and Chinese as well.

[pukkandan]

You can probably bypass this issue using a format filter. If you can show me the output of -F, I can help you write a correct filter.

To actually fix this however, I couldn't find a free VPN/proxy. If you can provide me some proxy details, I can test and (probably) fix it

The extracted was added from ytdl-org/youtube-dl#13600 by @mariuszskon. Maybe he would be interested in helping

[LunaGNUisance]

I had read the manual page and found the section on the format filter, however, I wasn't sure if it was actually possible to filter out the IDs in the way that I needed to, so I hadn't attempted it.

Output of -F:

[AnimeLab] Downloading login page
[AnimeLab] wonder-egg-priority-episode-2: Downloading URL https://www.animelab.com/player/wonder-egg-priority-episode-2/subtitles
[AnimeLab] 58821: Downloading m3u8 information
[AnimeLab] 58821: Downloading m3u8 information
[AnimeLab] 58821: Downloading MPD manifest
[AnimeLab] 58821: Downloading MPD manifest
[AnimeLab] wonder-egg-priority-episode-2: Downloading URL https://www.animelab.com/player/wonder-egg-priority-episode-2/dubbed
[AnimeLab] 58821: Downloading m3u8 information
[AnimeLab] 58821: Downloading m3u8 information
[AnimeLab] 58821: Downloading MPD manifest
[AnimeLab] 58821: Downloading MPD manifest
[info] Available formats for 58821:
ID                                       EXT  RESOLUTION |   TBR PROTO | VCODEC        VBR ACODEC     ABR  ASR    NOTE
---------------------------------------- ---- ---------- - ----- ----- - ----------- ----- --------- ---- ------- -----------------------------
185040_yeshardsubbed_ja-JP-audio-ENGLISH m3u8 audio only |       m3u8  |                   unknown                [en]
185045_yeshardsubbed_ja-JP-audio-ENGLISH m3u8 audio only |       m3u8  |                   unknown                [en]
192360_nothardsubbed_en-audio-ENGLISH    m3u8 audio only |       m3u8  |                   unknown                [en]
192361_nothardsubbed_en-audio-ENGLISH    m3u8 audio only |       m3u8  |                   unknown                [en]
185046_yeshardsubbed_ja-JP-0             m4a  audio only |  207k https |                   mp4a.40.2 207k 48000Hz DASH audio, m4a_dash, 48000Hz
185047_yeshardsubbed_ja-JP-0             m4a  audio only |  207k https |                   mp4a.40.2 207k 48000Hz DASH audio, m4a_dash, 48000Hz
192362_nothardsubbed_en-0                m4a  audio only |  211k https |                   mp4a.40.2 211k 48000Hz DASH audio, m4a_dash, 48000Hz
192363_nothardsubbed_en-0                m4a  audio only |  211k https |                   mp4a.40.2 211k 48000Hz DASH audio, m4a_dash, 48000Hz
185040_yeshardsubbed_ja-JP-579           mp4  426x240    |  579k m3u8  | avc1.640028  579k                        
185045_yeshardsubbed_ja-JP-579           mp4  426x240    |  579k m3u8  | avc1.640028  579k                        
192360_nothardsubbed_en-580              mp4  426x240    |  580k m3u8  | avc1.640028  580k                        
192361_nothardsubbed_en-580              mp4  426x240    |  580k m3u8  | avc1.640028  580k                        
185046_yeshardsubbed_ja-JP-1             mp4  426x240    |  671k https | avc1.640028  671k                        DASH video, mp4_dash
185047_yeshardsubbed_ja-JP-1             mp4  426x240    |  671k https | avc1.640028  671k                        DASH video, mp4_dash
192362_nothardsubbed_en-1                mp4  426x240    |  688k https | avc1.640028  688k                        DASH video, mp4_dash
192363_nothardsubbed_en-1                mp4  426x240    |  688k https | avc1.640028  688k                        DASH video, mp4_dash
185043_yeshardsubbed_ja-JP               mp4  360p       |       https | unknown           unknown                [ja-JP]
192357_nothardsubbed_en                  mp4  360p       |       https | unknown           unknown                [en]
185040_yeshardsubbed_ja-JP-888           mp4  640x360    |  888k m3u8  | avc1.640028  888k                        
185045_yeshardsubbed_ja-JP-888           mp4  640x360    |  888k m3u8  | avc1.640028  888k                        
192360_nothardsubbed_en-889              mp4  640x360    |  889k m3u8  | avc1.640028  889k                        
192361_nothardsubbed_en-889              mp4  640x360    |  889k m3u8  | avc1.640028  889k                        
185046_yeshardsubbed_ja-JP-2             mp4  640x360    | 1342k https | avc1.640028 1342k                        DASH video, mp4_dash
185047_yeshardsubbed_ja-JP-2             mp4  640x360    | 1342k https | avc1.640028 1342k                        DASH video, mp4_dash
192362_nothardsubbed_en-2                mp4  640x360    | 1344k https | avc1.640028 1344k                        DASH video, mp4_dash
192363_nothardsubbed_en-2                mp4  640x360    | 1344k https | avc1.640028 1344k                        DASH video, mp4_dash
185042_yeshardsubbed_ja-JP               mp4  480p       |       https | unknown           unknown                [ja-JP]
192356_nothardsubbed_en                  mp4  480p       |       https | unknown           unknown                [en]
185040_yeshardsubbed_ja-JP-1298          mp4  853x480    | 1298k m3u8  | avc1.640028 1298k                        
185045_yeshardsubbed_ja-JP-1298          mp4  853x480    | 1298k m3u8  | avc1.640028 1298k                        
192360_nothardsubbed_en-1300             mp4  853x480    | 1300k m3u8  | avc1.640028 1300k                        
192361_nothardsubbed_en-1300             mp4  853x480    | 1300k m3u8  | avc1.640028 1300k                        
192362_nothardsubbed_en-3                mp4  854x480    | 2207k https | avc1.640028 2207k                        DASH video, mp4_dash
192363_nothardsubbed_en-3                mp4  854x480    | 2207k https | avc1.640028 2207k                        DASH video, mp4_dash
185046_yeshardsubbed_ja-JP-3             mp4  854x480    | 2210k https | avc1.640028 2210k                        DASH video, mp4_dash
185047_yeshardsubbed_ja-JP-3             mp4  854x480    | 2210k https | avc1.640028 2210k                        DASH video, mp4_dash
185044_yeshardsubbed_ja-JP               mp4  720p       |       https | unknown           unknown                [ja-JP]
192358_nothardsubbed_en                  mp4  720p       |       https | unknown           unknown                [en]
185045_yeshardsubbed_ja-JP-1798          mp4  1280x720   | 1798k m3u8  | avc1.640028 1798k                        
192361_nothardsubbed_en-1798             mp4  1280x720   | 1798k m3u8  | avc1.640028 1798k                        
185047_yeshardsubbed_ja-JP-4             mp4  1280x720   | 3337k https | avc1.640028 3337k                        DASH video, mp4_dash
192363_nothardsubbed_en-4                mp4  1280x720   | 3364k https | avc1.640028 3364k                        DASH video, mp4_dash
185041_yeshardsubbed_ja-JP               mp4  1080p      |       https | unknown           unknown                [ja-JP]
192359_nothardsubbed_en                  mp4  1080p      |       https | unknown           unknown                [en]
185045_yeshardsubbed_ja-JP-2818          mp4  1920x1080  | 2818k m3u8  | avc1.640028 2818k                        
192361_nothardsubbed_en-2819             mp4  1920x1080  | 2819k m3u8  | avc1.640028 2819k                        
192363_nothardsubbed_en-5                mp4  1920x1080  | 5436k https | avc1.640028 5436k                        DASH video, mp4_dash
185047_yeshardsubbed_ja-JP-5             mp4  1920x1080  | 5469k https | avc1.640028 5469k                        DASH video, mp4_dash

I couldn't find any free VPNs or Proxies for Australia or New Zealand; I don't think many would provide a free service for those countries, especially because we have essentially nothing of value someone who wasn't from here would want to bypass geo-restrictions to access. Personally, I use ProtonVPN for bypassing geo-restrictions for Crunchyroll and Hidive, but I'm sure there are better deals out there.

This is a bit of a niche extractor, which again, is why I'm surprised there's an extractor at all for it.

[LunaGNUisance]

Thank you for that clue, @pukkandan, I've found a filter that works:

yt-dlp --add-metadata --write-sub --cookies cookies.txt -ciw -o "%(title)s.%(ext)s" https://www.animelab.com/shows/wandering-witch-the-journey-of-elaina -f "[language=ja-JP]"

[Jules-A]

@Jules-A I would first like to note that, as far as I'm aware, Animelab is only accessible within Australia and New Zealand. It is essentially Funimation for those regions (they're owned by the same company - well, Crunchyroll is too, but Animelab and Funimation are more tightly integrated).

Put simply, the subtitles are a part of the JP-video version, and they can't be removed. In the English dub version, however, there are no subtitles at all, which is what I downloaded and paired up with the Japanese audio.

I'm from Australia but right now I'm watching a backlog of CR and Netflix dubs but I do plan to sub to AnimeLab soonish.
So does that mean they don't translate stuff like sign posts? That doesn't sound right? If they do, would that even be called a hard-sub?

[LunaGNUisance]

@Jules-A I would first like to note that, as far as I'm aware, Animelab is only accessible within Australia and New Zealand. It is essentially Funimation for those regions (they're owned by the same company - well, Crunchyroll is too, but Animelab and Funimation are more tightly integrated).
Put simply, the subtitles are a part of the JP-video version, and they can't be removed. In the English dub version, however, there are no subtitles at all, which is what I downloaded and paired up with the Japanese audio.

I'm from Australia but right now I'm watching a backlog of CR and Netflix dubs but I do plan to sub to AnimeLab soonish.
So does that mean they don't translate stuff like sign posts? That doesn't sound right? If they do, would that even be called a hard-sub?

For Wonder Egg Priority, at least, they do subtitle signposts, letters, and such. In the Japanese version, at least. I haven't really watched the English dub for any shows, so I couldn't say whether they subtitle those sorts of things on that version.

The only thing that constitutes whether it's a hard sub is whether the subtitles are burned into the video.

Edit: Even if only some things are subtitled, it's still hard-subbed. Soft sub if the subtitles in the video use an .srt/.vtt/.ass file for supplying subs, hard sub if the subs are 'burned' into the video.

[LunaGNUisance]

For completeness, this command will download any series (that isn't a movie, you need the direct link to the movie rather than the show page) on Animelab with the Japanese version or English version; no mismatched audio and video, with a preference for the Japanese version:

yt-dlp -o "%(title)s.%(ext)s" -f "[language=ja-JP]/[language=en]" https://www.animelab.com/shows/SHOWNAME

Swap ja-JP around with en if you have a preference for the English dub. I have this aliased in my ~/.bashrc as:

alias ytdlpal="yt-dlp --add-metadata --write-sub --cookies cookies.txt -ciw -o '"'%(title)s.%(ext)s'"' -f '"'[language=ja-JP]/[language=en]'"'"

[mariuszskon]

Hey, as @pukkandan mentioned I am the original author of the extractor.
I never expected this extractor to be merged in with a fork 😅 but I'm glad it has found some use.
Indeed it is a shame that the default mismatches video and audio, but luckily my extractor is... verbose enough with the formats that you can somehow wrangle it to be good.
I might be able to fix this however I won't be able to work on it for the foreseeable future. So if someone else is willing to take on the challenge of fixing my code, feel free 😄

[pukkandan]

@mariuszskon I have a pretty good idea how to fix this. However, due to geo-restriction, I am unable to test the code. If I write a potential fix for it, will you be able to test it?

[Jules-A]

You could try UrbanVPN but it's P2P so not sure if it would work or you could try a trial version of something like Expressvpn.

Probably worth noting AnimeLab plans to merge in to Funimation... Zzz... So I don't think AL will be around for very long.

[mariuszskon]

See ytdl-org/youtube-dl#13600 (comment) for why the AnimeLab extractor may no longer be needed.

[LunaGNUisance]

@pukkandan This can be closed. Animelab has been permanently shut down and absorbed into FUNimation (which as of last month is being slowly absorbed into Crunchyroll). The Animelab site will redirect you to FUNimation to login, and doesn't do any streaming anymore.

https://blog.funimation.com/2021/11/02/a-new-anime-era-begins-as-animelab-retires-on-9-december-2021/