Support non-Unicode Microsoft cmaps

[Masaiki]

problem occured on two traditional Chinese font. libass found the fonts, but can't render the charactors.

logs show that graphs of the charcodes coundn't be found.
here are the ass files and font files

[TheOneric]

Are you sure you attached the correct file? I can't seem to find the line from your screenshot, after sorting events by start time I get: nvm, I missed the other file

Dialogue: 0,0:03:23.49,0:03:25.80,sign_uma,Sign,0,0,0,,{\c&H55CAED&\an5\fs36\frz270\fad(255,1)\pos(513,266.444)}重炮{Mayano\NTop\NGun}
Comment: 0,0:03:24.17,0:03:26.17,Default,,0,0,0,,（重炮）
Dialogue: 0,0:03:37.56,0:03:39.36,Default,,0,0,0,,蜂蜜偏硬偏濃多加點
Dialogue: 0,0:03:39.36,0:03:41.11,Default,,0,0,0,,好 偏硬偏濃多加點是吧

You're testing with Aegisub, but Aegisub overwrites our DirectWrite font provider in a way that is known to cause problems with fallbacks and other things, see eg. #464 or about ttc files specifically #503 (comment) .
Can you test with something else, eg mpv or vlc?
Also a few days back changes to our DirectWrite provider were merged, from the log it appears you do not use those commits yet, can you test with libass master?

---

When I attempt to manually try to retype the visible and important bits from the screenshot (retyped 訐 and 天 are missing a stroke on top/beyond the highest horizontal line; not sure if just a font difference or different characters), fontconfig can't find anything for the name 華康圓體注音 instead I need to use DFYuanChuIn.

[Script Info]
ScriptType: v4.00+
ScaledBorderAndShadow: yes

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle
, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Default,DejaVu Sans,48,&H00FFFFFF,&H000000FF,&H00000000,&H00000000,0,0,0,0,100,100,0,0,1,2,2,2,10,10,10,1

[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:00.00,0:00:05.00,Default,,0,0,0,,{\shad0\bord2\fs55\fn華康圓體注音\c&H2C2DF9&\b1}訐解{\r\fs45\fnTT-JTCウインZ5}古天(グフ)
Dialogue: 0,0:00:05.01,0:00:10.00,Default,,0,0,0,,{\shad0\bord2\fs55\fnDFYuanChuIn\c&H2C2DF9&\b1}訐解{\r\fs45\fnTT-JTCウインZ5}古天(グフ)

With the changed font name, it first selects the font, then fails to find glyph 0x8A10 訐 and selects a fallback. Are you sure DFYuanChuIn contains 0x8A10? Fontforge can't find the glyph either, but it also complained about the lookup-table and other things when opening this font.

[astiob]

The font you’re using does not contain the characters you’re trying to display. You can see this in the very log file that you attached.

As mentioned, your Aegisub build disables font fallback, so you don’t see the characters at all. This serves as a warning to you, the subtitle author, that you’re using a wrong font. (Although this is not Aegisub’s original intention and will stop happening in the future.)

Also as mentioned, the font attached does not have that name, so I’m not sure why it finds the font at all for you. The only name the attached font contains is DFYuanChuIn, and that’s what you should use to ensure it works on non-Windows, if this really is the right font file. (This may be a GDI compatibility issue we want to fix in libass: it might be adding the font file name as a secondary name for East Asian fonts? I don’t remember seeing this though.)

I’m not sure why there’s blank horizontal space though! Can anyone explain that?

---

Using \fnDFYuanChuIn, this is what I get with standalone current libass master using Core Text on macOS 10.13 with Japanese-biased language preferences:

Relevant lines from the log:

[ass] libass source: commit: 0.15.1-41-g5733e1c28b92ebbf090bea8f25e252aa40e0b9c4
[ass] Shaper: FriBidi 1.0.10 (SIMPLE) HarfBuzz-ng 2.7.0 (COMPLEX)
[ass] Added subtitle file: '測試字幕.ass' (26 styles, 413 events)
[ass] Loading font file './文鼎新藝體.ttf'
[ass] Loading font file './華康圓體注音.ttc'
[ass] Using font provider coretext
[ass] fontselect: (華康粗圓體(P), 400, 0) -> /System/Library/Fonts/Helvetica.ttc, -1, Helvetica
[ass] Glyph 0x54C8 not found, selecting one more font for (華康粗圓體(P), 400, 0)
[ass] fontselect: (華康粗圓體(P), 400, 0) -> /System/Library/Fonts/ヒラギノ角ゴシック W4.ttc, -1, HiraginoSans-W4
[ass] fontselect: (華康中黑體(P), 700, 0) -> /System/Library/Fonts/Helvetica.ttc, -1, Helvetica-Bold
[ass] fontselect: (華康棒棒體W5, 700, 0) -> /System/Library/Fonts/Helvetica.ttc, -1, Helvetica-Bold
[ass] fontselect: (DFYuanChuIn, 700, 0) -> DFYuanChuIn-Bd-BPMW-BF, 0, DFYuanChuIn-Bd-BPMW-BF
[ass] fontselect: (TT-JTCウインZ5, 700, 0) -> /System/Library/Fonts/Helvetica.ttc, -1, Helvetica-Bold
[ass] Glyph 0x300A not found, selecting one more font for (華康棒棒體W5, 700, 0)
[ass] fontselect: (華康棒棒體W5, 700, 0) -> /System/Library/Fonts/ヒラギノ角ゴシック W7.ttc, -1, HiraginoSans-W7
[ass] Glyph 0x8A3B not found, selecting one more font for (DFYuanChuIn, 700, 0)
[ass] fontselect: (DFYuanChuIn, 700, 0) -> /System/Library/Fonts/ヒラギノ角ゴシック W7.ttc, -1, HiraginoSans-W7
[ass] Glyph 0x53E4 not found, selecting one more font for (TT-JTCウインZ5, 700, 0)
[ass] fontselect: (TT-JTCウインZ5, 700, 0) -> /System/Library/Fonts/ヒラギノ角ゴシック W7.ttc, -1, HiraginoSans-W7
[ass] Glyph 0x4F7F not found, selecting one more font for (華康中黑體(P), 700, 0)
[ass] fontselect: (華康中黑體(P), 700, 0) -> /System/Library/Fonts/ヒラギノ角ゴシック W7.ttc, -1, HiraginoSans-W7

---

Are you sure you attached the correct file? I can't seem to find the line from your screenshot

There are two ASS files in the archive. The lines in the screenshot are in the other one, 測試字幕.ass. (Look for “explain” inside the file.)

retyped 訐 and 天 are missing a stroke on top/beyond the highest horizontal line; not sure if just a font difference or different characters

FYI: different characters indeed. The right ones are 註 (there’s also a stroke on the bottom right—also, this is the very U+8A3B!) and 夫.

[astiob]

Correction.

The font does have that name, and it might contain those characters. But it’s not using a Unicode character map. libass only supports Unicode character maps (and names).

[astiob]

I previously mentioned non-Unicode cmaps in #503 (comment). That’s buried in a comment thread, so let’s instead use this issue to track that.

I briefly tried to look at those Chinese character sets back then, but I couldn’t figure it out: everything on the Web is about the encoding (going from code units to Unicode) but I want the code points. Or so I think. At any rate, this still seems painful (and I really wish FreeType did this, as well as name decoding).

[Masaiki]

this is how it display with xy-VSFilter.
and i tested it with mpv(kikoplayer maybe 0.15.1), and xysubfilter with libass(master and 0.15.1)
the problem occurs with 華康圓體注音 on 測試字幕.ass and 文鼎新藝體 on [FLsnow][umamusumeS2][01][AVC_AAC][WEBRIP][1080p].cht.ass
my bad, i didn't say it clearly