Noto CJK: unnecessary mapping of 0000..001F

[MahaHas]

Characters (U+0000 to U+001F) have length of 1 and map to space.

[behdad]

I agree this is not desired in OpenType fonts. cc @kenlunde

[kenlunde]

I am curious as to what problem or problems are caused by the presence of these mappings. Android may not depend on them, but in our experience, other platforms may depend on the presence of these mappings. If no problems are caused, I would argue that it is safer to leave them in.

All of Adobe's CJK fonts include such mappings, which are present in the UTF-32 CMap resources, and which are specified in the following CMap resource snippet:

1 beginnotdefrange
<00000000> <0000001f> 1
endnotdefrange

In addition, thousands of non-Adobe CJK fonts, particularly Japanese ones, include the same mappings for this range by virtue of using the UTF-32 CMap resources that we make available.

[behdad]

I think the thinking was that we prefer users to see a tofu if there's an unexpected control character in text. If there's any font on the system that claims to support these (Noto Sans CJK in this case), it gets used by the fallback process, and as a result character is rendered invisible.

[kenlunde]

@behdad: Given what you wrote, if a font is intended to be used stand-alone, it is okay—and arguably safer—to include the mappings for U+0000 through U+001F, but if a font is intended to be used as part of a font fallback system, these mappings should be removed. Is that an accurate statement?

[behdad]

@kenlunde I think it's accurate to say for standalone use, it's harmless; I wouldn't say safer. I agree with the rest of your assessment.

[roozbehp]

@kenlunde, please also note that different advances are recommend for some of these. From https://www.microsoft.com/typography/otspec/recom.htm:

"Additional recommendations:

• Glyph 1 should have no contours and zero advance width.
• Character U+000D (carriage return) should map to a glyph with a positive advance width.
• Characters U+0001-001F (misc ASCII control codes) and U+007F (delete) should be mapped to glyph 0 (with some exceptions noted below).
• Characters U+0000 (null), U+0008 (backspace) and U+001D (group separator) should map to glyph 1.
• Characters U+0009 (horizontal tabulation), U+0020 (space) and U+00A0 (no-break space) should map to a glyph with no contours and a positive advance width.
• Characters U+0009 and U+0020 should map to a glyph with the same width."

[kenlunde]

@roozbehp: We thoroughly investigated those particular recommendations in the OpenType Specification about three years ago, and although they are meant to apply only to TrueType-based, not CFF-based, OpenType fonts, we didn't find any TrueType consumers that actually depended on such settings or conditions. In other words, those recommendations are antiquated and can effectively be ignored for modern environments.

[davelab6]

@kenlunde cool! has that been reported to the ot spec maintainers? :)

[kenlunde]

@davelab6: Not yet, but your question is a good nudge for me to do so. ☺

[davelab6]

🎱

[kenlunde]

Done.

[dougfelt]

So, since there's been no further comment on this, Adobe is providing these fonts, and Ken prefers the current mapping, it appears that the status quo wins and the characters remain mapped. @behdad please reopen if you think the mapping should be changed.