Skip to content
This repository has been archived by the owner. It is now read-only.

Support for non-latin font fallbacks #118

Closed
andersnoren opened this issue Sep 8, 2019 · 67 comments
Closed

Support for non-latin font fallbacks #118

andersnoren opened this issue Sep 8, 2019 · 67 comments
Assignees

Comments

@andersnoren
Copy link
Contributor

@andersnoren andersnoren commented Sep 8, 2019

Following in the tradition of this issue for Twenty Nineteen and this one for Twenty Seventeen, we’d love to get some help from all of the polyglots and i18n folks out there to make sure that Twenty Twenty has awesome support for non-latin alphabets. To start with, we need help with finding good non-latin fallbacks. Later on, we'll need lots and lots of help with testing.

Kudos to @nielslange for this handy table from last year, which I'm stealing outright. This is the list of alphabets supported in twentyseventeen and twentynineteen. I'll update with font stacks, testers and PRs for Twenty Twenty as we go.

Language ISO Codes Fonts Tester Status
Arabic ar, ary, azb, ckb, fa-IR, haz, ps Tahoma, Arial, sans_serif; @elazzabi
Chinese Simplified (China) zh_CN "PingFang SC", "Helvetica Neue", "Microsoft YaHei New", "STHeiti Light", sans-serif @yuliyang
Chinese Traditional (Taiwan) zh_TW "PingFang TC", "Helvetica Neue", "Microsoft YaHei New", "STHeiti Light", sans-serif; @yuliyang
Chinese (Hong Kong) zh_HK "PingFang HK", "Helvetica Neue", "Microsoft YaHei New", "STHeiti Light", sans-serif; @yuliyang
Cyrillic bel, bg-BG, kk, mk_MK, mn, ru-RU, sah, sr_RS, tt_RU, uk "Helvetica Neue", Helvetica, "Segoe UI", Arial, sans_serif;
Devanagari bn_BD, hi_IN, mr, ne_NP Arial, sans_serif;
Greek el "Helvetica Neue", Helvetica, Arial, sans_serif;
Gujarati gu Arial, sans_serif;
Hebrew he-IL "Arial Hebrew", Arial, sans_serif;
Japanese ja sans-serif; @nukaga, @miminari
Korean ko-KR "Apple SD Gothic Neo", "Malgun Gothic", "Nanum Gothic", Dotum, sans-serif; @donpark
Thai th "Sukhumvit Set", "Helvetica Neue", Helvetica, Arial, sans-serif;
Vietnamese vi "Libre Franklin", sans-serif; @philiparthurmoore, @dinhtungdu
@kjellr
Copy link

@kjellr kjellr commented Sep 9, 2019

A quick note from an implementation perspective: If possible, it would be great to get these loaded as individual stylesheets, and call them only if they're needed. In Twenty Nineteen, each of these additional styles are included in the main stylesheet by default, and add quite a bit of filesize.

Loading

@andersnoren
Copy link
Contributor Author

@andersnoren andersnoren commented Sep 9, 2019

@kjellr Thanks for the note! Yes, finding a good way to avoid that would be great. Getting a discussion going about the performance impact of using alphabet specific stylesheets and only loading the one that is needed would be awesome.

Loading

@kjellr
Copy link

@kjellr kjellr commented Sep 9, 2019

This was touched upon in Slack today, but just making note here: The most difficult aspect of this is separate from the development hurdle: it's finding appropriate font substitutes that carry the same feel as the primary latin fonts. Ideally designers familiar with each non-latin language would help making these fallback stack decisions, since many of us aren't familiar designing in languages other than our own.

Loading

@nukaga
Copy link

@nukaga nukaga commented Sep 10, 2019

For Japanese, I think the title (h1-h6) choose to be Noto Sans JP and the text use be "Hiragino Kaku Gothic ProN" or "sans_serif" only.
Web fonts in Japanese fonts are heavy and are not suitable for setting all posts. And since the environment of Japanese fonts is various, I thought it would be good to make use of the fonts of each device.

Loading

@westonruter
Copy link
Member

@westonruter westonruter commented Sep 11, 2019

it would be great to get these loaded as individual stylesheets, and call them only if they're needed.

This it would be great to use wp_add_inline_style() for this. The mappings of languages to fonts can be stored in PHP and then it can call wp_add_inline_style() to just output what is required. This would avoid the frontend overhead of another render-blocking stylesheet.

Loading

@nielslange
Copy link
Collaborator

@nielslange nielslange commented Sep 11, 2019

@andersnoren I've collected the following potential non-latin fallback fonts on Google Fonts. Would you mind going through the list to take a look which fonts would be suitable for:

  1. Titles
  2. Other texts

Potential fonts

ℹ️ Noto Sans might be a good fit as it supports most of the locals listed above.

Loading

@andersnoren
Copy link
Contributor Author

@andersnoren andersnoren commented Sep 11, 2019

@nukaga Thank you for the reply, and sorry that I'm late in getting back to you. I've added 'Noto Sans JP', 'Hiragino Kaku Gothic ProN', sans-serif to the Japanese row.

@nielslange Noto Sans looks like it would substitute well for Inter in terms of x-height and overall letterform width, so that seems like a strong candidate at least. I don't feel comfortable making suggestions for most of the specific alphabets listed though, since I have no idea if the fonts I would propose would be at all readable.

Loading

@nielslange
Copy link
Collaborator

@nielslange nielslange commented Sep 11, 2019

@andersnoren Thanks for your reply! In that case, I start by finding folks that can test this issue and can suggest readable fonts. How does that sound to you?

Loading

@andersnoren
Copy link
Contributor Author

@andersnoren andersnoren commented Sep 11, 2019

@nielslange Sounds good!

Loading

@dinhtungdu
Copy link

@dinhtungdu dinhtungdu commented Sep 17, 2019

FYI, here is the list of all characters have diacritical marks in Vietnamese. (We can use it for testing purpose)

"a" => "á|à|ạ|ả|ã|ă|ắ|ằ|ặ|ẳ|ẵ|â|ấ|ầ|ậ|ẩ|ẫ|Á|À|Ạ|Ả|Ã|Ă|Ắ|Ằ|Ặ|Ẳ|Ẵ|Â|Ấ|Ầ|Ậ|Ẩ|Ẫ",
"o" => "ó|ò|ọ|ỏ|õ|ô|ố|ồ|ộ|ổ|ỗ|ơ|ớ|ờ|ợ|ở|ỡ|Ó|Ò|Ọ|Ỏ|Õ|Ô|Ố|Ồ|Ộ|Ổ|Ỗ|Ơ|Ớ|Ờ|Ợ|Ở|Ỡ",
"e" => "é|è|ẹ|ẻ|ẽ|ê|ế|ề|ệ|ể|ễ|É|È|Ẹ|Ẻ|Ẽ|Ê|Ế|Ề|Ệ|Ể|Ễ",
"u" => "ú|ù|ụ|ủ|ũ|ư|ứ|ừ|ự|ử|ữ|Ú|Ù|Ụ|Ủ|Ũ|Ư|Ứ|Ừ|Ự|Ử|Ữ",
"i" => "í|ì|ị|ỉ|ĩ|Í|Ì|Ị|Ỉ|Ĩ",
"y" => "ý|ỳ|ỵ|ỷ|ỹ|Ý|Ỳ|Ỵ|Ỷ|Ỹ",
"d" => "đ|Đ",

Loading

@nielslange
Copy link
Collaborator

@nielslange nielslange commented Sep 17, 2019

Potential testers:

  • Arabic: @elazzabi and @youknowriad Would you mind assisting me in finding and testing fonts for Arabic?
  • Chinese: @yuliyang Would you mind assisting me in finding and testing fonts for Chinese?
  • Cyrillic: @slaFFik Would you mind assisting me in finding and testing fonts for Cyrillic?
  • Devanagari: TBD
  • Greek: @chriskmnds Would you mind assisting me in finding and testing fonts for Greek?
  • Gujarati: @galakhyati
  • Hebrew: @danieldanilov and @yoavf Would you mind assisting me in finding and testing fonts for Hebrew?
  • Japanese: @nukaga and @miminari Would you mind assisting me in finding and testing fonts for Japanese?
  • Korean: @donpark: Would you mind assisting me in finding and testing fonts for Korean?
  • Thai: @imnok Would you mind assisting me in finding and testing fonts for Thai?
  • Vietnamese: @dinhtungdu and @philiparthurmoore: Would you mind assisting me in finding and testing fonts for Vietnamese?

Loading

@dinhtungdu
Copy link

@dinhtungdu dinhtungdu commented Sep 17, 2019

@nielslange Yes, sure :)

Loading

@nielslange
Copy link
Collaborator

@nielslange nielslange commented Sep 17, 2019

Thanks, @dinhtungdu! 😀

Loading

nielslange added a commit to nielslange/twentytwenty that referenced this issue Sep 17, 2019
nielslange added a commit to nielslange/twentytwenty that referenced this issue Sep 17, 2019
@philiparthurmoore
Copy link

@philiparthurmoore philiparthurmoore commented Sep 18, 2019

Vietnamese: @dinhtungdu and @philiparthurmoore: Would you mind assisting me in finding and testing fonts for Vietnamese?

@dinhtungdu Let me know if you need me to test also - happy to help. :)

Loading

@yuliyang
Copy link

@yuliyang yuliyang commented Sep 18, 2019

Chinese Simplified (China) - Noto Sans SC
https://fonts.google.com/specimen/Noto+Sans+SC

Chinese Traditional (Taiwan) - Noto Sans TC
https://fonts.google.com/specimen/Noto+Sans+TC

Chinese (Hong Kong) - Noto Sans HK
https://fonts.google.com/specimen/Noto+Sans+HK

I can help with testing. 😄

Loading

@elazzabi
Copy link

@elazzabi elazzabi commented Sep 18, 2019

Happy to help test Arabic <3

Loading

@nukaga
Copy link

@nukaga nukaga commented Sep 18, 2019

I will be happy to help you!

Loading

@nielslange
Copy link
Collaborator

@nielslange nielslange commented Sep 18, 2019

Thanks, @philiparthurmoore, @yuliyang, @elazzabi and @nukaga for helping out!

@philiparthurmoore Would https://fonts.google.com/specimen/Noto+Sans work for Vietnamese?

@yuliyang Thanks for suggesting the fonts for Chinese!

@elazzabi Would https://fonts.google.com/specimen/Cairo work for Arabic?

@nukaga Would https://fonts.google.com/specimen/Noto+Sans+JP work for Japanese?

Loading

nielslange added a commit to nielslange/twentytwenty that referenced this issue Sep 18, 2019
@nielslange
Copy link
Collaborator

@nielslange nielslange commented Sep 18, 2019

@yuliyang Based on #118 (comment) I created the PR nielslange@e95fce6. Feel free to start testing and let me know if something needs to be adjusted. Also, is it common to format text bold and italic in Chinese or is that rather uncommon?

Loading

@donpark
Copy link

@donpark donpark commented Sep 18, 2019

@andersnoren it'd be nice to have some basic info on what "finding and testing fonts" means and how one should go about doing it.

  1. "Twenty Twenty" theme is not listed and WP.org theme pages under https://wordpress.org/themes/twentytwenty/ 404s so I had to git clone, zip and upload to an Atomic test site to see the theme.
  2. Should I be switching site language to target language or set it to some known problematic language? IOW, are we testing site languages using fonts designed for target language or just target language and font combo which is a much smaller search space?
  3. Should I be testing a font by overriding site font via * {font-family:"Foobar"} Custom CSS to simulate fallback?
  4. What are some of the most common issues in non-latin font fallbacks that I should be looking out for?
  5. Are there specific pages/posts/widgets/blocks/etc that cause non-latin fallback font problems?
  6. Do we have a single test page with all problematic parts and controls to switch site language and fonts?

I'd like to help but, as a developer, I think I'm missing some really basic stuff here.

Loading

@yuliyang
Copy link

@yuliyang yuliyang commented Sep 19, 2019

@yuliyang Based on #118 (comment) I created the PR nielslange@e95fce6. Feel free to start testing and let me know if something needs to be adjusted. Also, is it common to format text bold and italic in Chinese or is that rather uncommon?

Chinese characters do not have italic. It's very uncommon unless the font itself has italic character designed. I double-checked Noto Sans for Chinese and none of them has italic. So ideally, no italic, please. :)

Loading

@andersnoren
Copy link
Contributor Author

@andersnoren andersnoren commented Sep 19, 2019

@yuliyang Thanks, updated! Do you have any suggestions on suitable fallbacks for Noto Sans for the Chinese fonts? The following were used for Twenty Nineteen:

'PingFang TC', 'Helvetica Neue', Helvetica, STHeitiTC-Light, Arial, sans-serif;

@donpark Sorry for the confusion. At this point, we're mostly looking for suggestions on fonts we can include for each alphabet. None of them are currently included in the theme. Once we have pull requests created for including the fonts for the respective alphabets, we would appreciate any help we can get with testing the pull requests by setting the site language to the language the PR includes fonts and styles for.

I'm afraid I'm not the best person to speak to point 4 and 5 (common issues and font problems). I wanted to get the ball rolling on non-latin fallbacks early, but I don't have much experience in this area personally.

Loading

@miminari
Copy link
Member

@miminari miminari commented Sep 19, 2019

@nielslange I’d love to do it.

Loading

@nielslange
Copy link
Collaborator

@nielslange nielslange commented Sep 19, 2019

Chinese characters do not have italic. It's very uncommon unless the font itself has italic character designed. I double-checked Noto Sans for Chinese and none of them has italic. So ideally, no italic, please. :)

Thanks for your reply, @yuliyang!

@nielslange I’d love to do it.

Great to hear that, @miminari! In #118 (comment) I've already asked @nukaga if https://fonts.google.com/specimen/Noto+Sans+JP would work for Japanese? Do you think that font will work?

Loading

@nielslange
Copy link
Collaborator

@nielslange nielslange commented Sep 29, 2019

@elazzabi, @yuliyang, @nukaga, @miminari, @donpark, @philiparthurmoore, @dinhtungdu: Based on #542 (comment) and #118 (comment) we're not using Google Fonts but only system fonts as non-latin fallback fonts. I'm implemented this feature.

Would you mind running some tests is the implementation works as expected? You can download the theme, which includes this feature, via https://github.com/nielslange/twentytwenty/archive/try/%23118.zip.

Please also let me know if you have any questions regarding the fonts or the testing or if you have any other questions or suggestions. Thank you very much in advance for offering to assist with testing this PR! 😀

Loading

nielslange added a commit to nielslange/twentytwenty that referenced this issue Sep 29, 2019
nielslange added a commit to nielslange/twentytwenty that referenced this issue Sep 29, 2019
nielslange added a commit to nielslange/twentytwenty that referenced this issue Sep 29, 2019
nielslange added a commit to nielslange/twentytwenty that referenced this issue Sep 29, 2019
nielslange added a commit to nielslange/twentytwenty that referenced this issue Sep 29, 2019
nielslange added a commit to nielslange/twentytwenty that referenced this issue Sep 29, 2019
nielslange added a commit to nielslange/twentytwenty that referenced this issue Sep 30, 2019
@elazzabi
Copy link

@elazzabi elazzabi commented Sep 30, 2019

@nielslange I added twentytwenty to a test site. In general, title and content looks good. Here are some things to fix however:

Screen Shot 2019-09-30 at 7 07 40 PM

The date should be transformed from: سبتمبر 30, 2019 to YEAR MONTH DAY, no commas between them (I can't write it as GH is giving me a hard time with Arabic :/)

Screen Shot 2019-09-30 at 7 07 57 PM

Screen Shot 2019-09-30 at 7 07 02 PM

Screen Shot 2019-09-30 at 7 05 12 PM

Screen Shot 2019-09-30 at 7 16 05 PM

Search, sorting, notification settings and 'To the top' aren't translated.

Loading

@joyously
Copy link

@joyously joyously commented Sep 30, 2019

The date should be transformed from: سبتمبر 30, 2019 to YEAR MONTH DAY,

Is that how you have your test site's date format set?

Search, sorting, notification settings and 'To the top' aren't translated.

The theme isn't translated yet, but those Notify options are from a plugin.

Loading

@nielslange
Copy link
Collaborator

@nielslange nielslange commented Oct 1, 2019

@elazzabi Thanks for testing! 🤗 Similar to what @joyously already mentioned, I also assume that the date format might come from your site settings> Can you double-check this? And the strings that you mentioned have indeed not been translated yet. This should happen within the next couple of days/weeks.

Loading

@elazzabi
Copy link

@elazzabi elazzabi commented Oct 1, 2019

Is that how you have your test site's date format set?

Oh forgot about that. All good then 👍

Loading

@nukaga
Copy link

@nukaga nukaga commented Oct 1, 2019

Hello. I found two problems during the test.
・ The page title is too large in Japanese.
・ In our language, the align centered page title is strange.

  1. Change the size of characters according to language
  2. For Japanese, left justify
    Is this possible?

スクリーンショット 2019-09-25 15 23 02

Loading

@nielslange
Copy link
Collaborator

@nielslange nielslange commented Oct 1, 2019

Thanks for testing this PR, @nukaga. If I understand you correctly, the page title should be left-aligned and the font size should be decreased. All other elements, e.g. tagline, meta information, body text, etc. remain untouched, correct?

Loading

@nukaga
Copy link

@nukaga nukaga commented Oct 1, 2019

@nielslange Thank you for understanding!

the page title should be left-aligned and the font size should be decreased

Yes. That's right.
Is adjustment possible?

For other elements, I will check.

Loading

@donpark
Copy link

@donpark donpark commented Oct 2, 2019

It'd be really nice to have something like headstart that populates a site with content that pushes non-latin support in themes to reasonable extremes for testing purpose. Hmm. troublestart or hellstart? 😀

Loading

@nielslange
Copy link
Collaborator

@nielslange nielslange commented Oct 2, 2019

@donpark That's a great idea. The theme review team has a theme unit test, in English, which covers pretty much all cases. It might be good to have that theme unit test file also available in other languages for testing purposes.

cc: @carolinan, @pattonwebz, @ianbelanger79

Loading

@joyously
Copy link

@joyously joyously commented Oct 2, 2019

There's actually an issue on the unit test about international content.

Loading

nielslange added a commit to nielslange/twentytwenty that referenced this issue Oct 4, 2019
nielslange added a commit to nielslange/twentytwenty that referenced this issue Oct 4, 2019
nielslange added a commit to nielslange/twentytwenty that referenced this issue Oct 5, 2019
@yuliyang
Copy link

@yuliyang yuliyang commented Oct 7, 2019

Same with Japanese, the page title is probably too large for all Chinese (simplified and traditional).
Other than that, everything looks good!

Tested: Chinese Simplified (China), Chinese Traditional (Taiwan), Chinese (Hong Kong)

Loading

@nukaga
Copy link

@nukaga nukaga commented Oct 12, 2019

We did a Japanese test today. ( with @atachibana @miminari @shinyabw )

First, please use only the title and text sans-serif.
The situation of Japanese devices varies, and some people use universal fonts. We thought that sans-serif only designations were the easiest to read.

title
・ The font size is large.
・ Character feed is clogged
・ Center alignment is strange

Text
・ We want the font to be sans-serif
・ Narrow line spacing

So we suggest the following.

title:

font-family: sans-serif;
font-size: 6.4em;
letter-spacing: normal;

p:

font-family: sans-serif;
line-height 1.6;

Loading

ianbelanger79 added a commit that referenced this issue Oct 28, 2019
* Block editor font sizes: Updated ”Regular” to match the front-end, and bumped ”Small”

* Block Editor Styles: Update Block: Columns with heading margins to match front-end

* Add non-latin language handling, and enqueue it for block styles, classic editor styles and front-end styles

* Replace sans_serif with sans-serif

Fixes #118
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.