Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions TeXmacs/tests/tmu/0158.en.tmu
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<TMU|<tuple|1.1.0|2026.2.5>>

<style|<tuple|generic|number-europe|preview-ref|british>>

<\body>
<tabular|<tformat|<twith|table-hmode|min>|<twith|table-width|1par>|<cwith|1|-1|1|-1|cell-hyphen|t>|<table|<row|<cell|№>|<cell|Uppercase>|<cell|Lowercase>|<cell|Name>>|<row|<cell|1>|<cell|А>|<cell|а>|<cell|а>>|<row|<cell|2>|<cell|Б>|<cell|б>|<cell|бэ>>|<row|<cell|3>|<cell|В>|<cell|в>|<cell|вэ>>|<row|<cell|4>|<cell|Г>|<cell|г>|<cell|гэ>>|<row|<cell|5>|<cell|Д>|<cell|д>|<cell|дэ>>|<row|<cell|6>|<cell|Е>|<cell|е>|<cell|е>>|<row|<cell|7>|<cell|Ё>|<cell|ё>|<cell|ё>>|<row|<cell|8>|<cell|Ж>|<cell|ж>|<cell|жэ>>|<row|<cell|9>|<cell|З>|<cell|з>|<cell|зэ>>|<row|<cell|10>|<cell|И>|<cell|и>|<cell|и>>|<row|<cell|11>|<cell|Й>|<cell|й>|<cell|и краткое>>|<row|<cell|12>|<cell|К>|<cell|к>|<cell|ка>>|<row|<cell|13>|<cell|Л>|<cell|л>|<cell|эль>>|<row|<cell|14>|<cell|М>|<cell|м>|<cell|эм>>|<row|<cell|15>|<cell|Н>|<cell|н>|<cell|эн>>|<row|<cell|16>|<cell|О>|<cell|о>|<cell|о>>|<row|<cell|17>|<cell|П>|<cell|п>|<cell|пэ>>|<row|<cell|18>|<cell|Р>|<cell|р>|<cell|эр>>|<row|<cell|19>|<cell|С>|<cell|с>|<cell|эс>>|<row|<cell|20>|<cell|Т>|<cell|т>|<cell|тэ>>|<row|<cell|21>|<cell|У>|<cell|у>|<cell|у>>|<row|<cell|22>|<cell|Ф>|<cell|ф>|<cell|эф>>|<row|<cell|23>|<cell|Х>|<cell|х>|<cell|ха>>|<row|<cell|24>|<cell|Ц>|<cell|ц>|<cell|цэ>>|<row|<cell|25>|<cell|Ч>|<cell|ч>|<cell|че>>|<row|<cell|26>|<cell|Ш>|<cell|ш>|<cell|ша>>|<row|<cell|27>|<cell|Щ>|<cell|щ>|<cell|ща>>|<row|<cell|28>|<cell|Ъ>|<cell|ъ>|<cell|твёрдый знак>>|<row|<cell|29>|<cell|Ы>|<cell|ы>|<cell|ы>>|<row|<cell|30>|<cell|Ь>|<cell|ь>|<cell|мягкий знак>>|<row|<cell|31>|<cell|Э>|<cell|э>|<cell|э>>|<row|<cell|32>|<cell|Ю>|<cell|ю>|<cell|ю>>|<row|<cell|33>|<cell|Я>|<cell|я>|<cell|я>>>>>
</body>

<\initial>
<\collection>
<associate|page-screen-margin|false>
<associate|prog-scripts|maxima>
</collection>
</initial>
14 changes: 14 additions & 0 deletions TeXmacs/tests/tmu/0158.zh.tmu
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<TMU|<tuple|1.1.0|2026.2.5>>

<style|<tuple|generic|chinese|table-captions-above|number-europe|preview-ref>>

<\body>
<tabular|<tformat|<twith|table-hmode|min>|<twith|table-width|1par>|<cwith|1|-1|1|-1|cell-hyphen|t>|<table|<row|<cell|№>|<cell|Uppercase>|<cell|Lowercase>|<cell|Name>>|<row|<cell|1>|<cell|А>|<cell|а>|<cell|а>>|<row|<cell|2>|<cell|Б>|<cell|б>|<cell|бэ>>|<row|<cell|3>|<cell|В>|<cell|в>|<cell|вэ>>|<row|<cell|4>|<cell|Г>|<cell|г>|<cell|гэ>>|<row|<cell|5>|<cell|Д>|<cell|д>|<cell|дэ>>|<row|<cell|6>|<cell|Е>|<cell|е>|<cell|е>>|<row|<cell|7>|<cell|Ё>|<cell|ё>|<cell|ё>>|<row|<cell|8>|<cell|Ж>|<cell|ж>|<cell|жэ>>|<row|<cell|9>|<cell|З>|<cell|з>|<cell|зэ>>|<row|<cell|10>|<cell|И>|<cell|и>|<cell|и>>|<row|<cell|11>|<cell|Й>|<cell|й>|<cell|и краткое>>|<row|<cell|12>|<cell|К>|<cell|к>|<cell|ка>>|<row|<cell|13>|<cell|Л>|<cell|л>|<cell|эль>>|<row|<cell|14>|<cell|М>|<cell|м>|<cell|эм>>|<row|<cell|15>|<cell|Н>|<cell|н>|<cell|эн>>|<row|<cell|16>|<cell|О>|<cell|о>|<cell|о>>|<row|<cell|17>|<cell|П>|<cell|п>|<cell|пэ>>|<row|<cell|18>|<cell|Р>|<cell|р>|<cell|эр>>|<row|<cell|19>|<cell|С>|<cell|с>|<cell|эс>>|<row|<cell|20>|<cell|Т>|<cell|т>|<cell|тэ>>|<row|<cell|21>|<cell|У>|<cell|у>|<cell|у>>|<row|<cell|22>|<cell|Ф>|<cell|ф>|<cell|эф>>|<row|<cell|23>|<cell|Х>|<cell|х>|<cell|ха>>|<row|<cell|24>|<cell|Ц>|<cell|ц>|<cell|цэ>>|<row|<cell|25>|<cell|Ч>|<cell|ч>|<cell|че>>|<row|<cell|26>|<cell|Ш>|<cell|ш>|<cell|ша>>|<row|<cell|27>|<cell|Щ>|<cell|щ>|<cell|ща>>|<row|<cell|28>|<cell|Ъ>|<cell|ъ>|<cell|твёрдый знак>>|<row|<cell|29>|<cell|Ы>|<cell|ы>|<cell|ы>>|<row|<cell|30>|<cell|Ь>|<cell|ь>|<cell|мягкий знак>>|<row|<cell|31>|<cell|Э>|<cell|э>|<cell|э>>|<row|<cell|32>|<cell|Ю>|<cell|ю>|<cell|ю>>|<row|<cell|33>|<cell|Я>|<cell|я>|<cell|я>>>>>
</body>

<\initial>
<\collection>
<associate|page-screen-margin|false>
<associate|prog-scripts|maxima>
</collection>
</initial>
71 changes: 71 additions & 0 deletions devel/0158.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# [0158] Cyrillic 字母在 smart_font 中的路由

## 1 相关文档
- [dddd.md](dddd.md) - 任务文档模板

## 2 任务相关的代码文件
- `src/Graphics/Fonts/smart_font.hpp`
- `src/Graphics/Fonts/smart_font.cpp`
- `tests/Graphics/Fonts/smart_font_test.cpp`

## 3 如何测试

### 3.1 确定性测试(单元测试)
```bash
xmake b smart_font_test
xmake r smart_font_test
```

### 3.2 非确定性测试(文档验证)
打开以下两个测试文档,验证所有俄语字母(大写和小写)能够正确渲染:
- `TeXmacs/tests/tmu/0158.en.tmu` — 英文文档(`british` 语言)
- `TeXmacs/tests/tmu/0158.zh.tmu` — 中文文档(`chinese` 语言)

```bash
xmake b stem
xmake r stem
```
在 Mogan 中分别打开上述两个文档,检查表格中的 33 个俄语字母是否正确显示。

## 4 如何提交

提交前执行以下最少步骤:

```bash
xmake b smart_font_test
xmake r smart_font_test
```

## 5 What

针对 Cyrillic Unicode range(U+0400 到 U+04FF)做全局 fallback 处理,使所有文档中的 Cyrillic 字母都能路由到 `default_chinese_font_name()` 对应的字体。

1. 在 `smart_font_rep::resolve(string c)` 中,当主字体不支持且字符属于 Cyrillic range 时,fallback 到 `cyrillic=<default_chinese_font_name()>`
2. 移除了 `smart_font_bis` 中 `sys-chinese` 的特殊 `cyrillic=` 前缀处理(由全局 fallback 统一覆盖)
3. `in_unicode_range` 保持导出并在测试中验证其对 Cyrillic range 的支持
4. 添加单元测试验证:
- `roman` 文档中的 Cyrillic 字符能正确 fallback
- `sys-chinese` 文档中的 Cyrillic 字符也能正确路由

## 6 Why

Cyrillic 字母之前没有全局的路由机制,在某些非中文文档中无法正确渲染。通过在 `resolve` 中添加全局 fallback,所有文档中的 Cyrillic 字符都能统一路由到指定字体(目前使用 `default_chinese_font_name()`,后续可独立配置)。

## 7 How

修改 `smart_font_rep::resolve(string c)`,在主字体检查之后、emoji 处理之后添加 Cyrillic fallback 逻辑:

```cpp
// Fallback Cyrillic characters to default Chinese font
if (range == "cyrillic") {
string chinese_name= default_chinese_font_name ();
if (chinese_name != "roman") {
for (int attempt= 1; attempt <= FONT_ATTEMPTS; attempt++) {
int nr= resolve (c, "cyrillic=" * chinese_name, attempt);
if (nr >= 0) return nr;
}
}
}
```

同时恢复 `sys-chinese` 的展开逻辑为:`cjk=<name>,roman`
13 changes: 12 additions & 1 deletion src/Graphics/Fonts/smart_font.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -397,7 +397,7 @@ is_cjk_punct (string_u8 c) {
return set->contains (c);
}

static bool
bool
in_unicode_range (string c, string range) {
string uc= strict_cork_to_utf8 (c);
if (N (uc) == 0) return false;
Expand Down Expand Up @@ -1182,6 +1182,17 @@ smart_font_rep::resolve (string c) {
}
}

// Fallback Cyrillic characters to default Chinese font
if (range == "cyrillic") {
string chinese_name= default_chinese_font_name ();
if (chinese_name != "roman") {
for (int attempt= 1; attempt <= FONT_ATTEMPTS; attempt++) {
int nr= resolve (c, "cyrillic=" * chinese_name, attempt);
if (nr >= 0) return nr;
}
}
}

if (math_kind != 0) {
string upc= substitute_upright (c);
if (upc != "" && fn[SUBFONT_MAIN]->supports (upc)) {
Expand Down
2 changes: 2 additions & 0 deletions src/Graphics/Fonts/smart_font.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -159,4 +159,6 @@ struct smart_font_rep : font_rep {
SI get_wide_correction (string s, int mode);
};

bool in_unicode_range (string c, string range);

#endif
45 changes: 45 additions & 0 deletions tests/Graphics/Fonts/smart_font_test.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ private slots:
void test_cursor_position_iii ();
void test_performance ();
void test_math_performance ();
void test_in_unicode_range_cyrillic ();
void test_roman_cyrillic_fallback ();
void test_sys_chinese_cyrillic ();
};

void
Expand Down Expand Up @@ -188,5 +191,47 @@ TestSmartFont::test_math_performance () {
fn->get_extents (math_text, ex);
}

void
TestSmartFont::test_in_unicode_range_cyrillic () {
// Cyrillic 字符不应该被当作 CJK
QVERIFY (!in_unicode_range ("<#400>", "cjk"));
QVERIFY (!in_unicode_range ("<#4FF>", "cjk"));
// Cyrillic 字符应该属于 cyrillic range
QVERIFY (in_unicode_range ("<#400>", "cyrillic"));
QVERIFY (in_unicode_range ("<#4FF>", "cyrillic"));
// Latin 字符不应该属于 cyrillic range
QVERIFY (!in_unicode_range ("a", "cyrillic"));
}

void
TestSmartFont::test_roman_cyrillic_fallback () {
// roman 文档中的 Cyrillic 字符应该 fallback 到 default_chinese_font_name
font fn= smart_font ("roman", "rm", "medium", "right", 10, 600);
smart_font_rep* fn_rep= (smart_font_rep*) fn.rep;
string c = utf8_to_cork ("А");
int nr = fn_rep->resolve (c);
QVERIFY (nr >= 0);
string chinese_name= default_chinese_font_name ();
if (chinese_name != "roman") {
// 确认没有 fallback 到 roman 的 ecrm 字体
QVERIFY (!occurs ("ecrm", fn_rep->fn[nr]->res_name));
}
}

void
TestSmartFont::test_sys_chinese_cyrillic () {
// sys-chinese 文档中的 Cyrillic 字符应该正确路由
font fn= smart_font ("sys-chinese", "rm", "medium", "right", 10, 600);
smart_font_rep* fn_rep= (smart_font_rep*) fn.rep;
string c = utf8_to_cork ("А");
int nr = fn_rep->resolve (c);
QVERIFY (nr >= 0);
string chinese_name= default_chinese_font_name ();
if (chinese_name != "roman") {
// 确认没有 fallback 到 roman 的 ecrm 字体
QVERIFY (!occurs ("ecrm", fn_rep->fn[nr]->res_name));
}
}

QTEST_MAIN (TestSmartFont)
#include "smart_font_test.moc"
Loading