Big bn::sprite_font exceeds compiler operation count limit for duplication check

[laqieer]

butano/butano/include/bn_sprite_font.h

Line 121 in 0e5c422

BN_ASSERT(! _duplicated_utf8_characters(), "There's duplicated UTF-8 characters");

It exceeds compiler operation count limit here for big bn::sprite_font with thousands of utf-8 characters.

butano/butano/include/bn_sprite_font.h

Lines 200 to 213 in 0e5c422

	for(int i = 0, size = _utf8_characters_ref.size(); i < size; ++i)
	{
	const string_view& a = utf8_characters_data[i];
	for(int j = i + 1; j < size; ++j)
	{
	const string_view& b = utf8_characters_data[j];
	if(a == b)
	{
	return true;
	}
	}
	}

The double for loop here may need optimization to reduce operation counts.

Examples:

Chinese font with 3500 chracters
Japanese font with 1945 characters

Fonts:

c1b8906

Errors:

main.cpp
In file included from C:/Users/laqie/Projects/gba-dev-best-practice/source/common-text/src/main.cpp:7:
c:\users\laqie\projects\butano\common\include\common_cn_variable_16x16_sprite_font.h:26:52:   in 'constexpr' expansion of 'bn::sprite_font(bn::sprite_items::common_cn_variable_16x16, bn::span<const bn::string_view>(common::cn::variable_16x16_sprite_font_utf8_characters), bn::span<const signed char>(common::cn::variable_16x16_sprite_font_character_widths))'
C:/Users/laqie/Projects/butano/butano/include/bn_sprite_font.h:85:71:   in 'constexpr' expansion of '((bn::sprite_font*)this)->bn::sprite_font::sprite_font((* & item), (* & utf8_characters_ref), (* & character_widths_ref), 0)'
C:/Users/laqie/Projects/butano/butano/include/bn_sprite_font.h:121:9:   in 'constexpr' expansion of '((bn::sprite_font*)this)->bn::sprite_font::_duplicated_utf8_characters()'
C:/Users/laqie/Projects/butano/butano/include/bn_sprite_font.h:209:22:   in 'constexpr' expansion of 'bn::operator==((* & a), (* & b))'
C:/Users/laqie/Projects/butano/butano/include/bn_string_view.h:425:28:   in 'constexpr' expansion of '(& b)->bn::string_view::size()'
c:\users\laqie\projects\butano\common\include\common_cn_variable_16x16_sprite_font.h:26:52: error: 'constexpr' evaluation operation count exceeds limit of 33554432 (use '-fconstexpr-ops-limit=' to increase the limit)
   26 |         variable_16x16_sprite_font_character_widths);
      |                                                    ^
In file included from C:/Users/laqie/Projects/butano/butano/include/bn_sprite_font.h:17,
                 from C:/Users/laqie/Projects/butano/butano/include/bn_sprite_text_generator.h:19,
                 from C:/Users/laqie/Projects/gba-dev-best-practice/source/common-text/src/main.cpp:4:
c:\users\laqie\projects\butano\common\include\common_jp_variable_16x16_sprite_font.h:26:52:   in 'constexpr' expansion of 'bn::sprite_font(bn::sprite_items::common_jp_variable_16x16, bn::span<const bn::string_view>(common::jp::variable_16x16_sprite_font_utf8_characters), bn::span<const signed char>(common::jp::variable_16x16_sprite_font_character_widths))'
C:/Users/laqie/Projects/butano/butano/include/bn_sprite_font.h:85:71:   in 'constexpr' expansion of '((bn::sprite_font*)this)->bn::sprite_font::sprite_font((* & item), (* & utf8_characters_ref), (* & character_widths_ref), 0)'
C:/Users/laqie/Projects/butano/butano/include/bn_sprite_font.h:121:9:   in 'constexpr' expansion of '((bn::sprite_font*)this)->bn::sprite_font::_duplicated_utf8_characters()'
C:/Users/laqie/Projects/butano/butano/include/bn_sprite_font.h:209:22:   in 'constexpr' expansion of 'bn::operator==((* & a), (* & b))'
C:/Users/laqie/Projects/butano/butano/include/bn_string_view.h:430:23: error: 'constexpr' evaluation operation count exceeds limit of 33554432 (use '-fconstexpr-ops-limit=' to increase the limit)
  430 |         const_pointer a_data = a.data();
      |                       ^~~~~~
make[2]: *** [/opt/devkitpro/devkitARM/base_rules:80: main.o] Error 1
make[1]: *** [/home/laqie/Projects/butano/butano/butano.mak:165: build] Error 2
make: *** [/home/laqie/Projects/butano/butano/butano.mak:160: all] Error 2

Note: This issue is opened according to #18 (comment).

[GValiente]

I'm working on a new font system that stores the UTF-8 characters map in ROM without character limits.

I have tested this new font system with a korean font with more than 2400 UTF-8 characters and it seems to work fine, but it would be great if I could test your fonts too.

In the examples you have posted I can't find the 3 files needed to create a font (*.bmp, *.json and *.h).

Could you link them?

Thanks.

[copyrat90]

I think op put them on the Fonts: commit just below the Examples:

Fonts:

c1b8906

[GValiente]

I'm blind :)

Thanks.

[laqieer]

I'm working on a new font system that stores the UTF-8 characters map in ROM without character limits.

That's great. I also have the same idea when I find that they are all loaded to WRAM at once, which is inefficient and consumes much time and space.

I think op put them on the Fonts: commit just below the Examples:

Yes, you are right.

[GValiente]

I think they look right :)

[GValiente]

Sadly, I have to break compatibility with old bn::sprite_font API, but I guess it is worth it.

I'll push a Butano 9.0.0 prerelease soon with the new API.

These header files have been updated to it: sprite_font.zip

[laqieer]

Could you please push it to a branch first so that I can merge it to mine directly? Thanks a lot.

[GValiente]

Here's your branch: https://github.com/GValiente/butano/tree/utf8_characters_map

[laqieer]

I find the changes on that branch are already on master, so I can merge changes from master directly instead.

[laqieer]

Test with 25000+ characters in 46055a0 and it works fine.

user_variable_32x32.fnt
common_sc_variable_16x16.fnt
common_jp_variable_16x16.fnt
common_kr_variable_16x16.fnt
common_tc_variable_16x16.fnt
common_hc_variable_16x16.fnt
    fonts/user_variable_32x32.fnt font header written in build/user_variable_32x32_sprite_font.h (character number: 22)
    ../../common/fonts/SimplifiedChinese/common_sc_variable_16x16.fnt font header written in build/common_sc_variable_16x16_sprite_font.h (character number: 3693)
    ../../common/fonts/Japanese/common_jp_variable_16x16.fnt font header written in build/common_jp_variable_16x16_sprite_font.h (character number: 2708)
    ../../common/fonts/Korean/common_kr_variable_16x16.fnt font header written in build/common_kr_variable_16x16_sprite_font.h (character number: 2449)
    ../../common/fonts/TraditionalChinese/common_tc_variable_16x16.fnt font header written in build/common_tc_variable_16x16_sprite_font.h (character number: 11250)
    ../../common/fonts/TraditionalChineseHK/common_hc_variable_16x16.fnt font header written in build/common_hc_variable_16x16_sprite_font.h (character number: 4914)
    Processed character number: 25036