Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for complex font rendering (Chinese, Arabic, ...) #1589

Open
adrianmay opened this issue Mar 30, 2014 · 29 comments
Open

Support for complex font rendering (Chinese, Arabic, ...) #1589

adrianmay opened this issue Mar 30, 2014 · 29 comments
Labels

Comments

@adrianmay
Copy link

@adrianmay adrianmay commented Mar 30, 2014

For languages like Chinese, it's not practical for an app to deliver its own font. Therefore there's a desire for libgdx to be able to use fonts delivered by the platform.

Some discussion already occurred here:
Roll-your-own Chinese font?
and here:
Development Forum: Supporting Asian fonts in BitmapFont

Earlier attempts at the same challenge have run into difficulties but there doesn't seem to be an exhaustive list of those difficulties. Perhaps that should be compiled here before attempting to develop a new plan.

@badlogic badlogic changed the title Use System Fonts Support for complex font rendering (Chinese, Arabic, ...) Mar 30, 2014
@badlogic

This comment has been minimized.

Copy link
Member

@badlogic badlogic commented Mar 30, 2014

Took the freedom to modify your issues title,

I believe we could create an extension that uses system facilities to render strings to textures. I'm not sure what an interface for something like that would look like.

Another alternative would be a LRU glyph cache with the same interface as BitmapFont. Open for other suggestions (and not saying i'm gonna implement this!)

@adrianmay

This comment has been minimized.

Copy link
Author

@adrianmay adrianmay commented Mar 30, 2014

If it's only about making textures, we could give Gdx.graphics a getSystemFont method taking the font characteristics and returning a Font which had a method like "Pixmap write(String s)".

But perhaps the whole edifice of widgets and dialogs ought to work with this, so how about introducing a Font interface or base class and then deriving BitmapFont and SystemFont from it. That would require changing all the BitmapFont references into Font references, but there's only about 600 of those ;-)

@jrenner

This comment has been minimized.

Copy link
Member

@jrenner jrenner commented Mar 31, 2014

how would the glyphs from the LRU cache be stored? Individual textures? in an atlas?

Is there anyway to do this that doesn't bind textures like crazy? I've been brainstorming but I haven't come up with anything.

Could we somehow use freetype font generator to dynamically add glyphs to a pre-rendered font, possibly with an internal texture atlas. for example:
myfont.png contains 100 chars I know I will use for buttons.
Users inputs n chars not in the original 100.
myfont2.png is created and generates those n chars into the first spots of the png, keeping track of position for writing next dynamically generated char.

It is just brainstorming, I don't know if it is a good idea.

@badlogic

This comment has been minimized.

Copy link
Member

@badlogic badlogic commented May 8, 2014

Your suggestion comes close to the system i wrote at work a few years ago. It's a big mess, especially if it has to be general purpose.

We should make this a higher priority imo, i want to be able to support the asian and arabic markets. For what it's worth, Unity has zero support for asian and arabic scripts...

Ultimately i think we need to expose a simple method that takes a string in utf and spits out a layouted texture/pixmap. Any glyph caching will fail on mobile due to layouting, The layouting and rendering can cache glyphs though via LRU on the heap. It's still going to be slow to generate textures for new strings.

based on this functionality, we can implement a special label in scene2d. for long blocks of text we'll need to be able to split the text up into multiple textures, which is going to be a pain. and of course, there's a limit to the amount of Ram se have available. the benefit of this method is that we can implement it per platform using systme libraries, including gwt.

i'll also try to find Androids implementation of text rendering and will try to get inspired by it. they actually have a glyph cache and freetype. we'll see.

@badlogic

This comment has been minimized.

Copy link
Member

@badlogic badlogic commented May 8, 2014

oh, and i think this functionality belongs in core.

@badlogic

This comment has been minimized.

Copy link
Member

@badlogic badlogic commented May 8, 2014

@NathanSweet

This comment has been minimized.

Copy link
Member

@NathanSweet NathanSweet commented May 9, 2014

From the discussion in the topic on the dev forum it seems layout is simple enough for CJK that we can use an atlas approach, same as we do for Latin languages. If that is true, we can get quite far by adding glyphs to the atlas on the fly. A lot of glyphs can fit on a 2048 texture. There can only be so many glyphs on the screen at once so if the texture were to get filled, simply clearing it would likely be sufficient.

For Arabic and RTL, I dunno, cache full strings I guess.

@davebaol

This comment has been minimized.

Copy link
Member

@davebaol davebaol commented May 9, 2014

AFAIK the number of letters/glyphs in the Arabic language is not a real problem.
They have 28 letters and their glyphs are about 4x since each letter can have up to 4 different shapes depending on whether it will be connecting with a preceding and/or a succeeding letter.
So the main problem is to properly render the letters in one of their conditional forms, depending on whether they are at the beginning, middle or end of a word. In particular, they may have 4 distinct forms: initial, medial, final and isolated.
A few letters have only the isolated or the final form though.
Also one more problem to take into account is that Arabic language is RTL but their numbers are LTR, just like ours. So a sentence containing numbers is a mix of RTL and LTR.

@cypherdare

This comment has been minimized.

Copy link
Contributor

@cypherdare cypherdare commented May 27, 2014

An idea following what Nate said...At startup, seed a glyph map with 2000 or so most common characters for the language, which seems to cover most text for these languages. Reserve the rest of the space for filling in additional glyphs (maybe reserve a fixed square for each of them to make it easy to overwrite them). As new strings are passed in for rendering, the font manager fills new characters into the empty spaces. (Not sure, but would an FBO be faster than new Pixmap-to-Texture if you are just writing a few sprites to it when it's updated? It wouldn't be getting cleared, so there'd still be a buffer restore each time.) This could minimize regeneration of the glyph texture...as the font manager is fed more and more strings, it becomes less and less likely to require updates to the texture.

Maybe also keep a usage count for each character with some kind of comparitor, so it will know which characters to overwrite first. Maybe even push overwritten ones to a second texture and smartly order the drawing of glyphs for only two texture binds if necessary.

@badlogic

This comment has been minimized.

Copy link
Member

@badlogic badlogic commented May 29, 2014

I'd like to start out with the simpler version that doesn't do an LRU glyph cache but instead renders/layouts an entire string to a texture. It's wasteful, but appearently good enough for cocos2d-x (Unity has zero support for CJK/arabic btw. And people pay for it...).

Questions:

  • What should the API be like?
  • Does this belong in core?
  • Should we try to make it somewhat interchangeable with BitmapFont? Or we could just create a new FixedLabel actor or something similar that folks can use instead of the BitmapFont based actor
  • Should we support system fonts?

I have a bit of time this week, so i'd like to get going with this asap.

@badlogic

This comment has been minimized.

Copy link
Member

@badlogic badlogic commented May 29, 2014

For reference, here's what Cocos2D-X does http://www.cocos2d-x.org/wiki/Text_Labels

Funny that that works out well for them. On a related note, we need to come up with a better way to do CJK/arabic CJK. Returning keycodes doesn't cut it with most custom keyboards on Android anymore. Should open a separate issue for that.

@xeoshow

This comment has been minimized.

Copy link

@xeoshow xeoshow commented May 30, 2014

support system fonts: vote +1, since this could reduce the size of apk and
make development easier ...

2014-05-29 20:29 GMT+08:00 Mario Zechner notifications@github.com:

For reference, here's what Cocos2D-X does
http://www.cocos2d-x.org/wiki/Text_Labels

Funny that that works out well for them. On a related note, we need to
come up with a better way to do CJK/arabic CJK. Returning keycodes doesn't
cut it with most custom keyboards on Android anymore. Should open a
separate issue for that.


Reply to this email directly or view it on GitHub
#1589 (comment).

@sridharsundaram

This comment has been minimized.

Copy link
Contributor

@sridharsundaram sridharsundaram commented Nov 28, 2014

I did some work using harfbuzz to support Indic and Arabic languages in Bitmap fonts in libgdx. This worked for Desktop and for Android. Not sure about IOS.

  1. BitmapFontCache had to be modified - there is not a 1-1 mapping between chars and glyphs.
  2. the charsequence is first converted into a glyphsequence, then the glyphs are shaped (reordered according to language specific rules), then rendered - this is done in a JNI call.
  3. harfbuzz + freetype library needs to be added.
  4. textwrapping was an issue.
@sabotagedninja

This comment has been minimized.

Copy link

@sabotagedninja sabotagedninja commented Dec 6, 2014

Where system fonts are concerned, Java AWT's TextLayout classes can be used to render just about any language and styles, inclusing full RTL Arabic. I successfully used this to support Arabic text in our product based on MT4j. We only need it to run on desktop (win/mac), so I can't say if it'll work on mobile devices. The implementation generates one texture for every string requested on the fly, so it's not efficient in the least, but it does work splendidly. Maybe this is another point of inspiration.

@sridharsundaram I'd be very interested to learn about the changes you had to make in libgdx to support Arabic. Is there any way you can share that?

@NathanSweet

This comment has been minimized.

Copy link
Member

@NathanSweet NathanSweet commented Dec 6, 2014

If you want to use AWT for desktop-only font layout, you might see libgdx's UnicodeFont:
https://github.com/libgdx/libgdx/blob/master/extensions/gdx-tools/src/com/badlogic/gdx/tools/hiero/unicodefont/UnicodeFont.java
It renders glyphs to an atlas texture as needed and renders strings glyph by glyph. It doesn't cache strings. It doesn't do RTL. Hiero makes for a nice demo:
https://github.com/libgdx/libgdx/wiki/Hiero

@sridharsundaram

This comment has been minimized.

Copy link
Contributor

@sridharsundaram sridharsundaram commented Dec 11, 2014

I had to modify BitmapFontCache in addToCache

....
if (isComplexScriptLayoutEnabled()) {
List glyphs = getGlyphsAfterShaping(str, start, end, data);
requireGlyphSequence(glyphs);
for (Glyph g: glyphs) {
addGlyph(g, x + g.xoffset * scaleX,
y + g.yoffset * scaleY,
g.width * scaleX,
g.height * scaleY);
x += g.xadvance * scaleX;
}
return x - startX;
}
....

// Here we make a JNI call to get the complex script shaping done
private List getGlyphsAfterShaping(CharSequence str, int start,
int end, BitmapFontData data) {
List glyphs = new ArrayList();
int[] glyphIndices = complexScriptLayout.getGlyphsAfterShaping(str,
start, end);
for (int glyphIndex: glyphIndices) {
Glyph glyph = data.getGlyph((char) glyphIndex);
if (glyph != null) {
glyphs.add(glyph);
}
}
return glyphs;

}

complexScriptLayout is a JNI based class which simply calls into Harfbuzz
to reorder the glyphs.
I can send more details if there is interest.

Entire strings are processed at a time because the glyphs need to get
reordered and multiple
glyphs are composed to render a single screen character.

On Sun, Dec 7, 2014 at 1:12 AM, Bastiaan de Jong notifications@github.com
wrote:

Where system fonts are concerned, Java AWT's TextLayout classes can be
used to render just about any language and styles, inclusing full RTL
Arabic. I successfully implemented this to support Arabic text in our
product based on MT4j. We only need it to run on desktop (win/mac), so I
can't say if it'll work on mobile devices. The implementation generates one
texture for every string requested on the fly, so it's not efficient in the
least, but it does work splendidly. Maybe this is another point of
inspiration.

@sridharsundaram https://github.com/sridharsundaram I'd be very
interested to learn about the changes you had to make in libgdx to support
Arabic. Is there any way you can share that?


Reply to this email directly or view it on GitHub
#1589 (comment).

@bill-lin

This comment has been minimized.

Copy link

@bill-lin bill-lin commented Jan 2, 2015

+1 to using system font
I am developing an app in Chinese and it is very painful to handle chinese input and cross platform.

@tdlrobin

This comment has been minimized.

Copy link

@tdlrobin tdlrobin commented Jan 23, 2015

+1 to using system font

@abdlquadri

This comment has been minimized.

Copy link

@abdlquadri abdlquadri commented Mar 9, 2015

@NathanSweet I edited Hiero to handle RTL. Details here. Tested only Arabic though. http://www.badlogicgames.com/forum/viewtopic.php?f=15&t=18592

But it seems that success does not extend to the core libraries because my test still renders LTR even on desktop.

@abdlquadri

This comment has been minimized.

Copy link

@abdlquadri abdlquadri commented Mar 9, 2015

@sridharsundaram I will be interested in how you implemented that. Also does if work for the other platforms?

@abdlquadri

This comment has been minimized.

Copy link

@abdlquadri abdlquadri commented Mar 9, 2015

Hi All,

Just wondering what progress has been made on using System Font and how I can contribute to this. Any pointer to where to start would be great.

Regards.

@sridharsundaram

This comment has been minimized.

Copy link
Contributor

@sridharsundaram sridharsundaram commented Mar 9, 2015

I had to modify BitmapFontCache in addToCache

....
if (isComplexScriptLayoutEnabled()) {
List glyphs = getGlyphsAfterShaping(str, start, end, data);
requireGlyphSequence(glyphs);
for (Glyph g: glyphs) {
addGlyph(g, x + g.xoffset * scaleX,
y + g.yoffset * scaleY,
g.width * scaleX,
g.height * scaleY);
x += g.xadvance * scaleX;
}
return x - startX;
}
....

// Here we make a JNI call to get the complex script shaping done
private List getGlyphsAfterShaping(CharSequence str, int start,
int end, BitmapFontData data) {
List glyphs = new ArrayList();
int[] glyphIndices = complexScriptLayout.getGlyphsAfterShaping(str,
start, end);
for (int glyphIndex: glyphIndices) {
Glyph glyph = data.getGlyph((char) glyphIndex);
if (glyph != null) {
glyphs.add(glyph);
}
}
return glyphs;

}

complexScriptLayout is a JNI based class which simply calls into Harfbuzz
to reorder the glyphs.
I can send more details if there is interest.

Entire strings are processed at a time because the glyphs need to get
reordered and multiple
glyphs are composed to render a single screen character.

On Mon, Mar 9, 2015 at 3:42 PM, abdlquadri notifications@github.com wrote:

@sridharsundaram https://github.com/sridharsundaram I will be
interested in how you implemented that. Also does if work for the other
platforms?


Reply to this email directly or view it on GitHub
#1589 (comment).

@abdlquadri

This comment has been minimized.

Copy link

@abdlquadri abdlquadri commented Mar 9, 2015

"I can send more details if there is interest." Please send more detail.

@NathanSweet

This comment has been minimized.

Copy link
Member

@NathanSweet NathanSweet commented Mar 17, 2015

@sridharsundaram if you post a project that builds the HarfBuzz JNI stuff with gdx-jnigen, I'd like to take a look.

@NathanSweet

This comment has been minimized.

Copy link
Member

@NathanSweet NathanSweet commented Mar 19, 2015

#2955 adds support for shaping characters into glyphs. It sets the stage to use HarfBuzz. Once merged I have code ready which enables FreeTypeFontData to render glyphs on the fly:

The red boxes are the backing textures, so the font has 4 pages of glyphs. The page size was made very small on purpose for testing.

Packing glyphs on the fly isn't super efficient with PixmapPacker's tree packing, since they aren't sorted. Might be better to use line by line packing, maybe using the max glyph height.

@Masterhame

This comment has been minimized.

Copy link

@Masterhame Masterhame commented Apr 17, 2017

Is this problem solved? any clear solution or any package to have the completed methods for writing with complex script languages?

@NathanSweet

This comment has been minimized.

Copy link
Member

@NathanSweet NathanSweet commented Apr 19, 2017

Not as far as I know. libgdx has on the fly glyph rendering, so Latin, Cyrillic, CJK, and similar languages are supported. RTL languages would need something like Harfbuzz for their complex layout.

@CrowniAPIs

This comment has been minimized.

Copy link

@CrowniAPIs CrowniAPIs commented Oct 7, 2017

RTL languages like Arabic not just direction from RIGHT to LEFT, but how connect their letters with each other and some letters as a block with 2 or more letters.
So, I do it with fully supporting RTL language.
https://github.com/CrowniAPIs/libGDX-RTL-Language

@Kieaer

This comment has been minimized.

Copy link

@Kieaer Kieaer commented Oct 22, 2019

Even in October 2019, the problem is not solved.
I still don't see the combination characters I'm typing when I type them, and I can't solve the problem that only appears when the combination characters are completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.