Text layout without blocking the UI

[dnfield]

There are a number of bugs that I've seen that boil down to this: if you have a large chunk of text, laying it out may be expensive enough where it blocks the UI for an unacceptable amount of time. One work around for this is to try to break up your text into smaller chunks, which works if you already have natural breaks (e.g. many small paragraphs laid out individually instead of all at once), but doesn't work as well if you just have a really long paragraph you want to layout and you don't know where the line breaks will fall. This may be a problem for, e.g. a reader for an academic journal that just has long paragraphs, or perhaps in some other languages where longer paragraphs may be more common.

It is not currently possible to layout text in a separate isolate, as all dart:ui methods must run in the main isolate. We should explore either making it possible to run in another isolate, or creating some async text layout methods.

Here are some example issues:

#23718
#30604
#41536 (not as strongly related but may be helped by such an API)

/cc @GaryQian @jason-simmons

[esDotDev]

#23718 is more related to the general ability to measure a text widget, big or small, and make layout considerations based on that.

For example, you may need to know if a single hz line can fit, and just hide it if it can't.

[dnfield]

That is already possible, but only practical in cases where the text is reasonable short.

[esDotDev]

I see, thanks!

[Hixie]

Do we know if these cases could be handled if the text measuring was done in parallel?

I was wondering if maybe what we could do is when we build a RenderParagraph, we have it immediately inform the engine about the text it cares about, so that the engine can warm its caches for that text, so that by the time we come to actually measuring the text, it can respond much quicker than now.

[dnfield]

I'm not sure what part you mean when asking in parallel.

If we did the whole text layout operation on another thread while we kept moving along, I think that's basically the API this issue is asking for. If we, alternatively, somehow parallelized text layout so that multiple threads could figure out how to lay out a paragraph, that might help too.

Are you suggesting that we implicitly call layout on the Paragraph returned by ParagraphBuilder.build, and that way could (potentially) avoid blocking on the call to Paragraph.layout? That could help, but only in cases where the user isn't already immediately calling layout anyway. It seems like it'd be better to make this more explicitly controlled anyway, so that you can do something like paragarph.asyncLayout.

[Hixie]

I'm suggesting that RichText.createRenderObject and RichText.updateRenderObject would call a new RenderParagraph.precache API, which would (via some new TextPainter API) call some new non-blocking dart:ui API to warm the caches for that text and style, so that by the time we got to layout, the call to TextPainter.layout would return faster. (The new API wouldn't return anything, it would just do whatever work the engine can do to lay out a piece of text that doesn't depend on knowing the constraints. For many scripts, that's a lot of work. For some, it may be significantly less.)

The "parallel" part applies in the sense that multiple such RichText/RenderParagraph/TextPainter groups would all call this new API during the widget build phase, before layout, such that all these text node would be warming up simultaneously in the engine.

We can't call TextPainter.layout because we don't know the constraints ahead of layout. For that same reason, an asyncLayout doesn't help with making a single frame faster because by the time we're ready to call it we need the answer synchronously.

[dnfield]

I don't think anyone has profiled which parts of this take a long time. However, this bug was inspired more by dealing with longer strings of text where the actual layout is the non-fixed cost (as opposed to loading font attributes).

This would have disadvantages where you may not know the full size of e.g. a sliver you want to layout, but I suspect in a lot of cases developers do know (or could make reasonable decisions) about what size to lay text out ahead of time.

[Hixie]

Almost all RenderParagraph nodes have no idea what size they'll be until layout time, as far as I can tell (at which time they need the answer synchronously).

[sagarkardani]

I am working on some custom textviewer and I faced #92173 issue. I also piled up all similar issues in this comment.

My point here is layout call is expensive at present. So if layout is done in seperate isolate or async and if it is expensive just like at present then also we will have to wait for layout details till layout call is complete. This would not be much beneficial in cases like #55722, #58478, #92173. My suggestion here is workaround should also be done to improve the performance of layout call.

[dnfield]

@Hixie - we could create a widet/RO that knows how to deal with async text layout, similar to how we deal with async image loading today. Developers with application domain knowledge could say "start at this size, which should be close", and text would eventually come in at the right size.

I'm not sure if it's possible to have any heuristics to guess something close to the right size, but maybe we could do something close to accurate with the length of the string.

[dnfield]

Imagine, for example, a list of small strings, each of which takes ~1ms to lay out and each of which will be shown with an ellipsis if they are too long to fit on onel ine. As soon as you have more than 8 of these strings you're over budget on text layout alone. Async text layout would help a lot here, because you could just have your list tiles render faster and eventually put the strings in as they become available, rather than losing 1 or more frames worth of budget waiting for an answer on initial layout.

[dnfield]

/cc @goderbauer @jason-simmons

[goderbauer]

#94419 contains an actual example that renders 9 list tiles with 3 RenderParagraphs each. On average on a low-end device, each RenderParagraph takes 1.0ms to lay out, which means 27ms alone are spend laying out text in this rather simple screen.

[goderbauer]

The idea of precomputing some text metrics asynchronously and kicking that off right when the RenderObject is configured (way ahead of the layout phase) sounds interesting. This would be similar to Android's PrecumputedText feature. According to their medium article at https://medium.com/androiddevelopers/prefetch-text-layout-in-recyclerview-4acf9103f438 about 90% of the layout work can be done ahead of time without knowing the width/height of the text box yet. During layout of RenderParagraph we would then finish up the layout by providing width/height of the text box. Overall, the text layout would still be synchronous, the size of the text is available in the same frame. But much of the layout work would be in parallel on a different thread ahead of the layout phase without blocking the UI thread for it.

[goderbauer]

//cc @Rusino

[goderbauer]

I've spun out the 2-step text layout idea into its own issue with some experimental data showing that in theory this could help with improving build/layout times: #96235