Opt plain-text character array into incremental summarization - #27588
Conversation
|
Hi! Thank you for opening this PR. Want me to review it? Based on the diff (44 lines, 1 files), I've queued these reviewers:
How this works
|
Joshua Smithrud (Josmithr)
left a comment
Joshua Smithrud (Josmithr)
left a comment
There was a problem hiding this comment.
Left a suggestion for documentation we should add. It would also probably be worth adding a changeset document to this PR, since we're surfacing improved functionality that requires user opt-in. We can likely re-use much of the same documentation for both purposes.
| "StringArray", | ||
| // Opt the character content into incremental summary optimization. | ||
| sf.types([SchemaFactory.string], { | ||
| custom: { [incrementalSummaryHint]: true }, |
There was a problem hiding this comment.
I was thinking we would just set this in the text editor schema above the TextNodes (both plain and formatted) so we would avoid reencoding the whole plain text when changing the formatted text for example.
Incremental summary chunks are pretty expensive, so even with our larger chunk sizes now, I'm not sure its a win (each one gets included in a top level list of all chunks in the document, complicates container level GC, and has to be referenced by a handle which has a non negligible size not to mention each incremental chunk itself has a lot of boilerplate wrapping it, including another copy of the needed shapes (not too bad for strings I guess). I also think we might not do a great job of encoding sequences at the root of chunks efficiently (I think they might all get the any shape encoder, adding per character overhead in this case).
Thus I think putting the opt in here is a good way to stress test incremental summary, but probably not a performance win in most cases.
Bundle size comparisonBase commit: Notable changesNo bundles changed by ≥ 500 bytes parsed. Per-bundle deltas
|
Description
Enables Incremental Summariation for plain text. Plain text is contained in large uniform chunks, and will benefit from incremental summarization. Formatted is currently not stored in a way where incremental summarization will provide a benefit compared to the overhead cost of incremental summarization.
Testing
Single test in textDomain.spec.ts that confirms the plain text character array is opted into incremental summarization.
Reviewer Guidance
The review process is outlined on this wiki page.