Repository navigation
v1.4.1
What's Changed
- perf: documents with big strings — tests and fewer large-object copies by @FransVanEk in #84
Details
Documents that carry a few very large strings (for example base64-encoded files of 250–350 KB) were reported to slow every command and function down. Measured: per-command cost does not depend on string size on any adapter (Newtonsoft, System.Text.Json, XML, YAML). What grows with the strings is the work around the commands — parse, convert, serialize — and the large-object-heap allocation it causes, which under concurrent load shows up as extra full GCs that pause every thread.
- Newtonsoft
JsonNodeAdapter.Parsereads through a pooled buffer: 12.2 MB → 1.7 MB allocated for a 2 MB document. - XML → JSON conversion allocates about half as much (15.3 MB → 7.5 MB for the same document).
- System.Text.Json
Serializereuses its options instead of creating new ones per call. - New tests (
BigObjectPerformanceTests) guard per-command cost, the full command/function sweep, linear conversion cost and allocation per stage.
Not solved: the JSON → XML direction is largely unchanged, and its cost under concurrent load remains. Hosts handling large documents should consider server GC, running a script in one execution rather than per command, and compiled scripts.
Full Changelog: v1.4.0...v1.4.1