Build template models in Jint's hidden-class representation, prepare scripts once, and bound script execution

[lahma]

docfx's Jint usage lives in three files under src/Docfx.Build/TemplateProcessors/Preprocessors/, plus one drive-by in Template.cs. This PR bumps Jint 4.6.1 → 4.15.3 and reworks that usage against the embedding surface 4.15 added.

Jint 4.15.3 is released and on nuget.org. A 4.15 pin is required, not optional: JsObject.CreateFromEntries and Options.AddLazyGlobal do not exist before 4.15.0, Engine.Advanced.GetInteropConversionDiagnostics not before 4.15.1, and Engine.Advanced.HasSharedShape not before 4.15.3.

The version jump itself is large — 4.6.1 is from ~2026-03 — and carries a lot of interop and interpreter work plus spec-conformance fixes that docfx gets for free. Some of it is directly relevant to how docfx uses Jint: resolved CLR members are now cached process-wide on the shared TypeResolver rather than per engine, CLR property/field access is compiled instead of reflected per hit, and host delegates no longer re-reflect their parameter metadata on every call. That last one is what require and the console / templateUtility members were paying.

---

1. Build model objects in Jint's hidden-class representation

JintProcessorHelper.ConvertObjectToJsValue built every model object with a jsObject.FastSetDataProperty(...) loop. Despite the name that is not the fast way to project host data into a new object: it stores a raw PropertyDescriptor, which is a dictionary-mode operation, so each model gets a property dictionary and a descriptor per property, and the object permanently forfeits Jint's shape (hidden class) inline cache.

JsObject.CreateFromEntries drives the incremental hidden-class path instead. This matters specifically for docfx because every document of a given document type is handed to the template with the same model layout, so repeated calls presenting the same key sequence intern and share one shape — which is what keeps the property reads in transform/getOptions monomorphic across a whole docset rather than megamorphic.

The IEnumerable<KeyValuePair<...>> overload is used deliberately: the source is an IDictionary that cannot be viewed as a span, and materialising a temporary array of the model's size just to call the span overload would cost the very allocation the shaped representation saves. The entries are streamed through a local iterator and walked exactly once.

Key sets the representation cannot express — integer-index-like keys, objects past the shape property cap, excessive layout fan-out — fall back to the ordinary dictionary representation mid-build, silently and correctly. That fallback is exercised by the existing TestJObjectConvertWithJToken test (its ValueDict is keyed "1" / "key") and by a new test that pins the resulting own-key order.

Because the fallback is silent, the tests assert it directly. The claim this rests on is not merely that a model is shaped but that documents of one type share a layout, and Engine.Advanced.HasSharedShape (4.15.3) is documented as the stable predicate for exactly that question — one a host may pin for the documented success case of the factory it called. The positive assertion uses it. The integer-index-like fallback keeps the finer-grained GetObjectRepresentation, which is deliberately not a compatibility contract, because naming the exact fallback representation is that assertion's whole point; it is annotated as expected to go red on a future Jint that learns to shape such keys — good news arriving as a red test, not a docfx defect.

One behavioural risk the version jump carries, now pinned. Jint's default for CLR arrays became a live view in 4.14 — script writing to such an array writes through to the CLR array itself — and this bump crosses that change. It is inert for docfx only because ConvertObjectToJsValue builds every array itself, via the JsArray ownership-transfer constructor, and never hands a CLR array to Jint's converter. What keeps that true is narrow: model arrays arrive as IList<object> (ConvertToObjectHelper produces object[], which qualifies), so a model conversion that ever yielded a typed array such as int[] would fall through and start handing script a live view onto docfx's own model object. 4.15.1's Engine.Advanced.GetInteropConversionDiagnostics makes that observable, and a new test asserts a representative model crosses zero CLR arrays under either mode — with a control line proving the counters are live rather than vacuously zero.

2. Prepare template scripts once instead of once per pool slot

Template sources and required module sources were executed from text via engine.Execute(source, path). PreprocessorWithResourcePool creates one preprocessor — and therefore one engine — per parallelism slot, so that was P parses of the template plus P parses of every module it requires.

They are now parsed once through Engine.PrepareScript and shared across the pool: a Prepared<Script> is immutable and documented as reusable and thread-safe, and Jint re-derives any per-node cached state when a prepared script runs against a different realm. Preparation also attaches static analysis data and constant folding, so the shared copy executes slightly faster too.

The cache is per template (created in PreprocessorLoader.Load and handed to each pooled instance), keyed by resource path — no process-wide static state, nothing that outlives a build. The existing public TemplateJintPreprocessor constructor is unchanged; an internal overload takes the shared cache.

3. Give the preprocessor engines execution constraints

CreateDefaultEngine called new Jint.Engine() with no options at all — no timeout, no statement limit, no cancellation. A while (true) {} in a .tmpl.js from the docset being built would pin a render thread for the lifetime of the process.

Every engine the preprocessor owns now registers a 30 s timeout, a 50,000,000 statement budget, and the build's CancellationToken (reachable via DocumentBuildContext, and correctly absent when there is no context). TemplateJintPreprocessor resets them all at the top of GetOptions/TransformModel, so they bound one document rather than the lifetime of a pooled preprocessor.

That reset is not optional, and getting it wrong is not a subtle inefficiency — it is a build-breaking bug, so it is worth stating why. Jint resets an engine's constraints when the engine is entered through a public API such as Engine.Invoke, and that covers the template's own engine. It does not cover the engines require builds: a module's exported function belongs to the module's engine, so common.transform(model) is an ordinary function call that charges the module engine's constraint instances without passing through any entry point that resets them. A timeout deadline is armed only on reset and a statement counter is only cleared on reset, so without an explicit reset a module engine would carry a deadline fixed at preprocessor construction — meaning every document handled by a pool slot older than 30 s would fail with a spurious TimeoutException — and a statement budget spanning the whole build. Two tests pin both directions: a module call still succeeds after the preprocessor has been aged past its own timeout, and a runaway loop inside a module is still stopped (which is why the constraints stay registered on module engines rather than being dropped from them).

The sample docsets keep every pool slot's active phase well under 30 s, so this failure mode is structurally invisible to the snapshot suite; the tests above are what cover it.

The values are deliberately real numbers: Jint treats saturated values (MaxStatements(int.MaxValue), TimeoutInterval(TimeSpan.MaxValue)) as "remove the constraint", so spelling "effectively unlimited" that way would silently produce an engine with no limit. The statement budget is orders of magnitude above what transforming even a very large model costs; a runaway loop hits it in about a second.

On cost: the timeout and cancellation constraints are amortized (checked at a bounded cadence, not per statement). The statement counter is exact, but because it is the only exact constraint registered here Jint charges it inline rather than disarming the interpreter's tight-loop lanes, so ordinary template execution is not slowed by having it. If you would rather not carry a statement budget at all, dropping that one line leaves the timeout doing the same job less deterministically.

A new test asserts that a runaway transform fails the document instead of hanging.

4. Register require per engine instead of copying a function object between engines

require gives every module its own Engine, and built it with CreateEngine(engine, RequireFuncVariableName) — which read the require function out of one engine and installed that same JsValue into another. A JsValue that is an ObjectInstance holds a hard reference to the engine and realm that created it; passing one to a different engine is not a supported arrangement in Jint, and is neither validated nor made safe. It happened to work because a DelegateWrapper mostly just forwards to its CLR delegate.

Each engine now registers require from the CLR delegate itself. Same delegate, same module cache, same resolution — only without a value crossing an engine boundary. CreateEngine is gone.

This does not finish the job: the module's exports object is still created in the module's engine and returned into the engine that called require, and the shipped templates then call functions off it — ManagedReference.html.primary.js does overwrite.transform(model) on a function that belongs to a different engine. Removing that needs the CommonJS wrapper-function change described under Follow-up, which is an observable semantic change and belongs in its own PR. This change removes the half that can be removed without one.

Two notes on what remains. First, the value side of that crossing does hold together under 4.15: the member inline caches key on tokens carried by the object (shape identity, properties version) rather than on the reading engine, mutations route through the object's own engine, and the shared TypeResolver cache is keyed by interop profile, which both engines share. What does not cross safely is execution-scoped state, and this PR introduces exactly that — which is why §3's reset has to walk every engine rather than relying on the entry point. Second, one small behaviour change falls out of registering require from the delegate: a module engine's require is now NonEnumerable, where copying the JsValue previously made it enumerable. It is observable only to a module script enumerating its own globals, no shipped template does, and the root and module engines now agree — but it is a behaviour change, so it is recorded rather than left implicit.

5. Register console and templateUtility lazily

Options.AddLazyGlobal defers building a global until a script reads the name. Most template scripts never mention console or templateUtility, and engines are not scarce in this design: the preprocessor pool builds one per parallelism slot and require builds one more per module — a bundle with 5 shared modules at MaxParallelism 16 is 96 engines.

The flags match: AddLazyGlobal's default is the configurable/enumerable/writable combination SetValue(string, object) produces, which is what these two used. exports stays eager, because it is read back off the engine after the script runs whether or not the script ever mentioned the name.

6. Drop the Newtonsoft round trip in Template.GetOptions

GetOptions did JObject.FromObject(obj).ToObject<TransformModelOptions>() — once per document, per template — to read two fields off a value the script engine already handed back as a property bag. It now reads isShared and bookmarks directly, matching names case-insensitively as the converter did. Anything that is not a property bag (a preprocessor that is not script based) still goes through the converter, so nothing that worked before stops working.

---

Jint 4.15 feature audit

Everything 4.15.0, 4.15.1 and 4.15.3 added that could plausibly apply here, and what was done with it. 4.15.2 is absent because it adds no API — it was a fix release (async/generator suspension state, bound-of-bound functions, inherited-accessor receivers, a built-in shape probe fast-miss). The version notes name the release each item landed in, not the pin.

Feature	Verdict	Why
JsObject.CreateFromEntries	Adopted	§1 — the model marshal is the one place docfx builds a fresh object per document.
Engine.Advanced.HasSharedShape (4.15.3)	Adopted (tests only)	§1 — the contract-stable predicate for "do these models share a layout", which is the property the PR's design rests on. Replaces the non-contract enum for the positive assertion.
Engine.Advanced.GetObjectRepresentation	Adopted (tests only)	§1 — retained only for the negative case, where naming the exact fallback representation is the point. Not used in production code; explicitly not a compatibility contract.
AppContext switch Jint.EnableHostContractVerification (4.15.3)	Declined	Every verifier it enables is gated on a host extension point — TryGetOwnPropertyValue (OwnValueHook), the built-in-shape probe, an Ordinary semantics claim, ArrayLikeObject.HasIndex — and docfx implements none of them; its models are plain engine-owned objects with no host contract to contradict. Enabling it would verify nothing while reading to a future maintainer as though it did. Worth adding the day docfx gains its first host object.
Engine.Advanced.AddLazyGlobal (post-construction) (4.15.3)	Declined	docfx's only post-construction global is require, which is the one nearly every shipped template does use, so deferring it would rarely pay; the two that genuinely often go untouched, console and templateUtility, are already deferred through the options-time API.
Options.AddImmutableCrossing (4.15.3)	Declined	It memoizes reads through ObjectWrapper, and docfx's hot traffic — the models — deliberately does not cross as wrappers at all. The only wrapped objects are console and templateUtility, read a handful of times per document at most and now lazy besides, and declaring compiler-generated anonymous types by runtime GetType() would be a fragile registration for that.
PropertyDescriptor CreateLazy (4.15.3)	Declined	docfx constructs no PropertyDescriptors; it hands Jint values, not descriptors.
Engine.Advanced.WithRestoredGlobals (4.15.3)	Declined	An ergonomic scope around capture/restore, which docfx declined on its merits below — there is no per-document global state to clear, so a nicer wrapper around it changes nothing.
Options.AddLazyGlobal	Adopted	§5.
Engine.Advanced.GetInteropConversionDiagnostics (4.15.1)	Adopted (tests only)	§1 — pins that the model marshal crosses no CLR array, so the live-view default this bump inherits from 4.14 cannot reach docfx's own model objects. Diagnostic only, never branched on.
"values do not cross engines" (newly documented rule)	Partly adopted	§4 — the require function no longer crosses; the module exports object still does, pending the follow-up.
Interop and interpreter work: shared TypeResolver member cache, compiled property/field access, delegate metadata caching, built-in fast-call lane, constraints no longer disarming the tight-loop lane	Adopted implicitly	Arrives with the bump, no code change. It is also what makes console / templateUtility / require cheap enough that reworking them by hand is not worth it — see JsObjectShape below.
JsObjectLayout + JsObject.Create	Skipped	Stronger than CreateFromEntries when the key set is known before the loop. docfx's is not: the model arrives as an IDictionary whose keys are data-driven, so a layout could only be found by a lookup keyed on the very key sequence you would have to walk to build it, and the values would have to be materialised into an array first. CreateFromEntries interns the same shape across documents by transition, without either cost.
JsObjectLayout.CreateBuilder().AddLazy(...) — lazy slots (4.15.1)	Skipped	Re-checked, because a deferral opportunity does exist in principle: ConvertObjectToJsValue recurses eagerly over the whole model graph, so a template reading only part of a large model pays for the rest. But lazy slots are declarable only on a layout, and Jint states the boundary directly — "CreateFromEntries is deliberately not extended: lazy members are declared on a layout, and a runtime key set has nowhere to declare them." docfx's key sets are runtime-derived, so the factory it can use is precisely the one that cannot carry lazy slots. Worth noting the value would be limited anyway: the CLR-side values are already materialised before Jint sees them, so a factory could only defer the conversion, and the shipped templates iterate model.children — the large part — on every document.
JsObjectShape	Skipped	Would replace the reflected anonymous objects behind console / templateUtility. Three reasons not to: a shape's Method delegates take JsValue arguments, so converting would change host-boundary argument coercion on a surface third-party templates depend on; templateUtility closes over per-build state and so cannot be the process-shared static readonly shape the API targets; and the per-call reflection that motivated the idea is already gone with the bump.
Engine.Advanced.CaptureGlobalSnapshot / RestoreGlobalSnapshot	Skipped	Jint positions it as a configuration-reuse primitive for hosts that need a clean global per evaluation. docfx needs the opposite: nothing is re-registered per document — the model crosses as a function argument — and the global surface is set up once per engine and then deliberately persists for the whole build. There is no per-document global state to clear, so restoring between documents would buy nothing and only add a way to lose state the templates rely on.
Prepared<T>.ReferencedGlobals	Skipped	Aimed at hosts whose ambient API is large and whose CLR-side construction is the expensive part. docfx's is three names — two of which §5 now defers anyway, and the third is a single delegate wrapper. It would also not be a sufficient input on its own, since require has to be installed on every module engine transitively.
JsonParser.Parse(ReadOnlySpan<char>) / (ReadOnlySpan<byte>), JsonSerializer.Serialize(…, IBufferWriter<byte>)	Skipped	No JSON crosses the boundary. Models cross as object graphs, and the transform result feeds the Mustache renderer, not a serializer. 4.15.1's added docs reinforce the skip rather than change it: an IBufferWriter caller who wants a string re-pays the transcode the overload exists to avoid, and the string overloads return JsValue.Undefined for a value with no JSON representation — so Serialize(v).ToString() would silently yield the literal text "undefined". Neither trap is reachable from here.
Engine.Advanced.GetPropertyAccessSemantics (4.15.1)	Not applicable	Lets a test observe the semantics the engine derived for a host type. docfx defines none.
PropertyFlag.NonWritable / OnlyConfigurable (4.15.1)	Not applicable	Additive names for two combinations; nothing renamed. docfx spells no flags — AddLazyGlobal is taken at its default, which remains ConfigurableEnumerableWritable, the combination SetValue(string, object) produces.
ObjectInstance.GetOwnProperties routing docs (4.15.1)	Nothing to reconcile	The new remarks warn that a host overriding only GetOwnProperties ships keys no script-visible enumeration can see. docfx overrides nothing, so the gotcha cannot bite; the method is on its path only as a caller (CLR conversion, i.e. the ToObject() on every transform result), which is unchanged.
ArrayLikeObject	Skipped	For live indexed collections. docfx's model arrays are snapshots, and copying a snapshot into a JsArray once — which ConvertObjectToJsValue already does, using the ownership-transfer constructor — is what that API's own guidance recommends instead.
AddObjectConverter(converter, params Type[])	Not applicable	docfx registers no object converters.
Options.Interop.EnumConversion / EnumConversionMode	Not applicable	No CLR enums cross the boundary.
NullPropagatingReferenceResolver + ReferenceResolverInterests	Skipped	Would turn a TypeError on a nullish base into undefined for every template in the ecosystem. That is a language-semantics change for third-party template authors, not an optimization.
Host-object hooks: PropertyAccessSemantics, TryGetOwnPropertyValue, ProbeOwnProperty, JsObjectShape.SetHostState	Not applicable	docfx defines no ObjectInstance subclass, and no custom Jint type at all.

---

Not done here — follow-up

require still creates a whole extra Engine per module, and the module's exports object — created in the module's engine — is still returned into the calling engine's script, which then calls functions off it. §4 removed the other half of that (the require function object no longer crosses), but the object graph still does.

The conventional CommonJS shim — wrapping the module source in (function (exports, require, module) { ... }) and executing it in the same engine, with the exports object cached before the body runs so the existing "unfinished copy" cycle behaviour is preserved — would remove both the extra engines and the remaining cross-engine values. It is left out of this PR because it is an observable semantic change (module code would see the host engine's globals, and reassigning exports rather than mutating it would behave differently), which deserves its own PR and its own discussion.

Measured effect

BenchmarkDotNet, default job, [MemoryDiagnoser], .NET 10.0.10, x64. 64 ManagedReference-shaped models per operation through TemplateJintPreprocessor.TransformModel — a deep marshal in, the JS call, and a deep unmarshal back out, once per document. BEFORE is this PR's merge base (2a436495ed, Jint 4.6.1); AFTER is the PR tip (Jint 4.15.1). Both sides ran back to back on an otherwise idle machine from identical harness sources.

Lane	BEFORE	AFTER	Time	Allocated
transform, self-contained template	2.078 ms · 3.09 MB	1.990 ms · 2.75 MB	−4.2 %	−11.0 %
transform, through a required module	2.118 ms · 3.10 MB	1.989 ms · 2.76 MB	−6.2 %	−11.0 %

The pin has since moved to 4.15.3; the table above was measured at 4.15.1 and has not been re-run, since neither later release changes a default execution path (4.15.2 was fixes, 4.15.3 additive API) and re-measuring would only add noise.

StdDev was ≈0.6 % of the mean on every row, so the time deltas are several times the run-to-run spread; the allocation figures are deterministic. Gen1 collections roughly halved (15.6 → 7.8 per 1 000 ops on the first lane, 19.5 → 7.8 on the second), which is the property-descriptor-per-property churn going away.

Four honesty notes:

• This delta conflates two things. BEFORE is the merge base, so it carries Jint 4.6.1; the measurement is "what merging this PR does", not "what CreateFromEntries does". Four minor versions of engine and interop work are in the same number, and none of it is separated out here.
• The require lane exists for fairness, not decoration. §3's constraint reset is per-call work proportional to the number of engines a preprocessor owns, and a template with no modules owns exactly one — the cheapest possible case for the AFTER side. Every shipped template uses require, so that lane is the representative one. It is also where the older arrangement was relatively worse: BEFORE, adding a module cost ~2 % (2.078 → 2.118 ms) because the module engine re-parsed its own source and a function object was copied across the engine boundary; AFTER, the two lanes are indistinguishable.
• §2's win is not in these numbers. Sharing one Prepared<Script> across a pool is a construction-time effect that only materialises across concurrent pool rents, which is not what this benchmark exercises. Treat §2 as unmeasured here rather than as measured-and-small.
• The harness is local and deliberately not part of this repository — docfx has no benchmark project, and adding one for a single review is not a change worth carrying. It uses only docfx's public API so the identical source compiles against both sides.

Verification

• Full solution, Release: builds clean, 0 warnings, against Jint 4.15.3 restored from nuget.org. Package provenance checked: the nuspec in jint.4.15.3.nupkg records commit="a304aa5dac…", which is what the v4.15.3 tag on sebastienros/jint points at.
• Docfx.Build.Tests, Release, net10.0: green (119 passed). Ten new tests cover this area — own-key order and nesting; zero CLR arrays crossing the model marshal, with a live-counter control; models sharing a layout, plus the hidden-class representation; the integer-index-like key fallback, plus the dictionary representation; getOptions reading; a chained require (a template requiring a module that itself requires another, which is the shipped-template pattern); console reaching the logger through the lazy global; the runaway-script guard; a module call surviving a preprocessor aged past its own timeout; and a runaway loop inside a module still being stopped. The last two were written red first — before the per-call reset, the aged-preprocessor test fails with TimeoutException thrown from TimeConstraint.Check() inside the module function's loop.
• Docfx.Build.Tests is granted InternalsVisibleTo (following Docfx.Build.Common.Tests, already in that list) so the two constraint tests can construct a preprocessor with a short timeout instead of sleeping for 30 s. The timeout is an optional parameter on the existing internal constructor; the public constructor and the default are unchanged.
• docfx.Snapshot.Tests is the important gate for a change to model marshalling and to require, since it renders the shipped templates — require chains and all — over the sample docsets. Because those tests AutoVerify locally, the suite was run repeatedly on the same machine, with and without the changes, and the resulting verified trees compared by content hash: 784 of 785 rendered snapshots are byte-identical across every pair. The one exception, BuildFromProject.SourceGenerator.html.view.verified.json, carries git-remote and absolute-path detection that varies between runs; re-running at unchanged source reproduced both of its observed contents, so the variation is environmental rather than behavioural. (Comparing against the committed baseline instead is not meaningful — a local run legitimately differs from the CI-produced files in _key / _tocKey paths, which is why the comparison is run-to-run.) The suite's blind spot is anything depending on wall clock or cumulative scale, which is exactly the constraint-reset failure mode above; the two unit tests cover that instead. It was re-run for the 4.15.1 and 4.15.2 pin moves — both trees byte-identical to their predecessor. 4.15.2 earned its run because two of its fixes changed existing property read and probe lanes and docfx's models are shape-mode receivers read on every document. 4.15.3 was not re-run, and deliberately so: all seven of its commits are additive API or documentation, every new behaviour is opt-in, and none of them alters an execution path docfx is already on.
• The two red checks on this PR are pre-existing and unrelated — neither is caused by these changes.
  • Lint (dotnet format --no-restore --verify-no-changes) fails on test/Docfx.Dotnet.Tests/SymbolUrlResolverUnitTest.cs(5,1): IDE0005. A clean worktree of main at this PR's merge base reproduces it exactly, and PRs chore(deps-dev): bump js-yaml from 4.1.1 to 4.2.0 in /templates #11081, chore(deps): bump dompurify from 3.3.1 to 3.4.11 in /templates #11080, chore(deps): bump esbuild and tsx in /templates #11077 and chore(deps): bump uuid and mermaid in /templates #11073 all fail Lint the same way. Docfx.Dotnet.Tests references only Docfx.Dotnet, neither of which this PR touches; dotnet format --verify-no-changes on the two projects it does touch exits 0. The repo pins no global.json, so this looks like SDK/analyzer drift on the runner image. Left out of this PR deliberately rather than smuggled in; it is sent separately as chore: remove unnecessary using to fix the Lint check #11085, which turns the whole-solution dotnet format --verify-no-changes green again. This PR's Lint check should pass once that merges.
  • test (macos-latest) fails only on XRefArchiveBuilderTest.TestDownload, which downloads http://dotnet.github.io/docfx/xrefmap.yml — a network flake. Every new test in this PR passed on that runner (Docfx.Build.Tests reported 117 passed / 1 failed / 3 skipped of 121, the one failure being TestDownload), and docfx.Snapshot.Tests was green there.
• The other build/template test projects are green too: Docfx.Build.ManagedReference.Tests, Docfx.Build.RestApi.Tests, Docfx.Build.RestApi.WithPlugins.Tests, Docfx.Build.SchemaDriven.Tests, Docfx.Build.UniversalReference.Tests, Docfx.Build.OverwriteDocuments.Tests, Docfx.Build.Common.Tests, and docfx.Tests (115/115).

🤖 Generated with Claude Code

https://claude.ai/code/session_016uV6H9cTntzsoKiaJRBn4f