GEP-27 reference implementation (including some parts currently planned for Groovy 7 but could possibly be moved forward)

[paulk-asert]

GEP-27: Compact Closure & Lambda Compilation — closure packing, capability analysis & typed static dispatch (draft/RFC)

Status

Draft for review of the GEP-27 direction. Two clean commits: the Groovy-6 lambda slice (GROOVY-12143, raised separately) as the base, and the larger closure-side work (originally scoped for Groovy 7) on top. Opt-in behind flags, off by default, byte-for-byte today's behaviour when disabled. Posted to validate the approach end-to-end and gather feedback.

What this delivers

Beyond the lambda leg (which eliminates the generated class for SAM lambdas — see GROOVY-12143), this branch implements closure packing: an eligible closure literal's body is hoisted into a synthetic method on the enclosing class and the literal is replaced by a single shared PackedClosure adapter — removing the per-closure generated class (and the deeply-nested $_closure1$_closure2$_closure3 name explosion) while the runtime value stays a real groovy.lang.Closure (so curry/memoize/trampoline/iteration keep working).

1. Capability analysis (PackStrategy, chooseStrategy). A single decision point selects FULL_CLASS (S0) vs PACKED_ADAPTER (S1) — shared vocabulary for the closure and lambda writers.

2. Two ways to trigger packing:

• @PackedClosures annotation (class/method) — the opt-in trust path (dynamic or static).
• Automatic under @CompileStatic — chooseStrategy proves delegate-independence using the type checker's own IMPLICIT_RECEIVER resolution paths (a path ending in owner is owner-resolved and safe; delegate, from @DelegatesTo/with, is not), so packing is sound with no annotation and no trust. Behind groovy.target.closure.pack.

3. Typed static dispatch (GEP-27 phase 4). The hoisted body reuses the expressions StaticCompilationVisitor already annotated, so under @CompileStatic it compiles statically — read-only captures pass as typed leading parameters, inferred types (@ClosureParams, implicit it) arrive via metadata, and the method takes the closure's inferred return type. Packed bodies are statically dispatched, zero invokedynamic — often fewer indy sites than today's closure class, which Reference-boxes captures as Object.

4. Written captures ride the compiler's existing holder machinery (the shared Reference passed as a parameter), so the body is never rewritten, compiles statically, and supports every write form (compound ops beyond +=, postfix value semantics) — replacing ~70 lines of AST-rewrite with fewer, more capable ones.

5. Soundness for the dynamic trust path:

• Runtime guard — PackedClosure fails fast (source-located message) if a delegate / delegate-consulting resolveStrategy is set on a trust-packed closure; a statically proven closure tolerates it as a no-op (as a @CompileStatic closure class does today).
• Compile-time escape decline — closures that visibly escape (stored to a field/property, returned, appended, in a collection literal — the Grails static constraints = { … } shape) stay classes.

6. Mechanism fidelity so a packed closure behaves like a generated closure class: correct owner/static dispatch in static methods; getParameterTypes() carries the real declared types; arguments adapted as reflective dispatch would (vararg collection, single List/Tuple destructuring, Closure→SAM / GString→String coercion); call() dispatches straight to doCall with invoker-exception unwrapping.

7. ClassNode.visitMethods — O(1) fixpoint. Visiting a method during code generation can add more (constructor-ref synthetic, lambda $deserializeLambda$, a hoisted body that hoists a nested one); dedup newly-added methods by identity via an IdentityHashMap set (O(1)) instead of ArrayList.removeAll (O(n²)). This fixes a compile-time freeze the O(n²) scan caused against method-generating visitors (e.g. groovy-contracts).

Flags

Flag	Effect	Default
groovy.target.lambda.hoist	SAM lambda hoisting (GROOVY-12143)	off (opt-in)
groovy.target.closure.pack	automatic @CompileStatic closure packing	off (opt-in)
@PackedClosures	annotation-scoped opt-in (dynamic/static)	n/a

The build forwards groovy.target.closure.pack to both the forked Groovy compiler (compile-time packing of sources) and the test JVM (runtime assertScript). Flags off ⇒ byte-for-byte today's behaviour.

Testing

• New: ClosurePackCapabilityTest, PackedClosuresTransformTest (delegate-independence proof, @DelegatesTo declines, auto-pack, zero-indy typed bodies, write forms, List/Tuple destructuring), plus LambdaHoistTest.
• Flag-on runtime stress: 0 failures across 816 tests (LambdaTest + every *Closure*/*DelegatesTo* static-compile suite + ReproducibleBytecodeBugs), diffed by failing-test set against the dynamic-body baseline (169 → 0 over the increments).
• Compile-time packing validated across 7 modules (flag reaching the compiler fork): groovy-core (678-test stress green), groovy-contracts (full suite green, incl. the case that previously froze), and groovy-json/groovy-xml/groovy-sql/groovy-templates/groovy-datetime (compileTestGroovy clean, zero VerifyErrors).
• Flag-off: 778 green (byte-identical); unit suites 22 green.

Known limitations / follow-ups

• RFC scope, not a merge request yet: this is the broad Groovy-7 direction; expect design discussion before any of it lands beyond the Groovy-6 lambda slice.
• Dispatch uses a cached reflective Method (prototype); an invokedynamic call site is the known performance upgrade.
• Nested-closure hoisting (packing a closure inside an already-packed body) is intentionally not pursued here — such nested closures stay classes, exactly as before their parent packed.
• A full clean test with the flag on across all ~60 modules, and interactive IDE/tooling verification (debugger step-into, breakpoints, variable inspection; coverage/decompiler tools), remain before any default flip.

Relationship to the lambda PR

Includes GROOVY-12143 as its base (lambda S2). If that lands first, this branch rebases to just the closure-side commit.

[testlens-app]

✅ All tests passed ✅

🏷️ Commit: d351610
▶️ Tests: 103560 executed
⚪️ Checks: 23/23 completed

---

Learn more about TestLens at testlens.app.