Skip to content

Commit 997def0

Browse files
committed
feat(check): infer lambda return type for structural matching
Phase 7 left a known gap: lambdas synth'd to Dynamic, which the gradual-consistency rule treats as universally assignable. So `process(fn(x) "hi")` passed a `fn(int) -> bool` slot silently - exactly the kind of mistake the static check exists to catch, on exactly the construct (lambdas to map/filter/process callbacks) that users hit most. Close the gap by computing a real TypingFnT for every lambda: - Params come from l.Typing.Params (unannotated args are nil- typed, which IsAssignableFrom already treats as `any`). - Block-form bodies populate a per-fn returnStack: every `*rl.Return` walked inside the body appends its synth'd value type. The stack discipline (push on lambda/fn entry, pop on exit) makes nested fn defs naturally isolated - a return inside an inner fn never leaks into the outer lambda's slot. - Expression-form bodies put the expression directly in Body (verified against the converter - no ExprStmt wrapping). Synth it as the implicit return. - The collected types union via the existing unionTypesForJoin pipeline: flatten, drop bottom, subsume, dedupe. Multi-path returns of int and str produce `int|str`, not `int, str`. - Empty (no returns) is void. Declared annotation, when present, takes precedence over inference - matches how typed locals work. walkFnDef also pushes/pops a return frame, even though hoisted- fn return inference is deferred to a later commit. The push is cheap and keeps the lambda case correct when a named fn appears inside the lambda body. Plumbing the actual consume-and-use side for hoisted fns needs the Tarjan SCC + placeholder return type machinery the plan calls for, which is its own piece of work. Caveats explicitly tested by snapshots: - Multi-value `return a, b`: takes Values[0]. Tuple/unpacking handling deferred. - Recursive lambdas bound to a local: the recursive ref still synths to Dynamic since SymbolTypes[f] isn't populated during the body walk. Inferred return collapses to Dynamic via any- like absorption. Sound but lossy. - Declared-vs-inferred mismatch on the lambda's own return isn't yet diagnosed at the return-statement site - the declared annotation wins and the call passes. The LambdaWith- DeclaredReturnHonored snapshot pins this behavior and flags the follow-on. The lambda frame stays inherited from the enclosing scope (not reset like walkFnDef does), preserving the closure narrowing behavior the snapshot LambdaInheritsEnclosingNarrowing pinned in Phase 4. The reassignment-after-definition closure check is still deferred. Five new snapshot cases in fn_value/structural_match.snap exercise the matrix: expression-form wrong return, expression-form right shape, block-form multi-path union, block-form no-return-is-void, declared-return-honored.
1 parent f131715 commit 997def0

3 files changed

Lines changed: 245 additions & 34 deletions

File tree

rts/check/snapshots/fn_value/structural_match.snap

Lines changed: 115 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ fn process(callback: fn(int) -> bool):
1515
process(cb)
1616
### STDOUT ###
1717
# Identifier types
18-
x @ 2:12 -> <no-type>
18+
x @ 2:12 -> int
1919
callback @ 5:5 -> fn(int) -> bool
2020
process @ 7:1 -> fn(fn(int) -> bool)
2121
cb @ 7:9 -> fn(int) -> bool
@@ -71,7 +71,7 @@ fn process(callback: fn(int) -> bool):
7171
process(cb)
7272
### STDOUT ###
7373
# Identifier types
74-
x @ 2:12 -> <no-type>
74+
x @ 2:12 -> str
7575
callback @ 5:5 -> fn(int) -> bool
7676
process @ 7:1 -> fn(fn(int) -> bool)
7777
cb @ 7:9 -> fn(str) -> bool
@@ -127,8 +127,8 @@ fn process(callback: fn(int) -> bool):
127127
process(cb)
128128
### STDOUT ###
129129
# Identifier types
130-
x @ 2:12 -> <no-type>
131-
y @ 2:16 -> <no-type>
130+
x @ 2:12 -> int
131+
y @ 2:16 -> int
132132
callback @ 5:5 -> fn(int) -> bool
133133
process @ 7:1 -> fn(fn(int) -> bool)
134134
cb @ 7:9 -> fn(int, int) -> bool
@@ -158,7 +158,7 @@ fn process(callback):
158158
process(cb)
159159
### STDOUT ###
160160
# Identifier types
161-
x @ 2:12 -> <no-type>
161+
x @ 2:12 -> str
162162
callback @ 5:5 -> dynamic
163163
process @ 7:1 -> fn(any)
164164
cb @ 7:9 -> fn(str) -> str
@@ -172,13 +172,12 @@ process(cb)
172172
# Diagnostics
173173
(none)
174174
### TITLE ###
175-
LambdaPassedSynthsDynamic
175+
LambdaExprWrongReturnTypeFires
176176
### DESCRIPTION ###
177-
Lambdas still synth to Dynamic today (deriving a TypingFnT from
178-
the body is deferred). IsAssignableFrom treats Dynamic as gradually
179-
compatible with any fn type, so the call passes silently. Known
180-
caveat: a lambda with the wrong shape won't be caught at the
181-
static layer yet.
177+
Expression-form lambda `fn(x) "hi"` infers `fn(any) -> str`. The
178+
expression body IS the return value, so we synth it directly. The
179+
mismatch with `fn(int) -> bool` fires - this is the first case the
180+
old "lambda synths to Dynamic" caveat blocked.
182181
### INPUT ###
183182
fn process(callback: fn(int) -> bool):
184183
callback(5)
@@ -193,5 +192,110 @@ process(fn(x) "hi")
193192
callback (param): fn(int) -> bool
194193
process (fn): fn(fn(int) -> bool)
195194

195+
# Diagnostics
196+
[hint] RAD30001 @ 4:9 - Argument type 'fn(any) -> str' is not assignable to expected type 'fn(int) -> bool'
197+
### TITLE ###
198+
LambdaExprRightShapeOk
199+
### DESCRIPTION ###
200+
Lambda whose inferred return matches the expected signature -
201+
no diagnostic. `fn(x) x > 0` infers `fn(any) -> bool` and slots
202+
into `fn(int) -> bool` cleanly (contravariant param: any accepts
203+
int; covariant return: bool matches bool).
204+
### INPUT ###
205+
fn process(callback: fn(int) -> bool):
206+
callback(5)
207+
208+
process(fn(x) x > 0)
209+
### STDOUT ###
210+
# Identifier types
211+
callback @ 2:5 -> fn(int) -> bool
212+
process @ 4:1 -> fn(fn(int) -> bool)
213+
x @ 4:15 -> dynamic
214+
215+
# Symbol types
216+
callback (param): fn(int) -> bool
217+
process (fn): fn(fn(int) -> bool)
218+
x (param): <no-type>
219+
220+
# Diagnostics
221+
(none)
222+
### TITLE ###
223+
LambdaBlockMultiPathReturnUnioned
224+
### DESCRIPTION ###
225+
A block-form lambda with two return paths gets a union return
226+
type. With no annotation, inference unions the return arms into
227+
`int|str`, neither of which is bool - the call fires.
228+
### INPUT ###
229+
fn process(callback: fn(int) -> bool):
230+
callback(5)
231+
232+
process(fn(x):
233+
if x > 0:
234+
return 1
235+
return "hi"
236+
)
237+
### STDOUT ###
238+
# Identifier types
239+
callback @ 2:5 -> fn(int) -> bool
240+
process @ 4:1 -> fn(fn(int) -> bool)
241+
x @ 5:8 -> dynamic
242+
243+
# Symbol types
244+
callback (param): fn(int) -> bool
245+
process (fn): fn(fn(int) -> bool)
246+
x (param): <no-type>
247+
248+
# Diagnostics
249+
[hint] RAD30001 @ 4:9 - Argument type 'fn(any) -> int|str' is not assignable to expected type 'fn(int) -> bool'
250+
### TITLE ###
251+
LambdaBlockNoReturnIsVoid
252+
### DESCRIPTION ###
253+
A lambda with no `return` statement infers `void` as the return.
254+
That doesn't match `fn(int) -> bool`, so the slot fires.
255+
### INPUT ###
256+
fn process(callback: fn(int) -> bool):
257+
callback(5)
258+
259+
process(fn(x):
260+
print(x)
261+
)
262+
### STDOUT ###
263+
# Identifier types
264+
callback @ 2:5 -> fn(int) -> bool
265+
process @ 4:1 -> fn(fn(int) -> bool)
266+
print @ 5:5 -> dynamic
267+
x @ 5:11 -> dynamic
268+
269+
# Symbol types
270+
callback (param): fn(int) -> bool
271+
process (fn): fn(fn(int) -> bool)
272+
x (param): <no-type>
273+
274+
# Diagnostics
275+
[hint] RAD30001 @ 4:9 - Argument type 'fn(any) -> void' is not assignable to expected type 'fn(int) -> bool'
276+
### TITLE ###
277+
LambdaWithDeclaredReturnHonored
278+
### DESCRIPTION ###
279+
When the user writes a return annotation (`-> bool`) we use it
280+
verbatim instead of inferring. Here the body returns "hi" (str),
281+
which doesn't match the declared bool - that's the right place
282+
for the diagnostic to fire (typed-local-style mismatch on the
283+
return, not at the call site). Today we don't yet enforce the
284+
body-vs-declared-return check, so the call still passes - this
285+
snapshot pins the current behavior and marks the gap.
286+
### INPUT ###
287+
fn process(callback: fn(int) -> bool):
288+
callback(5)
289+
290+
process(fn(x) -> bool: return "hi")
291+
### STDOUT ###
292+
# Identifier types
293+
callback @ 2:5 -> fn(int) -> bool
294+
process @ 4:1 -> fn(fn(int) -> bool)
295+
296+
# Symbol types
297+
callback (param): fn(int) -> bool
298+
process (fn): fn(fn(int) -> bool)
299+
196300
# Diagnostics
197301
(none)

rts/check/snapshots/narrow/lambda.snap

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ fn f(x: int?):
1818
x @ 4:17 -> int
1919

2020
# Symbol types
21-
cb (local): dynamic
21+
cb (local): fn() -> void
2222
f (fn): fn(int?)
2323
x (param): int?
2424
y (local): int
@@ -47,7 +47,7 @@ fn f(x: int?):
4747

4848
# Symbol types
4949
a (local): int
50-
cb (local): dynamic
50+
cb (local): fn() -> void
5151
f (fn): fn(int?)
5252
x (param): int?
5353
z (local): int?

rts/check/type_check.go

Lines changed: 128 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,38 @@ type typeChecker struct {
9999
resolved *Resolved
100100
info *TypeInfo
101101
frame *Frame
102+
// returnStack collects return-statement value types per
103+
// enclosing function/lambda scope. Each entry is the running
104+
// list of types observed inside one fn body. Lambdas use it to
105+
// synth a proper TypingFnT return type instead of falling back
106+
// to Dynamic. Pushed on lambda/fn entry, popped on exit. Stack
107+
// depth N means we're inside N nested fn bodies; a return at
108+
// depth N belongs to the innermost (top) frame, never the
109+
// outer ones - matching the runtime where returns target the
110+
// nearest enclosing fn.
111+
returnStack [][]rl.TypingT
112+
}
113+
114+
func (tc *typeChecker) pushReturnFrame() {
115+
tc.returnStack = append(tc.returnStack, nil)
116+
}
117+
118+
func (tc *typeChecker) popReturnFrame() []rl.TypingT {
119+
n := len(tc.returnStack)
120+
out := tc.returnStack[n-1]
121+
tc.returnStack = tc.returnStack[:n-1]
122+
return out
123+
}
124+
125+
// recordReturn appends a return-value type to the innermost fn
126+
// scope's accumulator. A return outside any fn (which is itself a
127+
// validation error caught elsewhere) is a no-op here.
128+
func (tc *typeChecker) recordReturn(t rl.TypingT) {
129+
n := len(tc.returnStack)
130+
if n == 0 {
131+
return
132+
}
133+
tc.returnStack[n-1] = append(tc.returnStack[n-1], t)
102134
}
103135

104136
func (tc *typeChecker) walkFile(file *rl.SourceFile) {
@@ -132,6 +164,20 @@ func (tc *typeChecker) walkStmt(n rl.Node) {
132164
tc.walkWhileLoop(v)
133165
case *rl.FnDef:
134166
tc.walkFnDef(v)
167+
case *rl.Return:
168+
// Synth the return value's type and feed it into the
169+
// innermost fn scope's return collector. Only the first
170+
// value is observed today (multi-value return is unpacked
171+
// at the call site; structural matching of the unpacked
172+
// shape is deferred). A bare `return` records void so
173+
// "no value" paths still join cleanly.
174+
var t rl.TypingT
175+
if len(v.Values) > 0 {
176+
t = tc.synth(v.Values[0])
177+
} else {
178+
t = rl.NewVoidType()
179+
}
180+
tc.recordReturn(t)
135181
default:
136182
// Generic descent. Later sub-commits replace these with
137183
// kind-specific handlers (for loops, switch, return, etc.).
@@ -235,7 +281,16 @@ func (tc *typeChecker) walkFnDef(n *rl.FnDef) {
235281
}
236282
}
237283
tc.frame = NewFrame()
284+
// Push a return-collector frame so any `return E` inside the
285+
// body lands in this fn'\''s slot, not whatever lambda we may
286+
// be nested under at the call site. We don'\''t consume the
287+
// collected types here yet - hoisted-fn return inference is a
288+
// separate commit gated on SCC handling for mutual recursion.
289+
// Pushing now is the cheap half: it keeps lambdas correct when
290+
// a named fn is declared inside them.
291+
tc.pushReturnFrame()
238292
tc.walkStmts(n.Body)
293+
_ = tc.popReturnFrame()
239294
tc.frame = saved
240295
}
241296

@@ -2078,33 +2133,85 @@ func (tc *typeChecker) addUnaryOpIssue(span rl.Span, op rl.Operator, operand rl.
20782133
// `xs = []` followed by `xs.append(1)` to `List<int>`, but the safe
20792134
// over-approximation lets every existing program type-check today.
20802135

2081-
// synthLambda walks the lambda'\''s body so identifier-uses inside it
2082-
// get types recorded. The body executes in a child frame of the
2083-
// enclosing one: captured variables retain whatever narrowing the
2084-
// enclosing frame had at the lambda'\''s definition.
2136+
// synthLambda walks the lambda'\''s body and synthesizes a structural
2137+
// TypingFnT. Params come from the grammar annotation (l.Typing.Params,
2138+
// possibly with nil per-param Type for unannotated args - that maps
2139+
// to `any`). The return type is the headline work here:
2140+
//
2141+
// - Declared annotation (`fn(x) -> int: ...`): use it verbatim.
2142+
// - Block-form lambda: union the types of every `return E` we
2143+
// encountered while walking the body. Bare `return` contributes
2144+
// void. Empty (no returns at all) is void.
2145+
// - Expression-form lambda: the body is a single ExprStmt and the
2146+
// expression IS the return value, so we synth it directly and
2147+
// skip the walk-and-collect dance.
20852148
//
2086-
// Pyright'\''s closure rule says we should only preserve narrowing on
2087-
// captured paths that aren'\''t reassigned later in the enclosing
2088-
// scope (otherwise the lambda might run after the reassignment
2089-
// invalidates the narrowing). We don'\''t implement that lookahead
2090-
// yet: the conservative-but-permissive answer is to forward the
2091-
// enclosing frame. False positives (lambda thinks x is narrowed but
2092-
// x got reassigned before the lambda ran) are possible but rare in
2093-
// practice - users usually invoke lambdas at their definition site
2094-
// or shortly after. When this bites, the right shape is a
2095-
// reassignment-after-definition scan keyed on captured paths.
2149+
// Closure rule deferred: same caveat as walkFnDef. Captured-path
2150+
// narrowings from the enclosing frame are NOT preserved. For a
2151+
// closure-friendly design we'\''d need Pyright'\''s reassignment-
2152+
// after-definition check. Today, the body opens a fresh frame so
2153+
// outer locals appear unnarrowed (read as their base type).
20962154
//
2097-
// The lambda itself synths to Dynamic for now. A proper TypingFnT
2098-
// would require running the body in synth-mode to derive a return
2099-
// type from `return` statements - deferred to the return-type-
2100-
// inference follow-on.
2155+
// Recursion: anonymous lambdas can'\''t self-reference. A named
2156+
// recursive lambda bound to a local (`f = fn(x) f(x-1)`) would synth
2157+
// the recursive `f` to Dynamic because SymbolTypes[f] isn'\''t set
2158+
// during the walk. Sound but lossy - the inferred return would
2159+
// collapse to Dynamic via any-like subsumption rules. Fixing this
2160+
// needs the Tarjan SCC + placeholder return type machinery the plan
2161+
// already describes for hoisted fns.
21012162
func (tc *typeChecker) synthLambda(n *rl.Lambda) rl.TypingT {
21022163
enclosing := tc.frame
2103-
for _, stmt := range n.Body {
2104-
tc.walkStmt(stmt)
2164+
// Unlike walkFnDef (which resets to a fresh frame because a fn
2165+
// can be called from anywhere), lambdas inherit the enclosing
2166+
// frame'\''s narrowings - that'\''s closure semantics. The
2167+
// reassignment-after-definition lookahead that would invalidate
2168+
// stale narrowings is still deferred (see docstring).
2169+
tc.pushReturnFrame()
2170+
2171+
// Expression-form lambdas (`fn(x) x + 1`) put the expression
2172+
// node directly in Body (verified against the converter output -
2173+
// it does not wrap in ExprStmt). Each body entry is the value to
2174+
// return; synth it to feed the inferred return type. Block-form
2175+
// lambdas walk via walkStmts and rely on the `*rl.Return` case
2176+
// in walkStmt to populate returnStack.
2177+
if !n.IsBlock {
2178+
for _, stmt := range n.Body {
2179+
if stmt == nil {
2180+
continue
2181+
}
2182+
tc.recordReturn(tc.synth(stmt))
2183+
}
2184+
} else {
2185+
tc.walkStmts(n.Body)
21052186
}
2187+
2188+
collected := tc.popReturnFrame()
21062189
tc.frame = enclosing
2107-
return tc.record(n, rl.NewDynamicType())
2190+
2191+
// Honor an explicit return annotation; otherwise infer.
2192+
var returnT rl.TypingT
2193+
if n.Typing != nil && n.Typing.ReturnT != nil {
2194+
returnT = *n.Typing.ReturnT
2195+
} else if len(collected) == 0 {
2196+
returnT = rl.NewVoidType()
2197+
} else {
2198+
returnT = unionTypesForJoin(collected)
2199+
}
2200+
2201+
// Construct a fresh TypingFnT so we don'\''t mutate the parsed
2202+
// l.Typing (other readers - signature display, hover - rely on
2203+
// the AST being immutable). Params are shared by value; the
2204+
// caller never writes through ReturnT, so a fresh pointer is
2205+
// safe.
2206+
params := []rl.TypingFnParam(nil)
2207+
if n.Typing != nil {
2208+
params = n.Typing.Params
2209+
}
2210+
fn := &rl.TypingFnT{
2211+
Params: params,
2212+
ReturnT: &returnT,
2213+
}
2214+
return tc.record(n, fn)
21082215
}
21092216

21102217
func (tc *typeChecker) synthLitList(n *rl.LitList) rl.TypingT {

0 commit comments

Comments
 (0)