You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Pre-existing integration test failures surfaced by #618
PR #619 fixes issue #618 (protocol script/eval was reporting success: true on compile errors due to langruntraperror's trap-and-continue behavior + a missed fllangerror signal). Fixing that surfaces ~107 integration tests that were silently green on develop because the eval-trap masked compile errors AND silent-success outcomes.
These are pre-existing bugs, not regressions from #618. The eval-trap was hiding them. Filing this as the umbrella issue to triage them.
Categories (counts after #618 + #586 + yaml fixture cleanup)
Category
Count
Sample failure
Likely root cause class
tcp
52
tcp.countConnections - track stream lifecycle returns "false" because during=2 (counts listener + stream) where test expects during=1 (stream only)
Verb-vs-test semantic mismatch. Need to decide: does countConnections count listeners?
menu
16
menu.setCommandKey("S") fails because headless overload signature is (@leaf, 'X'), not (key)
Test fixtures written for Mac-UI cursor-based API; need rewrite for headless overload
HTTP
7
webserver/inetd request handling
needs investigation
html
5
sanitization / framework
needs investigation
inetd
4
listener-related
needs investigation
webserver
3
response-related
needs investigation
cursor
3
menu cursor navigation
likely same as menu
persistence
2
db.save/db.load of menu data
menu serialization round-trip
workflow
2
end-to-end menu
composite
misc
13
mainresponder, html.entityencode, deep tests
scattered
Recommended triage approach
Per-category PR or worktree. Don't try to fix them all in one chain — each category likely has its own root cause family.
Until then
Integration suite should NOT block #619 / #617 from landing — the failures pre-existed and the eval-trap is a worse hazard than visible red tests. Two options to keep CI green:
Mark the 107 tests as skip: with reason "blocked on #N for category-X cleanup" (where N is a per-category sub-issue of this). Same pattern Frontier already uses for 188 prior skipped tests.
Pre-existing integration test failures surfaced by #618
PR #619 fixes issue #618 (protocol script/eval was reporting success: true on compile errors due to
langruntraperror's trap-and-continue behavior + a missedfllangerrorsignal). Fixing that surfaces ~107 integration tests that were silently green on develop because the eval-trap masked compile errors AND silent-success outcomes.These are pre-existing bugs, not regressions from #618. The eval-trap was hiding them. Filing this as the umbrella issue to triage them.
Categories (counts after #618 + #586 + yaml fixture cleanup)
tcp.countConnections - track stream lifecyclereturns "false" becauseduring=2(counts listener + stream) where test expectsduring=1(stream only)countConnectionscount listeners?menu.setCommandKey("S")fails because headless overload signature is(@leaf, 'X'), not(key)db.save/db.loadof menu dataRecommended triage approach
Per-category PR or worktree. Don't try to fix them all in one chain — each category likely has its own root cause family.
Until then
Integration suite should NOT block #619 / #617 from landing — the failures pre-existed and the eval-trap is a worse hazard than visible red tests. Two options to keep CI green:
skip:with reason "blocked on #N for category-X cleanup" (where N is a per-category sub-issue of this). Same pattern Frontier already uses for 188 prior skipped tests.Recommended: option 1, because it makes the debt visible AND auditable.
Source of analysis
Labels
tech-debt, P1, scope/large, testing