Skip to content

Burndown: ~107 pre-existing integration test failures surfaced by #618 fix #620

Description

@jsavin

Pre-existing integration test failures surfaced by #618

PR #619 fixes issue #618 (protocol script/eval was reporting success: true on compile errors due to langruntraperror's trap-and-continue behavior + a missed fllangerror signal). Fixing that surfaces ~107 integration tests that were silently green on develop because the eval-trap masked compile errors AND silent-success outcomes.

These are pre-existing bugs, not regressions from #618. The eval-trap was hiding them. Filing this as the umbrella issue to triage them.

Categories (counts after #618 + #586 + yaml fixture cleanup)

Category Count Sample failure Likely root cause class
tcp 52 tcp.countConnections - track stream lifecycle returns "false" because during=2 (counts listener + stream) where test expects during=1 (stream only) Verb-vs-test semantic mismatch. Need to decide: does countConnections count listeners?
menu 16 menu.setCommandKey("S") fails because headless overload signature is (@leaf, 'X'), not (key) Test fixtures written for Mac-UI cursor-based API; need rewrite for headless overload
HTTP 7 webserver/inetd request handling needs investigation
html 5 sanitization / framework needs investigation
inetd 4 listener-related needs investigation
webserver 3 response-related needs investigation
cursor 3 menu cursor navigation likely same as menu
persistence 2 db.save/db.load of menu data menu serialization round-trip
workflow 2 end-to-end menu composite
misc 13 mainresponder, html.entityencode, deep tests scattered

Recommended triage approach

Per-category PR or worktree. Don't try to fix them all in one chain — each category likely has its own root cause family.

Until then

Integration suite should NOT block #619 / #617 from landing — the failures pre-existed and the eval-trap is a worse hazard than visible red tests. Two options to keep CI green:

  1. Mark the 107 tests as skip: with reason "blocked on #N for category-X cleanup" (where N is a per-category sub-issue of this). Same pattern Frontier already uses for 188 prior skipped tests.
  2. Accept temporary red — land fix(lang+parser): honor fllangerror (#618) + bare-LF parser fix (#586) + test cleanup #619 + fix(parser): bare-LF support in // comments and whitespace (#586) — DRAFT, do not merge #617, treat the 107 as the new floor, work them down PR by PR.

Recommended: option 1, because it makes the debt visible AND auditable.

Source of analysis

Labels

tech-debt, P1, scope/large, testing

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority/p1High priority - address soonscope/largeMulti-week effort, significant design and implementationtype/testTest infrastructure, test cases, or test-related changes

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions