Skip to content

edu/ast_lol: ast.lol — auto-graded AST course from expressions to a SQL optimizer - #1451

Merged
aaylward merged 4 commits into
mainfrom
claude/ast-parsing-learning-app-qli1yg
Aug 25, 2026
Merged

edu/ast_lol: ast.lol — auto-graded AST course from expressions to a SQL optimizer#1451
aaylward merged 4 commits into
mainfrom
claude/ast-parsing-learning-app-qli1yg

Conversation

@aaylward

Copy link
Copy Markdown
Collaborator

New client-side web app for ast.lol (Cloudflare Workers, same stack as 1d4_web) at domains/edu/apps/ast_lol: a tiered course on AST parsing and transformation for experienced programmers, building to SQL query parsing and rule-based plan optimization. 6 tiers, 13 lessons, 16 challenges; Tiers 1–2 build a toy expression language, Tiers 3–5 build "AstQL" (tokenize → Pratt parse → resolve → plan → execute with 3VL → optimize). The capstone optimizer is graded on row-for-row equivalence on two databases plus a hard cost budget from an instrumented executor.

The grader runs submissions in a sandboxed Web Worker (timeout by termination); the same harness runs under vitest, which is what makes the CI contract real: every reference solution passes its own bank, every starter fails, custom-test placeholders build inputs the oracle accepts, capstone budget literals match the formula, and the SQL parser replays a frozen corpus. Failure output is the product: first-difference paths, per-test console capture, error lines mapped to user code, per-test hints, and user-authored custom tests graded by the reference solution as oracle.

What a reviewer can't see from the diff:

  • The review panel ran against d5b2de3; 0f3060c carries its surviving findings (degenerate bench-DB RNG, AND/OR/NOT coercion soundness, an uncloneable-args false pass in the grader, capstone budgets moved off the module-load path). Findings kept deliberately are pinned by tests and recorded in docs/CURRICULUM.md.
  • Verified beyond vitest: Playwright E2E in the real Workers runtime (grading pass/fail, custom tests, capstone, mobile overflow, progress persistence).
  • Cloudflare deploy config is included but the dashboard/DNS hookup for ast.lol is manual (README).

Deliberately not covered (see docs/ROADMAP.md): TypeScript/functional-language submissions (the grader's language field is the seam), code splitting, GROUP BY/subqueries/outer joins.

Also touches: .github/workflows/branch.yml (new test-ast-lol job mirroring the other web apps) and the web-app row in docs/WORKING_AGREEMENT.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01C7snNHsXg6msYZkBUHsViv


Generated by Claude Code

claude added 2 commits August 25, 2026 01:46
New client-side web app for ast.lol (Cloudflare Workers, same stack as
1d4_web): 6 tiers, 13 lessons, 16 challenges building from an expression
language to AstQL — a SQL subset with resolution, logical plans, an
instrumented executor, and a capstone optimizer graded on result
equivalence plus a cost budget. Submissions run in a sandboxed Web
Worker; failures report first-difference paths, per-test console output,
mapped error lines, and hints; users can add oracle-graded custom tests.
CI job test-ast-lol runs typecheck, vitest (224 tests incl. a frozen SQL
parser corpus), and the build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7snNHsXg6msYZkBUHsViv
benchDb: replace the raw LCG (its low bit alternates, confining even-modulus
columns to half their domains and emptying two capstone queries) with
mulberry32; totals now derive from the ordered product. Executor: AND/OR/NOT
coerce operands consistently, restoring TRUE AND x -> x soundness; pinned by
new soundness and coercion tests. Grader: validate structuredClone before the
grading try (uncloneable custom args passed any submission); worker-creation
and rejection paths surface as error reports. Capstone: budgets are literals
pinned to the formula by CI (nothing executes plans at module load; q1 is an
explicit no-regression guard), and the solution is the composition of the
three pass solutions. Plus doc/test-name accuracy fixes; 259 tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7snNHsXg6msYZkBUHsViv
Comment thread domains/edu/apps/ast_lol/src/grader/harness.ts Dismissed
Comment thread domains/edu/apps/ast_lol/src/grader/harness.ts Dismissed
The two new Function calls are the product — the learner's own code and
test inputs, run in their own browser's worker, with no server and no
cross-user boundary. Documented where CodeQL flags them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7snNHsXg6msYZkBUHsViv

Copy link
Copy Markdown
Collaborator Author

On the two CodeQL js/code-injection alerts (#9, #10, both in src/grader/harness.ts): these flag the app's core function rather than a vulnerability, and I'm deliberately not "fixing" them in code.

Both sites execute the learner's own input in the learner's own browser — the submission, and their custom-test inputs — inside a Web Worker, same trust model as jsfiddle/codepen/leetcode-style client-side runners. There is no server-side execution, no persistence beyond that browser's localStorage, and no path by which one user's code reaches another user. Removing the dynamic evaluation would remove the product; no sanitization makes new Function acceptable to this query. 8098c55 documents the threat model at both sites.

Recommended resolution: dismiss both alerts as won't fix (by design) in the code-scanning UI, which turns the CodeQL check green on the next run. The alternative — a path exclusion for src/grader/ in codeql.yml — would also silence genuine future findings in that file, and changing the security workflow config seemed like your call, not mine. (The check's "2 configurations not found" warning for go/java-kotlin is just this PR touching neither language.)


Generated by Claude Code

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 25, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
1d4-web 452f9dc Commit Preview URL

Branch Preview URL
Aug 25 2026, 10:03 AM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 25, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
iili 452f9dc Commit Preview URL

Branch Preview URL
Aug 25 2026, 10:03 AM

@cursor

cursor Bot commented Aug 25, 2026

Copy link
Copy Markdown

Review (Bugbot + manual pass)

CI is green (21/21). Bugbot on the PR head reported no findings. A deeper pass over the grader client, ChallengeView drafts, SQL engine, and capstone found three issues worth folding in before merge — none block the curriculum/engine core, but two can mislead learners and one can mis-report timeouts near the budget.

Survivors (see line threads):

  1. runGrader timeout path never clears running on test-result — a timer firing in the gap after the last result (or between tests) can mark an already-completed test as timeout, duplicate its id, and flip the report status even though the worker finished.
  2. Timeout/skip rows hardcode custom: false — hung/skipped custom tests lose the “yours” badge.
  3. Draft effect ignores return-to-starter — editing back to the starter text (without Reset) leaves a stale localStorage draft that reappears on remount.

Looked at and not raised: empty-column Project from prune under unused join sides (allowed by validatePlan, equivalence holds); CodeQL js/code-injection on the grader Function sites (threat model already stated in-code and on the PR); setState-after-navigate mid-grade (low; remount key contains bleed).

Panel note: prior panel findings on d5b2de3 look folded into 0f3060c as described in the PR body; this pass is on 8098c55.

Comment thread domains/edu/apps/ast_lol/src/grader/client.ts Outdated
Comment thread domains/edu/apps/ast_lol/src/views/ChallengeView.tsx Outdated
Comment thread domains/edu/apps/ast_lol/src/grader/client.ts
A timeout firing after a test's result no longer blames the finished test
(running clears on test-result, and the synthesized row is guarded against
completed ids). Timeout and skipped rows keep custom-test flags, so the
'yours' badge survives a hang. Editing back to the exact starter clears
the stored draft instead of resurrecting it on the next visit. All three
pinned by tests; 262 total.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7snNHsXg6msYZkBUHsViv

Copy link
Copy Markdown
Collaborator Author

All three findings verified and fixed in 452f9dc, each with a pinning test:

  1. Timeout mis-blamerunning now clears on test-result, and the synthesized timeout row is additionally guarded against ids already in completed, so a test that finishes just under the wire stays passed and no id appears twice. New test drives the exact window (result applied, timer fires before done).
  2. Custom flags on timeout/skip rowsplannedTests entries carry custom, and both synthesized row kinds use it, so the "yours" badge survives a hang.
  3. Edit-back-to-starter — the draft debounce now clears the stored draft when the editor content equals the starter, instead of only on Reset.

Generated by Claude Code

@cursor

cursor Bot commented Aug 25, 2026

Copy link
Copy Markdown

452f9dc closes the three review findings. Each has a pinning test; client.test.ts + views.test.tsx pass locally (19/19), and test-ast-lol is already green on this head. Remaining CI is the C++ sanitizer / full-repo jobs, unrelated to this diff. Nothing further from this pass.

@cursor

cursor Bot commented Aug 25, 2026

Copy link
Copy Markdown

CI on 452f9dc is green — 21/21, including test-ast-lol. The three review findings stay closed.

@aaylward
aaylward merged commit 2b269ef into main Aug 25, 2026
21 checks passed
@aaylward
aaylward deleted the claude/ast-parsing-learning-app-qli1yg branch August 25, 2026 11:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants