benchctl: clargs + sqlite_linq rewrite, sql/ benchmarks, where+count fold

[borisbat]

Three coupled changes that fall out of modernizing utils/benchctl/ onto the new SQL surface.

1. utils/benchctl/ rewrite

• Hand-rolled argv parsing (flags.das) replaced with daslib/clargs — single [CommandLineArgs] struct, positional command + files, structured filter flags. Help via -? (script-host eats --help).
• Raw sqlite3_* calls in bench_sql.das replaced with [sql_table] Benchmark, with_sqlite RAII, db |> insert(rows) bulk transactions, _sql(... |> select_from(type<Benchmark>) |> _where(...)) for both query and compare.
• --select / --select_old / --select_new raw-WHERE flags removed. Replaced with structured composable filters: --commit, --tag (insert-side: repeatable; query-side: single-value, bracket-free), --old-commit, --old-tag, --new-commit, --new-tag. Filters compose with AND via the _sql || empty-string short-circuit pattern (one call site, all flag combinations). Workflow becomes compare --old-tag before --new-tag after.
• --colors false → --no-color (clargs convention).
• For one-off filters the structured flags don't cover (string_allocs > 0, etc.) — README points users at the sqlite3 shell directly. No raw-SQL escape hatch in the tool itself.
• bench_sql.das and flags.das deleted; bench_table.das (the [sql_table] struct) and bench_args.das (clargs wrapper) added. BenchmarkEntry collapses into the [sql_table] Benchmark struct.
• Side fixes: added options gen2 to utils.das and table_fmt.das (transitive macro requirements force gen2 parser mode in callers); read_file opens in "rb" mode (Windows CRLF was tripping fread's byte-count check on the JSON ingest path).

Behavioral hardening (review round 2)

• run_subcommand validates cmd against {reset, insert, query, compare} before opening the SQLite DB. Typos like qurey no longer create an empty benchdata.db as a side effect.
• validate_tag_chars helper rejects [ / ] in tag values consistently across insert/query/compare (insert-side originally; now query and compare too). Empty tags are skipped on insert.
• --tag for query rejects multiplicity > 1 with a clear error (insert-side --tag remains repeatable; compare-side flags are scalar already).
• run_compare_cmd post-filters --new rows against --old IDs (table<int;bool> lookup) — restores the overlap-exclusion semantic the old code had via WHERE id NOT IN (...). Without it, identical/empty filters on both sides would compare a result set against itself.
• run_reset_cmd uses try_drop_table_if_exists / try_create_table and returns an error string; the implicit-init path threads the error through run_subcommand instead of crashing on a corrupt or locked DB.

Workaround for an upstream typer bug

bench_args.das exists because combining daslib/clargs and sqlite/sqlite_boost in the same module makes Option<string> ambiguous in instantiation contexts — three mod-aliased views of the same daslib/option.das:30 template collide. Filed as #2598 with a minimal repro. The wrapper requires clargs privately and returns a plain struct with no Option fields, keeping clargs::Option out of main.das's namespace.

2. benchmarks/sql/ (new)

Comparison suite mirroring tests/dasSQLITE/parity_check_*.das but oriented to throughput. Four files (select_where, select_where_order_take, count_aggregate, indexed_lookup) plus a _common.das fixture with a [sql_table] Car struct + fixture_db / fixture_array helpers. Three modes per file:

Mode	Source	Form
m1	:memory: SQLite	_sql — compile-time SQL emission, work pushed to the engine
m3	pre-populated array<Car>	plain LINQ chain — materializes intermediate filter/sort arrays
m3f	pre-populated array<Car>	_fold from daslib/linq_boost — fuses the chain into a single pass

File	Headline finding
select_where.das	Filter chain over 10K rows. Modest asymmetry.
select_where_order_take.das	Filter + sort + limit. SQL ORDER BY + LIMIT bounds work; m3 sorts the full filtered set.
count_aggregate.das	count() after _where over 1M rows. m3f wins 12× over _sql and 4× over m3 — the where+count fold beats SQLite's COUNT(*) roundtrip on already-materialized data.
indexed_lookup.das	_where(_.id == K) against PRIMARY KEY over 1M rows. Inverse-asymmetry: SQLite wins by ~1000× (m1 ≈ 3.1 µs, m3 ≈ 9 ms, m3f ≈ 3.4 ms per lookup) — daslang has no indexed-lookup primitive on array<T>, so it pays full O(n). Documents where indexed storage earns its keep.

3. daslib/linq_boost — where + count fold

New ["where_", "count"] FoldSequence + fold_where_count macro function. Emits an inlined single-pass loop (var n = 0; for it in src; if <pred> n++; return n) with the predicate spliced via fold_linq_cond. No intermediate filter array, no block-call overhead.

Numbers from count_aggregate_1m_m3f (1M Cars, predicate match rate ~50%):

	INTERP	JIT
Before fold rule (materializing where)	25.5 ns/op, 19.7 B/op	—
With fold rule (helper-fn first attempt, block call)	28.4 ns/op, 0 B/op	—
With fold rule (inlined loop, predicate spliced)	8 ns/op, 1 B/op	3 ns/op, 1 B/op

JIT-compiled m3f at 3 ns/op is ~12× faster than _sql (38 ns) at 1M rows. Per-element cost is flat from 100K → 1M; all three modes scale linearly.

Regression test in tests/linq/test_linq_fold.das::test_where_count_fold covers half-match, zero-match, all-match, and empty-source cases.

Test plan

• Lint clean (utils/lint/main.das on every changed .das file)
• Full tests/ suite (7944 passed, 0 failed/errors) — no regressions in tests/linq/ or tests/dasSQLITE/
• AOT suite (7338 passed, 0 failed/errors)
• All .das files MCP-formatted
• Detect-dupe scan against daslib/utils/benchmarks corpus — no actionable findings
• End-to-end benchctl flow: reset → insert --tag before → re-run → insert --tag after → compare --old-tag before --new-tag after produces clean Welch's t-test output
• Subcommand-typo smoke test: qurey rejected without creating benchdata.db

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR modernizes utils/benchctl to use the newer daslib/clargs and typed sqlite_* APIs, adds a dedicated SQL/LINQ throughput benchmark suite, and introduces a new _fold optimization that fuses where + count into a single pass.

Changes:

• Rewrites benchctl CLI parsing to daslib/clargs and replaces raw sqlite3_* calls with [sql_table] + SqlRunner + select_from/_where/_sql.
• Adds benchmarks/sql/ suite comparing _sql vs non-macro DB LINQ vs pure in-memory LINQ (materializing vs _fold).
• Adds a new daslib/linq_boost fold rule to fuse _where(p) |> count() into an inlined single-pass counter loop.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
utils/benchctl/utils.das	Enables gen2 and reads input files in binary mode for cross-platform JSON ingest consistency.
utils/benchctl/table_fmt.das	Enables gen2 for benchctl table formatting helpers.
utils/benchctl/README.md	Updates docs for structured filters, --no-color, and removal of raw SQL selection flags.
utils/benchctl/main.das	New clargs-based flow and typed SQLite interactions for reset/insert/query/compare.
utils/benchctl/flags.das	Removes legacy hand-rolled flag parsing.
utils/benchctl/benchstat.das	Switches from BenchmarkEntry to the new typed [sql_table] Benchmark.
utils/benchctl/bench_table.das	Introduces typed [sql_table] Benchmark schema for benchctl storage.
utils/benchctl/bench_sql.das	Removes legacy raw SQL helpers and schema strings.
utils/benchctl/bench_args.das	Adds clargs wrapper module to avoid Option<string> ambiguity with sqlite modules.
daslib/linq_boost.das	Adds where_ + count fold rule emitting an inlined single-pass loop.
benchmarks/sql/_common.das	Shared fixture (Car table + DB/array setup helpers) for SQL throughput benchmarks.
benchmarks/sql/select_where.das	Adds benchmark for _where filtering across DB/array modes.
benchmarks/sql/select_where_order_take.das	Adds benchmark for _where + _order_by + take across DB/array modes.
benchmarks/sql/count_aggregate.das	Adds benchmark for _where + count, showcasing new fold optimization.
benchmarks/README.md	Documents the new benchmarks/sql/ suite and its mode matrix.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.