Skip to content

feat(security): W3.1 PR A — SQL parameter classifier + prepared template rewriter (helpers only)#37

Merged
jrosskopf merged 1 commit into
mainfrom
feature/gh-25-prepared-helpers
May 16, 2026
Merged

feat(security): W3.1 PR A — SQL parameter classifier + prepared template rewriter (helpers only)#37
jrosskopf merged 1 commit into
mainfrom
feature/gh-25-prepared-helpers

Conversation

@jrosskopf
Copy link
Copy Markdown
Contributor

Summary

First slice of the W3.1 prepared-statement refactor (#25). This PR adds the two pure helpers the rest of W3.1 will build on — NO integration with the query path yet, NO behavioural change for any existing endpoint. The aim is to let reviewers verify the security-critical logic in isolation before the larger query-path change lands.

PR B (separate, follow-up) will wire these into DatabaseManager behind a per-endpoint opt-in flag with the full integration test surface (see below).

What ships in this PR

1. SqlParameterClassifier — pure helper

Maps a RequestFieldConfig validator to (bindable, SqlParameterType):

  • int/integerInteger
  • number/float/doubleDouble
  • boolean/boolBoolean
  • dateDate
  • timeTime
  • uuid/string/email/enumVarchar
  • no typed validator → not bindable (conservative; Mustache stays the safe default)
  • unknown validator name → not bindable (forward-compat: a future validator type can't accidentally land on the prepared path)

2. PreparedTemplateRewriter — pure helper

Scans a Mustache template, rewrites {{ params.X }}? for bindable X at section depth 0, records binding order. Leaves alone:

  • Triple-brace {{{ params.X }}} (operators migrate these separately)
  • Anything inside {{#X}}...{{/X}} or {{^X}}...{{/X}} (any depth)
  • Params with no typed validator
  • Non-params.* references like {{ conn.X }}, {{ env.X }}, {{ auth.X }}
  • Malformed / unterminated tags (passthrough, no crash)

Test plan (~45 Catch2 cases, all green)

Classifier ([security][prepared][classifier])

Every validator type, multiple-validator fallback (first known wins), case sensitivity, whitespace intolerance, large validator lists, determinism across calls, empty fields, forward-compat (unknown name before known doesn't block).

Rewriter ([security][prepared][rewriter] + [edge])

Empty template, no params, single bindable param, triple-brace passthrough, unknown field passthrough, section suppression (positive {{#}}, inverted {{^}}, nested), stray {{/X}} clamps to depth 0, unterminated tags (passthrough, no crash), no-whitespace form {{params.X}}, very wide binding count (25 distinct), multi-line templates, idempotency, repeated param becomes two bindings, and a pinning test proving an injection-style value would land in a bound ? rather than being expanded into syntax.

ctest -R "PreparedTemplateRewriter|SqlParameterClassifier"46/46 pass.

What this PR does NOT ship (deliberate, lands in PR B)

  • DatabaseManager::executeQuery integration (no duckdb_prepare / duckdb_bind_* yet)
  • EndpointConfig.use-prepared-statements opt-in flag
  • Cache-layer interaction
  • Pagination interaction (LIMIT / OFFSET binding)
  • Write-path integration (executeWrite, executeWriteInTransaction)
  • Companion W3.3 work (demoting the regex SQL-injection validator)
  • Full integration tests against a real flapi binary

PR B testing plan (committed deliverable, NOT part of this PR)

To address the "thoroughly tested with full integration tests" concern up front:

  • C++ integration test running real DuckDB prepared queries for each SqlParameterType with a value that would have been an injection attempt under Mustache (e.g., 1; DROP TABLE customers -- bound as Varchar → zero rows, no DROP).
  • C++ integration test comparing rendered SQL + result rows of the same endpoint with and without use-prepared-statements: true for a corpus of templates copied from test/integration/api_configuration/sqls/.
  • Python E2E tests that boot a real flapi server, hit endpoints in both modes, and diff responses.
  • Cache-enabled endpoint test, write-endpoint test, paginated endpoint test — each in both modes.
  • Adversarial test corpus: historical SQL-injection payloads applied to bindable params, asserting each is rendered inert.

Closes part of #25 (PR A of three planned)
Refs #21

…ate rewriter (helpers only) (#25)

First slice of the prepared-statement refactor. This PR adds the two
pure helpers the rest of W3.1 will build on — NO integration with the
query path yet, NO behavioural change for any existing endpoint. The
file inventory and call graph make the integration risk visible to a
reviewer before the larger query-path change lands.

Why split this out:
The full W3.1 change touches `QueryExecutor`, `DatabaseManager::executeQuery`,
`executeWrite`, the cache layer, and every endpoint's render path. Done
as a single PR it would be unreviewable AND introduce regression risk to
every endpoint at once. Shipping the helpers first lets reviewers verify
the security-critical logic in isolation; PR B then wires them in behind
a per-endpoint opt-in flag with the full integration test surface.

What this PR ships:

1. `SqlParameterClassifier` — pure helper mapping a `RequestFieldConfig`
   validator to `(bindable, SqlParameterType)`. int/integer →
   Integer; number/float/double → Double; boolean/bool → Boolean;
   date → Date; time → Time; uuid/string/email/enum → Varchar.
   Without a typed validator: NOT bindable (conservative; Mustache
   path remains the safe default). Unknown validator names also fall
   back to non-bindable for forward-compat.

2. `PreparedTemplateRewriter` — pure helper that scans a Mustache
   template and rewrites `{{ params.X }}` to `?` for bindable X at
   section depth 0, recording the binding order. Leaves alone:
   - Triple-brace `{{{ params.X }}}` (operators migrate these later)
   - Anything inside `{{#X}}...{{/X}}` or `{{^X}}...{{/X}}` (depth > 0)
   - Params with no typed validator
   - Non-`params.*` references like `{{ conn.X }}`, `{{ env.X }}`,
     `{{ auth.X }}`
   - Malformed / unterminated tags (passthrough, no crash)

3. ~45 unit test cases across both helpers (Catch2), with explicit
   edge-case coverage:
   - Classifier: every validator type, multiple-validator fallback,
     case sensitivity, whitespace handling, large validator lists,
     determinism, empty fields, forward-compat with future names.
   - Rewriter: empty template, no params, single bindable param,
     triple-brace passthrough, unknown field passthrough, section
     suppression (positive and inverted), nested sections, stray
     `{{/X}}` handling, unterminated tags, no-whitespace form, very
     wide binding count (25 distinct), multi-line templates,
     idempotency, repeated occurrence becomes two bindings, and a
     pinning test proving an injection-style value would land in a
     bound `?` rather than being expanded into syntax.

What this PR does NOT yet ship (deliberate, lands in PR B):

- `DatabaseManager::executeQuery` integration — no DuckDB
  `duckdb_prepare`/`duckdb_bind_*` calls yet.
- `EndpointConfig.use-prepared-statements` opt-in flag.
- Cache-layer interaction.
- Pagination interaction (LIMIT/OFFSET binding).
- Write-path integration (`executeWrite`,
  `executeWriteInTransaction`).
- The companion W3.3 work — demoting the regex SQL-injection
  validator to a soft warning, conditional on the prepared path
  being on.
- Full integration tests against a real flapi binary
  (parameter-type roundtrip, write paths, cache, pagination,
  error paths).

PR B testing plan (committed deliverable, not part of this PR):

- C++ integration test running real DuckDB prepared queries for each
  `SqlParameterType` (Integer, Double, Boolean, Date, Time, Varchar)
  with a value that would have been an injection attempt under
  Mustache (e.g., `1; DROP TABLE customers --` bound as Varchar →
  zero rows, no DROP).
- C++ integration test comparing rendered SQL + result rows of the
  same endpoint with and without `use-prepared-statements: true`
  for a corpus of templates copied from
  `test/integration/api_configuration/sqls/`.
- Python E2E tests that boot a real flapi server, hit endpoints in
  both modes, and diff responses.
- Cache-enabled endpoint test, write-endpoint test, paginated
  endpoint test — each in both modes.
- Adversarial test corpus: a list of historical SQL-injection
  payloads applied to bindable params, asserting each is rendered
  inert (returns the expected zero-row result, never executes
  attacker syntax).

Skipped pre-commit hook per the existing precedent in commit e1b465e —
the bd-shim calls 'bd hook pre-commit' (singular) which is missing
from the installed bd binary (only 'bd hooks' plural exists).
@jrosskopf jrosskopf force-pushed the feature/gh-25-prepared-helpers branch from 0911ecc to 2f7d311 Compare May 16, 2026 17:31
@jrosskopf jrosskopf merged commit 8bf073d into main May 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant