Skip to content

v0.3.1 — Fix 5 bugs from Codex adversarial review

Latest

Choose a tag to compare

@robertoecf robertoecf released this 29 Apr 12:14
· 97 commits to main since this release

Patch release. v0.3.0 shipped with a critical DoS bug in /registry/lookup that we caught only after the release, via adversarial review by an external LLM (gpt-5.4 xhigh via tmux). All five issues are fixed here, with regression tests for each.

What was broken in v0.3.0

# Bug Severity What happened
1 FTS5 reserved words crashed the endpoint 🔴 Critical GET /registry/lookup?q=OR returned 500 Internal Server Error (sqlite3 syntax error). Public DoS.
2 FTS5 syntax injection 🟠 High q=PETR4 NOT VALE3 returned PETROBRAS — user could inject FTS5 operators to steer results.
3 Prefix / fragment search didn't work 🟠 High q=PETROBR returned 0 hits despite the contract claiming "company-name fragment" / "prefix searches".
4 Labeled identifiers stopped resolving 🟡 Medium q="CNPJ: 33.000.167/0001-01", q="FIP 05886", q="CVM 9512" all returned 0.
5 cdb series in BCB SGS catalog was secretly historical 🟢 Low Last observation 2012-12; users picking findata bcb get cdb got ancient data with no warning.

How they were fixed

A single rewrite of _normalize_query in src/findata/registry/store.py:

_FTS5_RESERVED = frozenset({"AND", "OR", "NOT", "NEAR"})

def _escape_token(t: str) -> str:
    if t in _FTS5_RESERVED:
        return f'"{t}"'   # phrase-quoted literal — no operator interpretation
    return f"{t}*"        # prefix wildcard — fragment / prefix matching

Plus a third "digits-only joined" variant in the OR'd query, so labels like CNPJ: don't contaminate the joined form.

For Bug 5: cdb in SERIES_CATALOG now self-documents as "Taxa média de CDB pré-fixado (histórico, descontinuado em 2012)".

Regressions added

test_lookup_fts5_reserved_word_does_not_crash      # Bug 1
test_lookup_fts_injection_is_neutralized           # Bug 2
test_lookup_prefix_match_finds_full_word           # Bug 3
test_lookup_labeled_cnpj_resolves                  # Bug 4
test_normalize_query_reserved_word_gets_quoted     # unit
test_normalize_query_injection_attempt_neutralized # unit
test_normalize_query_labeled_cnpj_includes_digits_only_variant
+ 2 unit updates for changed normalize behavior

168 → 176 tests. Ruff + mypy --strict clean. Pre-push hook validated.

Upgrade

pip install -U findata-br==0.3.1

Adversarial review takeaway

The five bugs were caught only because a separate model reviewed the diff after the release. Doing this review before v0.3.0 shipped would have saved the patch release — the workflow now treats Codex review as a release gate, not a follow-up.

🤖 Generated with Claude Code