test: add FFI grammar corpus

[membphis]

Summary

• Add a table-driven FFI grammar corpus for eager-mode JSON grammar behavior.
• Assert qjson_parse_ex error codes and byte offsets for number, string, literal, trailing-content, and structural-boundary parse cases.
• Include success cases for valid number/string/literal grammar, including long numeric input and surrogate pairs.

Closes #135

Test Plan

• cargo test --release --test ffi_grammar_corpus
• cargo test --release --no-default-features --test ffi_grammar_corpus
• cargo test --release
• cargo test --release --no-default-features
• cargo test --features test-panic --release
• make lint
• PATH=/Users/yuanshengwang/.luarocks/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/pkg/env/global/bin:/Library/Apple/usr/bin:/Users/yuanshengwang/.codex/tmp/arg0/codex-arg02NX0WU:/Users/yuanshengwang/.local/bin:/Users/yuanshengwang/.opencode/bin:/Users/yuanshengwang/.bun/bin:/Users/yuanshengwang/.luarocks/bin:/Users/yuanshengwang/.cargo/bin:/Applications/Codex.app/Contents/Resources make test

Note: the default /opt/homebrew/bin/busted on this machine is a broken Lua 5.5 wrapper; the Lua suite was run with the LuaJIT/5.1 busted wrapper in ~/.luarocks/bin. vendor/lua-cjson/cjson.so was built locally before the full make test run.

Summary by CodeRabbit

• Tests
  • Expanded test coverage for the FFI parsing API, validating eager parsing behavior and error reporting across numbers, strings, literals, and edge cases including trailing content detection and boundary offset accuracy.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a4d4ffb4-5d44-4d3d-ae2a-d3ca1d207d8b

📥 Commits

Reviewing files that changed from the base of the PR and between 6225ce2 and 5a92e85.

📒 Files selected for processing (1)

• tests/ffi_grammar_corpus.rs

---

📝 Walkthrough

Walkthrough

This PR adds a comprehensive Rust test corpus (tests/ffi_grammar_corpus.rs) that validates FFI eager-mode JSON parsing behavior. The 414-line file establishes test infrastructure (FFI wrappers, assertion utilities, test data structures) and then exercises eager parsing across five grammar categories: numbers, strings, literals, trailing content, and offset boundaries. All test cases verify both success/failure conditions and exact error-offset reporting.

Changes

Eager parsing grammar corpus

Layer / File(s)	Summary
Test infrastructure setup tests/ffi_grammar_corpus.rs	Module imports, test case structs (OkCase, ErrorCase), eager-mode configuration, and FFI wrapper functions (parse_eager_ok, parse_eager_error) with core assertions around document validity, error codes, and offset validation. Adds assert_ok_cases and assert_error_cases utilities for data-driven test iteration.
Number grammar validation tests/ffi_grammar_corpus.rs	Tests accept valid numeric forms (zero, signed integers, fractions, exponents) and very long tokens; rejects invalid grammars (leading zeros, plus sign, dot, missing exponent digits, NaN/Infinity) with exact error-offset assertions.
String grammar validation tests/ffi_grammar_corpus.rs	Tests accept ASCII, escaped controls, Unicode escapes, and surrogate pairs; rejects invalid escapes (truncated \u, bad surrogates, invalid UTF-8) with exact error offsets.
Literal grammar validation tests/ffi_grammar_corpus.rs	Tests accept true/false/null standalone and in arrays; rejects truncated, misspelled, and interrupted literals with exact parse-error offsets.
Trailing content and offset boundary validation tests/ffi_grammar_corpus.rs	Tests rejection of trailing content after root-level values and validates reported parse-error offsets correspond to object keys, array elements, and malformed syntax boundary positions.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• api7/lua-qjson#51: Adds FFI eager-mode validation and depth-handling changes that directly exercise the eager-mode parsing behavior and error-offset patterns tested in this corpus.

🚥 Pre-merge checks | ✅ 6

✅ Passed checks (6 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'test: add FFI grammar corpus' directly and clearly describes the main change: adding a test corpus for FFI grammar validation.
Linked Issues check	✅ Passed	The PR successfully implements all coding requirements from issue #135: grammar corpus covering number/string/literal/trailing-content/offset categories with qjson_parse_ex FFI validation.
Out of Scope Changes check	✅ Passed	All changes are focused on adding the FFI grammar corpus test file as specified in issue #135; no unrelated modifications detected.
E2e Test Quality Review	✅ Passed	Grammar corpus with 50+ test cases covering issue #135: numbers/strings/literals/trailing-content/offsets. E2E, proper memory management, clear naming, no dependencies, real FFI.
Security Check	✅ Passed	Test-only PR adding JSON grammar validation tests. Audited against 7 security categories with no vulnerabilities found in code, credentials, auth, resources, TLS, isolation, or secrets.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/issue-135-grammar-corpus

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}