perf: add stateful object iterator

[membphis]

Summary

• add a stack-allocated qjson_iter FFI iterator for object entries
• switch Lua LazyObject pairs/materialization paths to qjson_iter_next
• add Rust FFI coverage for empty, ordered, nested, and type-mismatch iterator cases
• add Lua pairs coverage for 1/10/100-key objects and early break

Closes #29

Verification

• cargo test --release
• cargo test --release --no-default-features
• cargo test --features test-panic --release
• cargo clippy --release --all-targets -- -D warnings
• Lua suite via local LuaJIT 5.1 busted: DYLD_LIBRARY_PATH=$(pwd)/target/release LUA_PATH=... LUA_CPATH=... /Users/yuanshengwang/.luarocks/bin/busted --lua=$(command -v luajit) tests/lua --lpath='./lua/?.lua' (169 successes, 3 pending for local LuaJIT without LUA52 compat)
• 500-key local LuaJIT microbench: old qjson_cursor_object_entry_at loop ~3.2k walks/s; new qjson.pairs loop ~22.7k walks/s; checksums matched

Note: full make bench was not runnable on this machine because /usr/local/openresty/bin/resty is not installed.

Summary by CodeRabbit

• New Features

  • Added stateful object iterator API for traversing JSON objects with efficient key/value pair access.

• Tests

  • Added comprehensive test coverage for the new iterator functionality, including edge cases and multiple object sizes.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR implements a stateful object iterator for the qjson library to eliminate O(N²) enumeration cost. A new iterator struct tracks position in an object's index array; Lua consumers call two FFI functions to initialize and step through key/value pairs, replacing slow index-based cursor reads. Both Rust and Lua test suites validate iterator semantics and integration.

Changes

Iterator API: Type Contract through Rust Implementation

Layer / File(s)	Summary
Iterator Type Contract and Declarations include/qjson.h	qjson_iter struct holds document pointer and position indices; qjson_iter_init and qjson_iter_next declarations define the initialization and stepping interface.
FFI Interface and Library Loading lua/qjson/lib.lua	FFI ffi.cdef declares the iterator struct and both functions; both marked as required exports so library load fails if absent.
Core Iterator Implementation in Rust src/ffi.rs	Rust struct with #[repr(C)]; qjson_iter_init validates cursor, skips whitespace, sets position state; qjson_iter_next decodes key into scratch buffer, returns value cursor, advances position or exhaustion state; scratch-buffer invalidation docs updated.

Lua Consumer: Pairs Iteration Refactoring

Layer / File(s)	Summary
Lazy Object Iteration and Pairs Integration lua/qjson/table.lua	New new_object_iter helper initializes iterator; lazy_object_iter refactored to call qjson_iter_next for stepping; LazyObject.__pairs returns iterator state; materialize_object_contents and ensure_object_order_state migrate to iterator-based walks while preserving duplicate-key semantics.

Test Coverage: Rust FFI and Lua Integration

Layer / File(s)	Summary
Rust FFI Iterator Tests tests/ffi_stateful_iter.rs	Unsafe FFI helpers for parsing, iterator initialization, and stepping; tests verify empty-object exhaustion, mixed-type key/value correctness, independent iterator position tracking, and type-mismatch rejection.
Lua Pairs Integration Tests tests/lua/lazy_table_spec.lua	make_object helper generates N-key JSON objects; parameterized tests validate key order for sizes 1/10/100; new test confirms early break does not prevent subsequent field access on the same object.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 6

✅ Passed checks (6 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'perf: add stateful object iterator' accurately describes the main change—introducing a stateful iterator to improve object enumeration performance from O(N²) to O(N).
Linked Issues check	✅ Passed	All requirements from issue #29 are met: stateful POD iterator struct added, qjson_iter_init and qjson_iter_next FFI functions implemented, C header and Lua ffi.cdef updated, LazyObject iteration refactored to use iterator, comprehensive tests added, and O(N) performance achieved.
Out of Scope Changes check	✅ Passed	All changes are directly aligned with issue #29 scope: iterator struct/functions, Lua wrapper updates, header/FFI updates, and corresponding tests. No unrelated modifications detected.
E2e Test Quality Review	✅ Passed	E2E tests span full API flows (FFI and Lua), cover boundary cases (empty/1/10/100 keys), error conditions, and early-break. All error codes checked, assertions readable, no unrelated changes.
Security Check	✅ Passed	Security check categories (credentials, auth, database, TLS) are not applicable to this JSON library optimization PR. Pointer validation is correct throughout.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/issue-29-stateful-iter

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

include/qjson.h (1)

92-95: ⚡ Quick win

Consider documenting the key_ptr scratch-lifetime contract.

qjson_iter_next returns key_ptr/key_len pointing into the document's input buffer or scratch, which any subsequent *_get_str or iterator call may invalidate. A brief comment here (mirroring qjson_cursor_object_entry_at usage) would make the must-copy-immediately requirement explicit for C consumers, matching the contract LuaJIT already enforces via ffi.string.

📝 Suggested doc comment

+/* key_ptr/key_len point into the doc's input buffer or scratch and are
+   invalidated by any later *_get_str or qjson_iter_* call; copy immediately. */
 int qjson_iter_next(qjson_iter* it,
                     const uint8_t** key_ptr, size_t* key_len,
                     qjson_cursor* value_out);

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@include/qjson.h` around lines 92 - 95, Document that qjson_iter_next may
return key_ptr/key_len that point into the document input buffer or into
internal scratch and that those bytes are transient: callers must copy the key
immediately before calling any other qjson functions (e.g., any *_get_str or
another qjson_iter_* call) because subsequent operations may invalidate the
pointer; add a brief comment next to the qjson_iter_next declaration mirroring
the note used for qjson_cursor_object_entry_at and reference qjson_iter_init and
qjson_iter_next in the comment for clarity.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@include/qjson.h`:
- Around line 92-95: Document that qjson_iter_next may return key_ptr/key_len
that point into the document input buffer or into internal scratch and that
those bytes are transient: callers must copy the key immediately before calling
any other qjson functions (e.g., any *_get_str or another qjson_iter_* call)
because subsequent operations may invalidate the pointer; add a brief comment
next to the qjson_iter_next declaration mirroring the note used for
qjson_cursor_object_entry_at and reference qjson_iter_init and qjson_iter_next
in the comment for clarity.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7141c1cc-51c7-4741-b70e-868599ccdf44

📥 Commits

Reviewing files that changed from the base of the PR and between 6aa02e4 and 8e0cd63.

📒 Files selected for processing (6)

• include/qjson.h
• lua/qjson/lib.lua
• lua/qjson/table.lua
• src/ffi.rs
• tests/ffi_stateful_iter.rs
• tests/lua/lazy_table_spec.lua