Skip to content

feat(bindings): put_documents / add_documents across python, nodejs, wasm, ruby, php (#866)#871

Merged
mosuka merged 2 commits into
mainfrom
feat/866-bindings-batch
Jul 15, 2026
Merged

feat(bindings): put_documents / add_documents across python, nodejs, wasm, ruby, php (#866)#871
mosuka merged 2 commits into
mainfrom
feat/866-bindings-batch

Conversation

@mosuka

@mosuka mosuka commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Summary

Exposes the batch ingestion API (#551 PR-4, the campaign's final PR) in all five language bindings, on top of the merged Engine::put_documents / add_documents.

Each binding gains put_documents / add_documents (putDocuments / addDocuments in the camelCase hosts), taking a list of (id, document) pairs — the most direct mirror of the core Vec<(String, Document)> signature and the natural batched form of each binding's existing singular put_document(id, doc). Semantics carry through from the core: sequential in-order application, one WAL fsync per batch, duplicate-id dedup within a put batch (last wins), chunk accumulation for add, and fail-fast with the applied prefix left in place (retrying is idempotent). Each binding's converter names the offending position on a malformed entry, and an empty batch is a no-op.

Binding Method shape Notes
Python put_documents([(id, dict), ...]) iterable of (str, dict) pairs
Node.js putDocuments([[id, obj], ...]) Vec<(String, Value)> napi tuple marshalling; index.d.ts regenerated
wasm putDocuments([[id, obj], ...]) JS array via serde_wasm_bindgen; .d.ts regenerated
Ruby put_documents([[id, hash], ...]) manual define_method registration added
PHP putDocuments([[$id, $doc], ...]) nested-array marshalling

The BatchIngest error surfaces message-only through each binding's existing error mapper (the message already carries failed_index / failed_id / applied via Display); no new host exception types.

Verification (real runtime tests, not just compile)

  • Pythonmaturin develop + pytest: 4/4 (test_batch_ingest.py: empty no-op, apply+dedup, chunk accumulation, malformed-entry position).
  • Node.jsnapi build + vitest: 58/58 (3 new in batch_ingest.spec.mjs); regenerated index.d.ts carries both methods.
  • Rubyrake compile + minitest: 4/4 (test_batch_ingest.rb, incl. ArgumentError naming documents[1]).
  • PHPcargo build --release + phpunit: batch tests green (LaurusTest.php).
  • wasmcargo build --target wasm32-unknown-unknown and wasm-pack build --target web succeed; putDocuments / addDocuments present in the regenerated pkg/*.d.ts. (This binding has no runtime test harness — build-test only, consistent with the rest of the wasm suite.)
  • fmt / clippy 1.95 + 1.97 across all four host bindings + wasm-target clippy clean; markdownlint clean on all 10 updated api_reference.md (en+ja × 5).

Generated artifacts (laurus-nodejs/index.d.ts, laurus-wasm/pkg/) are git-ignored, so they are regenerated by each binding's build rather than committed.

Docs

docs/{src,ja/src}/laurus-{python,nodejs,wasm,ruby,php}/api_reference.md: put_documents / add_documents rows (or headings for wasm) with the pair-list shape and fail-fast/no-rollback semantics.

Closes #866. Refs #551 (final PR of the batch-ingestion campaign; the #551 umbrella is closed during wrap-up once PR-3 (#865) and this PR are both merged).

mosuka added 2 commits July 15, 2026 23:19
…wasm, ruby, php (#866)

Expose the Engine::put_documents / add_documents batch API (#551) in all
five language bindings. Each gains put_documents / add_documents
(putDocuments / addDocuments in the camelCase hosts) taking a list of
(id, document) pairs — the direct mirror of the core
Vec<(String, Document)> signature and the batched form of each binding's
existing singular put_document(id, doc).

Semantics carry through from the core: sequential in-order application,
one WAL fsync per batch, duplicate-id dedup within a put batch (last
wins), chunk accumulation for add, and fail-fast with the applied prefix
left in place (retrying is idempotent). Each converter names the
offending position on a malformed entry, and an empty batch is a no-op.
The BatchIngest error surfaces message-only through each binding's
existing error mapper (the message carries failed_index / failed_id /
applied via Display).

Per-binding marshalling: Python iterates (str, dict) pairs; Node uses
napi Vec<(String, Value)> tuple marshalling; wasm deserializes a JS
array via serde_wasm_bindgen; Ruby adds manual define_method
registration and validates [String, Hash] pairs; PHP walks nested
arrays. The generated laurus-nodejs/index.d.ts and laurus-wasm/pkg/ are
git-ignored, so they are regenerated by each build rather than
committed.

Runtime-tested: Python 4/4 (pytest), Node 58/58 (vitest, 3 new), Ruby
4/4 (minitest), PHP batch tests (phpunit); wasm builds for
wasm32-unknown-unknown and via wasm-pack with both methods in the
regenerated .d.ts (build-test only, matching the wasm suite).

Closes #866. Refs #551
The CI Format job runs stable (1.97) rustfmt, which formats two spots in
the #866 bindings differently than the local 1.95 rustfmt did: the Ruby
pairs_to_documents signature collapses onto one line, and the wasm
map_err closure re-wraps. No logic change.

Refs #866
@mosuka
mosuka merged commit 7a5e1cd into main Jul 15, 2026
17 checks passed
@mosuka
mosuka deleted the feat/866-bindings-batch branch July 15, 2026 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(bindings): put_documents / add_documents across python, nodejs, wasm, ruby, php (#551 PR-4)

1 participant