Sync: @huggingface/transformers v4 migration + release 0.9.30-chronode.4 - #13
Merged
Conversation
…ation Ports rohitg00/agentmemory@8c90741 onto the Chronode fork: - @xenova/transformers ^2.17.2 -> @huggingface/transformers ^4.2.0 (optionalDependencies; onnxruntime-node/web now transitively provided). - src/providers/embedding/{local,clip}.ts: import specifiers, module typed as typeof import("@huggingface/transformers"), explicit { dtype: "q8" } at all pipeline call sites (v4 defaults to fp32 on Node), split import try/catch so model-load errors propagate unmasked. Fork's processingLocation field preserved. - src/state/reranker.ts: specifier + { dtype: "q8" }; drops the fork-local pipeline cast that existed only to satisfy the removed xenova.d.ts stub. - src/xenova.d.ts deleted; __mocks__/@huggingface/transformers.ts manual mock added for missing-package tests. - New tests: local-embedding-provider (3), clip-embedding-provider (5), env-guarded local-embedding-smoke (RUN_HF_SMOKE=1); reranker gains a doMock positive path. ci/r13-test-manifest.json restamped 176 -> 179. - CI matrix [20,22] -> [20,22,24,26]; docs/specifier swaps across README, 11 README translations, SECURITY.md, benchmark docs + script. - Release 0.9.30-chronode.4; CHANGELOG documents the one-time local embedding model re-download caused by the v4 cache-dir change.
…, Node 26 profile First CI round on the expanded matrix exposed three fork-side gaps: - integrations/filesystem-watcher: fs.watch on Node >= 23 Linux reports a missing root via the async handle 'error' event instead of throwing synchronously, so start() silently attached nothing and the "could not watch any of the configured roots" contract never fired. An explicit existsSync check before attach makes it deterministic; @agentmemory/fs-watcher 0.1.1 -> 0.1.2. - CI: run npm test with RUN_HF_SMOKE=1 — the R-13 gate treats any skipped test as failure, and upstream's env-guarded real-model embedding smoke skips by default. - scripts/r13/lib.mjs: promote Node 26 to an accepted R-13 profile so matrix cells produce clean pass receipts (--require-pass rejects provisional-pass waivers); harness self-test updated with the new declared-profile table (v26 accepted, v27 rejected).
Sequential forks hand off between test files: an exiting worker can still be visible in a single 250ms telemetry sample while the next file's worker starts (native modules like onnxruntime delay teardown once loaded). The sampler now flags peak concurrency only when >1 worker persists across consecutive samples; a real maxWorkers>1 misconfiguration stays flagged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports upstream rohitg00/agentmemory
8c90741(rohitg00#1096) onto the Chronode fork and ships release 0.9.30-chronode.4.Port
@xenova/transformers^2.17.2 →@huggingface/transformers^4.2.0 (optionalDependencies).onnxruntime-node/onnxruntime-webdropped from direct declaration — they ship transitively with v4. Lockfile regenerated vianpm install.src/providers/embedding/local.ts,src/providers/embedding/clip.ts,src/state/reranker.ts: import specifiers updated; module typed astypeof import("@huggingface/transformers"); explicit{ dtype: "q8" }at all pipeline call sites (v4 defaults to fp32 on Node — without this, downloads grow ~3.5x); import try/catch split so model-load/network errors propagate their real message instead of the generic install hint. Fork'sprocessingLocationprovider field preserved.src/xenova.d.tsdeleted (v4 ships its own types); fork-local reranker pipeline cast (needed only for the old stub) removed. Manual mock added at__mocks__/@huggingface/transformers.tsfor missing-package tests.local-embedding-provider(3),clip-embedding-provider(5), env-guarded real-model smoke (RUN_HF_SMOKE=1, skipped by default), plus avi.doMockpositive path inreranker.ci/r13-test-manifest.jsonrestamped 176 → 179 files.[20, 22]→[20, 22, 24, 26]; specifier/prose swaps across README, 11 README translations, SECURITY.md, benchmark docs + script.The v4 package resolves its model cache inside its own directory (
node_modules/@huggingface/transformers/.cache/) instead of xenova's. Local embeddings re-download once (~23 MB q8) on first use after upgrade, then cache as before. Documented in CHANGELOG.Verification battery (all green)
Xenova/all-MiniLM-L6-v2load through v4 produced a 384-dim finite vector (~9.6 s incl. download); also executed inside the R-13 gate viaRUN_HF_SMOKE=1.npx vitest run: 1951 passed / 0 failures; only the 3 known file-level suite errors identical to base (integration secret gate; twonode:test.mjssuites vitest cannot load).@xenovaTS2307s are gone.AGENTMEMORY_SECRET=dummy AGENTMEMORY_PROJECT_CAPABILITY_SECRET=dummy node scripts/r13/run.mjs→R-13 passed(receipt1787689621784-0-675b0486, zero skips).standalone.mjsversion stamp).