Skip to content

Releases: mcn92/pikelet

Build encoder weights: BGE-small u8 v1

Choose a tag to compare

@mcn92 mcn92 released this 27 Oct 22:19

BAAI/bge-small-en-v1.5 (MIT) as an inline-encoder u8 weight blob, 37,538,112 bytes.

sha256: 3be1ede506f1826717a76593e6d614a15e15de8caa34fa209ab0089f0d613385

Exported from Hugging Face revision 5c38ec7c405ec4b44b94cc5a9bb96e735b38267a with examples/one-file-search/encoder-spike/export_encoder_blob.py --model BAAI/bge-small-en-v1.5 --pooling mean --bits 8, which reproduces these bytes exactly. pikelet create (embedding mode workers-ai) downloads it on the first build to embed passages, pinned by sha256, and caches it in ~/.cache/pikelet/models. Override the source with PIKELET_BGE_WEIGHTS_URL.

Pride and Prejudice knowledge pack v1

Choose a tag to compare

@mcn92 mcn92 released this 17 Sep 00:33

Compiled .pikelet knowledge pack for Jane Austen's Pride and Prejudice (public domain), sourced from Project Gutenberg.

Mount it: npx pikelet mcp --pack <asset-url>#<identity>
Or write your MCP client config: npx pikelet mcp install --client claude-code --pack <asset-url>#<identity>

pack records identity
pride-prejudice.pikelet 595 (61 chapters) 6d4298f345c011b78459ad74fcb63dc25e8b0ee8472e22218fcf2c9fc0ba6759

Self-calibrated abstention (kind-3 inline encoder): 5-fold cross-validated AUC 0.859 against hard negatives, using ablation plus a corpus-entity-swap negative class (needed here since a single-novel corpus is thematically uniform enough that excluding one chapter often leaves a question answerable from another — entity/character swaps supply the rest of the hard-negative signal). The fit is saturated on this corpus (probe classes separate almost perfectly), so treat abstention boundaries with some caution — narrative prose is a harder fit for this calibration approach than reference/technical corpora.

Packs are range-read off this release's URL — never downloaded whole. A mount that finds different bytes than the pinned identity refuses to serve.

Wikipedia knowledge pack v1

Choose a tag to compare

@mcn92 mcn92 released this 14 Sep 21:59

Self-contained kind-3 .pikelet artifact: 456,153 Simple English Wikipedia records, embedded MiniLM-L6 query encoder, semantic + lexical (BM25) hybrid retrieval, per-record integrity, calibrated abstention.

No embedding API, no vector database, no search server required to query it — the file is the search environment. Mount it directly over static HTTP:

pikelet mcp --pack https://github.com/mcn92/pikelet/releases/download/wikipedia-pack-v1/wikipedia.pikelet

Or open it locally with openPikeletFile from pikelet-wasm/complete.

size 648.5 MiB (680,029,254 bytes)
records 456,153
dim 384
encoder inline-transformer-v1 (MiniLM-L6, quantized)
identity ef70ccaaebc54726...
sha256 359e5b25edf8dd895b7fce3fbe76803de4c92206156f35b3bbfd650fd7cab38e
recall@10 95.0% (199/200 strong)

Veyra demo packs v1

Choose a tag to compare

@mcn92 mcn92 released this 14 Sep 03:07

Compiled .pikelet artifacts for the abstention/containment demo (examples/05-one-file-search/web/).

Source: examples/05-one-file-search/web/veyra-corpus/ (gen.mjs deterministically generates the fictional Station Veyra corpus; make-chamber43.mjs derives the single-fact-contradiction variant).

pack records demo compares
veyra.pikelet 94 full corpus
veyra-ablated.pikelet 88 6 facts removed — their questions must abstain
veyra-chamber43.pikelet 94 one fact (loc-tovash.md) edited to contradict every other record

Fetch and verify with node examples/05-one-file-search/web/veyra-corpus/fetch-veyra.mjs (checks each download against its pinned SHA-256 before writing).

Knowledge packs v1

Choose a tag to compare

@mcn92 mcn92 released this 02 Sep 23:24

Identity-pinned .pikelet knowledge packs for MCP mounting.

Mount one: npx pikelet mcp --pack #
Or write your MCP client config: npx pikelet mcp install --client claude-code --pack #

pack records license manifest identity
astro-docs.pikelet 3,255 (docs.astro.build, en) MIT e4d17aee1ec9298eb60f8f127b1fed7bdbcccc08fa2195ea7333eff77f1997e0
rust-book.pikelet 1,035 (doc.rust-lang.org/book) MIT OR Apache-2.0 21f86277770e02b574ec18ceaa708c0526a93843fece63345cb8ba67551c9e03

Packs are range-read off this release's URLs — never downloaded whole. A mount that finds different bytes than its pinned identity refuses to serve.

Inline Wiki Pancake Artifact v4

Choose a tag to compare

@mcn92 mcn92 released this 31 Aug 15:17

Complete kind-3 search artifact: Simple English Wikipedia, 456,153 records, self-contained inline MiniLM encoder.

  • identity: 1b180adf4c6cebb2dcd5615256df6a25dac5fda8738dbbc11d60af86046f97f3
  • size: 680,029,254 bytes (648.53 MiB)
  • augmented recall@10: 95.2% at the recommended C=200 (97.0% at C=600) vs exact fp32 brute-force ground truth, 200-query pre-registered eval
  • v4 recovers the pack's canonical build: k-means cluster-ordered row layout + the 192-dim 2:1-pooled sketch (v1-v3 embedded a 96-dim 4:1 sketch in unpermuted order — the 82.8% era), adds the hybrid BM25 lexical segment (kind 5), a format-2 sketch with per-row read verification, and manifest fetch hints (recommendedRerank 200, recommendedGap 16384): ~127 HTTP range requests per query, down from ~458
  • readers: pancake-wasm >= 0.6.0 for hybrid + lazy encoder open; older readers serve vector-only and eager open from the same file
  • eval ground truth ids are in the permuted (data-perm) id space

v1-v3 stay published for older checkouts.

Inline encoder weights v1

Choose a tag to compare

@mcn92 mcn92 released this 20 Aug 05:29

MiniLM-L6-v2 quantized derivative (apache-2.0), 25,474,368 bytes.

sha256: 3b14685a73bd7f30477be8dad89902b6e4bb55e49ec325c9e071c462cf89089b

Downloaded automatically by create-pancake-search when the blob is absent from an npm install (it is excluded from published tarballs); byte-identical to the wiki-inline artifact's encoder segment. Override the source with PANCAKE_ENCODER_WEIGHTS_URL.

Inline Wiki Pancake Artifact v2

Choose a tag to compare

@mcn92 mcn92 released this 20 Aug 05:45

Kind-3 complete artifact (Simple English Wikipedia, 456,153 chunks, MiniLM inline). Supersedes artifact-wiki-inline-v1, which stays published for older checkouts.

identity (manifest sha256): b25ff90d074fe889f02f6249ca5d4ce95099f2e1b04b9c1f71bd23f6d61b3828
size: 562,737,657 bytes

Changes from v1: the encoder declaration now states the kernel's real sequence policy (maxTokens 128, longInputs windowed-mean-pool — v1 declared the teacher's 256), and carries three verification vectors (contract section 4.4 mode 1), including a >128-token probe that pins the windowed mean-pool path. Index, corpus, and evaluation segments are unchanged; recall@10 stays 82.4%.

Inline Wiki Pancake Artifact v1

Choose a tag to compare

@mcn92 mcn92 released this 15 Aug 20:20

Complete .pancake artifact for the inline-transformer-v1 wiki demo.

Manifest identity:
77a08937d1414409d55e28b17bccf43a4ef374e76223eaefec590451ad9afba1

Verify:
node examples/05-one-file-search/compile.mjs --inspect pancake-wiki-inline.pancake
node examples/05-one-file-search/test-inline.mjs