Repository navigation
Added
-
Accept custom
os.PathLike[str]objects consistently for local model files, model folders, caches, extra files, stores and sinks. Path objects are always local model references; only strings may identify a Hub repository. Byte paths remain invalid, and existing validation error classes are preserved.Import
PathLike,ModelRefandWorkerCountfromfse.typingfor common input annotations.Accept one-dimensional NumPy text arrays as batches in encoding and prediction, including document IDs and position-based resume across stores of different lengths. Unicode, object and StringDType arrays use the existing dimension and row checks. Results retain the same bits as list batches. Score sink targets and store row locations expose their guaranteed array dtypes.
-
The lint check verifies local file references in Markdown and notebooks without network access. Architecture guidance links to the configured layers and guardrail paths, and contributor rules require validating public inputs before conversion.
-
The speed study compares ten README encoding configurations against released fse 2.0.0, with five paired rounds, fixed ST and paragraph inputs, production memory admission and a shared version-study budget from setup through allocation steps.
-
The versions speed study now pairs the 100-million-sentence writing cell against fse 2.0.0, with full warmups, alternating arms, persisted code and scale sample bit checks, and a streaming memory preflight.
-
Run version measurements, CPU profiles and memory profiles as steps of one allocation, sharing the saved time and money budget. Select the deciding candidate child, isolate profiler installations, and retain missing profile evidence in sealed results.
-
The README has a new section with a picture and a short example for several writing machines on a network drive, with one store for each writer and one view for the reading machine (DR-351: Mehrere Schreiber auf einem Netzlaufwerk in der README mit Grafik zeigen).
-
Complete public static return types before 2.1 (DR-323: Öffentliche Typen und vollständige statische Rückgabetypen vor 2.1 anbieten). Import
Encoding,LengthLevel,ScoreHead,SinkRowsandTokenizerfromfse.typing. Factories preserve subclass types where they already return subclasses;Encoder.quantizeandWordVectors.quantizereturn their base classes. Overloads distinguish array, int8 and sink results, vectors and length reports. Public batch iterators exposeclose()throughGenerator, and guaranteed array dtypes are explicit. Batch fields now include their possibleNonevalues, so typed callers must check them, for exampleassert batch.rows is not None.Headsmapping values have concrete model types. Values, validation errors, arithmetic and artifact formats stay unchanged. -
Enable the int8 prefilter on arm64 CPUs for searches without
rows=(DR-340: int8-Vorfilter für die Suche ohne rows= auf ARM freischalten, part of DR-168: Suche mit int8-Vorfilter und beweisbarer Fehlerschranke beschleunigen). Exact searches of int8 stores (mode="exact", and"auto"up to 8 queries; 1 to 8 queries,kup to 100, 256 to 1024 values per row, at least 65,536 rows) of a store or a view on arm64 CPUs with the dot-product extension now skip the rows that a provable int8 bound rules out and rescore the rest through the exact path; ids and score bits are those of the search without it. A search withrows=ordoc_ids=, withmax_per_doc, a float store, a smaller store, a largerkor more queries takes the previous path unchanged, as do x86 CPUs (the AVX2 variant stays off) and the numpy backend. A local measurement on Apple silicon found large savings for whole-store searches (not published here) and none forrows=, which is whyrows=is excluded. The log eventstore.prefilter(DEBUG) records each call that enters. No public API, arithmetic or persistence name changes. -
Add a local Aegis prompt-head preparation recipe (DR-318: Prompt-Köpfe aus dem Aegis-2.0-Datensatz außerhalb von fse trainieren und mit Modellkarte vorbereiten). It compares two pinned encoders, trains binary safety and ten-category heads outside fse, and prepares complete artifacts for grouped inference, with source reports, subset metrics and model cards. No upload or library change. File-only recovery preserves the frozen test result; the deliverable scan audits its narrow exception for pinned source URL literals.
-
Add publication-study search cells for exact float32, exact int8 and binary search, plus an explicit prefilter comparison selection (DR-308: Suchzellen mit Zeit je Anfrage in der Veröffentlichungsstudie anbieten). Whole-pool blocks share a calibrated pass count and report time per request with median and IQR. Check the measured final pass against exact score bits and ids, record binary recall and prefilter path evidence, and admit model groups against phase memory plans and the allocation budget. No library change or published speed numbers.
-
Add an offline Civil Comments classifier preparation tool under
tools/hub/heads/(DR-311: Toxizitäts-Classifier aus google/civil_comments außerhalb von fse trainieren und mit Modellkarte vorbereiten). It compares two pinned encoders on train and validation, trains a six-label head with scikit-learn outside fse, verifies complete and standalone artifacts, and renders a model card from recorded test metrics. Models and data stay outside the checkout. The final prepared model retains toxicity and insult from the full streaming SGD head, with validation-selected thresholds and one frozen official test evaluation. No upload or library change. -
Add two native-file classification headline cells for 100 million texts (DR-314: Kopfzeile für 100 Millionen klassifizierte Texte in der Veröffentlichungsstudie anbieten), with labels and probabilities, configurable head labels (default: toxicity and insult), resource admission before input preparation, per-round warmup, and sample bit checks against cpu and numpy. The untrained placeholder is restricted to functional rehearsals. Publication waits for a pinned real toxicity head. Protocols expose comparison-arm, resource-plan, and quiet-cell properties. Measurement tooling only; no library change.
-
Profile an addressed development or fse speed child with
speed.py --profileand the strictprofile@1contract (DR-313: Profiljob für einen adressierten Kindjob unter der Heavy-Sperre ausführen). One heavy lock covers access checks, offline input reconstruction and execution. Profiles have a separate deadline, fresh folders, checked origin hashes and executing identities, explicit coverage evidence and diagnostic receipts that comparison summaries refuse. Measurement tooling only. -
Reuse the shared pool for text scans and add
pool=tofse.add_sign_bits(DR-309: Gemeinsamen Pool auch für die Text-Zerlegung und add_sign_bits nutzen). File, encoded-string and Arrow scans use the pool threads and task places before encoding. Batch iterators scan on their iteration pool;StreamSourcecollections still scan inline. Sign-bit backfills share the pool limits and leave it open, including after failures. Chunks, error order and output bytes stay unchanged. -
Evaluate a named
cells@1decision across development cells (DR-298: Gesamtentscheidung über mehrere Zellen im Mess-Kern auswerten). Main cells must pass their existing noise and confirmation gates plus the declared practical saving. Edge cells must meet their factor bound in comparison and confirmation. Four explicit changes: optional automatic early stop after a final rejection (disabled in the local int8 study), exit 1 for an overall rejection or inconclusive result including partial selections,decision.jsonalso after preparation failure or budget exhaustion, and per-edge factor bounds without a global minimum. Cell decisions are unchanged. Measurement tooling only. -
Document named linear heads, score stores, document IDs and later filtering, add the Civil Comments filtering recipe, and extend the classification and folding recipes (DR-230: Kopf-Dokumentation und Toxizitäts-Beispielkopf vorbereiten). Documentation only; no change to the library.
-
Add
VectorStore.search(..., mode="binary", oversample=None)(DR-267: Binäre Suche mit search(mode="binary", oversample=) anbieten, part of DR-255: Binäre Vorsuche mit exaktem Nachrechnen als Opt-in anbieten). Stage 1 ranks the selected rows by the Hamming distance of their sign bits to the bits of the query (numpy,fse._reference_bits) and keeps the bestoversample * kper query (default 40, provisional until DR-264: Recall der Varianten und Kosten der Bitsuche gegen den int8-Vorfilter messen); stage 2 scores only those rows exactly with the path ofrescore, so the result issearch(mode="exact", rows=sorted(C))for the candidatesCbit for bit, andoversampleat least the selection givesmode="exact".rows=,doc_ids=,max_per_doc=1,workers=andpool=work; a shard without sign bits is derived from its rows with onestore.bits_derivedWARNING per store. New eventsstore.binaryandstore.bits_derived."binary"andoversampleare public values and freeze with 2.1; no new persistence name;scoreand score stores are unchanged. -
Add
ShardSink(sign_bits=True)(DR-266: Vorzeichenbits beim Schreiben mit ShardSink(sign_bits=True) mitschreiben, part of DR-255: Binäre Vorsuche mit exaktem Nachrechnen als Opt-in anbieten). Default False; without the switch the bytes written are those of before. With it each shard gets itssign_bits-<first row>artifact: the bits are derived in the hash thread from the codes committed, and published as a step of its own after the shard is renamed and its rows are counted, with the durability of the sink (nonetags the artifact). A failure of the bits is an error of the sink that never changes the rows already counted; a stop between a shard and its bits leaves a valid store. A sink made withresume=Truewrites the bits that the published shards it keeps lack (checked asfse.add_sign_bitschecks them), so any stop resumes to the bytes of an uninterrupted run; a sink that only appends does not. The bits equal those offse.add_sign_bits. Adds the fieldssign_bits(store.sink_opened) andsign_bits_shards(store.sink_closed). No new persistence name. No timing was measured; the write cost follows later. -
Add
fse.add_sign_bits(path, *, durability="full", workers=None)and the companion artifactsign_bits-<first row>of the sign bits of a shard (DR-265: Vorzeichenbits als Begleitdatei je Shard lesen und mit fse.add_sign_bits nachrüsten, part of DR-255: Binäre Vorsuche mit exaktem Nachrechnen als Opt-in anbieten). The artifact is an NpyContainer with the tensorsign_bits(rulesign_bits@1), bound to its shard byfse.shard_fingerprint; shards andstore.jsonare unchanged.VectorStore.open,refresh,verify, views andrepair_storeread and check it (a bound, damaged or unknown artifact isCorruptArtifactErrororUnsupportedArtifactError, never skipped; a shard without it stays valid; repair moves bits before their shard, and alone when they do not fit).add_sign_bitswrites the missing artifacts after hashing the vector bytes against the checksum of the shard, and returns their number. Adds the eventstore.sign_bits_addedand the fieldssign_bits_shards,sign_bits_bytes(store.opened),added_sign_bits(store.refreshed) andmoved_sign_bits(store.repaired).VectorStore.search(..., mode="binary")searches with them. New persistence names, which freeze with 2.1:sign_bits-,sign_bits,fse.kind=sign_bits,fse.bits=sign_bits@1,fse.shard_fingerprint. v2.0.0 reads such a store and passes over the entries. -
VectorStore.search,scoreandrescoretakepool=(DR-283: Gemeinsamen Pool für search, score, rescore und Köpfe über einen Store annehmen): with anfse.Pooltheir tasks run on its threads, at mostworkers(defaultpool.workers) at once, instead of threads of each call, so concurrent calls of a service share one set of threads. Bit-defined calls (mode="exact","auto"for at most 8 vectors, the int8 prefilter,rescore, head store mode) give the bits of a call without a pool,mode="blas"stays close as before, and a call withoutpool=behaves as before.workersabovepool.workersraisesValueError, a closed poolRuntimeError, before a row is read. A call takes a place of the pool per task until the task ends, and an error, a prefilter that gives up or a stream ended early cancels its unstarted tasks, waits for its running ones and leaves the pool open. Fixed: the store mode of a head (predict,predict_proba,decision_functionandpredict_batchesof aClassifierorRegressorgiven aVectorStore) acceptedpool=and ignored it; it now runs on that pool. See docs/architecture/store.md, "Concurrency". -
Measure multilingual quality with
benchmarks/quality.py(DR-276: Mehrsprachige statische Modelle auf multilingualen Benchmarks vergleichen): MTEB(Multilingual, v2) and NanoBEIR-multilingual of the pinned MTEB 2.26.25 join the benchmarks, and every task keeps the benchmark it came from, so a task name of several benchmarks (STS17 is English only in MTEB(eng, v2)) is refused unless written as"Benchmark/Task". Each task's data is unloaded after its evaluation. Local class K runs need explicitHF_HOMEandHF_DATASETS_CACHE, stop at the next task below a free disk floor, abort above a resident set limit, refuse tasks with a larger memory estimate and take a shared study deadline. Rows record the MTEB training relation, the execution path and the snapshot file hashes; summaries add coverage by set equality, tables per language and language pair, a common zero-shot set and reproduction against published MTEB results. New output probes compare fse with the native libraries. Benchmark tooling only; no change to the library. -
Add recipe 12 (
recipes/12-fastapi-search.ipynb) (DR-285: Rezept für FastAPI mit async /search über einen Store schreiben): a FastAPI service withlifespan, a store opened in each process, async/searchthroughanyio.to_thread.run_syncwith aCapacityLimiterof the size of one sharedfse.Pool,mode="exact", a view that is opened again at a fixed distance and swapped atomically, and a shutdown that waits for running searches; the second part replays three writers withshard=(i, 3)and one view over a shared folder. It checks that parallel requests give the bits of serial ones and that the view gives the bits of one store (claim class correctness, no timing). Network drives stay outside the crash contract. Pinsfastapi,uvicornandhttpxinrecipes/requirements.txt. Documentation only; no change to the library. -
Add recipe 11 (
recipes/11-reranker-candidates.ipynb, helpers inrecipes/_rerank.py) and the reranker study ofbenchmarks/quality.py(DR-277: Rezept und Experiment für fse-Kandidaten mit nachgelagertem Reranker erstellen): fse finds candidates in a store with document IDs, the texts come from the user's own source, and any local Sentence Transformers cross-encoder orders the top N again; an optional section fuses fse with BM25 by Reciprocal Rank Fusion (k = 60). The study (cells of kindrerankinbenchmarks/config/rerank-local.json, run with--run --local-run) measures NDCG@10 and recall of these arms on the 13 NanoBEIR tasks, class K. Documentation and measurement only; no change to the library, no reranker and no API client in fse. -
Exact searches of int8 stores (
mode="exact", and"auto"up to 8 queries) on arm64 CPUs with the dot-product extension skip rows that a provable int8 bound rules out and rescore the rest through the exact path ofrescore; ids and score bits are unchanged, and anything outside the proven domain takes the previous path (DR-168: Suche mit int8-Vorfilter und beweisbarer Fehlerschranke beschleunigen, part 3, DR-278: int8-Vorfilter in die exakte Suche einbinden). The domain is 1 to 8 queries,kup to 100, 256 to 1024 values per row and at least 65,536 selected rows, withoutmax_per_doc,rows=anddoc_ids=. Adds the log eventstore.prefilter(DEBUG:queries,k,rows,pairs,fallback,workers). No public API and no persistence name changes. -
Read sentence-transformers models whose table is float16, such as
hotchpotch/bekko-static-embedding-v1, as fse float16 tables (DR-274: float16-Tabellen von sentence-transformers-Modellen als fse-float16 laden):Encoder.from_pretrainedkeeps the table unchanged (half the memory of float32), the kernel widens it exactly and every sum runs in float32, so the rows are the bits of the encoder of the widened table;truncate_dimandnormalizework as for float32. sentence-transformers itself takes the mean of such a table in float16, so the comparison is with its weights made float32, within the bound of the float32 tables. Additive: models that loaded load as before, bit for bit; int8 and float64 tables of sentence-transformers models and every non-float32 table of Model2Vec models are still refused. No persistence name is added or changed. -
Create the GitHub release in CI for every
v*tag (DR-261: GitHub-Release bei jedem Release automatisch anlegen und den Ablauf als Skill festhalten): a new jobgithub-releaseruns after the PyPI upload and is the only job withcontents: write. It names the releasefse X.Y.Z, uses the## [X.Y.Z]section of this file as its text, attaches the sdist and the wheels, and marksaN,bNandrcNversions as pre-releases.tools/release_notes.py(withtests/test_release_notes.py) also runs before the PyPI upload and stops the release if the tag is malformed, the section is missing, empty or duplicated, or a built file does not carry the version of the tag. New repository skill.agents/skills/release. No change to the library. -
Allow
Encoder.from_pretrained(..., truncate_dim=k)for Model2Vec models (DR-260: truncate_dim auch für Model2Vec-Modelle erlauben): the encoder keeps the first k dimensions of the table, as it already does for sentence-transformers models, beforenormalize@1, andfse.sourcerecords thedimsof the cut, whichsave,load,save_pretrainedandquantizekeep."model"(the default) and None keep all dimensions, k equal to the width too, a larger k raises. The columns a cut drops are checked for NaN and infinity, so a corrupt table is refused with or without a cut (also for sentence-transformers models). fse encoders still refusetruncate_dim; without the argument nothing changes. -
Add
fse.Pool(workers=None, *, start=False, max_tasks=None)and the keyword-onlypool=of the ten methods that tokenize text (DR-241: Heißen Pfad vor 2.1 profilieren und die besten ein bis zwei Engpässe beheben, rank 2, with DR-185: Optionalen Thread-Pool über mehrere Aufrufe anbieten). A pool is a set of threads that runs the tasks of any number of calls, so a server that encodes many small requests, or a loop over many small batches, no longer starts threads for every call;start=Truestarts them when the pool is made. A stream (encode_batches,predict_batches) now runs all its batches on one pool that lives for the iteration and is closed when it ends, fails, is closed or is collected, where each batch made its own threads before (the profile of the stream cells showed 59 to 81 % waiting and 15,000 to 28,000 thread entries for 3,660 calls). The default stays that no thread outlives a call; a pool passed withpool=is closed by its owner.workers=may not exceedpool.workers. Bits, errors and the order of commits are unchanged. Additive:v2.0.0behaves exactly as before. -
Add the guide External indexes on fse shards and recipe 10 (
recipes/10-external-ann-indexes.ipynb, helpers inrecipes/_ann.py) (DR-169: Shards optimal für externe ANN-Indizes vorbereiten, part 4): how FAISS, USearch and hnswlib take the rows ofiter_batches(VectorBatch.normalized(), int8 codes without a copy), howVectorStore.rescoregives their hits the exact score, and how to add rows, continue after a stop, repair and use views. Documentation only; no change to the library. -
Add
VectorBatch.inv_normsandVectorBatch.normalized()(DR-169: Shards optimal für externe ANN-Indizes vorbereiten, part 3): every batch ofVectorStore.iter_batchesnow holds the stored inverse norms of its rows as a read-only float32 view, andnormalized()returns a new float32 array of approximately unit-length rows for an external index (FAISS, hnswlib):float32(stored) * inv_normper element, int8 from the codes without the scale. The bits are fixed in the default IEEE environment; a batch built by hand is checked and refused withValueErrororTypeErrorif it does not fit. No new persistence name. -
Add
VectorStore.rescore(queries, candidates, /, k=10, *, workers=None, max_per_doc=None)(DR-169: Shards optimal für externe ANN-Indizes vorbereiten, part 2), which scores the hits of an external index (FAISS, hnswlib, USearch) exactly: for every query it returns the bestkrows among its own candidate positions (-1marks a gap; the lists may be unsorted, hold duplicates, be empty or shorter thank), bit for bit assearch(query, k, mode="exact", rows=sorted(candidates without -1)), also withmax_per_doc=1and on views. A matrix result is filled up on the right with ids-1, scores-infand document IDs-1; a single query is not. uint64 candidates are rejected. It adds the log eventstore.rescore(DEBUG:queries,k,pairs,read_rows,workers,max_per_docwhen set), and no persistence name. -
Add resuming of endless streams at a position that can be read again (DR-252: Unendliche Streams an einer wiederlesbaren Position fortsetzen, part 5 of DR-165: Shards und Resume für große Jobs einführen):
Encoder.encode_batches(batches, sink=, positions=True)andpredict_batches(..., positions=True)ofHeads,ClassifierandRegressortake batches(texts, position)(or(texts, doc_ids, position)) into sinks made withresume=True, a store of the new modefse.resume=source_position@1. A position is an opaque text of 1 to 65,536 UTF-8 bytes (a Kafka offset, a byte offset); fse stores it and never reads it. Every shard keeps the position of the batch it ends in underfse.source_positionandfse.source_position_row, and the root keeps the start of the stream. After a crashsink.source_position, alsoVectorStore.source_positionandScoreStore.source_position, is(P, skip): seek the source toP, give its batches again, and the firstskiprecords, which the store holds, are dropped without being encoded; every other record is written as an uninterrupted run writes it, bit for bit. Every batch of the replay gives one result:first_rowis the replay row of its first computed record,n_rowsthe records computed (0 for a batch the store holds) anddurable_rowsthe replay rows that are durable, so a consumer commits its input up to there. A vector store and a score store written together resume together, also when they hold different numbers of rows (seek by the one that holds fewer). A store of this mode has astore.jsonof up to 1 MiB,membersof a view show thepositionof each member, and a view reads stores of streams in the order given. A store of one resume mode is not resumed by the other:encode(texts, shard=)andpredict(..., shard=)writesource_rows@1, the batches withpositions=Truewritesource_position@1. -
Add resumable score stores and resumable heads (DR-250: Score-Stores und Köpfe wiederaufnehmen, part 3 of DR-165: Shards und Resume für große Jobs einführen):
fse.ScoreSink(..., resume=True)writes a score store of the modefse.resume = source_rows@1(the same six keys as a vector store).Heads.predict,Classifier.predictandRegressor.predicttake the new keyword-onlyreturn_vectors=,vector_sink=,score_sink=,doc_ids=andshard=: with a sink made withresume=Truethe call resumes, runs only the rows the stores do not hold yet, bit for bit as an uninterrupted run, and returnsNone(withlengthsthe pair(None, LengthReport)). A vector store and a score store of one call may hold different numbers of rows: a first phase catches the shorter one up, a second runs the rest into both. The outputs of the heads are dropped, so the memory of a call does not grow with the rows. A sink made withresume=Trueis refused bypredict_batchesand by every other entry. Without the new keywords nothing changes. -
Add resumable vector stores and fixed partitions (DR-248: Vektor-Stores wiederaufnehmen und feste Partitionen schreiben, part 1 of DR-165: Shards und Resume für große Jobs einführen):
fse.ShardSink(..., resume=True)writes a store of the new modefse.resume=source_rows@1, andEncoder.encode(texts, sink=sink, shard=(i, n))takes partiofnof a source that can be read again (a list, tuple or array ofstr, token lists or the path of a text file; parts are whole blocks of 16,384 rows). The call hashes the whole source and compares its fingerprint, the encoder, the dtype and the partition with the store: another source or setup is aValueErrorbefore any row is reserved. A stopped run continues with only the missing rows, bit for bit as an uninterrupted run, and a store that is complete is left alone. Underdurability="full"the sink syncs the store directory once when it appends. A report oflengths=covers the rows of that call and itsfirst_rowis their source row. Streams and Arrow text columns are refused with aTypeError;encode_batchesand the heads refuse a sink made withresume=True. The format (six keys:fse.source_fingerprint,fse.source_form,fse.execution_fingerprint,fse.source_first_row,fse.source_rows,fse.source_shard) is also read and repaired for score stores; their sink follows. These names freeze with 2.1. Recipe 03 uses the new calls. -
Add
VectorStore.open_view(paths)andScoreStore.open_view(paths)(DR-251: Mehrere Stores als eine Sicht lesen, part 4 of DR-165: Shards und Resume für große Jobs einführen), which read several stores as one store whose rows follow each other, without merging anything on disk. Search and score withmode="exact",take_*,get_*,iter_batchesandpredict_batchesgive the bits and the order of ties of one store that holds the same rows (BLAS blocks follow the shards, somode="blas"and"auto"above 8 queries differ in the last bits, as between two shard sizes). The stores must hold the same encoder, dtype, dimension and document-ID mode (scores: the same heads, columns and paths, and the same thresholds across each boundary); stores written withEncoder.encode(shard=)must be parts of one source (each part at most once, not mixed with stores without a source) and are read in the order of the parts; parts may be missing or unfinished (a snapshot, positions are the concatenation,sourceisNone), and only if all parts are there and finished is a position a row of the source. A view has nopathand norefresh(ValueError),verifyruns over each store, andclosecloses them all. New read-only helpers on both store classes and views:paths,members(one mapping per store:path,first_row,rows,source),source(a mapping for a store written from a source, and for a view of all its parts) andlocate(ids), which names the store and the row of each position. New eventsstore.view_openedandscores.view_opened. -
Add
VectorStore.refresh()andScoreStore.refresh(), which take the shards a writer has published since the store was opened into the store itself and return the number of new rows (an int; 0 if there are none). A reader of a store that is still written sees new shards without opening it again, and only the new shards are checked and, if taggeddurability="none", hashed. Searches, scores and reads that run, and generators already made, keep the rows they started with: every public read now takes its shards and row count once, when it is called (a generator ofiter_batchesorpredict_batchesat the call, not at its first block). A gap, an orphan or a break of the chain raisesCorruptArtifactError, a store whose root or last shard changed since it was opened raisesRuntimeError; in both cases the rows stay as they were. A repair meanwhile is detected at best (repair_storestays offline).VectorStore.closeraises while a refresh runs. New eventsstore.refreshedandscores.refreshed(DEBUG). (DR-249: Stores während des Schreibens mit refresh() nachladen) -
Add
lengths=,count_truncated=andon_truncation=to the ten methods that tokenize text (Encoder.encodeandencode_batches,Heads.predictandpredict_batches,Classifier.predict,predict_proba,decision_functionandpredict_batches,Regressor.predictandpredict_batches), which show what the length rule of the model cut.lengths="summary"or"per_row"returns(result, fse.LengthReport)(a 2.0 return type becomes a pair, with a sink(None, report)), makes every batch of a*_batchescall the pair(batch, report)and givesHeadsBatchthe fieldlengths;EncodedBatchandPredictedBatchare unchanged. The report holds the sums (rows cut by characters, by tokens, empty rows, tokens, unknown ids dropped) and with"per_row"the same per row;count_truncated=Trueadds the tokens and characters cut.on_truncation="raise"raises the newfse.TruncationError(ValueError)(row,cut,limit) for the first sentence the rule would cut. Without the keywords nothing is recorded and every result is bit for bit the same; aVectorStoreinput with them raises before a row is read. A sentence that reachesmax_tokenscosts at most one more word for the flags, and counting tokenizes the rest of a cut sentence. (DR-163: Sichtbar machen, was beim Tokenisieren gekürzt wurde) -
Let
Classifier.predict_batchesandRegressor.predict_batcheswrite to sinks withoutfse.Heads(DR-243: Umwege über fse.Heads bei einzelnen Köpfen entfernen): the new keyword-onlyvector_sink=,score_sink=anddoc_ids=work as onHeads, and a single head and a group of one write the same files and append to each other's stores.sink=keeps exactly its role and behavior of 2.0 and excludes the new keywords; aVectorStoreinput refuses them (the outputs of a store keep going tosink=).ScoreSink(path, model, name=...)andScoreStore.open(path, heads=model, name=...)take a single head; the name is required, because it is written to the store.ScoreSink(store=...)is nowstored=...(unreleased, no alias). NewLinearModel.path(return_vectors=...)names the path a head takes, asHeads.pathsdoes for a group. One validator checksdoc_idsforencode_batches,Heads.predict_batchesand single heads: a wrong type is aTypeError, a wrong value aValueError, and the message names the method. Shard sinks of fse in document-ID mode are refused when the call brings no IDs, but a sink of your own withhas_doc_idsTrue runs as in 2.0, also inencodeandencode_batches(DR-240 had refused it). -
Add
VectorStore.take_vectors(rows, /)andtake_doc_ids(rows, /), which read the rows at a list of store positions (a list or an int64 array of one or two dimensions, soresult.idsworks as it is) in that order with duplicates kept, andVectorStore.iter_batches(block_rows=, start=, stop=)with the newfse.VectorBatch, which gives blocks of read-only views of the mapped shards (int8 asQuantizedRows). Negative positions raise. docs/document-ids.md lists store positions apart from run positions. (DR-237: Vektoren und Dokument-IDs nach einer Liste von Zeilen und blockweise lesen) -
Add durable scores of a group of heads (DR-162, Scores und Vorhersagen dauerhaft speichern):
Heads.predict_batches(..., score_sink=sink, doc_ids=...)writes the scores of every head to a score store (fse.ScoreSink(path, heads, stored=None, doc_ids=False, shard_rows=None, durability="full")), andfse.ScoreStorereads it without an encoder:iter_batchesgives the stored values as read-only views,predict_batchesgives labels or values asHeadsdoes, with new multilabel thresholds throughthresholds=, without new inference. A classifier stores its decision (the float32 logits, from which probabilities and labels follow bit for bit) or its probabilities, a regressor its values. The formatprediction_scores1 (store_version2) holds one tensorscoresfor all heads, optionaldoc_idsandthresholds.<head>;store.jsonmay be 1 MiB (stores of vectors keep 64 KiB); a head keeps the path it ran on (folded,unfolded, ...) and an append with another path is refused.HeadsBatch.durable_rowsreports the durable rows of a run;fse.ScoreBatchcarriesstore_row,outputsanddoc_ids. TheEncoderrefuses aScoreSink, and so doessink=ofClassifierandRegressor, with a hint toscore_sink=.fse.repair_storerepairs both kinds of stores, andstore.repairedgains the fieldkind. New eventsscores.sink_opened,scores.shard_published,scores.sink_closed,scores.openedandscores.verifiedonfse.store. A root whosefse.kindnames no kind of fse is now anUnsupportedArtifactError; with keys of its own it was aCorruptArtifactErrorbefore. -
Add
vector_sink=toHeads.predict_batches(DR-229: Vektoren und Scores in einem Lauf in zwei Stores schreiben): one pass writes the vectors of the encoder to aShardSink(or anyfse.Sink) and, withscore_sink=, the scores to aScoreSink, two folders with the samedoc_ids;HeadsBatch.durable_rowsis the minimum of both sinks.vector_sinkneedsreturn_vectors=True(heads on raw means take the unfolded path). Each folder survives a process crash on its own, and a power loss underdurability="full"; rows beyond the confirmed count may be in one folder only and are written again on a replay. Two shard stores of unequal length take a run only withdoc_ids.ClassifierandRegressortake the samevector_sink=(DR-243: Umwege über fse.Heads bei einzelnen Köpfen entfernen). -
Export
fse.DocumentSearchResult, the typeVectorStore.searchreturns with document IDs;ThresholdInputis no longer exported (it was never released). (DR-240: Kleine API-Korrekturen aus dem zweiten API-Review vor 2.1 umsetzen) -
Add
fse.Headsandfse.HeadsBatch: an immutable, read-only group of named heads of one encoder (Heads(quality=q, toxic=t), names[a-z][a-z0-9_]*, equal encoder fingerprints, all complete or all standalone) whosepredictandpredict_batchestokenize each text once and score all heads in one pass, with the bits of each head alone (softmax, multilabel, regression, folded, normalized and mixed heads, with or without the vectors).Heads.pathsshows the path each head takes. A loneClassifierorRegressornow runs through the same internal runtime with unchanged public API, outputs and sinks. The group has no store mode and no file format of its own; its sinks are thevector_sink=andscore_sink=of later entries. (DR-161: Mehrere Köpfe in einem Durchgang rechnen (Softmax und Mehrfach-Labels)) -
Import finished linear Model2Vec classifier heads (model2vec 0.10):
Classifier.from_pretrainedalso reads the export of aStaticModelPipeline(local folder or Hub repository at one commit), and the newClassifier.from_model2vectakes aStaticModelPipelineor a trainedStaticModelForClassification(onlyto_pipelineruns). One linear layer with softmax (softmax@1) or sigmoid (multilabel@1, cutoffs 0.5, changeable withwith_thresholdsorthresholds=), the fine-tuned table with its token weights multiplied in,normalizeand the classes in order. Hidden layers, regression heads, quantization, separate token weights, non-float32 heads, a legacypipeline.skopsexport (never opened) and unknown keys raise with a reason; nothing is trained and no pickle or skops is opened. Float64 tables and separate token weights of any Model2Vec model now raiseUnsupportedArtifactErrorinEncoder.from_pretrainedtoo (before:KeyError: 'F64'and the vocabulary quantization message). (DR-164: Lineare Model2Vec-Köpfe importieren) -
Add
max_per_doc=1toVectorStore.search: the k best documents per query, each by its best-scoring row (ties to the smaller position), combinable withdoc_idsandrowsfilters. Only the value 1 is allowed in this version; larger values may follow. Behavior change:searchnow returns at mostkresults,min(k, candidates), also whenkis larger than the number of rows of an unfiltered search, which earlier raised aValueError; empty results have the usual shapes and types. Exact mode keeps the bits of the ordinary search for the returned rows. (DR-218: Suche mit höchstens einem Treffer je Dokument anbieten) -
Import Platt-calibrated scikit-learn classifiers (CalibratedClassifierCV with method="sigmoid" and ensemble=False, binary or one-vs-rest) by folding the calibration into the head's weights and bias in float64 under the kernels' floating-point environment, rounded once to float32. Add softmax temperature to Classifier.from_arrays (finite, greater than zero). Both write linear_head format 2 with the label rule linear@2, now also defined for single-label heads as the first class of largest probability, so predict costs more than under linear@1; no new metadata key. ensemble=True or "auto", isotonic and temperature calibration, and calibrated OneVsRestClassifier estimators are refused. Existing format 1 heads keep their identifiers, fingerprints and predictions. (DR-227: Kalibrierte Klassifikatoren importieren und Softmax-Temperatur unterstützen)
-
Add pydantic-validated thresholds, read-only classifier thresholds and immutable
with_thresholdsupdates. Allow explicit class names for all sklearn classifier imports, preserving output order, and document external training through deployment. (DR-226: Mehrfach-Labels mit benannten Schwellen unterstützen) -
Add independent multilabel classifiers (
multilabel@1,linear@2) with strict per-label thresholds, including partial named overrides such asthresholds={"sport": 0.4}. Import fitted multilabel OneVsRestClassifier models with supported affine sigmoid heads. New models use linear_head format 2 andhead.thresholds; existing format 1 models keep their identifiers, fingerprints and predictions. (DR-199: Lineare Köpfe zum vollwertigen Bestandteil von fse machen) -
Add document-ID write, unfiltered-search and filtered-search cost profiles to shared local diagnosis, including normalized public-call latency and filter-workspace measurements. Numerical checks, timing floors and resource guards remain enforced.
-
Add optional int64 document IDs to vector stores, per-batch ID input, and exact filtered search by document IDs or physical rows. Duplicate document IDs retain every matching row. New ID-enabled stores use format 2 while vector-only stores remain format 1. DR-215 (Eigene Dokument-IDs im Store speichern (row_ids)).
-
Add an evidence-grounded fse walkthrough skill with offline synthetic demonstrations and one-item pacing. Skill-only instruction and metadata changes skip heavy PR and ordinary master CI jobs; release publication still requires full master CI.
-
Add a shared
fse-benchmarkskill for Codex and Claude Code, with measurement planning, validity checks and sourced lessons.
Changed
-
The documentation, the docstrings and the recipes now match fse 2.0 and the code of master (DR-326: Veraltete Dokumentation und Mac-Tempozahlen vor 2.1 bereinigen). The recipes label their timings with the Apple M1 Pro MacBook that produced them, dead links and local paths in notebook output are gone, and the migration guide, the security policy and the architecture pages describe the current behavior.
-
Correct the remaining contradictions between the documentation and the code (DR-336: Weitere Widersprüche der Dokumentation aus der Prüfung vor 2.1 korrigieren). The README marks which examples need fse 2.1 and lists all recipes, the architecture pages state the tokenizer, dtype, resume and shard-chain rules as the code applies them, the new
docs/README.mdindexes the guides, and the docstrings of public properties,Encoder.encode,ShardSink,Classifier.loadandRegressor.loaddescribe their parameters. Text, docstrings and two recipe code cells only; no library behavior changes. -
Collect user-visible changes as separate towncrier fragments instead of editing the shared Unreleased section (DR-343: CHANGELOG-Einträge als Fragmente mit towncrier sammeln). Release preparation assembles them into CHANGELOG.md, and publishing refuses leftover fragments.
-
Raise
fse_revisioninbenchmarks/config/pins.jsonfrom a1e9bf5 to 9a63400, the master after the fixes of the user stress test of DR-289 (Nutzer-Stresstest über alle Funktionen von 2.0 bis 2.1 mit Codex durchführen), so the Linux studies install the 2.1 candidate that the version comparison against fse 2.0.0 measures (DR-350: fse_revision auf den 2.1-Kandidaten für die Benchmarks setzen). Historical results keep their own pin. Measurement tooling only; no change to the library. -
Document that binary search with
oversample=Noneuses factor 40 throughout fse 2.x,
including minor releases. Other factors require an explicit positive integer. -
Reject unordered sets and mappings as class labels, boolean elements mixed into head weights, biases or word-vector indices, and masked arrays before conversion discards their masks, including temperature imports. These stricter checks affect existing 2.0 APIs. Ordered dictionary keys remain accepted. Valid inputs retain their float32 bits.
-
Narrow the new 2.1 surface before the freeze (DR-322: Neue öffentliche 2.1-Oberfläche vor dem Freeze bereinigen (Pool, score_schema, dropped_total)).
fse.Poolkeeps its constructor,workers,max_tasks,closed,close()and thewithblock;submit,try_acquire,acquire,release,check_openandinsideare internal (submitbypassedmax_tasks).ScoreSink.score_schemais internal; read what a store holds throughScoreStore.heads.LengthReport.dropped_totalis nowdropped_unknown_total, the sum ofdropped_unknown(unknown ids removed byfse.unknown=drop; tokens cut by the length rule aretruncated_tokens_total). All three names were never released, so there are no aliases; no persistence names change. -
Raise
fse_revisioninbenchmarks/config/pins.jsonfrom 624251d (fse 2.0) to a1e9bf5, the master after DR-317 (DR-320: Pin der fse-Revision für die Linux-Studien auf den aktuellen master heben), so the Linux studies install an fse withfse.Heads. Historical results keep their own pin. The speed allocation gains abaseenvironment (fse 2.0 at 624251d, lockspeed-base.lock.txt) passed as--base-python, so--study allis no longer refused for itsversionscells; the plans of the existing cells are unchanged. The developmentDockerfilenow uses the image ofpins.json(CPython 3.13.18) instead of 3.13.16-slim. The speed rehearsal machine check allows 6 GiB instead of 4 GiB, because the bekko cells fail in 4 GiB with aMemoryError(maintainer approval 2026-11-08). Measurement tooling only; no change to the library. -
Start sink threads only with their first work (DR-317: Sink-Threads eines nicht geschlossenen Sinks nicht unbefristet weiterlaufen lassen). Constructing or inspecting a
ShardSinkorScoreSinkstarts no thread. Write and hash start at the first commit, publish at the first finished shard. Used sinks still requireclose()or awithblock. Thread-start failures follow the shared writer error and cleanup protocol. -
Serve development lock waiters in registration order and report queue position (DR-315: Wartende an der Heavy-Sperre in Ankunftsreihenfolge bedienen). Canceled and crashed waiters leave the queue; existing lock owners are never taken over. FIFO requires draining old waiting controllers and wrappers before rollout. Measurement tooling only.
-
Name the claim classes of measurements correctness, descriptive, regression, optimization and publication (previously K/D/R/O/P) (DR-294: Messklassen mit lesbaren Namen statt Einzelbuchstaben benennen), in the contributor guides, the measurement guide, the study descriptions and the local quality runs, whose rows now record
"class": "correctness". Results written before keep their one-letter value;benchmarks/quality.py --summarizemaps it to the same class, so old and new rows of one configuration give the same summary. -
Make the task of a head give up the GIL far less often (DR-241: Heißen Pfad vor 2.1 profilieren und die besten ein bis zwei Engpässe beheben, rank 1): the checks of the backend methods compared arrays with
np.may_share_memory, which releases the GIL, up to 20 times per task offse.Heads(11 for oneClassifier); they now compare the byte bounds without a numpy call (fse._backends._base_checks.overlaps, and_overlapsin the extension), with the same result. A task whose heads read only the folded table also pools in one section without the GIL, asEncoder.encodedoes, instead of building and checking aBatchwith about ten more numpy passes over its tokens; a failed task is run again on the old path, which raises the error of its row. No arithmetic, API or format changes: the outputs are bit-identical to those of the numpy reference. -
VectorStore.closenow raisesRuntimeError("the store is in use ...")while a search, score or read runs; the message said "being searched" before. (DR-237: Vektoren und Dokument-IDs nach einer Liste von Zeilen und blockweise lesen) -
ScoreStore.iter_batchesandpredict_batchesraiseRuntimeErrorafterclose()also before the first block; they ended quietly before. The functions are unreleased; their entry for DR-162 (Scores und Vorhersagen dauerhaft speichern) stands in this section. (DR-242: ScoreStore.iter_batches nach close() vor dem ersten Block mit Fehler beenden) -
A second run into a
ShardSinkraises an error that says a sink takes exactly one run. AddEncodedBatch.vectors, an alias ofrows. Explain store positions, run positions, batch lengths, durable prefixes and document IDs in docs/document-ids.md. (DR-240: Kleine API-Korrekturen aus dem zweiten API-Review vor 2.1 umsetzen) -
Document which rows the
sinkofpredict_batchestakes with and withoutreturn_vectorsin a table, and name that role in the error for a sink of the wrong width. (DR-240: Kleine API-Korrekturen aus dem zweiten API-Review vor 2.1 umsetzen) -
A sink is in document-ID mode only with
has_doc_idsexactly True; other values (0, 1, None) are ignored unless the call brings document IDs, where a value that is no bool raises aTypeError, so sinks of 2.0 keep running.encode(doc_ids=True)raises aTypeErrorthat names the difference toencode_batches(doc_ids=True). A shard sink of fse in document-ID mode is refused before a batch is read when the call brings no IDs, with a hint for the working route; a sink of your own withhas_doc_idsTrue runs as in 2.0 (DR-243: Umwege über fse.Heads bei einzelnen Köpfen entfernen). Documenthas_doc_idsandcommit(..., doc_ids=)offse.Sink. (DR-240: Kleine API-Korrekturen aus dem zweiten API-Review vor 2.1 umsetzen) -
Classifier.thresholdsis None for heads without cutoffs instead of raising;with_thresholdsstill raisesValueErrorthere. (DR-240: Kleine API-Korrekturen aus dem zweiten API-Review vor 2.1 umsetzen) -
Rename the unreleased document-ID persistence names of store format 2 from
row_idstodoc_ids(doc_ids.npy,fse.doc_ids,fse.doc_ids_min,fse.doc_ids_max,fse.doc_ids_sorted); stores written by a pre-release of 2.1 are refused withUnsupportedArtifactError(also byverify()) and must be written again. AddVectorStore.has_doc_idsandVectorStore.get_doc_ids(start=, stop=).DocumentSearchResultkeepsdoc_idsin_replace,_asdictandrepr, and_makeraises instead of dropping them.encode_batchestakesdoc_ids=Truefor(texts, doc_ids)batches and checks it against the sink when called. DR-235 (Dokument-ID-Persistenz vor 2.1 in doc_ids umbenennen und Lesewege ergänzen). -
Refuse local
speed.py --diagnosetiming runs whose controller does not run at nice 0, before any directory, preparation or budget is used, and name zsh's default BG_NICE (which adds 5 to the nice value ofcmd &jobs) as a usual cause. Diagnostic rehearsals and the prepare phase are exempt, measured children still require nice 0 and the heavy wrapper for builds and checks still targets nice 10. (DR-234: zsh-Falle BG_NICE bei lokalen Messungen abfangen und dokumentieren) -
Require a Linux speed comparison only for published speed numbers and release guarantees (class P) or when the maintainer has explicitly approved a Linux machine for the ticket. Pull requests name their claim classes and their actual Linux comparison status, and state explicitly when none was run. Numerical conformance, bit identity and persistence guarantees are unchanged.
-
Make all macOS PR jobs optional, including wheels. Add a targeted manual macOS scope bound to an open PR and its immutable current head SHA, with explicit source/artifact identities and stale-head rejection. Preserve automatic master/tag coverage, full manual verification and release gates. CI summaries fail closed on failed, cancelled, abandoned or unknown started jobs.
-
Couple measurement effort to claim classes. Add candidate-only descriptive costs, a conservative five-pair regression screen, small independent adapter probes and a total local wall-time admission budget including preparation and load/lock waits. Existing numerical, persistence, optimization and Linux publication guarantees remain unchanged.
-
Limit macOS tests on pull requests and release tags to Python 3.13, alongside arm64 wheel builds. Master pushes retain the full Python 3.11 to 3.15 macOS matrix and sentence-transformers comparison. Release publication now requires successful full master CI for the exact tagged commit, verified through the GitHub API.
-
Avoid a second stored-row copy when rescoring candidates, including stage 2 of binary search (DR-310: Doppelte Zeilenkopie beim exakten Nachrechnen in rescore und der binären Suche vermeiden). Sort private candidate sections in place and gather ascending parts directly into their output. Scores, IDs and tie order keep the same bits. Public candidate arrays remain unchanged.
Removed
- From the next release, macOS Intel (x86_64) wheels are no longer built, tested or published. macOS wheels target Apple Silicon (arm64) only. Source distributions remain available, but builds on Intel Macs are unverified and no longer tested in CI. Existing 2.0.0 wheels remain available.
Fixed
-
Describe the thresholds of
ScoreStore.predict_batchescorrectly (DR-244: Doku und CHANGELOG für 2.1 auf den Endstand bringen): the defaultthresholds=Nonekeeps the stored cutoffs of every head, and only a head mapped toNoneresets its labels to 0.5. The behavior is unchanged. Thefsemodule docstring now also namesHeads,ShardSink,VectorStore,ScoreSink,ScoreStoreanddoc_ids. -
MemorySink, ShardSink and ScoreSink now reject boolean, floating-point, negative and oversized row arguments before allocation, reservation or commit state changes. Target sizes must fit the platform array index and byte-size limits, including sinks used by Heads. Targets spanning shards allocate their own buffer before reserving any shard rows, so a failed allocation cannot create a large chain of reservations. Python and NumPy integers remain supported.
-
FastText parameters accept NumPy integers like other integer API parameters, check their
types and bounds before conversion, and store them as Python integers. Invalid types,
including booleans and floats, raise an error naming the affected parameter instead of
failing during JSON serialization.Batch calls with a resume sink now explain the supported
positions=Trueroute for
Encoder, Classifier, Regressor and Heads, alongside the eager and non-resume alternatives. -
Correct the documentation of nested calls on a pool: inline text encoding and prediction calls are allowed, while reentry into an already active pool guard is refused, including single-task store calls with more than one worker. An outer inline text call does not activate that guard. Clarify that views returned by VectorStore and ScoreStore remain readable after close, and that an unknown keyword to StaticModel.from_pretrained raises pydantic.ValidationError.
-
Recipe helpers reject masked ANN candidate keys, hit counts, and validity flags before conversion, and the reranker recipe downloads complete pinned model snapshots into its temporary workspace. Integration instructions reflect the published fse and adapter packages, and the recipe guide includes installation with fse from PyPI. Recipe provenance records the installed fse version and permits release archives without Git metadata.
-
Store search and rescoring now reject masked queries and booleans mixed into
numeric queries. Store scoring also rejects these inputs in weights and bias,
before conversion can discard a mask or turn a boolean into a number.
Mappings and sets are rejected before conversion where an ordered sequence is required. -
Reject masked arrays with
TypeErrorbefore copying or taking ownership of word-vector tensors inWordVectors.from_arraysandEncoder.from_arrays, including scales, ngrams and vocabulary arrays. The gensim adapter also rejects masked source rows before conversion. Classifier labels and internal encoder weights reject masks instead of silently discarding them. Scikit-learn and Model2Vec imports also validate original weights, biases, labels and embedding tables before conversion; scikit-learn numeric attributes reject boolean arrays and scalars. -
The first table fold for a classifier, regressor or group of heads now respects the call's worker limit and shared pool. The caller helps when pool capacity is occupied, avoiding deadlocks with nested sink calls. Concurrent first calls share one completed fold; later calls keep their own execution settings.
-
Loading structurally damaged WordVectors, encoders and linear heads now raises
CorruptArtifactError. Unknown formats still raiseUnsupportedArtifactError, and array constructors retain their argument errors. -
Poolnow validates constructor arguments consistently with the other public
entry points, raisingpydantic.ValidationErrorfor invalid worker counts,
task limits or start flags. Positive Python and NumPy integer counts remain accepted.
Its docstring now distinguishes task places held until commit during text encoding
and prediction from store task places released at task completion, with finished
store results bounded by each call's own window. -
Fix the unreleased 2.1 ScoreStore outputs of a one-label multilabel classifier to keep shape
(n, 1), including empty selections and joined views. Head and store outputs now share one shape rule. Threshold updates validate supplied label keys before overlaying current cutoffs. -
Fix the type check of
kandcapacityin the int8 prefilter pass under Cython 3.1.0 and 3.1.1 (DR-329: Typprüfung von k und capacity im int8-Vorfilter unter Cython 3.1.0 reparieren). Those versions compileisinstance(x, int | np.integer)to a check that is always false (Cython issue 6420, fixed in 3.1.2), soint8_pass(..., k=10)raisedValueError. Both checks use the tuple form now, the build requiresCython>=3.1.2, and the three measurement locks (speed-main,speed-base,quality, and the inputquality.in) pincython==3.1.8instead of3.1.0, which changes the package identity of the measurement state. The prefilter is not enabled on master, so no user was affected. -
Return
(result, vectors)fromHeads.predict,Classifier.predictandRegressor.predictwithreturn_vectors=Trueand no sinks (DR-312: return_vectors=True in Heads.predict ohne Sinks nicht still übergehen); previously the call raised ValueError. With lengths the return is(result, vectors, LengthReport). Outputs and encoder vectors matchpredict_batches(return_vectors=True)bit for bit, including the unfolded path of raw-mean heads. Resume sink returns stay unchanged. Standalone heads, VectorStore inputs andRegressor.predict(sink=...)refuse this switch. -
Write
manifest.jsonof npy artifacts and store shards with LF line endings on every platform; Windows wrote CRLF before. Files, checksums and fingerprints are unchanged on Linux and macOS, and CRLF manifests written by 2.0.0 on Windows stay readable. (DR-238: Manifeste auf allen Plattformen mit LF-Zeilenenden schreiben) -
Preserve subnormal multilabel decisions under foreign floating-point settings and reject Hub folders whose declared format version disagrees with the model. Correct the one-label multilabel output description. (DR-226: Mehrfach-Labels mit benannten Schwellen unterstützen)
-
Reject already released Arrow document-ID schemas before reading their fields.