equivalence

Python → C# equivalence table

Filled in as we go: a row is added at the same time as each function is implemented, never retrofitted at the end (§6.1 of the brief).

Lodestar.Text — distances & similarity

Python	Library	C#	Differences
Levenshtein.distance(a, b)	rapidfuzz	Levenshtein.Distance(a, b)	Compares UTF-16 units by default; pass TextElement.CodePoint for exact parity with Python on non-BMP characters (emoji…). Weights (1,1,1).
Levenshtein.normalized_distance(a, b)	rapidfuzz	Levenshtein.NormalizedDistance(a, b)	distance / max(len(a), len(b)), 0 if both empty. Identical.
Levenshtein.normalized_similarity(a, b)	rapidfuzz	Levenshtein.NormalizedSimilarity(a, b)	1 - normalized_distance. Two empty strings ⇒ 1. Identical.
OSA.distance(a, b)	rapidfuzz	Osa.Distance(a, b)	Optimal String Alignment (restricted Damerau): adjacent transposition allowed, no substring re-edited. Differs from full Damerau ("CA"/"ABC" ⇒ 3 vs 2). Not a metric: the triangle inequality fails.
OSA.normalized_similarity(a, b)	rapidfuzz	Osa.NormalizedSimilarity(a, b)	1 - dist/max(len). Identical.
DamerauLevenshtein.distance(a, b)	rapidfuzz	DamerauLevenshtein.Distance(a, b)	Unrestricted Damerau (Lowrance-Wagner). "CA"/"ABC" ⇒ 2. At unit costs it is a true metric, unlike Osa, so it is the one to index with.
DamerauLevenshtein.normalized_similarity(a, b)	rapidfuzz	DamerauLevenshtein.NormalizedSimilarity(a, b)	1 - dist/max(len). Identical.
hamming_distance(a, b)	jellyfish	Hamming.Distance(a, b)	Differing positions + length difference. Matches jellyfish on normal inputs; documented divergence on combining marks (decision 0005).
Indel.distance(a, b)	rapidfuzz	Indel.Distance(a, b)	Insertions/deletions only = len(a)+len(b)-2·LCS. Basis of fuzz.ratio.
Indel.normalized_similarity(a, b)	rapidfuzz	Indel.NormalizedSimilarity(a, b)	1 - dist/(len(a)+len(b)). ×100 = fuzz.ratio.
jaro_similarity(a, b)	jellyfish	Jaro.Similarity(a, b)	Empty ⇒ 0. Matches jellyfish except combining-mark quirks (decision 0005).
jaro_winkler_similarity(a, b)	jellyfish	JaroWinkler.Similarity(a, b)	Prefix boost only when Jaro > 0.7 (Winkler threshold), weight 0.1, prefix ≤ 4.
SequenceMatcher(None,a,b).find_longest_match(...).size	difflib	Lcs.SubstringLength(a, b)	Longest common (contiguous) substring. Same tie-break as difflib.
— (classic LCS)	—	Lcs.SubsequenceLength(a, b)	Longest common subsequence (order-preserving, non-contiguous). Basis of Indel.
SequenceMatcher(None,a,b).ratio()	difflib	RatcliffObershelp.Similarity(a, b)	Gestalt 2·M/T. autojunk not replicated (identical for ≤ 200 elements; decision 0006).

Lodestar.Text — set similarity (q-gram multisets)

Python	Library	C#	Differences
Jaccard(qval=1).normalized_similarity(a, b)	textdistance	Jaccard.Similarity(a, b)	Multisets (bags) of q-grams, qval=1 by default. |A∩B|/|A∪B|.
Sorensen(qval=1).normalized_similarity(a, b)	textdistance	SorensenDice.Similarity(a, b)	2·|A∩B|/(|A|+|B|).
Overlap(qval=1).normalized_similarity(a, b)	textdistance	Overlap.Similarity(a, b)	|A∩B|/min(|A|,|B|).
Tversky(qval=1).normalized_similarity(a, b)	textdistance	Tversky.Similarity(a, b)	α=β=1 by default (⇒ Jaccard).
Cosine(qval=1).normalized_similarity(a, b)	textdistance	Cosine.Similarity(a, b)	|A∩B|/√(|A|·|B|). Pass qval:2 for character bigrams.

textdistance raises on some empty inputs; Lodestar defines them cleanly: both empty ⇒ 1, one empty ⇒ 0. The one exception is a zero Tversky weight, which leaves the denominator empty as well — its entry says when. The oracle covers non-empty pairs (qval=1); edges are covered by unit tests.

Lodestar.Text — phonetic encoding

Python	Library	C#	Differences
soundex(s)	jellyfish	Soundex.Encode(s)	Initial letter + 3 digits. Exact parity (402 words).
metaphone(s)	jellyfish	Metaphone.Encode(s)	Parity on real words; jellyfish non-word quirks not reproduced (decision 0007).
nysiis(s)	jellyfish	Nysiis.Encode(s)	Non-truncated variant. Exact parity (402 words).

Lodestar.Text — sparse vectorization

Python	Library	C#	Differences
CountVectorizer()	scikit-learn	new CountVectorizer()	Sorted vocabulary, token_pattern \b\w\w+\b (single characters dropped), lowercase by default. Parity across 10 configs.
CountVectorizer(ngram_range=(1,2))	scikit-learn	new CountVectorizer(new(){ NgramRange=(1,2) })	Word n-grams joined by a space.
CountVectorizer(analyzer="char"/"char_wb")	scikit-learn	Analyzer = AnalyzerKind.Char / CharWordBoundary	Character n-grams (with/without crossing word boundaries).
CountVectorizer(min_df=…, max_df=…)	scikit-learn	MinDf, MaxDf	<1 = proportion, ≥1 = absolute count (sklearn _limit_features semantics).
CountVectorizer(strip_accents="unicode")	scikit-learn	StripAccents = true	NFKD decomposition + removal of combining marks.
CountVectorizer(stop_words="english")	scikit-learn	StopWords = StopWords.English	sklearn's 318-word list (identical). Any custom collection accepted.
nltk.corpus.stopwords.words("french")	nltk	StopWords.French	Not identical. The shipped lists are Snowball's, not nltk's, for licensing reasons (decision 0010). Same for German, Portuguese, Spanish; Italian matches nltk word for word.
scipy.sparse (CSR)	scipy	CsrMatrix	Home-grown CSR: ToDense, L1/L2 norms, NormalizeRows, matrix-vector product.
TfidfVectorizer()	scikit-learn	new TfidfVectorizer()	smooth_idf + L2 normalization on by default. idf = ln((1+n)/(1+df)) + 1. Parity across 7 configs.
TfidfTransformer()	scikit-learn	new TfidfTransformer()	use_idf, smooth_idf, sublinear_tf, norm (L1/L2/none).
HashingVectorizer()	scikit-learn	new HashingVectorizer()	Hashing trick, no vocabulary. MurmurHash3-32 (seed 0) reproduced; alternate sign + L2 normalization by default.

Lodestar.Text — model persistence

Python	Library	C#	Differences
joblib.dump(vec, path) / pickle.dump(vec, f)	joblib / pickle	vec.Save(path) / vec.Save(stream)	Versioned JSON, not a pickle: data only, never code. Applies to CountVectorizer, TfidfVectorizer and HashingVectorizer. UTF-8 without BOM; the idf vector is base64-encoded raw IEEE-754 bits, the rest is readable JSON.
joblib.load(path) / pickle.load(f)	joblib / pickle	TfidfVectorizer.Load(path, options?)	Static, not a constructor. Bounded by ArtifactLoadOptions — pickle.load has no equivalent, since it trusts the file by design (decision 0011).
— (no equivalent)	—	ArtifactLoadOptions	Deliberate addition, not a port: caps vocabulary size, token length, JSON depth, total bytes and array length. Over a limit ⇒ InvalidDataException naming limit and value.

Lodestar.Text — stemming

Python	Library	C#	Differences
PorterStemmer(mode=ORIGINAL_ALGORITHM).stem(w)	nltk	PorterStemmer.Stem(w)	Porter (1980) algorithm, 5 steps. Exact parity (86 words).
SnowballStemmer("english").stem(w)	nltk	EnglishSnowballStemmer.Stem(w)	Porter2: R1/R2 regions, exceptions. Exact parity (190 words).
SnowballStemmer("french").stem(w)	nltk	FrenchSnowballStemmer.Stem(w)	French Snowball: RV region, 6 steps, NFC-normalized input. Exact parity (152 words).
SnowballStemmer("spanish").stem(w)	nltk	SpanishSnowballStemmer.Stem(w)	Spanish Snowball: attached-pronoun step 0, accents stripped last. Exact parity (127 words).
SnowballStemmer("portuguese").stem(w)	nltk	PortugueseSnowballStemmer.Stem(w)	Portuguese Snowball: nasal a~/o~ expansion, accents kept. Exact parity (105 words).
SnowballStemmer("italian").stem(w)	nltk	ItalianSnowballStemmer.Stem(w)	Italian Snowball: acute→grave folding, u/i marking. Exact parity (96 words); enza→te follows nltk over the published text, see 0008.
SnowballStemmer("german").stem(w)	nltk	GermanSnowballStemmer.Stem(w)	German Snowball: ß→ss, u/y marking, R1 floored at 3, no RV region. Exact parity (88 words).

Lodestar.Embeddings — sub-word tokenization & pooling

Python	Library	C#	Differences
Tokenizer(WordPiece(vocab)).encode(t)	tokenizers (HF)	new WordPieceTokenizer(vocab).Encode(t)	Greedy longest match, ## continuation, [UNK]. Pre-tokenization \w+|[^\w\s]+. Exact parity. The added_tokens table is matched as text, ahead of the pre-tokenizer, rather than folded into the vocabulary as whole-word entries — with lstrip, rstrip and single_word honoured, and the same AddedTokenScanner BpeTokenizer uses, so a flag cannot mean two things. Which text an entry is matched against is decided by its normalized field and not by special: non-normalized entries run in an outer pass over the raw input and emit the raw slice, normalized ones have their own content normalized and run over the lowercased gaps the first pass left. A special-but-normalized entry is lowercased like any other — decision 0022. WordPieceVocabulary.Count counts Vocab alone and therefore under-counts what Encode can emit, as BpeVocabulary already did.
Tokenizer(Unigram(...)).encode(t) / sp.encode(t)	tokenizers / sentencepiece	new SentencePieceTokenizer(vocab).Encode(t)	Unigram via Viterbi (max log-probability), preceded by the model's own precompiled_charsmap and its whitespace flags. Exact parity over four vocabularies and four different character maps — stock XLM-R's nmt_nfkc (the map every stock T5, ALBERT and camemBERT also carries, byte for byte), a nmt_nfkc_cf model, a hand-written three-rule map, and tiny_sp.model, which has none. remove_extra_whitespaces collapses runs of U+0020 only; a run of uncovered characters comes back as one unknown piece, as in Python.
sp.normalize(t)	sentencepiece	vocab.Normalizer.Normalize(t)	The precompiled_charsmap alone: a longest-match walk over a darts-clone trie. Covers every built-in rule and any --normalization_rule_tsv, because they all compile to that one blob. Not reimplemented on string.Normalize(FormKC), which would drift: the map is frozen at the Unicode version that compiled it and the two already differ on 181 code points — decision 0014.
sp.encode(t) over the XLM-R vocabulary	sentencepiece	new SentencePieceTokenizer(SentencePieceModelLoader.Load("xlmr_fairseq.model")).Encode(t)	250 002 pieces with <s>=0, <pad>=1, </s>=2, <unk>=3, <mask>=250001 — the layout HuggingFace gives XLM-R, and the one an id-based control filter gets wrong. Identical segmentation over Latin, Cyrillic and Japanese input, including text naming all five markers literally: none is ever matched as text. Fixture built by tools/fetch_xlmr_vocab.py.
sp.encode(t) with an all-positive-score vocabulary	sentencepiece	idem	The unknown piece is scored min(0, min_score) - 10 where Python uses min_score - 10. Identical for every real model (scores are log-probabilities, so the floor never binds); Lodestar penalises the unknown piece more where it does. Decision 0013.
Tokenizer(BPE(vocab, merges)).encode(t) with a ByteLevel pre-tokenizer	tokenizers (HF)	new BpeTokenizer(vocab).Encode(t)	Lowest-ranked-merge-first, over a doubly-linked list of symbols and a priority queue rather than a rescan-and-shift loop — see the scaling figures in decision 0017. Added-token matching before merging — the whole added_tokens table, including the special tokens model.vocab also declares, with each entry's lstrip, rstrip and single_word flags honoured as measured (decision 0022); the raw-versus-normalized pass that flag table also carries normalizes each gap between raw added tokens in isolation and matches the normalized half of the table inside that gap, raw entries against raw text and normalized entries against normalized text — ignore_merges, and add_prefix_space, which is applied per added-token-delimited segment and only where the segment does not already begin with a space, as ByteLevel does in Python. End-to-end parity over GPT-2's vendored 50 257-entry vocabulary and merge table (byte-level) and a self-trained model (the classic, non-byte-level lineage); BpePatterns.Llama3 and BpePatterns.Qwen2 are proven at the split level only, against the vocabulary the caller supplies — decision 0017 again. A byte-level model missing one of the 256 alphabet characters from model.vocab while declaring it as an added token now throws ArgumentException from ByteLevelSymbols rather than silently folding the added token's id in, as it did before issue #130; the reference does neither, dropping the uncovered byte instead since there is no unk_token to substitute — measured, aQa with byte Q missing from model.vocab and present only as an added token is ['a', 'a'] there — so this swaps one divergence for another, throwing rather than returning a wrong token stream.
BPE(..., fuse_unk=True)	tokenizers	new BpeVocabulary(vocab, merges) { FuseUnk = true }	A run of consecutive uncovered characters is one unknown token, not one each. The run stops at a pre-tokenizer boundary, so "aZ Za" under Whitespace keeps two. Fusing happens before merging, so a fused symbol can take part in a merge. With no UnkToken the flag does nothing, because an uncovered character is dropped rather than substituted; on a byte-level model it does nothing either, because all 256 characters are covered.
tokenizer.decode(ids)	tokenizers (HF)	BpeTokenizer.Decode(ids)	Byte-level: every complete, well-formed UTF-8 byte sequence round-trips exactly, including malformed-looking sequences that came from Encode. A byte sequence that is not well-formed UTF-8 — a token split across a multi-byte character's boundary, decoded on its own, is the ordinary way this happens — substitutes U+FFFD, matching HuggingFace rather than throwing, decision 0023. Decode_of_one_id_at_a_time_matches_the_reference proves it against tokenizers 0.23.1's own per-id output over CJK, emoji and accented text, #149. skipSpecialTokens defaults to false — the opposite of Python's skip_special_tokens=True — so Decode(Encode(x)) == x holds without passing an extra argument; pass true to drop added tokens. It drops exactly the added tokens whose added_tokens entry is special, carried on AddedToken.Special, matching Python's skip_special_tokens. Proven over the byte-level and classic corpora above, decode direction, including a GPT-2 corpus whose text names <|endoftext|> at the id model.vocab gives it. An added token carrying lstrip (or rstrip) breaks the byte-exact round trip, here and in Python alike: the absorbed whitespace is consumed into the match and is not restored, so 'a <mask> b' decodes to 'a<mask> b'. Following HuggingFace is the parity; restoring the space would be the divergence — decision 0022.
tokenizer.add_tokens([...])	tokenizers	BpeVocabulary.AddedTokens	An added token is matched as literal text and carries an id, but it is not a model vocabulary entry: a character it spells that model.vocab does not declare is still substituted with the unknown token. Measured, aQa with Q an added token absent from model.vocab and single_word on is ['a', '[UNK]', 'a']. TryGetId and Decode still see it, matching token_to_id and decode.
— (refused)	tokenizers	new BpeTokenizer(…) throws ArgumentException	Two shapes the reference also refuses while reading the document: a merge naming a token model.vocab does not declare — measured, Token `Q` out of vocabulary — and a merge whose result is absent, refused here too but with a message of Lodestar's own, since the reference panics there instead of raising: range end index 2 out of range for slice of length 1. A third shape, an unk_token present only in added_tokens, the reference does not refuse to build: it loads the file, answers token_to_id, and encodes text the model already covers, raising only from encode and only on text needing a substitution the vocabulary cannot supply. Lodestar refuses it here, at construction — earlier than the reference, a divergence in timing rather than outcome.
mean pooling + F.normalize	sentence-transformers	Pooler.MeanPoolAndNormalize(...)	Masked mean (padding excluded) + L2 normalization.
util.semantic_search / corpus @ query	sentence-transformers / numpy	new EmbeddingIndex(dim).Search(q, k)	Exhaustive SIMD-vectorized cosine. Top-k, index-ascending tie-break.
mean pooling over a [batch, seq, dim] tensor	sentence-transformers	Pooler.MeanPoolBatch(...) / MeanPoolAndNormalizeBatch(...)	Each row pooled against its own slice of the mask. Vectorized with Vector<float> on net10.0, scalar on netstandard2.0, and the two are bit-identical — asserted with float equality, not a tolerance, because one frozen corpus serves both builds.
tokenizer(texts, padding=True, truncation=True, max_length=n)	tokenizers (HF)	new BatchEncoder(tokenizer, options).EncodeBatch(texts)	Inserts the template's special tokens, truncates inside a budget that counts them as HuggingFace does, pads each batch to its own longest row (padding="longest", never "max_length") and builds the attention mask. Ids and mask replayed against encode_batch for equality, not within a tolerance. TruncationStrategy.None refuses an over-long text; HuggingFace's truncation=False returns it untruncated.
TemplateProcessing(single="[CLS] $A [SEP]")	tokenizers (HF)	SpecialTokenTemplate.Bert / .Roberta / .T5 / .None	The wrapping as data. Tokens are named, never numbered — the id comes from the model's vocabulary through ISubwordTokenizer.TryGetId, so a vocabulary placing [CLS] anywhere works and one lacking it throws at construction. A pair template ($A/$B) is not supported: Lodestar encodes one sequence at a time.
onnxruntime.InferenceSession(...).run(...) + pooling	onnxruntime	new OnnxTextEmbedder(path).Embed(ids, mask)	Loads an ONNX model (weights not redistributed), runs it, mean-pool + L2. Feeds token_type_ids only if the model declares it. Takes ReadOnlySpan<long> since 0.3.0, where it took IReadOnlyList<long>. Refuses an output whose rank is neither 3 nor 2, and any input or output name the model does not declare.
SentenceTransformer.encode(texts, batch_size=n, normalize_embeddings=True)	sentence-transformers	new OnnxTextEmbedder(path, tokenizer).EmbedBatch(texts, options)	The whole chain in one call: encode, sub-batch, pad, run, mean-pool, normalize, restore the caller's order. SortByLength buckets by length between sub-batches and changes nothing observable. Agreement with a float64 reference is bounded near 1e-7, not 1e-9: ONNX Runtime returns float32 and the vector is normalized in float32. convert_to_tensor, show_progress_bar and the pooling modes other than mean have no equivalent.

Lodestar.Embeddings — vocabulary loaders

Python	Library	C#	Differences
BertTokenizer(vocab_file=…) vocabulary loading	transformers	VocabTxtLoader.Load(path, …)	One token per line, id = line number. Reproduces two quirks of the Python loop: a blank line is a token whose string is empty, and a repeated token keeps the last id. A UTF-8 BOM is stripped rather than absorbed into the first token.
Tokenizer.from_file("tokenizer.json") (WordPiece)	tokenizers (HF)	TokenizerJsonLoader.LoadWordPiece(path)	Reads model.vocab, unk_token, continuing_subword_prefix, and derives lowercase from the normalizer. The whole added_tokens table lands in WordPieceVocabulary.AddedTokens with all five flags — lstrip, rstrip, single_word, special, normalized — instead of being folded into model.vocab: a folded entry is an ordinary whole-word vocabulary member and cannot honour a flag. normalized absent falls to !special, which is Rust's AddedToken::from default rather than a measured behaviour — tokenizers refuses a file omitting the field, so no corpus can reach that path. Refuses a pipeline it does not reproduce — NFKC/Precompiled normalizers, a non-Whitespace pre-tokenizer, any post_processor, truncation or padding — rather than ignoring it, and refuses an unk_token that model.vocab does not define even when added_tokens does: the table is matched as text ahead of the model, so an unknown token declared only there is one the model can never fall back to.
Tokenizer.from_file("tokenizer.json") (Unigram)	tokenizers (HF)	TokenizerJsonLoader.LoadUnigram(path)	Reads the [piece, score] pairs and unk_id. tokenizer.json records no piece types, so they are derived: the special entries of added_tokens become Control, the piece at unk_id becomes Unknown. A Precompiled normalizer is read — it is the same blob a spiece.model carries, base64-encoded — through the same interpreter, so the two formats describe the same model identically. NFKC is still refused: it asks for the runtime's Unicode tables where the model asked for a frozen map. Pre-tokenizer must be Metaspace with ▁.
models.BPE.from_file(vocab, merges)	tokenizers (HF)	BpeFilesLoader.Load(vocabPath, mergesPath)	Reads the pre-tokenizer.json vocab.json + merges.txt pair GPT-2 (and Llama-3, Qwen2) ship. Neither file carries a pipeline, so byteLevel (default true, GPT-2's own default) and the split pattern (BpePatterns.Gpt2 when byte-level, BpePatterns.Whitespace when not) are parameters, not read from the files — the second is named on the vocabulary now that leaving it unset no longer means the classic split. Proven over GPT-2's vendored vocab.json/merges.txt — decision 0017.
Tokenizer.from_file("tokenizer.json") (BPE)	tokenizers (HF)	TokenizerJsonLoader.LoadBpe(path)	Reads model.vocab/model.merges, the whole added_tokens table (the entries model.vocab also declares included — that is where every special token is — together with all five of each entry's flags — lstrip, rstrip, single_word, special and normalized), ignore_merges, end_of_word_suffix, unk_token, fuse_unk, a normalizer of NFC, NFKC, NFD, NFKD or a Sequence of those — empty included, which normalizes nothing — applied in the order declared, and derives byte-level-ness and the split pattern from pre_tokenizer — a bare ByteLevel (stock GPT-2), Whitespace (classic lineage), or a Sequence of Split then ByteLevel (the Llama-3/Qwen2 shape), or no split at all — which an absent pre_tokenizer and a bare ByteLevel with use_regex off both mean, read as BpeVocabulary.NoPreTokenizer (row below). Refuses, naming what it found: byte_fallback, a non-zero dropout — a training-time regularizer; set it to null to load the file, and see decision 0034, a non-empty continuing_subword_prefix on a byte-level pre-tokenizer, a normalizer other than those four forms or Sequence — Replace by name, since its pattern may be a Rust regex whose flavour .NET does not share, and anything else by name too — a ByteLevel block declaring no add_prefix_space in any of the three positions one can appear in — top-level pre_tokenizer, a Sequence step, the decoder — since tokenizers has no default for that field and refuses the file itself, truncation, padding, a post_processor, any other pre-tokenizer shape, and a decoder whose byte-level-ness disagrees with the model's own — which would not decode what it encodes, in Python either. Accepts the values that provably change nothing: a dropout of 0.0 and an end_of_word_suffix of "", which reads back as absent since an empty marker marks nothing — bpe_no_op_settings.json replays tokenizers producing the same tokens with each of the two as without it. An omitted use_regex and an omitted trim_offsets are accepted too: the first has a default in the reference, the second is never read here. All four forms agree between String.Normalize and tokenizers over unicode_forms.json's 56 cases. With a normalizer declared, Decode(Encode(x)) returns the normalized text rather than x, exactly as in Python — bpe_normalizer.json measures it, U+FFFD substitution included. Proven over all three pipeline shapes: byte-level GPT-2, the classic lineage, and a Sequence pattern shaped like Qwen2's — split-level only for Llama-3/Qwen2 themselves, decision 0017. Unrecognized top-level properties — the file's own version included — are accepted in silence here and by LoadWordPiece/LoadUnigram alike, where an artifact Lodestar itself wrote would reject them: tokenizers gains fields between releases, and refusing every one would refuse files that tokenize identically; what is checked is only the set of sections that change tokenization.
Sequence([Split(pattern), ByteLevel(use_regex=True)])	tokenizers	new BpeVocabulary(vocab, merges) { ByteLevel = true, PreSplit = new BpeSplitStep(pattern, SplitBehavior.Isolated, Invert: false), PreTokenizerPattern = BpePatterns.Gpt2 }	Both patterns apply, in order: the Split step's produces the pieces and ByteLevel's own re-splits each of them. With use_regex: false the second is absent and the Split pattern is the only one. The difference is not cosmetic — GPT-2's pattern knows only the contractions 's, 't, 're, 've, 'm, 'll, 'd, so aujourd'hui is three pieces with the second split and two without it, while don't is the same either way. All five behavior values and invert are reproduced now — see the Split row below — and the loader refuses a Split step declaring an absent or unknown one, as tokenizers 0.23.1 does. add_prefix_space goes on every piece the Split step produces, unless the piece already begins with a space, which is where HuggingFace puts it too — once per piece handed to the ByteLevel step, not once per input. tests/oracles/bpe_prefix_space.json measures it over 35 cases across five models.
Split(pattern, behavior=…, invert=…)	tokenizers	new BpeVocabulary(vocab, merges) { PreSplit = new BpeSplitStep(pattern, SplitBehavior.Isolated, Invert: false) }	All five behaviours and both invert values. Isolated keeps every match and every gap, Removed keeps the gaps alone, the two merge directions attach each match to the gap before or after it, and Contiguous is Isolated with adjacent matches joined — so Split("X", Contiguous) over "aXXb" gives ['a','XX','b'] where Isolated gives ['a','X','X','b']. invert swaps the roles of match and gap, which makes it a no-op for Isolated and Contiguous and exchanges the two merge directions. Empty pieces are dropped. behavior and invert are required fields: a file omitting either is refused here and by tokenizers 0.23.1, and the behaviour is read in the file's PascalCase spelling, not the Python constructor's snake_case. The pattern itself has two spellings and both are read: pattern.Regex, and pattern.String — the literal pre_tokenizers.Split("|", "isolated") writes, which being the shortest correct call is the likelier file to meet. A literal is escaped rather than interpreted, and \d is where that shows: Split(String "\d") cuts at a backslash followed by a d and leaves a digit alone, where Split(Regex "\d") cuts at every digit. A pattern node declaring both keys, or neither, is refused naming both — tokenizers writes exactly one, so neither shape is a file it produces, and the both case is refused on the two keys being present rather than on both values being readable. tests/oracles/bpe_split_literal.json measures 36 cases over 12 models, each literal beside its escaped twin. The escape is Regex.Escape, which does not produce the same string as Python's re.escape: measured, it leaves -, ], }, ~, & and the vertical tab bare where re.escape backslashes them, and spells tab, newline, carriage return and form feed as \t, \n, \r and \f where re.escape backslashes the character itself. Each output still matches the same literal under its own engine, so the reproduction holds — but for those characters it holds on .NET's documented escaping rather than on this corpus, whose six literals (\d, a.c, |, ab, an astral emoji and the empty string) carry none of them. In a Sequence, behavior/invert govern this step only — a following ByteLevel step (BpeVocabulary.PreTokenizerPattern) re-splits every piece this one produces, always Isolated with invert off, since the format gives ByteLevel no behavior field of its own.
pre_tokenizers.Whitespace()	tokenizers	new BpeVocabulary(vocab, merges) { PreTokenizerPattern = BpePatterns.Whitespace }	The classic, non-byte-level lineage's own split — word runs, punctuation isolated. BpePatterns.Whitespace is the pattern BpeTokenizer used to supply when a vocabulary named none, and naming it is now how a caller asks for it: a vocabulary declaring no PreSplit, no PreTokenizerPattern and no NoPreTokenizer is refused by the constructor rather than given this one, since that same shape is what a model splitting nothing at all would look like. TokenizerJsonLoader.LoadBpe sets it for a Whitespace pre-tokenizer, BpeFilesLoader.Load for a non-byte-level vocab.json/merges.txt pair. Measured, "aZ Za" is four tokens under it — ['a', '[UNK]', '[UNK]', 'a'] — where the same vocabulary with no split gives three (bpe_no_split.json, models whitespace and absent).
pre_tokenizer: null / ByteLevel(add_prefix_space=…, use_regex=False)	tokenizers	new BpeVocabulary(vocab, merges) { NoPreTokenizer = true }	Nothing is split: the text reaches the merge loop whole, so a merge may span what a pattern would have cut at. Two file shapes mean it and TokenizerJsonLoader.LoadBpe reads both — a tokenizer.json declaring no pre_tokenizer at all, and a bare ByteLevel whose use_regex is off — with ByteLevel following the shape, off for the first and on for the second. "Whole" is per added-token segment, not per text: with <sep> added, "o o<sep>o o" is ['oĠ', 'o', '<sep>', 'oĠ', 'o'] where the same model with use_regex on gives ['o', 'Ġ', 'o', '<sep>', 'o', 'Ġ', 'o'] — the oĠ merge spans the space the pattern cuts at, and the added token still ends the segment. add_prefix_space applies once, to the segment rather than to each piece: "hello world" gains a single leading Ġ and decodes to " hello world", while a text already beginning with a space gains nothing. Decode(Encode(x)) returns x on the byte-level shape, leading and trailing spaces included. Declaring it beside PreSplit or PreTokenizerPattern is refused, the two being contradictory. 22 cases over 7 models in bpe_no_split.json, against tokenizers 0.23.1.
BPE(..., continuing_subword_prefix="##")	tokenizers	new BpeVocabulary(vocab, merges) { ContinuingSubwordPrefix = "##" }	On the classic, non-byte-level lineage: every symbol after the first of each pre-tokenized piece is looked up prefixed, so "ab ab" gives ['a', '##b', 'a', '##b'] — the second word starts bare. There is no fallback: a non-initial symbol whose prefixed form is absent is substituted or dropped like any uncovered character, and the bare form is not consulted, which is what the reference does. A merge's result is its left side plus its right side without the prefix, so ("##b", "##c") produces ##bc and not bc; the reference refuses to build a file whose vocabulary carries the concatenated form instead. An end_of_word_suffix on that right side stays on: ("a", "##b</w>") produces ab</w>. The prefix composes with end_of_word_suffix, prefix then characters then suffix. An empty prefix reads as absent. Pairing a non-empty prefix with ByteLevel is refused by name, by BpeTokenizer's constructor and by TokenizerJsonLoader.LoadBpe — byte-level symbols are never prefixed while a merge's right side is still stripped, so the two halves would disagree, and the byte-level alphabet spells 0x23 as #, which lets the disagreement land on another existing id rather than raise. An end_of_word_suffix paired with ByteLevel is the opposite case: the two are independent properties, so a vocabulary can declare both, and where it does the suffix is silently ignored on the byte-level path rather than applied or refused — nothing here measures what tokenizers does with that pairing, so this is a documented gap, not a refusal.
sentencepiece_model_pb2.ModelProto().ParseFromString(…)	sentencepiece	SentencePieceModelLoader.Load(path)	Hand-written minimal protobuf reader (varint, length-delimited, fixed32). Pieces, scores, types, and unk/bos/eos/pad ids from trainer_spec. Scores are 32-bit floats widened to double, exactly as the Python binding does. The normalizer_spec is read, not merely inspected: its precompiled_charsmap becomes a PrecompiledNormalizer. Refuses a normalizer named without a map to apply, or a map that will not parse — nothing is decided from normalizer_spec.name.
sp.id_to_piece(i) / sp.get_score(i)	sentencepiece	vocab.Pieces[i].Piece / .Score	Identical; scores compared at 1e-9 in the oracle.
sp.IsControl(i) / sp.IsUnknown(i)	sentencepiece	vocab.Types[i], vocab.IsMatchable(i)	The type comes from the file. The previous constructor inferred it from ids 0/1/2, which is wrong for any model laying out differently — that constructor is now [Obsolete].

Lodestar.Embeddings — index persistence

Python	Library	C#	Differences
numpy.save(path, matrix)	numpy	index.Save(path) / index.Save(stream)	Versioned JSON whose vector block is base64-encoded raw little-endian IEEE-754 bits, not a .npy memory dump. Carries the normalization flag and an optional id per vector — a .npy header already carries shape/dtype, so the per-vector dimension is recoverable as shape[1], but the flag and the ids have nowhere to live in it.
numpy.load(path)	numpy	EmbeddingIndex.Load(path, options?)	Static, not a constructor. Returns a queryable index rather than an array, and bounds every count against ArtifactLoadOptions before it sizes a buffer — the vector block by MaxTotalBytes in bytes before parsing, the rest by MaxArrayLength in elements.
faiss.write_index(idx, path) / faiss.read_index(path)	faiss	index.Save(path) / EmbeddingIndex.Load(path)	Comparable in purpose, not in structure: Lodestar's index is exhaustive (IndexFlatIP-shaped), so there is no graph or quantizer to serialize. An approximate index is a separate decision, not made.
— (a parallel list[str] the caller keeps)	—	index.Add(vector, id) / index.GetId(i)	Deliberate addition: without ids in the file, a reloaded index is a wall of anonymous integers.

Lodestar.Fuzzy — applied fuzzy matching

Python	Library	C#	Differences
fuzz.ratio(a, b)	rapidfuzz	Fuzz.Ratio(a, b)	Indel similarity ×100. Case-sensitive (no preprocessing, like rapidfuzz).
fuzz.partial_ratio(a, b)	rapidfuzz	Fuzz.PartialRatio(a, b)	Best sliding window (shorter over longer; both directions when lengths are equal).
fuzz.token_sort_ratio(a, b)	rapidfuzz	Fuzz.TokenSortRatio(a, b)	Sort tokens then ratio.
fuzz.token_set_ratio(a, b)	rapidfuzz	Fuzz.TokenSetRatio(a, b)	Shared tokens vs differences.
fuzz.WRatio(a, b)	rapidfuzz	Fuzz.WRatio(a, b)	Weighted combination based on the length ratio.
process.extract(q, choices, limit=…, score_cutoff=…)	rapidfuzz	Process.Extract(q, choices, limit:…, scoreCutoff:…)	Default scorer WRatio, score-descending order (index tie-break), cutoff, short-circuit.
process.extractOne(q, choices)	rapidfuzz	Process.ExtractOne(q, choices)	Best candidate or null.
blocking deduplication	— (application pattern)	Deduplicator.FindClusters(...)	Partition by blocking key + transitive closure (union-find). Avoids O(n²).

Lodestar.Metrics — classification metrics

Python	Library	C#	Differences
accuracy_score(y_true, y_pred)	scikit-learn	Accuracy.Score(yTrue, yPred)	Identical, normalize included. The overload taking a ConfusionMatrix scores only the samples that matrix kept.
hamming_loss(y_true, y_pred, sample_weight=…)	scikit-learn	HammingLoss.Score(…)	Identical, both shapes. On single-label input it is one minus Accuracy.Score and agrees with ZeroOneLoss.Score; on a label matrix it counts wrong labels where that one counts wrong rows — measured, 0.3333… against 1 on two samples over three labels. The 2-D form arrives row-major with a labelCount, as the other 2-D metrics take theirs, and sampleWeight is per row rather than per value.
zero_one_loss(y_true, y_pred, normalize=…, sample_weight=…)	scikit-learn	ZeroOneLoss.Score(…)	Identical, normalize included as a bool as Accuracy.Score's already is. With weights and normalize=False the answer is the weight of the wrong samples rather than how many there are — measured 2.0 against a count of 1 — which is the reference's behaviour. On a label matrix a row is wrong if any of its labels is.
jaccard_score(y_true, y_pred, labels=…, pos_label=…, average=…, sample_weight=…, zero_division=…)	scikit-learn	JaccardScore.Score(…), .PerClass(…)	Identical on the four averaging modes Precision.Score already implements, weights and labels included — it is that metric's shape with a different ratio, and shares its machinery. ZeroDivision.NaN and ZeroDivision.Throw have no counterpart: jaccard_score admits only 0, 1 and 'warn', and refuses nan with an InvalidParameterError; the two extra members are this package's. average='samples' is not offered, for the reason AveragePrecision gives. An absent pos_label under Averaging.Binary raises here, which is the refusal Precision.Score already makes rather than anything new.
confusion_matrix(y_true, y_pred, labels=…)	scikit-learn	ConfusionMatrix.Compute(…)	Rows are true labels. Label order is the sorted union, or the caller's order left unsorted. Counts are double because sampleWeight is supported (decisions/0016).
multilabel_confusion_matrix(y_true, y_pred, sample_weight=…, labels=…, samplewise=…)	scikit-learn	MultilabelConfusionMatrix.Compute(…)	Identical, both shapes and samplewise included. Returns a stack of ConfusionMatrix rather than a type of its own: each entry is one class against everything else, which a two-label matrix already is, and its cells land where the reference puts them because its labels are 0 and 1 in that order. samplewise is structurally confined to the matrix overload, where scikit-learn refuses it at run time with "Samplewise metrics are not available outside of multilabel classification" — the call cannot be written here rather than being rejected. Under it a row's weight applies to each of that row's labels, since the matrix counts labels there.
class_likelihood_ratios(y_true, y_pred, labels=…, sample_weight=…, replace_undefined_by=…)	scikit-learn	LikelihoodRatios.Compute(…)	Identical, all four undefined shapes included. Returns a small sealed type with named Positive and Negative rather than a tuple, which would carry neither names nor documentation. replace_undefined_by is two parameters here: it takes a scalar or a mapping of {"LR+": …, "LR-": …} there, a union C# has no equivalent of, and passing the same value to both reproduces the scalar form. A truth with no positive sample refuses the replacement on both sides and answers nan whatever was asked for, where a truth with no negative sample takes it — measured (nan, nan) against (1, 1) with the replacement set to 1, and nothing in the reference's signature says so. More than two distinct labels is ArgumentException carrying scikit-learn's own sentence, where it raises ValueError.
hinge_loss(y_true, pred_decision, labels=…, sample_weight=…)	scikit-learn	HingeLoss.Score(…), .MultiClass(…)	Identical on every input either side defines, binary and one-decision-per-class alike. It reads a decision function rather than a label or a probability, the only member here that does, and charges until a margin of 1 — a prediction that is right but barely still costs something where ZeroOneLoss.Score counts it free. The multiclass form charges the true class's decision less the best of the others, Crammer and Singer's margin. posLabel is a parameter defaulting to 1 where scikit-learn infers the two classes; only the decision's sign is compared against it, so relabelling cannot move the number. One divergence, on a truth carrying a single class: scikit-learn maps every label to -1 through a LabelBinarizer with nothing to contrast and returns a value computed against the wrong side — measured 1.65 where the margins give 0.35, which is what an explicit posLabel answers here.
precision_score(…, average=…)	scikit-learn	Precision.Score(…, Averaging…)	All four modes. average=None is Precision.PerClass, a method rather than an enum member: it returns one value per class, not a scalar.
recall_score(…, average=…)	scikit-learn	Recall.Score(…, Averaging…)	As above.
f1_score(…, average=…)	scikit-learn	F1.Score(…, Averaging…)	As above.
fbeta_score(…, beta=…)	scikit-learn	FBeta.Score(…, beta, …)	Finite beta ≥ 0; scikit-learn also accepts inf, which throws here.
classification_report(…)	scikit-learn	ClassificationReport.Compute(…), .ToText(digits)	Structured and character-exact text. ZeroDivision.NaN renders NaN where Python writes nan; the numbers still match.
zero_division=0/1/np.nan	scikit-learn	ZeroDivision.Zero/One/NaN	Values identical. The UndefinedMetricWarning has no equivalent; ZeroDivision.Throw is the opt-in replacement.
roc_auc_score(y_true, y_score)	scikit-learn	RocAuc.Score(…)	Binary. posLabel is explicit here (default 1) where scikit-learn infers it.
roc_curve(y_true, y_score, pos_label=…, sample_weight=…, drop_intermediate=…)	scikit-learn	RocCurve.Compute(…)	Identical, the leading point at an infinite threshold included — no sample is above it, so both rates are 0 there and the reference prepends it rather than deriving it. drop_intermediate defaults to true here as it does there, where the other two curves default to false; the asymmetry is reproduced rather than normalised, and this curve's rule is the collinear one — a point the curve does not bend at. Measured, a ten-sample fixture goes from 11 points to 5. A class absent from the input gives a NaN rate rather than a division by zero, which is what the reference warns about and returns. Returned as a sealed class per decision 0040.
precision_recall_curve(y_true, y_score, pos_label=…, sample_weight=…, drop_intermediate=…)	scikit-learn	PrecisionRecallCurve.Compute(…)	Identical, including that Thresholds is one shorter than Precision and Recall: the curve carries an endpoint at recall 0 and precision 1 that no threshold produces, and padding it would invent one. drop_intermediate defaults to false and drops a point whose true-positive count matches both neighbours — a different rule from roc_curve's, and the same one det_curve uses; measured, 11 points to 8. With no positive sample the recall is taken as 1 at every threshold, the same substitution AveragePrecision.Score reproduces.
det_curve(y_true, y_score, pos_label=…, sample_weight=…, drop_intermediate=…)	scikit-learn	DetCurve.Compute(…)	Identical. The shortest of the three on the same input — 3 points where the ROC curve has 5 — because neither endpoint is carried: the curve starts where false positives stop being zero and stops where false negatives reach zero. Its points run by ascending threshold, the reverse of the other two. drop_intermediate defaults to false and shares precision_recall_curve's rule.
auc(x, y)	scikit-learn	Auc.Trapezoid(…)	Identical, direction included: a curve given right to left gives the same magnitude as the same curve given left to right. x that neither increases nor decreases is ArgumentException where scikit-learn raises ValueError, and fewer than two points likewise. Over RocCurve.Compute's output it equals RocAuc.Score exactly — an invariant no oracle states, asserted over every fixture. Over a precision-recall curve it is deliberately not average precision: the trapezoid reads 0.7916666666666666 where the step sum reads 0.8333333333333333.
brier_score_loss(y_true, y_proba, sample_weight=…, pos_label=…, scale_by_half=…)	scikit-learn	BrierScore.Score(…), .MultiClass(…)	Identical, both shapes and scale_by_half included. That parameter's 'auto' reads the input's shape — halving a one-dimensional binary probability and not halving a matrix — so it is a bool whose default differs per entry point rather than a string: scaleByHalf: true on Score and false on MultiClass, which reproduces both numbers. Measured, one matrix scores 0.245 unhalved and 0.1225 halved. pos_label is a parameter defaulting to 1 where scikit-learn infers the greater label present and refuses to guess for non-numeric labels — as RocAuc.Score's already is. A probability outside [0, 1] is ArgumentException carrying the reference's own sentence, which says less than 0 here and lower than 0 in log_loss; both wordings are kept.
log_loss(y_true, y_proba, normalize=…, sample_weight=…, labels=…)	scikit-learn	LogLoss.Score(…), .MultiClass(…)	Identical, normalize included as a bool as Accuracy.Score's already is. The clip is machine epsilon, 2.220446049250313e-16, measured rather than assumed because it has moved across versions: a predicted 0 for the true class contributes -log(eps), anything below the clip scores the same as 0, and a perfect prediction reads 2.2204460492503136e-16 rather than 0 because the top is clipped too. A row that does not sum to 1 is neither refused nor renormalised — the reference warns and scores the values as given, and there is no warning channel here, so only the number carries it; RocAuc.MultiClass is stricter than its own reference on exactly that point and this is not. posLabel is a widening: log_loss has none, a one-dimensional column always describing the greater label, and passing labels reversed only warns and returns the same number — scoring about the other class is the same call on the complement, which the corpus pins.
calibration_curve(y_true, y_prob, pos_label=…, n_bins=…, strategy=…)	scikit-learn	CalibrationCurve.Compute(…)	Identical. It is sklearn.calibration, not sklearn.metrics — the one member of the calibration family that lives in the other module, named here rather than filed beside its siblings. Both arrays share a length and that length is not nBins: an empty bin is dropped, so it depends on the data — measured, four probabilities over five uniform bins return four points and four probabilities inside one bin return one. strategy is an enum rather than a string, and Quantile reads its edges from np.percentile's linear interpolation, not from the weighted percentile decision 0024 pinned for the medians — the two disagree, and reusing the weighted one would move the third decimal. Repeated probabilities collapse quantile edges onto each other and empty bins rather than balancing them, which the corpus pins. posLabel is a parameter defaulting to 1 where the reference infers it, as BrierScore.Score's already is. There is no sample_weight: the reference has none for this curve. Returned as a sealed class per decision 0040.
roc_auc_score(…, multi_class=…)	scikit-learn	RocAuc.MultiClass(…, MultiClassRocOptions)	ovr and ovo. Separate method: the overloads would be ambiguous. Strategy, averaging, labels and weights travel in MultiClassRocOptions, which also carries MaxDegreeOfParallelism — no scikit-learn equivalent, opt-in, sequential by default. sampleWeight refused for ovo, as in scikit-learn.
balanced_accuracy_score(…, adjusted=…)	scikit-learn	BalancedAccuracy.Score(…)	Averages over the classes with a true sample, as scikit-learn does; adjusted divides by that same kept count, and returns NaN or -∞ when only one class is kept — the same two values scikit-learn returns. The overload taking a ConfusionMatrix scores only the classes that matrix holds: with an explicit labels subset, a dropped sample counts nowhere, not even in a denominator. balanced_accuracy_score has no labels parameter, so there is no reference value for that case (decisions/0020).
matthews_corrcoef(…)	scikit-learn	MatthewsCorrelation.Score(…)	scikit-learn hard-codes 0.0 when the denominator collapses; here it is ZeroDivision, defaulting to that value, with Throw available. An extension beyond parity, not a divergence in value. The overload taking a ConfusionMatrix scores only the classes that matrix holds; matthews_corrcoef has no labels parameter, so there is no reference value for a restricted matrix (decisions/0020).
cohen_kappa_score(…, weights=…)	scikit-learn	CohenKappa.Score(…, KappaWeighting…)	weights renamed weighting, because sampleWeight shares the signature. replace_undefined_by maps onto ZeroDivision, defaulting to NaN — scikit-learn's value; it also covers a view that holds no weight at all, where scikit-learn returns the same. The weighted forms depend on label order. The overload taking a ConfusionMatrix scores only the classes that matrix holds; cohen_kappa_score does take labels, so a reference value exists here, and on the fixture the tests pin the two agree (decisions/0020).
confusion_matrix(…, normalize=…)	scikit-learn	ConfusionMatrix.ToArray(Normalization)	A projection, not a parameter on Compute: several metrics here read a matrix, and fractions would make them silently wrong (decisions/0020).

Lodestar.Metrics — regression metrics

Python	Library	C#	Differences
mean_squared_error(…, multioutput=…)	scikit-learn	MeanSquaredError.Score(…), .PerOutput(…)	multioutput is the choice of method plus an optional outputWeights span, not an enum — raw_values changes the return type, which decisions/0016 already ruled cannot be an enum member, and decisions/0021 applies that ruling here. 2-D targets arrive row-major with outputCount; there is no 2-D overload, because a span cannot carry one. Two refusals every metric in this block shares, both reproduced with the message their Python layer prints: a sampleWeight that is zero throughout gives check_array's "Sample weights must contain at least one non-zero number." — the rule is every weight zero, not the sum, so [-1, -2, -3] still scores — and outputWeights summing to zero give numpy.average's "Weights sum to zero, can't be normalized.", where the rule is the sum, so [1, -1] is refused and [-1, -1] scores. ValueError and ZeroDivisionError both become ArgumentException. The accumulation behind the mean is Neumaier-compensated (issue #127), so the answer is at least as accurate as numpy's pairwise reduction, not merely close to it.
root_mean_squared_error(…)	scikit-learn	RootMeanSquaredError.Score(…), .PerOutput(…)	A type of its own: scikit-learn removed mean_squared_error(squared=False) in 1.6. The root is taken per output, before the reduction, so on more than one output the result is not the root of MeanSquaredError.Score — that is scikit-learn's order too. The underlying mean is Neumaier-compensated (issue #127), at least as accurate as numpy's pairwise reduction rather than merely close to it.
mean_absolute_error(…)	scikit-learn	MeanAbsoluteError.Score(…), .PerOutput(…)	As above for multioutput. The accumulation is Neumaier-compensated (issue #127), at least as accurate as numpy's pairwise reduction rather than merely close to it.
median_absolute_error(…)	scikit-learn	MedianAbsoluteError.Score(…), .PerOutput(…)	With sampleWeight, an averaged weighted percentile: the mean of the first value whose cumulative weight reaches half the total and the one just past the last that comes within one machine epsilon of it. That tolerance is scikit-learn's own (fraction_above > np.finfo(float64).eps) and it is load-bearing, not decoration: on sample_weight = [0.1] * 10 an exact comparison returns 4.0 where scikit-learn returns 4.5. A uniform weight is therefore usually the ordinary median but not always — measured, [0.7] * 10 gives 5.0 on the weighted path against 4.5 on the unweighted one, because there the overshoot is wider than an epsilon. Both sides agree, divergently, with scikit-learn.
mean_absolute_percentage_error(…)	scikit-learn	MeanAbsolutePercentageError.Score(…), .PerOutput(…)	The denominator is clamped at numpy's machine epsilon, 2**-52 — not double.Epsilon, which is 292 orders of magnitude smaller. mean_absolute_percentage_error([0], [1]) is therefore 4503599627370496.0 on both sides. The accumulation behind the mean is Neumaier-compensated (issue #127), at least as accurate as numpy's pairwise reduction rather than merely close to it.
max_error(y_true, y_pred)	scikit-learn	MaxError.Score(yTrue, yPred)	No sampleWeight and no multioutput, because max_error has neither and refuses 2-D input. A worst case is not an average.
mean_squared_log_error(…)	scikit-learn	MeanSquaredLogError.Score(…), .PerOutput(…)	Refuses a target at or below −1 on either side, as scikit-learn does — ArgumentException for its ValueError; the message additionally names the side, which costs no parity because no value is returned either way. The logarithm is numpy's log1p, reached through Kahan's identity rather than Math.Log(1.0 + x): on targets around 1e-9 the latter is out by 1.7e-8 relative, where this agrees with scikit-learn to a unit in the last place. The mean itself is Neumaier-compensated (issue #127), at least as accurate as numpy's pairwise reduction rather than merely close to it.
root_mean_squared_log_error(…)	scikit-learn	RootMeanSquaredLogError.Score(…), .PerOutput(…)	As above, and the root is taken per output before the reduction as in root_mean_squared_error. The underlying mean is Neumaier-compensated (issue #127), at least as accurate as numpy's pairwise reduction rather than merely close to it.
r2_score(…, force_finite=…)	scikit-learn	R2.Score(…), .PerOutput(…), .VarianceWeighted(…)	Two independent undefined cases, deliberately kept apart. Fewer than two samples is ZeroDivision, defaulting to NaN — scikit-learn's value, recorded in decisions/0020 — while a truth of zero variance over two or more samples is forceFinite. They do not overlap. One shape divergence: on fewer than two samples with more than one output, PerOutput returns one NaN per output, where r2_score returns a single scalar nan before it ever consults multioutput. No number differs — every scalar-returning path here still gives nan — and a one-element array would break PerOutput's own contract of one value per output. Both of R2's passes are Neumaier-compensated (issue #127): the answer is at least as accurate as numpy's pairwise reduction, not merely close to it — load-bearing on an ill-conditioned target, where a sequential sum measured 357× outside the oracle's tolerance.
explained_variance_score(…)	scikit-learn	ExplainedVariance.Score(…), .PerOutput(…), .VarianceWeighted(…)	Takes forceFinite but no ZeroDivision: it has no fewer-than-two-samples case to route, so explained_variance_score([3], [5]) is 1.0, not nan, and PerOutput matches scikit-learn exactly there — the divergence noted for r2_score is r2_score's alone. Its five accumulations are Neumaier-compensated (issue #127) for the same reason R2's are, at least as accurate as numpy's pairwise reduction rather than merely close to it.
mean_pinball_loss(…, alpha=…)	scikit-learn	PinballLoss.Score(…, alpha, …), .PerOutput(…)	Named for the loss rather than for the Python identifier's mean_ prefix, matching the other ten. alpha outside [0, 1] throws ArgumentOutOfRangeException where scikit-learn raises InvalidParameterError. The underlying mean is Neumaier-compensated (issue #127), at least as accurate as numpy's pairwise reduction rather than merely close to it.
mean_tweedie_deviance(y_true, y_pred, sample_weight=…, power=…)	scikit-learn	TweedieDeviance.Score(…)	Identical, all five regimes and their domains included: below 0 only the prediction must be strictly positive, at 0 nothing is constrained, in [1, 2) the truth must be non-negative and the prediction strictly positive, and from 2 up both must be strictly positive. Each refusal carries scikit-learn's own sentence naming the power, as an ArgumentException where it raises ValueError. A power in the open interval (0, 1) names no distribution and is ArgumentOutOfRangeException where scikit-learn raises InvalidParameterError. At power 0 the number is MeanSquaredError.Score's exactly. y·log(y/ŷ) is taken as 0 at y = 0, numpy's xlogy, which is what makes a zero truth legal in [1, 2). No multioutput, because the reference has none on this function.
mean_poisson_deviance(y_true, y_pred, sample_weight=…)	scikit-learn	PoissonDeviance.Score(…)	Identical. TweedieDeviance.Score at power 1, which is how the reference defines it too — asserted across the whole frozen corpus rather than on one pair. A zero truth is accepted and a zero prediction refused, with the power-1 sentence.
mean_gamma_deviance(y_true, y_pred, sample_weight=…)	scikit-learn	GammaDeviance.Score(…)	Identical. TweedieDeviance.Score at power 2. Both operands must be strictly positive — a zero truth is refused here where the Poisson accepts one — and the number is unchanged by scaling both arguments together, measured 0.09824107126307435 on the worked case and on the same case times ten.
d2_tweedie_score(y_true, y_pred, sample_weight=…, power=…)	scikit-learn	D2Tweedie.Score(…)	Identical on every input either side defines. At power 0 it is R2.Score exactly. A truth that never varies diverges in form, not in outcome: scikit-learn divides by the zero denominator and raises ZeroDivisionError, and this raises UndefinedMetricException naming the cause — where d2_absolute_error_score masks the same case and answers 0 on both sides. Fewer than two samples is nan with a warning there and ZeroDivision.NaN here, the parameter R2.Score already takes for the identical case. Multioutput is offered by neither: the reference raises "Multioutput not supported in d2_tweedie_score".
d2_pinball_score(y_true, y_pred, sample_weight=…, alpha=…, multioutput=…)	scikit-learn	D2Pinball.Score(…), .PerOutput(…)	Identical, weights and both multioutput modes included. A column whose truth never varies scores 0 on both sides, the reference masking that denominator here where d2_tweedie_score divides by it. Fewer than two samples is nan, adjustable through ZeroDivision. Which of two candidate order statistics the denominator's quantile takes is unobservable — they differ only where the quantile is ambiguous and the pinball loss is flat there — measured over four fixtures at five alphas each. alpha outside [0, 1] throws ArgumentOutOfRangeException where scikit-learn raises InvalidParameterError, as PinballLoss.Score's already does.
d2_absolute_error_score(y_true, y_pred, sample_weight=…, multioutput=…)	scikit-learn	D2AbsoluteError.Score(…), .PerOutput(…)	Identical. D2Pinball.Score at alpha = 0.5, an invariant no oracle states and a test asserts across every fixture — the two reach their baseline through different code, a quantile at one half and a median. Compares against the median where R2.Score compares against the mean, so an outlier in the truth does not flatter the model.

Lodestar.Metrics — clustering metrics

Python	Library	C#	Differences
adjusted_rand_score(labels_true, labels_pred)	scikit-learn	AdjustedRand.Score(labelsTrue, labelsPred)	Identical, degenerate cases included: an empty input and a single sample both score 1, and two independent partitions of four samples score -0.5.
normalized_mutual_info_score(…)	scikit-learn	NormalizedMutualInformation.Score(…)	average_method is not a parameter: the arithmetic mean, scikit-learn's default, is the only normalizer reproduced. The other three (min, geometric, max) have no oracle row and are refused by absence rather than by exception.
fowlkes_mallows_score(labels_true, labels_pred)	scikit-learn	FowlkesMallows.Score(labelsTrue, labelsPred)	Identical, and the degenerate cases split from the rest of this family: an empty input and a single sample score 0 here where the other five score 1, because there is no agreeing pair to count. Grouped as sqrt(tk/pk)·sqrt(tk/qk), the reference's own associativity.
adjusted_mutual_info_score(…)	scikit-learn	AdjustedMutualInformation.Score(…)	Identical. average_method is not a parameter, as for normalized_mutual_info_score above. The expected mutual information uses a cumulative log(k!) table rather than a gammaln series. That table accumulates rounding: parity is established below about 20 000 samples and not above it (8.4e-10 relative there, against this corpus's 1e-9).
rand_score(labels_true, labels_pred)	scikit-learn	RandIndex.Score(labelsTrue, labelsPred)	AdjustedRand.Score before the correction for chance: on [0,0,0,1,1,1] against [0,0,1,2,2,2] this scores 0.867 where AdjustedRand.Score scores 0.706. Uncorrected, so two independent labellings score well above zero, unlike the adjusted form.
mutual_info_score(labels_true, labels_pred)	scikit-learn	MutualInformation.Score(labelsTrue, labelsPred)	NormalizedMutualInformation.Score before the [0,1] normalisation, in nats. Unbounded above. One divergence, deliberate: on an empty input scikit-learn 1.9.0 raises ValueError (log(0) inside mutual_info_score), not a documented refusal; this returns 0.0, matching every other metric in the family — see decision 0039.
cluster.pair_confusion_matrix(labels_true, labels_pred)	scikit-learn	PairConfusionMatrix.Compute(labelsTrue, labelsPred)	The pair counts both Rand forms are computed from, as four named long properties rather than a ConfusionMatrix — that type counts labels, this counts pairs of samples, and reusing either the name or the type would have named the wrong thing. ToArray() reproduces numpy's [[C00,C01],[C10,C11]] shape for ported code.
homogeneity_score(…)	scikit-learn	Homogeneity.Score(…)	Identical.
completeness_score(…)	scikit-learn	Completeness.Score(…)	Identical, and implemented as Homogeneity with the two labellings exchanged, which is scikit-learn's own definition.
v_measure_score(…, beta=1.0)	scikit-learn	VMeasure.Score(…)	beta is not a parameter: the default of 1 is the harmonic mean, and no reference value exists here for another weighting.
homogeneity_completeness_v_measure(…)	scikit-learn	the three calls above	No combined call: three doubles would be a tuple or a record, and each metric is cheap enough to ask for on its own. The contingency table is rebuilt per call, which is O(n) each time.
silhouette_score(X, labels)	scikit-learn	Silhouette.Score(labels, features, featureCount)	Euclidean only. scikit-learn takes some twenty metric= names; each admitted here would be a parity claim to keep, so another metric goes through the precomputed path. 2-D samples arrive row-major with a feature count, as the regression metrics take 2-D targets.
calinski_harabasz_score(X, labels)	scikit-learn	CalinskiHarabasz.Score(…)	Identical, degenerate clusterings included. Clusters with no spread at all score 1 rather than dividing by zero, which is the reference's own guard — measured, four identical points in two clusters and two well-separated points each duplicated both read 1. A label count outside [2, n - 1] is ArgumentException carrying scikit-learn's sentence, "Number of labels is k. Valid values are 2 to n_samples - 1 (inclusive)", the same range silhouette_score and davies_bouldin_score refuse. No metric parameter and no precomputed-distance form, because the reference has neither: the score reads cluster centroids and a distance matrix does not carry them. 2-D input arrives row-major with a featureCount, as Silhouette.Score's does.
davies_bouldin_score(X, labels)	scikit-learn	DaviesBouldin.Score(…)	Identical. Lower is better, the opposite direction to every other clustering score here. Two clusters sharing a centroid contribute 0 rather than an infinity: the reference substitutes infinity for the zero distance before dividing, so the pair drops out of the maximum — measured, a perfect clustering and a fully degenerate one both read 0. Same refusal range and sentence as calinski_harabasz_score, and no precomputed-distance form for the same reason.
silhouette_score(D, labels, metric='precomputed')	scikit-learn	Silhouette.ScoreFromDistances(labels, distances)	A method of its own, not an overload: a matrix and a feature block are both a span of double, so the two signatures would collide (decisions/0021 applied to an input).
silhouette_samples(X, labels)	scikit-learn	Silhouette.PerSample(…), .PerSampleFromDistances(…)	Identical, cluster of one sample included: it scores 0, measured, rather than dividing by zero. The refusal outside [2, n-1] distinct labels is scikit-learn's ValueError, reproduced as ArgumentException with its sentence.

Lodestar.Metrics — ranking metrics

Python	Library	C#	Differences
dcg_score(y_true, y_score, k=…, log_base=…, sample_weight=…, ignore_ties=…)	scikit-learn	Dcg.Score(…)	Identical, every parameter included since #216. The gains are linear — Σ relevance / log(rank + 1) — as scikit-learn's are, not the 2^relevance − 1 form much of the literature uses: on [3, 2, 1, 0] ranked perfectly that is 4.7618595071429155 against 9.392789260714373. 2-D input arrives row-major with a labelCount, as the regression and clustering metrics take theirs; there is no 2-D overload, because a span cannot carry one. A k past the label count scores the whole row, as it does in scikit-learn; a k below 1 is ArgumentOutOfRangeException where scikit-learn raises InvalidParameterError, and a logBase outside (0, ∞) — zero, negative, NaN or infinite — is refused the same way, against the same constraint scikit-learn prints as "must be a float in the range (0.0, inf)"; a base below 1 is inside that interval and accepted on both sides, taking the score negative. A negative relevance is accepted and can give a negative score, which dcg_score does too. sampleWeight weights the mean over queries and cancels over a single one; a vector summing to zero is ArgumentException in numpy.average's own sentence where the reference raises ZeroDivisionError from that same call, and a negative weight is accepted on both sides and takes the mean outside the range the page promises — frozen in ranking_weighted.json, 1.0840593484403573 where the unweighted mean is 3.8424094674672755, and -3.0 at k = 2.
ndcg_score(y_true, y_score, k=…, sample_weight=…, ignore_ties=…)	scikit-learn	Ndcg.Score(…)	Identical since #216, sample_weight included. No log_base, because ndcg_score has none — the discount cancels in the ratio only when both halves share a base, and scikit-learn shares base 2. A row where nothing is relevant scores 0 rather than dividing by zero, which is scikit-learn's value too. The ideal is computed without tie averaging on both sides. A negative relevance is refused with scikit-learn's own sentence, "ndcg_score should not be used on negative y_true values." — unguarded, y_true = [1, -1] against y_score = [0.1, 0.9] scores -1.
ignore_ties=False (the default)	scikit-learn	ignoreTies: false	The closed form, not a permutation enumeration: within a tied group the mean relevance times the sum of the discounts of the positions it occupies is the average over the permutations, which is what scikit-learn computes. Load-bearing rather than cosmetic — on a row whose four scores are equal it gives 0.8069136566720543 against 0.6138273133441086. Ties are exact equality of the score on both sides; a tolerance would merge scores np.unique keeps apart.
ignore_ties=True	scikit-learn	ignoreTies: true	Not a parity claim on a row that has ties. _dcg_sample_scores reaches this path through a bare np.argsort, whose default is an unstable quicksort, so the order scikit-learn gives a tied group is undefined. Here it is defined — equal scores rank by descending index, the order top_k_accuracy_score's explicit kind="mergesort" gives. The two agree on every row of the frozen corpus, which is 4 and 6 documents wide; on a wider one they may not, and the frozen values would then depend on the numpy build that captured them. Untied rows are unaffected, and they are what ignore_ties is for.
top_k_accuracy_score(y_true, y_score, k=…, normalize=…, sample_weight=…)	scikit-learn	TopKAccuracy.Score(…)	One widening, no divergence in value, sample_weight included since #216 — with weights normalize=False returns the sum of the weights of the hits rather than how many there are, measured 7.0 against the unweighted 3.0, and because that path never divides it does not refuse a zero-sum vector at all, where the fraction does — what it returns there is the weighted sum of the hits, 3.0 on weights [1, 1, 1, -3] whose total is zero. scikit-learn infers the class set from y_true and refuses a score row wider than what it found unless given labels; here classCount is a parameter, so a class no sample carries raises nothing. On any input scikit-learn accepts, the two agree — ties included, because both take the top k of a stable sort. k below 1 is ArgumentOutOfRangeException where scikit-learn raises InvalidParameterError, and an empty y_true or a class outside [0, classCount) is ArgumentException where scikit-learn raises ValueError.
— (no counterpart)	—	ReciprocalRank.Score(…)	Not verified against a reference. Measured on scikit-learn 1.9.0, dir(sklearn.metrics) carries nothing matching reciprocal, so there is no corpus to freeze; the definition is pinned by tests instead, under decisions/0036, which also says what would retire the exception. The definition: the reciprocal of the rank of the first relevant document, averaged over queries, a query with no relevant document contributing 0 rather than being dropped from the average.
coverage_error(y_true, y_score, sample_weight=…)	scikit-learn	CoverageError.Score(…)	Identical, degenerate rows included: a sample with no relevant label contributes 0 rather than the label count, so the mean can sit below 1 — measured, 0.5 on two samples one of which is empty. A single label column is refused with scikit-learn's own sentence, "binary format is not supported", where label_ranking_average_precision_score accepts one and returns 1; that divergence is the reference's, reproduced rather than smoothed. A sample_weight summing to zero raises ArgumentException with numpy's "Weights sum to zero, can't be normalized." where scikit-learn raises ZeroDivisionError from the same numpy.average, and where label_ranking_average_precision_score returns NaN instead. A negative weight is accepted on both sides and takes the result out of its range — measured, 5.0 on the two-sample worked case. 2-D input arrives row-major with a labelCount, as the other 2-D metrics take theirs.
label_ranking_average_precision_score(y_true, y_score, sample_weight=…)	scikit-learn	LabelRankingAveragePrecision.Score(…)	Identical, the two places the reference disagrees with its siblings included. A single label column is accepted and scores 1, where coverage_error and label_ranking_loss refuse it — the reference validates this one differently, and making the three agree would invent a divergence instead of copying one. A sample_weight summing to zero gives NaN rather than raising, because the reference divides by the weight sum directly here instead of calling numpy.average. A negative weight is accepted and takes the result out of [0, 1] — measured, -0.33333333333333337. A sample where every label or no label is relevant scores 1 on both sides.
label_ranking_loss(y_true, y_score, sample_weight=…)	scikit-learn	LabelRankingLoss.Score(…)	Identical, tie handling included: an irrelevant label sharing a relevant one's score counts as outranking it, so a sample whose scores are all equal scores 1 rather than 0.5. A single label column is refused with "binary format is not supported", where label_ranking_average_precision_score accepts one. A sample_weight summing to zero raises ArgumentException with numpy's "Weights sum to zero, can't be normalized.", where the average precision returns NaN. A negative weight is accepted and takes the result out of [0, 1] — measured, 2.0. A sample where every label or no label is relevant holds no pair to order and contributes 0 on both sides.
average_precision_score(y_true, y_score, average=…, pos_label=…, sample_weight=…)	scikit-learn	AveragePrecision.Score(…), AveragePrecision.PerLabel(…)	Identical on every input either side defines, binary and label matrix alike, and it is a sum over the steps of the precision-recall curve rather than the trapezoid auc(recall, precision) takes — measured 0.8333333333333333 against 0.7916666666666666 on the worked case, and 0.5 against 0.75 on a row of tied scores. With no positive sample scikit-learn warns "No positive class found in y_true, recall is set to one for all thresholds" and returns 0.0; that value is reproduced rather than refused, where RocAuc.Score on the same walk throws. average='macro', 'micro' and 'weighted' are Averaging.Macro, Averaging.Micro and Averaging.Weighted; average=None is PerLabel. average='samples' is not offered — Averaging has no such member, and it is shared with Precision, Recall, F1 and FBeta, none of which implements it either. Three weight vectors diverge, all measured: one summing to zero, [1, 1, 1, -3], gives 0.5 there through a numpy divide-by-zero warning and -0 here; every weight 0 raises ValueError there and returns 0 here; and a pos_label no sample carries raises ValueError there and is the no-positive case here, 0 — posLabel being a parameter where scikit-learn infers it, as TopKAccuracy.Score's classCount already is. A negative weight leaving the total positive agrees: 0.75 on both.

Conventions

• Comparison unit. Unless stated otherwise, string distances compare char values (UTF-16 units), which is the native .NET choice and fastest. Python libraries (rapidfuzz, jellyfish) iterate over code points: to reproduce their values exactly on supplementary text (emoji, rare ideographs), pass TextElement.CodePoint. See decisions/0002-unicode-comparison-unit.md.
• ReadOnlySpan<char>. All computation signatures accept spans; string literals convert implicitly, so Levenshtein.Distance("a", "b") works with no allocation.
• Culture. No operation is culture-sensitive by default. Overloads accepting a CultureInfo are added where case/accents matter (tokenization).
• Stop words. StopWords.English is scikit-learn's list; the other five are Snowball's, because the nltk corpus carries no usable licence (decisions/0010). This is the one place where the library knowingly does not match nltk, so the gap is measured rather than described: French 154 words vs nltk's 157 (13 / 16 words apart), German 231 vs 232 (4 / 5), Portuguese 203 vs 207 (0 / 4), Spanish 308 vs 313 (2 / 7), Italian identical. Matching is ordinal against the analyzer's output, so StripAccents = true also stops accented entries from matching — as it does in scikit-learn.