Initial release.
Added
LocalLexicon— a fast, single-threaded string interner (&mut selfinterning,
live&selfresolution).ThreadedLexicon— a concurrent string interner:&selfinterning from many
threads, a cheapCloneArc-backed handle, fill-then-freeze.Sym— a 4-byte,Copy, niche-optimized handle;as_u32/from_u32and
From<Sym> for u32conversions;iter()over(Sym, &str).Reader— a sealed trait for the frozen,Send + Sync, lock-free read form,
produced byLocalLexicon::freeze/ThreadedLexicon::freeze; supportsiter().SymMap/SymSet/SymBuildHasher— fastSym-keyed maps and sets via an
identity-style hasher.ExtendandFromIteratorfor both interners.- Generic over the
BuildHasher, defaulting to a fast non-cryptographic hasher. no_stdsupport: the crate isno_std+allocwithout the defaultstd
feature (which enablesThreadedLexicon).- Optional
serdefeature: a string-based, interner-aware serialization model.
#[derive(SerializeIn)]/#[derive(DeserializeIn)]resolve eachSym
against aReader/ interner so values round-trip as their strings
(serialize aSymto its string, deserialize a string back into aSym),
making corpora portable across processes.SerializeReadersnapshots a whole
frozen corpus.Symitself has no plainSerialize/Deserialize(a raw
handle is meaningless outside its lexicon), and the liveThreadedLexiconis
deserialize-only — serialize the frozenReaderinstead.