Text 0.4.0 overlap

Lodestar.Text 0.4.0. This page is frozen at that release. Read the current documentation for what main says now. A link to a decision or a migration page follows main, and leaves the archive.

Overlap

Shared q-grams over the smaller bag: |A∩B| / min(|A|, |B|). Also the Szymkiewicz-Simpson coefficient.

public static class Overlap

Example — containment, which is what this measure is for.

using Lodestar.Text.Similarity;

double contained = Overlap.Similarity("apple", "pineapple");  // => 1

Remarks — the size gap between the two inputs is divided away rather than charged for, which makes Overlap the right answer to "does the shorter one appear in the longer" and the wrong answer to "do these two agree".

That is also its trap, and it is worth stating plainly: every bag contained in another scores 1, however much extra the other holds. A one-character query scores 1 against any text containing that character. Reach for Cosine when the extra material should still cost something, or Tversky when the containment should be measured in one direction only.

Applies to — net10.0, netstandard2.0.

See also — Cosine, Tversky, the set-similarity index.

Members

Member	What it does
Overlap.Similarity	Shared grams over the smaller of the two bags.