Text 0.5.0 hashingvectorizer transform

Lodestar.Text 0.5.0. This page is frozen at that release. Read the current documentation for what main says now. A link to a decision or a migration page follows main, and leaves the archive.

HashingVectorizer.Transform

Hash a corpus into the fixed columns.

public CsrMatrix Transform(IEnumerable<string> documents)

Parameters — documents is the corpus to vectorize. Nothing is learned from it.

Returns — CsrMatrix, one row per document and exactly NumFeatures columns, normalized by HashingVectorizerOptions.Norm.

Exceptions — ArgumentNullException when documents is null.

Example — the width is the option, not the corpus.

using Lodestar.Abstractions;
using Lodestar.Text.Vectorization;

var hv = new HashingVectorizer(new HashingVectorizerOptions { NumFeatures = 16 });

CsrMatrix first = hv.Transform(["the cat eats"]);
CsrMatrix second = hv.Transform(["an entirely different corpus about boats"]);

int width = first.ColumnCount;      // => 16
int alsoWidth = second.ColumnCount; // => 16

Remarks — no state carries between calls, so two corpora vectorized separately are directly comparable — the same term lands in the same column both times, and on another machine too. That is the property the count vectorizers cannot offer without saving and shipping a fitted model.

A row can hold negative values when AlternateSign is on, which is the default. That is not a defect: it is what makes two colliding terms tend to cancel rather than sum, and it means a CsrMatrix from here is the one place in this namespace where RowL1Norm's absolute values matter.

Applies to — net10.0, netstandard2.0.

See also — HashingVectorizer.FitTransform, HashingVectorizerOptions.