Embeddings 0.4.0 specialtokentemplate gethashcode

Lodestar.Embeddings 0.4.0. This page is frozen at that release. Read the current documentation for what main says now. A link to a decision or a migration page follows main, and leaves the archive.

SpecialTokenTemplate.GetHashCode

A hash consistent with that equality.

public int GetHashCode()

Returns — int, over the prefix tokens, the suffix tokens and the pad token.

Example — the same template hashes alike.

using Lodestar.Embeddings.Tokenization;

SpecialTokenTemplate template = SpecialTokenTemplate.Bert;

bool equal = template.Equals(SpecialTokenTemplate.Bert);  // => True
bool hashesAlike = template.GetHashCode() == SpecialTokenTemplate.Bert.GetHashCode();  // => True

Remarks — the token contents contribute, not the list identities, so a template rebuilt from the same strings hashes to what the ready-made one does. That is what makes a template usable as a dictionary key when caching encoders per model.

Applies to — net10.0, netstandard2.0.

See also — SpecialTokenTemplate.Equals.