ConvertTo‑ShpTokenCount

SYNOPSIS

Estimates the number of model tokens in a piece of text.

SYNTAX

ConvertTo-ShpTokenCount [-Text] <String[]> [<CommonParameters>]

DESCRIPTION

Returns an approximate token count for the supplied text using a pure PowerShell heuristic, so a caller can size a prompt - and, with Get-ShpCostEstimate, its likely cost - before sending it. No exact tokenizer is used (that would need a compiled dependency, which this module avoids); the estimate blends a character-per-token ratio with a word count and is deliberately conservative. Treat the result as a guide, not an exact figure: the service's reported usage is authoritative. Accepts text from the pipeline.

EXAMPLES

EXAMPLE 1

ConvertTo-ShpTokenCount -Text 'Summarise PowerShell splatting in two lines.'

Returns an approximate token count for the prompt.

EXAMPLE 2

Get-Content .\prompt.txt -Raw | ConvertTo-ShpTokenCount

Estimates the token count of a prompt held in a file.

PARAMETERS

-Text

The text to estimate. Accepts one or more strings (each is estimated and the counts are summed). Mandatory.

Type: String[]
Parameter Sets: (All)
Aliases:

Required: True
Position: 1
Default value: None
Accept pipeline input: True (ByValue)
Accept wildcard characters: False

CommonParameters

This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable, -InformationAction, -InformationVariable, -OutVariable, -OutBuffer, -PipelineVariable, -Verbose, -WarningAction, and -WarningVariable. For more information, see about_CommonParameters.

INPUTS

OUTPUTS

System.Int32

The estimated token count.

NOTES

RELATED LINKS