Skip to content

engine(streaming): token-id callback cannot express stop-string-trimmed output #424

Description

@jamesburton

Observation

Found while addressing review feedback on #296, which trims a stop-string suffix from the final
token at a character boundary.

TextGenerator's per-token callback is Action<int> — it receives a token id. Once a token has
been partially trimmed, there is no id that represents the trimmed text, so the callback cannot
express what was actually emitted.

The reviewer suggested invoking the callback for the kept portion. That cannot work as stated:
passing the original id would render the untrimmed text and leak the stop string back to the
consumer, contradicting StopStringCondition's contract; omitting the token loses the kept
characters entirely.

Impact

Consumers driving output from the token callback see a slightly different stream from consumers
reading the returned text: the final partial token is either absent or, if naively re-rendered,
includes the stop string that was meant to be removed. The non-streaming path is correct; it is the
callback surface that cannot represent the result.

The same limitation applies to any future transformation that operates on text rather than whole
tokens.

Proposed change

Add a text-level streaming callback alongside the id-level one, e.g.
Action<ReadOnlySpan<char>> (or an overload taking both id and text, with the text authoritative).
The trimming path then emits the kept characters with no id, and ordinary tokens emit both.

Worth deciding at the same time whether the id-level callback should remain for consumers that
genuinely want ids (logprobs, speculative-decoding instrumentation) — I would keep it, and document
that it is the pre-trim stream.

Acceptance criteria

  • A consumer driven purely by the streaming callback receives exactly the text returned by the
    non-streaming path, including the trimmed final token.
  • The stop string never appears in the callback stream.
  • Existing id-level consumers are unaffected, or the break is documented if the surface changes.
  • A test drives generation with a stop string that splits a token and asserts callback-assembled
    text equals the returned text.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions