You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
pack_chat(): budget a conversation and get messages back rather than
one string. Chat models take role-tagged message lists, and the role is not
decoration -- it drives the chat template. Roles, name, and caller
metadata survive packing intact; content is byte-exact, including content
containing the separator.
Message: a validated chat turn (system / user / assistant / tool).
Provider-style {"role", "content"} dicts are accepted anywhere a Message is, so existing message lists work unchanged.
ChatResult: the packed messages plus the full underlying audit trail, with to_openai() for the payload every major provider SDK accepts. Wraps rather
than subclasses PackResult -- a chat result has no .text to advertise.
tokens_per_message: charge the per-message framing overhead providers bill
for, so the budget reflects the real invoice instead of visible text only.
pin_last_user (default True): the most recent user message is protected
from eviction. Dropping the question being answered is never the intent.
HuggingFaceTokenizer: exact counts for open-weights models (Llama, Mistral,
Qwen, Gemma). Truncation slices real token ids. Pass an already-loaded
tokenizer to skip the download entirely.
AnthropicTokenizer: exact Claude counts via the API with per-string
caching, or a calibrated offline heuristic when no client is given. An API
failure degrades to the heuristic rather than taking the caller's packing
down with it.
SectionReport.items: the items that survived packing, in render order.
Rendered text is a lossy view -- once items are joined the boundaries are
gone -- and rebuilding messages by splitting on the separator would corrupt
any content legitimately containing it.
Notes
pack_chat() does not reimplement allocation. It maps messages onto
sections, calls the existing pack(), and maps the survivors back, so every
guarantee already proved for pack() holds for chat packing too. The
property tests assert them against both APIs.
Fully backwards compatible: no existing signature or behaviour changed.