Repository navigation
v0.3.5
A new way back for names and fixes for agents, and nothing that worked before stops working: a
patch release, as CONTRIBUTING.md describes. The Polish note of token_instructions now asks the
model to write a person's case after a token; a token without one restores as before. Where the
Polish model read English as names it now finds less, so an agent's system prompt comes out with
fewer tokens, and the gateway masks the prose of tools it used to send as written.
Added
- A person comes back in the case the model asks for. A token stood for the form the text
wrote, so "wniosek pracownika Jana Kowalskiego" gave an answer "Szanowny Panie Jana
Kowalskiego". The model may now write the case after a colon,<OSOBA_1:wołacz>, and the name
comes back declined: "Szanowny Panie Janie Kowalski". The Polish note oftoken_instructions
asks for it, with<OSOBA_1:mianownik>in a tool's arguments, and every restore understands
it:MaskedTurn.restoreandrestore_data,Anonymizer.restore,StreamRestorerwhen a
token and its case arrive cut apart, the Haystack components and both APIs of the gateway, a
tool's arguments staying valid JSON. Declined are names of one or two words the rules read for
sure, a listed first name and a surname such as Kowalski, Nowak, Bednarek, Mickiewicz or Duda,
written in capitals, in lowercase or without Polish letters as the value was. A surname alone
that may be a man's or a woman's, a foreign name, a company and a place come back as written,
and the case is dropped. A token without a case restores as before, and an invented one with
a case is reported byunknown_tokensandsweepas the token it declines.
Fixed
- Ordinary English words are no longer masked as names by the Polish model. Reading an agent's
English system prompt through the gateway, it tagged "result", "NOT", "Safety" and "the user's
own" as companies, people and places: 89 tokens in one call, a prompt the model could hardly
read, and, since the same words stood in the tool descriptions, a call the gateway's last check
refused. On a plainly English line, one with English words, few Polish ones and no Polish
letter, a span the Polish model finds is kept only as a capitalised name in mid-sentence, such as
"Maria Lindqvist" in "ask Maria Lindqvist for it". A name that opens an
English sentence is left to the rules, which find a Polish first name anywhere. Polish lines,
in lowercase or without diacritics, and an English spaCy model are untouched; the benchmark is
unchanged. - Names from code and lines with no language are no longer taken for people. An agent's
tools have parameters such asreplace_all, which the Polish model tagged as a person: the tool
schema holds the name as a key, which cannot be masked, so the gateway refused every call. A
span with a backtick, or with a "_" and no Polish first name, is now code on any line, so
"jan_kowalski" stays masked. On a line too short to show a language, such as "Max results" or
"Skills for", a span with a word in lowercase is kept only when it holds a Polish first name.
A line counts as Polish when its Polish words outnumber its English ones, and "ten", "pan",
"pod" and "ale", English words too, no longer count; after a colon on an English line, a name
is kept: "Contact: John Smith". The benchmark masks what it did, with one stray span fewer. - The gateway masks the prose of tools and answer formats. Tool descriptions went out as
written, so a value they held reached the provider, and a value the messages held was found
there by the last check, which refused the call: an agent with 18 tools could not get a single
call through. Now the descriptions and titles of the tools, the legacyfunctions, a JSON-schema
response_formatand the Responses API'stext.formatare masked, and so are the values of
enum,const,defaultandexamples; a model answering with such a token gets it back in
the arguments. Tool names, types and the names of parameters stay, so the provider still checks
answers against the schema. A value found in a tool is masked in the messages, and one found in
the messages in every tool.