Repository navigation
v0.3.3
Fixes only, and nothing that worked before stops working: a patch release, as CONTRIBUTING.md
describes. More is masked than under 0.3.2, so a text may carry more tokens, and with
WELL_KNOWN_NAMES the names of a few public institutions stay readable.
Fixed
- A token-shaped number longer than six digits in the input, such as
<OSOBA_50810112354>,
no longer sets the number of every later token, and one of thousands of digits no longer stops
masking with aValueError. A new token still skips every token the mapping holds, so
<OSOBA_999999>followed by a new person never hands out one token twice. Contributed by Sonike
in #24. - A spaCy model named as pip writes it, such as
pl-core-news-md, loads, and a model directory
withoutmeta.jsonandconfig.cfgis reported when the guards are built, without a download
command that could not work. An installed model still wins over a file or folder of the same
name in the working directory. Contributed by Sonike in
#27. - A password named by a word or two before its colon, "Hasło administratora: Zima2025!Lod",
"hasło użytkownika admin:", "Hasło do panelu klienta:", is masked: only "do", "dla" or "od" and
one word could stand there, so the value went out in clear. With words in between, the value
must look like a password, so "Hasło administratora zostało zmienione." and a date after "Hasło
ważne do:" stay as they are. - A PESEL with a typo right after its label is masked: "PESEL: 15501416181", with a wrong
check digit, and "pesel 78059938421", with day 99, went out in clear, since the checksum was
checked before the label. Without the label the checksum still decides, so an order number of
eleven digits stays as it is. - A feminine surname in another case is masked after the full name: once "Natalia Bielska"
was masked, "Bielskiej" and "Bielską" went out in clear, in the same text and in the next turn,
since no case of a surname in -ska, -cka or -dzka but the nominative begins with it. In
lowercase, "w bielskiej firmie" stays an adjective. - A street without "ul." and without a postcode is masked after an address label, "Adres:
Japczaka 17", "zam. Długa 5", "mieszka przy Długiej 5", and after "z", "na", "przy" or "do"
with a town right after the number, "z Długiej 17 w Białymstoku". Both went out in clear.
"Windows 11 w Polsce", "na Windows 11 w firmie" and "Excel 365 w biurze" stay as they are. - An address is found in linear time in a long run of capitalised words: each word started a
street that ran to the end of the run, so 5,000 words such as "Ulica Ulica ..." or "I I ..."
took several seconds, and ten times as many a hundred times longer. A street's name now takes
at most eight words and three titles, more than any Polish street needs. - ZUS, Płatnik and the institutions staff name every day are on
WELL_KNOWN_NAMES: with the
allowlist, "Jak to rozliczyć w ZUS?" and "zna program Płatnik" went out as<FIRMA_1>, since
the language model reads them as organisations. Added: ZUS, PUE ZUS, eZUS, Płatnik, KRUS, NFZ,
GUS, KSeF, ePUAP, CEIDG and PFRON. The list releases names only, so no identifier is unmasked,
and "pani Płatnik" stays a person.