Skip to content

v0.3.0

Choose a tag to compare

@github-actions github-actions released this 10 Nov 23:10
· 106 commits to main since this release

Added

  • MaskedTurn, from anonymizer.turn(): one object for everything a call to a model sends and
    gets back. It masks the history, the question and the results of tools with one mapping, so a
    value keeps one token across them, restores the arguments of tool calls and the answer, also
    while it streams, and reports invented tokens. Values the caller already knows are sensitive,
    such as its clients' names, go in known= and are masked wherever they appear. When a detector
    fails, every mask method raises MaskingUnavailableError and returns nothing half masked.
    PolCRM and PolSales each kept a copy of this as a Masker of their own.
  • MaskedTurn.mask_messages() masks a whole history written for the Anthropic or the OpenAI
    API: text, tool_use input, tool_result and "tool" content, the arguments of tool_calls.
    Tool results and arguments written as JSON stay valid JSON, and ids and tool names are left
    alone. A value first recognised in a later message is masked in the earlier ones too. A block
    nothing can mask, such as an image or a model's signed reasoning, is refused, or left out with
    unknown_blocks="drop".
  • token_instructions("pl" | "en"): a note for the system prompt that tells the model what the
    tokens are and to copy them exactly, the one both apps wrote out for themselves.
  • MaskingUnavailableError, the one error an app catches to refuse sending text it could not
    mask, and NerModelUnavailableError, the one of them raised when spaCy or its model is
    missing. ner_available() runs the same check as default_guards() without loading
    anything, for a settings page.

Changed

  • default_guards(with_ner=True) without spaCy or its model raises NerModelUnavailableError
    instead of a plain ModuleNotFoundError. It is a subclass of ModuleNotFoundError with the same
    message, so except ModuleNotFoundError and except ImportError still catch it. Through
    MaskingUnavailableError it is also a RuntimeError, so an except RuntimeError around
    default_guards() now catches it too, and a check of the exact type, such as
    type(exc) is ModuleNotFoundError, no longer matches. Nothing else in the 0.2.2 API changed:
    every name and signature is still there, and every benchmark document masks to the same text,
    tokens and spans, with the language model and without it.