Skip to content

lexindex v4.2.0

Choose a tag to compare

@github-actions github-actions released this 23 Sep 11:20
· 171 commits to main since this release
v4.2.0
8632e39

Added

  • StringIndex.occurrences(text): every key that occurs in a text, in one call. A dictionary
    segmenter — jieba and its ports among them — asks for the keys that start at each character of a
    sentence, and from Python that was a common_prefix call per character, each one a crossing of
    the binding for a walk that is over within a few characters. occurrences runs the same walk at
    every character on the Rust side and returns (start, end, id) triples, with text[start:end]
    the key: starts ascend, and within a start the shortest key comes first — the order jieba's word
    graph is built in. Offsets count characters, as a str is indexed, not UTF-8 bytes. The empty
    key, which would occur at every position, is not reported.
  • StringIndex::for_each_common_prefix, the walk under common_prefix and longest_prefix,
    is public: f(end, id) for every key that is a prefix of the query, shortest first, with the
    key &query[..end] and nothing allocated. The walk stops where no key continues, so the query
    can be the whole rest of a text.

Fixed

  • lexindex.inspect's docstring opened with somebody else's paragraph, a note on the
    extension module's gil_used = false that has sat above inspect since inspect was added in
    2.0.0 — so help(lexindex.inspect) began with it, and so did inspect's entry on the API page,
    which is rendered from the same docstring. The note is back beside the #[pymodule] it explains,
    as a plain comment that cannot become a docstring again.
  • The API page carried 73 links to nowhere and 26 Sphinx roles printed as typed. The binding's
    doc comments in src/python.rs are Python docstrings — help() and the API page both read them
    — but they were written as rustdoc: [id](Self::id) is a dead link in Markdown, and
    [PyStringIndex::to_bytes] names a Rust type no Python caller meets. Those are code spans now,
    and DictIndexIterator's docstring, which was a note on its lock, describes the iterator. The
    stub and lexindex/__init__.py used Sphinx roles (:meth:, :class:, 105 of them), which
    Markdown shows verbatim; they are double-backtick literals now, as the rest of those docstrings
    already were. A test holds every docstring the runtime or the stub carries to that.
  • docs.yml: its release: published trigger never fired, in 24 releases — release.yml
    creates the GitHub Release with GITHUB_TOKEN, whose events start no workflow — and its push
    filter missed two files the site is built from: polars/README.md, which is the Polars page, and
    src/python.rs, whose doc comments are the docstrings the API page renders for every compiled
    class and function. The trigger is gone and the filter names both, so a change to either
    redeploys the site on the push that carries it.