Skip to content

0.3.9

Choose a tag to compare

View all tags
@SergeyShk SergeyShk released this 23 Oct 19:23
· 90 commits to master since this release
0.3.9
6b4436d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

A patch release of anyTS, the language-independent core of the text statistics libraries ruTS (Russian) and esTS (Spanish).

Requirements

  • Python 3.11 or newer.
  • spaCy 3.8.15 or newer within 3 (new upper bound).
  • numpy 1.26, pandas 2.1, scipy 1.11, matplotlib 3.8 and graphviz 0.20.3 or newer.
pip install anyts

Added

  • anyts.utils.check_bool(value, what). It checks a switch, and a numpy bool_ passes too. Now these switches raise ParameterError for a value that is not a bool:
    • normalize of BasicStats;
    • filter_punct, filter_nums, use_lexemes and lowercase of WordsExtractor;
    • lowercase and within_words of CharNgramsExtractor;
    • by_lemma and ignore_case of kwic.
  • anyts.utils.is_token(value). It checks whether a value is a spaCy Token without importing spaCy, as is_doc, is_span and is_language do.

Behaviour changes

  • SyntaxStats keeps no Doc of the user.
    • sents holds the tokens of a copy of the Doc without its tensor and user data, and a pickle stores the compressed bytes of that copy.
    • On 30k characters, a pickle goes from 885 KB to 141 KB.
    • A component no longer makes a reference cycle through the Doc.
    • A pickle of 0.3.8 does not load on 0.3.9.
  • Clauses.
    • Some models attach the infinitive of a periphrasis as conj or csubj (Tiene que intentar salir). The walk now goes up through such an infinitive, so the infinitive under it keeps its clause.
    • is_predicate accepts a word with a copula (cop). In a language that drops the subject, a coordinated nominal predicate (y estaba cansado) now heads its clause.
    • On the texts of esTS, clause counts only went up.
  • Connectors.
    • The rules of ConnectorMatcher find a connector by its normalized form, however the dictionary or the list writes it. Connector.text stays as the dictionary writes it.
    • find(breaks=) rejects a position that is not that of a word after the first, with ParameterError.
    • is_subordinator raises SourceTypeError for a value that is not a Connector.
  • Token predicates. These raise SourceTypeError for a value that is not a Token: predicate_form, is_predicate, is_clause_head, is_infinitive_clause, is_subordinate_clause_head, is_infinitive_clause_head, is_passive, has_agent and is_agentless.
  • describe gives None for any key of a dictionary statistic (c_lengths_4, pos_ADJ), not only for the keys of this object.
  • MESSAGES is read-only (a MappingProxyType).
    • An assignment no longer changes the messages of every class.
    • A library builds its own mapping, MappingProxyType({**MESSAGES, **own}).
  • bootstrap_median_diff takes labels of one kind only, strings or numbers.

Quality

  • 2170 tests pass with 100% line and branch coverage.
  • CI runs on Python 3.11–3.14 on Linux, Windows and macOS, plus a job with the lowest versions of the dependencies.

Links

Assets 2
Loading