0.4.5

A patch of anyTS for esTS 0.9, from its quality check: the cohesion component takes blank_lines, and CohesionStats keeps less memory.

Requirements

The same as anyTS 0.4.0: Python 3.11 or newer, spaCy 3.8.15 or newer within 3, preshed 3.0.11 or newer except 3.0.14.

pip install anyts==0.4.5

Fixes

• blank_lines of the cohesion component. Since 0.4.2 a paragraph of CohesionStats is a line by default, and blank_lines=True takes the parts between blank lines, which a text wrapped inside its paragraphs needs. CohesionStatsComponent(nlp, name, blank_lines=False) passes it to its class now, so a pipeline can do the same: nlp.add_pipe("<library>_cohesion", config={"blank_lines": True}).
• Memory of CohesionStats. The overlaps with the next two sentences and with the next paragraph were computed on first use, from sets the object kept for every sentence and, in a text wrapped inside its paragraphs, for nearly every word. They are computed when the object is created now, and nothing else is kept: an esTS CohesionStats of a parsed novel of 44.6 thousand tokens holds about 5 MB instead of 12.8 MB, its pickle 575 KB instead of 1375 KB, in the same time. No number changes.

Incompatible changes

• A CohesionStats pickled by an earlier version of 0.4 has to be computed again: pickled before get_stats, it lacks the overlaps with the next sentences and paragraphs; one of 0.4.2-0.4.4 lacks n_paragraphs as well, and carries the sets of the old version.

Links

• Documentation: https://sergeyshk.github.io/anyTS/
• PyPI: https://pypi.org/project/anyts/

0.4.4

A patch of anyTS for esTS 0.9: the morphological statistics of a parsed Doc read every analysis once again, and two cards of the docs say what their measures do.

Requirements

The same as anyTS 0.4.0: Python 3.11 or newer, spaCy 3.8.15 or newer within 3, preshed 3.0.11 or newer except 3.0.14.

pip install anyts==0.4.4

Fixes

• Speed of MorphStats on a Doc. The default analyze gave every token a dictionary of its own, so the cache of the object, which checks and reads a mapping of features once for all the words that share it, never hit. The words of one analysis share one dictionary now: an esTS MorphStats of a parsed Doc of 44.6 thousand tokens takes 0.065 s instead of 0.16 s. A library whose own analyze gives the words of one form one mapping gets the same, as the table of the hooks says.
• Docs. The card of Wheeler-Smith: ten times the number of the words of two syllables or more per sentence. The similarity of sentences: the edit distance reads only the order of the parts of speech (an object-first and a subject-first sentence are alike when that order is the same), the dependency triples read no order of the words at all, and the order of the subject and the object is read by p_preverbal_subjects and p_preverbal_objects.

No number changes.

Links

• Documentation: https://sergeyshk.github.io/anyTS/
• PyPI: https://pypi.org/project/anyts/

0.4.3

A patch of anyTS with the fixes found by the review of ruTS 0.19 on Russian corpora: the rule of subordinating marks from 0.4.2 gets a hook for the context of a mark, and the kinds of the coordinated clauses follow the words themselves.

Requirements

The same as anyTS 0.4.0: Python 3.11 or newer, spaCy 3.8.15 or newer within 3, preshed 3.0.11 or newer except 3.0.14 (3.0.15 fixes the tokenizer again).

pip install anyts==0.4.3

Features

• SyntaxStats.is_subordinating_mark(token), a hook every dependent mark of a conjunct or a parataxis passes to make the clause subordinate, so a library reads the context of a word that is a conjunction and a pronoun (Russian «что ж», «что называется», «когда ..., то ...»). The default is the new is_subordinating_mark(token, subordinating_marks) of anyts.syntax_stats: the words of the mark with those of its fixed expression in the set. is_subordinate_clause_head and is_infinitive_clause_head take the check as is_subordinating_mark.

Changes of numbers

• Conjuncts of a marked clause. A conjunct with a subject of its own, or an infinitive one, belongs to the marked clause it is coordinated to; one sharing the subject does not, as a model attaches the predicates of the main clause to the marked clause too.
• The form of the word itself. An infinitive clause needs an infinitive of its own: a finite verb attached as a conjunct of an infinitive was an infinitive clause. A finite conjunct with a subject of its own of an infinitive takes the kind and the T-unit of the clause of the first word above the infinitives that is none (he came to help and she stayed: two T-units).
• On a Spanish sample checked by the review, the clause heads without a kind are as many as in 0.4.2.

Fixes

• Docs: the parts of split_paragraphs for dispersion are between blank lines; for a text with a paragraph on every line, its lines are the paragraphs, as in CohesionStats.

Links

• Documentation: https://sergeyshk.github.io/anyTS/
• PyPI: https://pypi.org/project/anyts/

0.4.2

A patch of anyTS with the fixes found by the quality check of ruTS 0.19 on Russian corpora: the definitions of the keys added in 0.4, the check of the parts of speech, and caveats in the cards of the docs. No library has been released on the keys of 0.4 yet, so their definitions change before one is.

Requirements

The same as anyTS 0.4.0: Python 3.11 or newer, spaCy 3.8.15 or newer within 3, preshed 3.0.11 or newer except 3.0.14.

pip install anyts==0.4.2

Changes of numbers

• Verb phrases of L2SCA. A participle or another non-finite form under amod or acl, or coordinated to one, without an auxiliary, a copula or a subject of its own is an attribute of its noun, not a verb phrase (the house built by workers), as in L2SCA. On the texts of ruTS by school grade, such participles made all the growth of VP/T with the grade.
• Clauses of L2SCA. C/T, CP/C and CN/C divide by the new n_l2sca_clauses: the clauses without the predicates sharing a subject (took the bucket and went) and without the infinitive clauses, as L2SCA counts finite clauses with a subject. C/S, MLC and the other metrics keep their clauses.
• Infinitive clauses. An infinitive clause head under csubj, advcl or ccomp (it is hard to say, he came to help) is an infinitive clause, as one under xcomp, not a subordinate one: subordinate_clauses_per_sent, p_complex_sents and p_complex_t_units fall and infinitive_clauses_per_sent grows on the texts with such infinitives.
• Infinitive under a conjunct of a periphrasis. In ready to come and help you live the clause of live is kept: the walk from it asks is_infinitive_periphrasis along the chain of infinitive conjuncts.
• Prepositional chains. A nominal predicate with a subject of its own and no copula (the present tense of Russian) starts a chain, like one with a copula.
• Paragraphs are lines, as in TAACO. They were the parts between blank lines, so the corpora with a paragraph on every line gave nan for every *_para. A paragraph without content words, such as the number of a chapter, joins the next one, the last such the one before it.
• In the snapshot of ruTS, 78 of 4,890 results change, by the rules above alone.

Features

• subordinating_marks of SyntaxStats, and the parameter of is_subordinate_clause_head and is_infinitive_clause_head: the subordinating conjunctions that make a clause under conj or parataxis a subordinate one (an infinitive one with an infinitive), since a model may attach a clause of because as a conjunct. Such a clause is no T-unit, nor is a clause under it, and it starts a coordination of its own. Empty by default.
• blank_lines of CohesionStats, find_paragraph_starts and find_unit_paragraph_starts: the paragraphs start after the blank lines alone, for a text whose lines are wrapped inside its paragraphs. LINE_BREAK in anyts.utils is the pattern of the line breaks. split_paragraphs (the parts of dispersion) is unchanged.
• has_pos(doc) in anyts.utils: whether the words of a Doc carry parts of speech.

Fixes

• Parts of speech. The attribute ruler of a pipeline without its morphologizer still tags whitespace SPACE, so has_annotation("POS") passed any text with a line break, and SyntaxStats, CohesionStats, MorphStats and the highlighting went on without parts of speech. They check by has_pos; no_pos asks not to disable the tagger or the morphologizer.
• get_stats raises ParameterError for a flat that is not a bool (get_stats("n_words") returned the flattened statistics), and MorphStats.get_stats for a filter_none or a flat that is not one.
• Docs. Caveats on Orlov's Z (unstable on texts shorter than about ten thousand words), burstiness on short texts, the bias of the entropies of word lengths, Dickes-Steiwer and the length of a text, Strain as 0.3 times Fucks, and the similarity of sentences, which sees no order of words.

Incompatible changes

• find_paragraph_starts, find_unit_paragraph_starts and CohesionStats start a paragraph after a line break by default; blank_lines=True keeps the blank lines.
• CohesionStats.n_paragraphs is a property and counts the paragraphs with content words.
• A SyntaxStats pickled by 0.4.1 lacks the infinitive clauses kept for L2SCA, and its get_stats raises AttributeError: it needs to be computed again.
• infinitive_clauses_per_sent is described as "Infinitive clauses per sentence".

Links

• Documentation: https://sergeyshk.github.io/anyTS/
• PyPI: https://pypi.org/project/anyts/

0.4.1

A patch of anyTS with the fixes found while moving ruTS onto anyTS 0.4, and one helper both libraries need for the syntax features of their comparison of corpora.

Requirements

The same as anyTS 0.4.0: Python 3.11 or newer, spaCy 3.8.15 or newer within 3, preshed 3.0.11 or newer except 3.0.14.

pip install anyts==0.4.1

Changes of numbers

• Verb phrases of L2SCA. An infinitive that makes one predicate with its head (SyntaxStats.is_infinitive_periphrasis), asked of the infinitive and of every infinitive conjunct above it, is part of the verb phrase of its head, as it is part of its clause: a modal or phasal verb with an infinitive gives one verb phrase, as English can leave does. verb_phrases_per_t_unit of ruTS falls by 1-16 % on 11 of its 36 parsed snapshot texts; nothing else of the snapshot changes.
• Words without syllables in the formulas of other languages. A word without syllables (a number, a preposition of one consonant) counts as monosyllabic in FORCAST and the Wiener Sachtextformeln and as easy in Linsear Write, as in textstat. The functions calc_* take the same counts as before; UnadaptedReadabilityStats passes the new ones.

Features

• set_sent_starts(doc, starts) in anyts.utils: sets the sentences of a Doc by the character offsets of their starts, those of the segmenter of a library, so that the parser, senter or sentencizer keeps them; the whitespace stays with the sentence before it.

Fixes

• keyness names why a corpus gives no frequencies: no words a FrequencyReference keeps, texts shorter than n words, or no words.
• MorphStats checks and reads a mapping of features shared by the words of one form once: a library that shares them builds the statistics faster (ruTS: 0.288 → 0.265 s on 39,000 words).
• Docs: the rule of verb phrases in the card of L2SCA and in the hooks of SyntaxStats, the rule of words without syllables for the formulas of other languages, the statistics (not counts) of BasicStats.print_stats, the tags of MorphStats without a visible backslash.

Links

• Documentation: https://sergeyshk.github.io/anyTS/
• PyPI: https://pypi.org/project/anyts/

0.4.0

The fourth release of anyTS, the language-independent core of the text statistics libraries ruTS (Russian) and esTS (Spanish).

New in this release:

• morphological statistics and readability formulas with their original coefficients;
• new measures of syntax, cohesion and lexical diversity;
• keywords by n-grams, rolling Delta, Zeta segments and three stylometric plots.

Requirements

• Python 3.11 or newer.
• spaCy 3.8.15 or newer within 3.
• preshed 3.0.11 or newer, except 3.0.14 (new). The wheels of 3.0.14 break the tokenizer of spaCy (explosion/spaCy#14058).
• numpy 1.26, pandas 2.1, scipy 1.11, matplotlib 3.8 and graphviz 0.20.3 or newer. No trained spaCy model is needed.
• Rendering the word tree and the network of collocations also needs the executables of Graphviz.

pip install anyts

ruTS 0.18 and esTS 0.8 require anyTS 0.3 (anyts<0.4).

Incompatible changes

• Messages (74 → 90 keys).
  • hook_result is removed, and check_messages refuses a dictionary that has it. The result of each hook has a key of its own instead: hook_doc, hook_lexical_words, hook_cohesion_words, hook_syllables, hook_sounds, hook_highlights and hook_morph_words.
  • New keys:
    • source_span;
    • connector_breaks, connector_breaks_short, connector_blocked, connector_blocked_empty and connector_type;
    • morph_features, morph_markers, morph_stats_desc and morph_values_desc.
  • source_type names the type, "The data source cannot be of type {type}". A Span gets source_span, "The data source cannot be a Span: pass span.as_doc()". anyts.utils.source_type_error gives either error for a class to raise.
  • layer_annotations lists the annotations after a colon, and the default HighlightedText.annotation_names are bare nouns ("parts of speech" instead of "the parts of speech").
  • The errors of ConnectorMatcher.find and is_subordinator take the messages of the class.
• New keys of the statistics.
  • BASIC_STATS_DESC gets 2 keys, DIVERSITY_STATS_DESC 2, COHESION_STATS_DESC 6 and SYNTAX_STATS_DESC 14.
  • A subclass with the stats_desc of the core gets them in get_stats, print_stats and to_dataframe, and the widths of print_stats change.
  • The units of L2SCA, the prepositional chains and the similarity of sentences, and the overlaps with the next two sentences and of the paragraphs, are computed on first use. A subclass without these keys costs nearly as before.
• CohesionWord has two more fields at the end, connector_blocked and paragraph_start. Unpacking it by position breaks.
• Behaviour.
  • SyntaxStats raises SourceError (no_pos) for a Doc with a parse but without parts of speech, as CohesionStats and MorphStats do.
  • PhonStats.mean_syllable_len counts the sounds of the classes of a syllable, by its CV pattern.
  • Quotes and the apostrophe are no longer marks between words in find_marks, find_unit_marks and find_connector_highlights.
  • BLANK_LINE takes every line break of Unicode: \r\n, \r, U+0085 and U+2028 besides \n, and U+2029 alone. The zero-width space and the byte order mark count as whitespace on a blank line. This bounds the sentences of SentsExtractor, start_sents_after_blank_lines, kwic and the paragraphs. The pattern moves to anyts.utils, and anyts.extractors still exports it.
  • positive of keyness must be a bool.
  • Equal scores of keyness and collocations are ordered by frequency, the scores compared to 12 significant digits.
  • In compare_features, every feature draws its bootstrap from a generator of its own. The intervals of a feature no longer depend on the other columns, and they change once, for the same seed.
  • Juilland's D of dispersion and calc_juilland_d is cut at 0.
  • A SyntaxStats or CohesionStats of the core pickled by 0.3.9 loads, but its get_stats, print_stats and to_dataframe raise AttributeError: it lacks the new keys. Such objects need to be computed again. A subclass that names only the keys of 0.3.9 is not affected, nor are the other classes.

Features

• Morphology: MorphStats with the analyze hook and MorphWord. It counts the parts of speech and the features of Universal Dependencies, word by word or by value, and gives the markers of a language library, explain_text and print_markers. The component is MorphStatsComponent.

• Unadapted readability: UnadaptedReadabilityStats and eleven formulas with their original coefficients:

  • FORCAST, and Linsear Write in the form of textstat;
  • Powers-Sumner-Kearl and McAlpine EFLAW;
  • the Wiener Sachtextformel in its four variants, and Dickes-Steiwer;
  • Gulpease, Fucks, Strain, ELF and Wheeler-Smith.

  The component is UnadaptedReadabilityStatsComponent.

• Syntax: 14 keys:

  • the words before the root;
  • the T-units and phrases of L2SCA: T/S, MLT, C/T, CT/T, VP/T, CP/C and CN/C;
  • the prepositional chains, with the is_prep_complement hook;
  • the share of nominal subjects and objects before the verb;
  • the similarity of adjacent sentences, by the edit distance of their parts of speech and by their dependency triples.

  Also calc_edit_distance, calc_multiset_similarity and dependency_triples.

• Cohesion: the overlaps of nouns, arguments and content words with the next two sentences and between paragraphs, in the manner of TAACO, by calc_forward_overlap. Paragraphs start after a blank line, by CohesionWord.paragraph_start, and the class gives n_paragraphs.

• Diversity: Orlov's Z (calc_orlov_z), burstiness (calc_burstiness, Kim and Jo 2016), and calc_mamtld(step=), faster by the step. BasicStats adds letters_entropy and syllables_entropy.

• Corpus:

  • keyness(n=) gives keywords by n-grams of a list of texts;
  • collocations(side=) gives directed windows, and a node no longer slows it down;
  • rolling_delta with statistics=, and zeta_segments;
  • split_paragraphs in anyts.extractors: the numbers of words of its paragraphs give the sizes of the parts of dispersion.

• Plots: rolling_delta_plot, zeta_scatter_plot and delta_histogram_plot.

• Helpers:

  • source_type_error, find_paragraph_starts and find_unit_paragraph_starts in anyts.utils;
  • ConnectorMatcher.find(blocked=), CohesionWord.connector_blocked and find_connector_highlights(blocked=): the positions where no connector starts, by a rule of a language that reads more than the words and their parts of speech, such as the parse.

For the libraries

• New sections of the docs to include (50).

  • components.md: MorphStatsComponent, UnadaptedReadabilityStatsComponent.
  • corpus/stylometry.md: rolling_delta, zeta_segments.
  • extractors/sentences.md: split_paragraphs.
  • stats/cohesion_stats.md: CohesionStats-forward_overlap, calc_forward_overlap.
  • stats/diversity_stats.md: calc_orlov_z, calc_burstiness.
  • stats/morph_stats.md: MorphStats, MorphStats-attributes, MorphStats-parameters, MorphStats-hooks, MorphStats-metrics, MorphStats-get_stats, MorphStats-print_stats, MorphStats-markers, MorphStats-explain_text, MorphWord.
  • stats/syntax_stats.md: SyntaxStats-words_before_root, SyntaxStats-l2sca, SyntaxStats-prep_chains, SyntaxStats-word_order, SyntaxStats-similarity, is_prep_complement, calc_edit_distance, calc_multiset_similarity, dependency_triples.
  • stats/unadapted_readability_stats.md: UnadaptedReadabilityStats, UnadaptedReadabilityStats-attributes, UnadaptedReadabilityStats-parameters, UnadaptedReadabilityStats-hooks, UnadaptedReadabilityStats-metrics, and the eleven calc_* of the formulas.
  • utils/messages.md: source_type_error.
  • utils/spacy.md: find_unit_paragraph_starts.
  • utils/words.md: find_paragraph_starts.
  • visualizers/stylometry.md: rolling_delta_plot, zeta_scatter_plot, delta_histogram_plot.

• Anchors the tables of metrics link to. The sections DiversityStats-metrics, CohesionStats-metrics and SyntaxStats-metrics link to the headings of new cards, which stand above their sections, outside them. Without these anchors, mkdocs build --strict fails. Above each of these sections, a library writes a heading with the anchor:

  • #forward_overlap above CohesionStats-forward_overlap;
  • #orlov_z above calc_orlov_z, and #burstiness above calc_burstiness;
  • #words_before_root, #l2sca, #prep_chains, #word_order and #similarity above SyntaxStats-words_before_root, -l2sca, -prep_chains, -word_order and -similarity.

  In the same way, UnadaptedReadabilityStats-metrics links to the cards of its eleven formulas: each anchor is the name of the calc_* section below it without calc_ (#forcast above calc_forcast).

• Paragraphs. An analyze of CohesionStats that builds its own words sets paragraph_start by find_paragraph_starts or find_unit_paragraph_starts. Without it, the keys *_para are nan.

• Corpus. keyness of several texts takes the list of texts, not their words joined. A mapping of the frequencies of n-grams goes with n=1.

Quality

• Tests: 2,390 tests with 100% line and branch coverage. Two tests of the docs check every parameter table against its signature and run the examples of 39 pages against their results.
• CI: Python 3.11-3.14 on Linux, Windows and macOS, a job on the lowest versions of the dependencies, and a job for the examples of the docs.
• Numbers of the libraries: through the classes of esTS 0.8 and ruTS 0.18, the keys of 0.3.9 give the same numbers bitwise. The exception is mean_syllable_len of ruTS, 2.369 → 2.333 on average over its texts. The snapshots of the li...

0.3.9

A patch release of anyTS, the language-independent core of the text statistics libraries ruTS (Russian) and esTS (Spanish).

Requirements

• Python 3.11 or newer.
• spaCy 3.8.15 or newer within 3 (new upper bound).
• numpy 1.26, pandas 2.1, scipy 1.11, matplotlib 3.8 and graphviz 0.20.3 or newer.

pip install anyts

Added

• anyts.utils.check_bool(value, what). It checks a switch, and a numpy bool_ passes too. Now these switches raise ParameterError for a value that is not a bool:
  • normalize of BasicStats;
  • filter_punct, filter_nums, use_lexemes and lowercase of WordsExtractor;
  • lowercase and within_words of CharNgramsExtractor;
  • by_lemma and ignore_case of kwic.
• anyts.utils.is_token(value). It checks whether a value is a spaCy Token without importing spaCy, as is_doc, is_span and is_language do.

Behaviour changes

• SyntaxStats keeps no Doc of the user.
  • sents holds the tokens of a copy of the Doc without its tensor and user data, and a pickle stores the compressed bytes of that copy.
  • On 30k characters, a pickle goes from 885 KB to 141 KB.
  • A component no longer makes a reference cycle through the Doc.
  • A pickle of 0.3.8 does not load on 0.3.9.
• Clauses.
  • Some models attach the infinitive of a periphrasis as conj or csubj (Tiene que intentar salir). The walk now goes up through such an infinitive, so the infinitive under it keeps its clause.
  • is_predicate accepts a word with a copula (cop). In a language that drops the subject, a coordinated nominal predicate (y estaba cansado) now heads its clause.
  • On the texts of esTS, clause counts only went up.
• Connectors.
  • The rules of ConnectorMatcher find a connector by its normalized form, however the dictionary or the list writes it. Connector.text stays as the dictionary writes it.
  • find(breaks=) rejects a position that is not that of a word after the first, with ParameterError.
  • is_subordinator raises SourceTypeError for a value that is not a Connector.
• Token predicates. These raise SourceTypeError for a value that is not a Token: predicate_form, is_predicate, is_clause_head, is_infinitive_clause, is_subordinate_clause_head, is_infinitive_clause_head, is_passive, has_agent and is_agentless.
• describe gives None for any key of a dictionary statistic (c_lengths_4, pos_ADJ), not only for the keys of this object.
• MESSAGES is read-only (a MappingProxyType).
  • An assignment no longer changes the messages of every class.
  • A library builds its own mapping, MappingProxyType({**MESSAGES, **own}).
• bootstrap_median_diff takes labels of one kind only, strings or numbers.

Quality

• 2170 tests pass with 100% line and branch coverage.
• CI runs on Python 3.11–3.14 on Linux, Windows and macOS, plus a job with the lowest versions of the dependencies.

Links

• Documentation: https://sergeyshk.github.io/anyTS/
• PyPI: https://pypi.org/project/anyts/
• Changes: 0.3.8...0.3.9

0.3.8

A patch release of anyTS, the language-independent core of the text statistics libraries ruTS (Russian) and esTS (Spanish).

Requirements

Python 3.11 or newer, numpy 1.26, pandas 2.1, scipy 1.11, spaCy 3.8.15, matplotlib 3.8 and graphviz 0.20.3 or newer.

pip install anyts

Added

• ConnectorMatcher.parcellation. This attribute lists the connectors, by their text, that are never subordinators whatever their part of speech: a sentence they open is linked to the one before. is_subordinator reads it, so a library no longer overrides the method for this.
• anyts.utils.is_language(value). It checks whether a value is a Language pipeline of spaCy without importing spaCy, like is_doc and is_span.

Behaviour changes

• Flat keys. describe gives None for every key of get_stats(flat=True), including the keys of a class that counts per-word values in its own get_stats (describe("pos_NOUN")). The statistics are computed only for a name that begins with the name of a statistic, so a typo still raises UnknownStatError at once.
• Narrowed presets. A subclass of ReadabilityStats may narrow presets and keep the scales by preset it inherits from a readability class. The scales a class sets itself or takes from a mixin are still checked when the class is defined.

Quality

2135 tests pass with 100% line and branch coverage. CI runs on Python 3.11–3.14, on Linux, Windows and macOS, and with the lowest versions of the dependencies.

Links

• Documentation: https://sergeyshk.github.io/anyTS/
• PyPI: https://pypi.org/project/anyts/
• Changes: 0.3.7...0.3.8

0.3.7

A patch release of anyTS, the language-independent core of the text statistics libraries ruTS (Russian) and esTS (Spanish).

Requirements

Python 3.11 or newer, numpy 1.26, pandas 2.1, scipy 1.11, spaCy 3.8.15, matplotlib 3.8 and graphviz 0.20.3 or newer.

pip install anyts

Added

• The form of an auxiliary in the clause checks. predicate_form reads an auxiliary that the model leaves without VerbForm through the language hook auxiliary_form. Until now the clause checks called it without the hook. As a result, a predicate whose finite auxiliary has no form read as a participle and headed no clause, as in the Spanish Sé que os habéis posado.
  • is_clause_head, is_infinitive_clause, is_subordinate_clause_head and is_infinitive_clause_head take auxiliary_form.
  • SyntaxStats has a hook of the same name. By default it returns an empty string, which skips the auxiliary as before.

Behaviour changes

• In is_infinitive_clause, is_subordinate_clause_head and is_infinitive_clause_head, the default of is_clause_head is now None. None means is_clause_head of the module with the hooks of the call. Before, a call with is_infinitive_periphrasis or auxiliary_form and no is_clause_head read the word it completes without those hooks.
• In split_windows, symmetric marks after a closing mark close the previous window only when whitespace follows them. A dash glued to the next word now goes with the line of dialogue it opens: cuatro.—¿Cinco? gives cuatro. and —¿Cinco?. A run of such marks moves as a whole, and marks glued to the last word close its window as before.
• The messages no_parse, no_lemmas and no_pos say how to get the annotation, for example "The data source has no parts of speech: parse the text with a model instead of a blank pipeline". The keys do not change.
• predicate_form with a hook that is not callable raises "The verb form of an auxiliary must be callable", like the clause checks.

Documentation

The cards of the clause checks and the hooks of SyntaxStats describe auxiliary_form. The section of split_windows gives the new rule for the marks, and kwic-parameters names the error of a window that is not an integer.

Quality

2128 tests pass with 100% line and branch coverage. CI runs on Python 3.11–3.14, on Linux, Windows and macOS, and with the lowest versions of the dependencies.

Links

• Documentation: https://sergeyshk.github.io/anyTS/
• PyPI: https://pypi.org/project/anyts/
• Changes: 0.3.6...0.3.7

0.3.6

A patch release of anyTS, the language-independent core of the text statistics libraries ruTS (Russian) and esTS (Spanish). It changes the documentation only; the package code is that of 0.3.5.

Requirements

Python 3.11 or newer, numpy 1.26, pandas 2.1, scipy 1.11, spaCy 3.8.15, matplotlib 3.8 and graphviz 0.20.3 or newer.

pip install anyts

Documentation

• Signatures and errors. Every page opens with the call signature of its class or function and ends the description with a Raises: list. A test compares every signature on the site with the code.
• Metric cards. A class of metrics gives an overview of its metrics: what each measures, its range and what a higher value means. A card per metric follows, with the signature of its function, its formula, its parameters and its errors. The families of measures of keyness, collocations and dispersion have cards too.
• One page per class. The pages of the metric functions of diversity, readability, cohesion, syntax, phonetics and style merge into their module pages.
• Examples. Examples sit in Code and Result tabs, and the results are real outputs.
• Hooks. The language hooks, which only a library needs, are folded at the end of a page.
• Theme. The site has a dark scheme, copy buttons on code, left-aligned tables and a home page of cards.

For language libraries

The libraries include the named sections of these pages. Their names stay, but the sections of the six pages of metric functions moved to the module pages, so includes such as stats/diversity_stats_funcs.md:calc_ttr become stats/diversity_stats.md:calc_ttr. The text of most sections changed, so their translations need an update. The classes of the template are in docs/css/extra.css.

Links

• Documentation: https://sergeyshk.github.io/anyTS/
• PyPI: https://pypi.org/project/anyts/