Skip to content

0.4.2

Choose a tag to compare

View all tags
@SergeyShk SergeyShk released this 26 Oct 00:09
· 21 commits to master since this release
0.4.2
846eff8
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

A patch of anyTS with the fixes found by the quality check of ruTS 0.19 on Russian corpora: the definitions of the keys added in 0.4, the check of the parts of speech, and caveats in the cards of the docs. No library has been released on the keys of 0.4 yet, so their definitions change before one is.

Requirements

The same as anyTS 0.4.0: Python 3.11 or newer, spaCy 3.8.15 or newer within 3, preshed 3.0.11 or newer except 3.0.14.

pip install anyts==0.4.2

Changes of numbers

  • Verb phrases of L2SCA. A participle or another non-finite form under amod or acl, or coordinated to one, without an auxiliary, a copula or a subject of its own is an attribute of its noun, not a verb phrase (the house built by workers), as in L2SCA. On the texts of ruTS by school grade, such participles made all the growth of VP/T with the grade.
  • Clauses of L2SCA. C/T, CP/C and CN/C divide by the new n_l2sca_clauses: the clauses without the predicates sharing a subject (took the bucket and went) and without the infinitive clauses, as L2SCA counts finite clauses with a subject. C/S, MLC and the other metrics keep their clauses.
  • Infinitive clauses. An infinitive clause head under csubj, advcl or ccomp (it is hard to say, he came to help) is an infinitive clause, as one under xcomp, not a subordinate one: subordinate_clauses_per_sent, p_complex_sents and p_complex_t_units fall and infinitive_clauses_per_sent grows on the texts with such infinitives.
  • Infinitive under a conjunct of a periphrasis. In ready to come and help you live the clause of live is kept: the walk from it asks is_infinitive_periphrasis along the chain of infinitive conjuncts.
  • Prepositional chains. A nominal predicate with a subject of its own and no copula (the present tense of Russian) starts a chain, like one with a copula.
  • Paragraphs are lines, as in TAACO. They were the parts between blank lines, so the corpora with a paragraph on every line gave nan for every *_para. A paragraph without content words, such as the number of a chapter, joins the next one, the last such the one before it.
  • In the snapshot of ruTS, 78 of 4,890 results change, by the rules above alone.

Features

  • subordinating_marks of SyntaxStats, and the parameter of is_subordinate_clause_head and is_infinitive_clause_head: the subordinating conjunctions that make a clause under conj or parataxis a subordinate one (an infinitive one with an infinitive), since a model may attach a clause of because as a conjunct. Such a clause is no T-unit, nor is a clause under it, and it starts a coordination of its own. Empty by default.
  • blank_lines of CohesionStats, find_paragraph_starts and find_unit_paragraph_starts: the paragraphs start after the blank lines alone, for a text whose lines are wrapped inside its paragraphs. LINE_BREAK in anyts.utils is the pattern of the line breaks. split_paragraphs (the parts of dispersion) is unchanged.
  • has_pos(doc) in anyts.utils: whether the words of a Doc carry parts of speech.

Fixes

  • Parts of speech. The attribute ruler of a pipeline without its morphologizer still tags whitespace SPACE, so has_annotation("POS") passed any text with a line break, and SyntaxStats, CohesionStats, MorphStats and the highlighting went on without parts of speech. They check by has_pos; no_pos asks not to disable the tagger or the morphologizer.
  • get_stats raises ParameterError for a flat that is not a bool (get_stats("n_words") returned the flattened statistics), and MorphStats.get_stats for a filter_none or a flat that is not one.
  • Docs. Caveats on Orlov's Z (unstable on texts shorter than about ten thousand words), burstiness on short texts, the bias of the entropies of word lengths, Dickes-Steiwer and the length of a text, Strain as 0.3 times Fucks, and the similarity of sentences, which sees no order of words.

Incompatible changes

  • find_paragraph_starts, find_unit_paragraph_starts and CohesionStats start a paragraph after a line break by default; blank_lines=True keeps the blank lines.
  • CohesionStats.n_paragraphs is a property and counts the paragraphs with content words.
  • A SyntaxStats pickled by 0.4.1 lacks the infinitive clauses kept for L2SCA, and its get_stats raises AttributeError: it needs to be computed again.
  • infinitive_clauses_per_sent is described as "Infinitive clauses per sentence".

Links

Assets 2
Loading