Skip to content

0.4.0

Choose a tag to compare

View all tags
@SergeyShk SergeyShk released this 26 Sep 21:06
· 195 commits to master since this release
0.4.0
1772c94
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

The fourth release of esTS adds style and sound: style metrics, phonostatistics, the scansion and rhyme of Spanish verse, highlighting of a text, and a dataset of Spanish sonnets.

Requires Python 3.11 or newer. Install with pip install pyests and import with import ests. The new statistics need no spaCy model. The one exception is the verbal nouns of the style metrics, which, like the syntactic layers of the highlighting, need a parsed text.

Incompatible changes

The API of 0.3 is unchanged. Two changes affect the numbers:

  • Syllabification of the diaeresis.
    • A ü outside gü and qü, and any ï, now mark a hiatus: sü-a-ve, crü-el, glo-rï-o-sa.
    • ï no longer takes the stress.
    • HIATUS_VOWELS and ACCENTED_VOWELS change to match.
    • count_syllables, word_stress and every measure built on them move for such words: the readability formulas, n_complex_words, the syllables of verse. In the 150 texts of the corpus this touches 131 occurrences of 72 word forms.
  • Negative zero. entropy, heaps_beta and cv_entropy of a degenerate text (a single lexeme, a single CV pattern) return 0.0 instead of -0.0.

No new dependencies.

Style metrics

  • StyleStats computes nine metrics.
    • The SEO indicators, as in the Advego and Text.ru services: classic and academic nausea, water, spam and Zipf naturalness, plus the keyword_density method.
    • The markers of the officialese style that Spanish plain-language guides warn about: verbal nouns, compound prepositions, parenthetical expressions and clichés. The lists behind them hold 42 compound prepositions, 55 parenthetical expressions and 74 clichés.
    • A cliché that starts with an infinitive also matches the forms of the verb: proceder a finds se procedió a.
  • The component is ests_style.

Phonostatistics

  • PhonStats computes 13 statistics on a rule-based transcription (transcribe, the standard of Spain with yeísmo and distinción):
    • the shares of vowels, sonorants, voiced and voiceless sounds;
    • the consonant-vowel ratio;
    • consonant clusters;
    • hiatus;
    • the entropy of the CV patterns;
    • hardness;
    • the alliteration and assonance indices;
    • open syllables and the mean syllable length.
  • The component is ests_phon.

Verse

  • VerseStats scans a poem in metrical syllables.
    • It counts synalepha and applies the law of the final stress.
    • Each line is fitted to the meter by hiatus, dieresis and synaeresis.
    • Compound verse is read by its hemistichs.
  • From the scansion it gives:
    • the meter;
    • the rhythmic stresses and the types of the endecasílabo;
    • the endings of the lines;
    • the rhyme, consonante and asonante;
    • the rhyme schemes in the Spanish notation (ABBA ABBA CDC DCD, -a-a);
    • 19 strophes, from the pareado to the décima;
    • the form of the poem: soneto, romance, silva, or the strophe shared by all stanzas.
  • seseo=True reads the rhyme with the seseo of Andalusia and America.
  • Accuracy on the sonnets of DISCO:
    • the length of a line agrees with DISCO in 97.0% of the lines and with rantanplan in 97.7%;
    • 99.1% of the rhyming pairs found are pairs of RhymeTagger, and 97.9% of its pairs are found.
  • The helper functions are detect_meter, rhyme_scheme, split_stanzas and accentuate. The component is ests_verse.

Highlighting

  • highlight marks the fragments behind the statistics in 15 layers of five groups:
    • readability: long sentences, complex and rare words;
    • syntax: passive, participial and gerund clauses, chains of de, split predicates;
    • officialese: verbal nouns, compound prepositions, clichés;
    • style: stopwords, parenthetical expressions, connectors;
    • phonics: alliteration.
  • HighlightedText.to_html() gives the markup with a legend, and a notebook shows it directly.

Dataset

  • SpanishSonnets holds 4,259 sonnets by 1,167 authors from the 15th to the 20th century.
    • They come from DISCO 5.0 (CC BY 4.0), with its automatic scansion and rhyme labels on every line.
    • Only the public-domain sonnets are kept.
    • Corrections to DISCO's metadata are listed in the archive.
  • The records can be filtered by period, author, country and gender.

Quality

  • 1876 tests with 100% line and branch coverage.
  • ruff and mypy are clean.
  • CI runs on Python 3.11 to 3.14 and on Linux, Windows and macOS.
  • The release went through its own QA pass, and all its findings are fixed.
  • The docstrings and the bilingual documentation were trimmed down to what a reader needs.

Documentation: https://sergeyshk.github.io/esTS/ · Commits: 0.3.0...0.4.0

Assets 2
Loading