Repository navigation
v4.0.0
The major version bump from v3 to v4 is due to backward-incompatible yet minor
API changes triggered by the switch from the PyLangAcq + wordseg
dependencies to Rustling for handing CHAT data, word segmentation, and
part-of-speech tagging.
Added
- Added the new function
jyutping_to_ipafor Jyutping-to-IPA conversion. - The
characters_to_jyutpingfunction can now take a list of strings as input
with user-provided word segmentation. - Added support for Python 3.11, 3.12, 3.13, and 3.14.
Changed
- Switched to Rustling as the underlying engine
for CHAT data parsing, word segmentation, and part-of-speech tagging;
dropped PyLangAcq and wordseg as dependencies. CHATReaderhas been renamedCHATand switched from PyLangAcq's legacy pure-Python
parser to a Rust-based parser from Rustling, with various API changes
for method names, arguments, etc.- The word segmentation model has been updated to a semi-supervised hybrid approach
that combines a DAG and hidden Markov model.
Thesegmentfunction for word segmentation no longer accepts an argument
for a custom segmenter. - For both word segmentation and part-of-speech tagging, the persisted models
shipped with the package are now zstd-compressed FlatBuffers binaries.
Removed
- Dropped support for Python 3.7, 3.8, and 3.9.
Fixed
- Fixed word segmentation so that spaces between English words
in the user input are now honored as word boundaries. - If
parse_textis given an empty input orNone,
now an emptyCHATinstance is returned. - If
parse_textis given a non-empty list of utterances,
then any empty utterance (e.g.,None,"")
will now be represented by an emptyUtteranceinstance inside the resultingCHAToutput. - Fixed the HKCanCor-to-UD mapping for
G1mapped toVERBnotV.