This tools loads two sets of time-aligned sequences and outputs several measures of accuracy between them.
This project was initially based on the following paper:
Okko Johannes Räsänen, Unto Kalervo Laine, and Toomas Altosaar: An Improved Speech Segmentation Quality Measure: the R-value
It can be viewed under this link: http://legacy.spa.aalto.fi/research/stt/papers/r_value.pdf
There is one minor addition to the paper above. All the segmentation measure papers seem to treat boundaries as entities described only by the time they occur in. The contents of the segments between the boundaries are not checked.
While this may be fine for most uses, this tool assigns a name to each boundary based on the segments located around them. Eg: if a boundary exists between segments titled 'a' and 'b', the boundary will be named "a_b". Hits are counted if and only if the time AND the names of the boundaries (reference and hypothesis) match.
- CTM
- TextGrid
- Hit rate
- Over-segmentation rate
- Precision
- Recall
- F-measure
- r1 (computed as a part of R-value)
- r2 (computed as a part of R-value)
- R-value
TextGrid input requires the installation of the TextGrid package, as described in the followin repo: https://github.com/kylebgorman/textgrid
It can easily be installed using pip:
pip install TextGrid
usage: AlignMeasure.py [-h] [--ref-tier REFTIER] [--hyp-tier HYPTIER] ref hyp
Calcualte various alignemnt accuracy measures.
positional arguments:
ref reference segmentation (CTM or TextGrid)
hyp studied segmentation (CTM or TextGrid)
optional arguments:
-h, --help show this help message and exit
--ref-tier REFTIER, -rt REFTIER
for TextGrid, use which tier for reference (default:0)
--hyp-tier HYPTIER, -ht HYPTIER
for TextGrid, use which tier for hypothesis
(default:0)
The input file type is determined based on the extension only!
Number of boundaries in reference segmentation: 6
Number of boundaries in studied segmentation: 7
Number of hits: 4
Hit rate (higher=>better_: 66.666667%
Over-segmentation rate (closer-zero=>better): 16.6666666667
Precision (higher=>better): 57.142857%
Recall (higher=>better): 66.666667%
F-measure (higher=>better): 61.538462%
r1 (closer-zero=>better): 37.267799625
r2 (closer-zero=>better): -35.3553390593
R-value (higher=>better): 63.688431%