Skip to content

read double digits for telephone grammar #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yzhang123 merged 4 commits into from
Feb 6, 2023
Merged

read double digits for telephone grammar #32

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7991bd0
read double digits for telephone grammar
LarisaKe Feb 3, 2023
cda59a5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 3, 2023
dcc0c1d
import zero graph instead of hard coding
LarisaKe Feb 4, 2023
9302ec1
Merge branch 'FR_ITN' of https://github.com/NVIDIA/NeMo-text-processi…
LarisaKe Feb 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions nemo_text_processing/inverse_text_normalization/fr/taggers/telephone.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,17 +52,36 @@ def __init__(self):
(graph_ties + delete_hyphen + graph_digit),
)

# accept `double zéro` -> `00`
single_digits = graph_digit | graph_zero
digit_words = pynini.union(graph_digit.optimize(), graph_zero).invert()

doubled_digit = pynini.union(
*[
pynini.cross(
pynini.project(str(i) @ digit_words, "output")
+ pynini.accep(" ")
+ pynini.project(str(i) @ digit_words, "output"),
pynutil.insert("double ") + pynini.project(str(i) @ digit_words, "output"),
)
for i in range(10)
]
)
doubled_digit.invert()
digit_twice = single_digits + pynutil.delete(" ") + single_digits
doubled_digit @= digit_twice

graph_first_pair = graph_zero + delete_space + graph_digit
graph_first_pair |= pynutil.insert("0") + graph_digit # if zero is omitted
graph_first_pair |= doubled_digit
graph_first_pair += (
delete_space + insert_space
) # delete_space since closure allows possible gaps to be removed

# All digits
single_digits = graph_digit | graph_zero

graph_pair_all_digits = single_digits + delete_space
graph_pair_all_digits += single_digits
graph_pair_all_digits |= doubled_digit

graph_all_digits = pynini.closure(graph_pair_all_digits + delete_space + insert_space, 3, 3)
graph_all_digits = graph_first_pair + graph_all_digits + graph_pair_all_digits
Expand Down
3 changes: 2 additions & 1 deletion tests/nemo_text_processing/fr/data_inverse_text_normalization/test_cases_telephone.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
zéro deux douze trente-deux trente trente~02 12 32 30 30
zéro deux une deux trois deux trois zéro trois zéro~02 12 32 30 30
deux douze trente-deux trente trente~02 12 32 30 30
deux une deux trois deux trois zéro trois zéro~02 12 32 30 30
deux une deux trois deux trois zéro trois zéro~02 12 32 30 30
double neuf douze trente-deux trente trente~99 12 32 30 30