[TN] WFST to normalize punctuation #4108

ekmb · 2022-05-04T20:19:48Z

What does this PR do ?

Added WFST graph to handle punctuation after verbalization step, parallel normalization, bug fixes

Collection: [TN]

Changelog

Add specific line by line info of high level changes in this PR.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

[ X] New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to # (issue)

Signed-off-by: ekmb <ebakhturina@nvidia.com>

…_threshold Signed-off-by: ekmb <ebakhturina@nvidia.com>

lgtm-com · 2022-05-04T20:32:55Z

This pull request introduces 2 alerts when merging 5f0ef3e into dc13c5d - view on LGTM.com

new alerts:

1 for Duplicate key in dict literal
1 for Wrong name for an argument in a class instantiation

Signed-off-by: ekmb <ebakhturina@nvidia.com>

lgtm-com · 2022-05-06T22:41:24Z

This pull request introduces 2 alerts when merging 64bbc44 into ddd8719 - view on LGTM.com

new alerts:

1 for Duplicate key in dict literal
1 for Wrong name for an argument in a class instantiation

Signed-off-by: ekmb <ebakhturina@nvidia.com>

lgtm-com · 2022-05-10T20:26:30Z

This pull request introduces 2 alerts when merging b5814f6 into 650718f - view on LGTM.com

new alerts:

1 for Duplicate key in dict literal
1 for Wrong name for an argument in a class instantiation

Signed-off-by: ekmb <ebakhturina@nvidia.com>

lgtm-com · 2022-05-13T23:21:34Z

This pull request introduces 2 alerts when merging d40929b into 08df199 - view on LGTM.com

new alerts:

1 for Unreachable code
1 for Wrong name for an argument in a class instantiation

Signed-off-by: ekmb <ebakhturina@nvidia.com>

yzhang123

thanks Evelina for the great effort!

nemo/collections/nlp/data/text_normalization/utils.py

yzhang123 · 2022-05-23T21:25:59Z

nemo_text_processing/text_normalization/en/data/roman/key_word.tsv

 chapter
-Class
-CLASS


why removing this?

nemo_text_processing/text_normalization/en/graph_utils.py

tests/nemo_text_processing/en/data_text_normalization/test_cases_cardinal.txt

yzhang123 · 2022-05-23T21:33:39Z

tests/nemo_text_processing/en/data_text_normalization/test_cases_word.txt

 $ and 5% or %~dollar and five percent or percent sign
 1~one
 1~one
-1, ~one ,
-1!!!!~one ! ! ! !
-(1)Hello~( one ) Hello
 !1~! one


this will not be included? only punctuation following semiotic and words?

moved to test_cases_punctuation.txt

yzhang123 · 2022-05-23T21:37:27Z

tests/nemo_text_processing/en/data_text_normalization/test_cases_punctuation.txt

 abbreviation USA,~abbreviation USA,
 23rd july, 1998~the twenty third of july, nineteen ninety eight
 April 29th’s meeting~april twenty ninth’s meeting
 ?,~?,
 ?,no~?,no
-I've 20' and 14/ they're I'm 16c.~I've twenty' and fourteen slash they're I'm sixteen c.
+I've 20' and 14/ they're I'm 16c.~I've twenty ' and fourteen slash they're I'm sixteen c.


why space between 20 and ' in normalization?

apostrophe is not handled if attached to a semiotic token

yzhang123 · 2022-05-23T21:40:17Z

nemo_text_processing/text_normalization/en/taggers/word.py

                pynini.difference(
-                    self.graph, pynini.union("$", "€", "₩", "£", "¥", "#", "$", "%") + pynini.closure(NEMO_DIGIT, 1)
+                    graph, pynini.union("$", "€", "₩", "£", "¥", "#", "$", "%") + pynini.closure(NEMO_DIGIT, 1)


why did we treat this separately?

reduced the complexity of this

yzhang123 · 2022-05-23T21:44:06Z

nemo_text_processing/text_normalization/en/taggers/word.py

+        # non_digit is needed to allow non-ascii chars, like in "Müller's"
+        non_digit = pynini.difference(NEMO_NOT_SPACE, NEMO_DIGIT).optimize()
+        at_least_one_alpha = (
+            pynini.closure(non_digit) + pynini.closure(NEMO_ALPHA, 1) + pynini.closure(non_digit)


why not just having closure(non_digit), why including NeMo_Alpha?

yzhang123 · 2022-05-23T21:44:29Z

nemo_text_processing/text_normalization/en/taggers/word.py

+
+        # punct followed by word and another punct mark: { "And, }
+        alpha_with_punct_graph = (
+            pynini.closure(punct_symbols)


why includding punct_symbols here?

yzhang123 · 2022-05-23T21:44:59Z

nemo_text_processing/text_normalization/en/taggers/word.py

+            pynini.closure(punct_symbols)
+            + at_least_one_alpha
+            + pynini.closure(punct_symbols, 1)
+            + pynini.closure(non_digit)


why requiring non_digit, NeMo_Alpha here?

Signed-off-by: ekmb <ebakhturina@nvidia.com>

yzhang123 · 2022-05-26T13:54:41Z

tests/nemo_text_processing/en/data_text_normalization/test_cases_word.txt

 $ and 5% or %~dollar and five percent or percent sign
 1~one
 1~one
-1, ~one ,
-1!!!!~one ! ! ! !
-(1)Hello~( one ) Hello
 !1~! one


Signed-off-by: ekmb <ebakhturina@nvidia.com>

ekmb added 4 commits May 3, 2022 13:25

roman and ipa symbols support

147a676

Signed-off-by: ekmb <ebakhturina@nvidia.com>

update punct

90f3d57

Signed-off-by: ekmb <ebakhturina@nvidia.com>

update test to remove spaces, add parallel to normalize_list, add cer…

6a457ff

…_threshold Signed-off-by: ekmb <ebakhturina@nvidia.com>

Merge branch 'main' into tn_tts_e

5f0ef3e

ekmb added 4 commits May 5, 2022 09:35

grammar updates based on bug reports

1320d09

Signed-off-by: ekmb <ebakhturina@nvidia.com>

merge main

cc9c275

Signed-off-by: ekmb <ebakhturina@nvidia.com>

bug fixes according to to_improve doc

7191985

Signed-off-by: ekmb <ebakhturina@nvidia.com>

Merge branch 'main' into tn_tts_e

64bbc44

ekmb added 3 commits May 9, 2022 07:11

add input chek

be1a1c9

Signed-off-by: ekmb <ebakhturina@nvidia.com>

update punct grammar to handle non-ascii

ad745e5

Signed-off-by: ekmb <ebakhturina@nvidia.com>

Merge branch 'main' into tn_tts_e

b5814f6

ekmb added 3 commits May 10, 2022 14:34

punct graph keys fix

2a2d683

Signed-off-by: ekmb <ebakhturina@nvidia.com>

add punct graph to default

4c2a013

Signed-off-by: ekmb <ebakhturina@nvidia.com>

post-processing separated

165dd57

Signed-off-by: ekmb <ebakhturina@nvidia.com>

ekmb changed the title ~~WFST to normalize punctuation~~ [TN] WFST to normalize punctuation May 12, 2022

ekmb added 3 commits May 13, 2022 15:41

move punct related test cases

38263d1

Signed-off-by: ekmb <ebakhturina@nvidia.com>

clean up

66e6d62

Signed-off-by: ekmb <ebakhturina@nvidia.com>

move punct related test cases

d40929b

Signed-off-by: ekmb <ebakhturina@nvidia.com>

ekmb added 3 commits May 15, 2022 15:58

all cpu tests green

fe97b1d

Signed-off-by: ekmb <ebakhturina@nvidia.com>

Merge branch 'main' into tn_tts_e

f589ae0

clean up and punct related tests moved to punct test cases

8ab07f0

Signed-off-by: ekmb <ebakhturina@nvidia.com>

ekmb requested a review from yzhang123 May 16, 2022 02:46

clean up

a7acdd4

Signed-off-by: ekmb <ebakhturina@nvidia.com>

yzhang123 suggested changes May 23, 2022

View reviewed changes

ekmb added 2 commits May 24, 2022 17:38

Merge branch 'main' into tn_tts_e

7a46cea

update jenkins grammars, review

8c65436

Signed-off-by: ekmb <ebakhturina@nvidia.com>

ekmb added 3 commits May 25, 2022 21:48

Merge branch 'main' into tn_tts_e

d67beaf

flatten result

9ec4d3c

Signed-off-by: ekmb <ebakhturina@nvidia.com>

Merge branch 'main' into tn_tts_e

4b5d376

yzhang123 previously approved these changes May 26, 2022

View reviewed changes

remove return from batch_norm, change PT container to 1.11.0

5771035

Signed-off-by: ekmb <ebakhturina@nvidia.com>

ekmb dismissed yzhang123’s stale review via 5771035 May 26, 2022 14:12

yzhang123 previously approved these changes May 26, 2022

View reviewed changes

add flag for audio-based tests

124fb2b

Signed-off-by: ekmb <ebakhturina@nvidia.com>

ekmb dismissed yzhang123’s stale review via 5771035 May 26, 2022 17:40

ekmb force-pushed the tn_tts_e branch from 34a8fcf to 5771035 Compare May 26, 2022 17:40

ekmb added 2 commits May 26, 2022 10:42

Merge branch 'main' into tn_tts_e

d625aa7

update jenkins test file name: now punct is fixed by default

82a3ed2

Signed-off-by: ekmb <ebakhturina@nvidia.com>

ekmb requested a review from yzhang123 May 26, 2022 19:28

yzhang123 approved these changes May 26, 2022

View reviewed changes

yzhang123 merged commit 49dff00 into main May 26, 2022

ekmb deleted the tn_tts_e branch August 3, 2022 07:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TN] WFST to normalize punctuation #4108

[TN] WFST to normalize punctuation #4108

ekmb commented May 4, 2022

lgtm-com bot commented May 4, 2022

lgtm-com bot commented May 6, 2022

lgtm-com bot commented May 10, 2022

lgtm-com bot commented May 13, 2022

yzhang123 left a comment

yzhang123 May 23, 2022

yzhang123 May 23, 2022

ekmb May 26, 2022

yzhang123 May 26, 2022

yzhang123 May 23, 2022

ekmb May 26, 2022

yzhang123 May 23, 2022

ekmb May 25, 2022

yzhang123 May 23, 2022

yzhang123 May 23, 2022

yzhang123 May 23, 2022

yzhang123 May 26, 2022

[TN] WFST to normalize punctuation #4108

[TN] WFST to normalize punctuation #4108

Conversation

ekmb commented May 4, 2022

What does this PR do ?

Changelog

Usage

Before your PR is "Ready for review"

Who can review?

Additional Information

lgtm-com bot commented May 4, 2022

lgtm-com bot commented May 6, 2022

lgtm-com bot commented May 10, 2022

lgtm-com bot commented May 13, 2022

yzhang123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment