Skip to content

Fix word range #334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Oct 12, 2025
Merged

Fix word range #334

merged 4 commits into from
Oct 12, 2025

Conversation

folivoramanh
Copy link

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Just separate range verbalizer instead of using word verbalizer to avoid double quote and hyphen error when inference

Before your PR is "Ready for review"

Pre checks:

  • Have you signed your commits? Use git commit -s to sign.
  • Do all unittests finish successfully before sending PR?
    1. pytest or (if your machine does not have GPU) pytest --cpu from the root folder (given you marked your test cases accordingly @pytest.mark.run_only_on('CPU')).
    2. Sparrowhawk tests bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...
  • If you are adding a new feature: Have you added test cases for both pytest and Sparrowhawk here.
  • Have you added __init__.py for every folder and subfolder, including data folder which has .TSV files?
  • Have you followed codeQL results and removed unused variables and imports (report is at the bottom of the PR in github review box) ?
  • Have you added the correct license header Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. to all newly added Python files?
  • If you copied nemo_text_processing/text_normalization/en/graph_utils.py your header's second line should be Copyright 2015 and onwards Google, Inc.. See an example here.
  • Remove import guards (try import: ... except: ...) if not already done.
  • If you added a new language or a new feature please update the NeMo documentation (lives in different repo).
  • Have you added your language support to tools/text_processing_deployment/pynini_export.py.

PR Type:

  • New Feature
  • Bugfix
  • Documentation
  • Test

If you haven't finished some of the above items you can still open "Draft" PR.

folivoramanh and others added 4 commits October 6, 2025 16:36
@folivoramanh
fix range and quote
fa76b95 
Signed-off-by: folivoramanh <palasek182@gmail.com>
@folivoramanh
fix quote in post process
5f9e379 
Signed-off-by: folivoramanh <palasek182@gmail.com>
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
451823f 
for more information, see https://pre-commit.ci
@folivoramanh
fix quote and range
2fd9d95 
Signed-off-by: folivoramanh <palasek182@gmail.com>
tbartley94
tbartley94 approved these changes Oct 6, 2025
View reviewed changes
Copy link
Collaborator

@tbartley94 tbartley94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tbartley94 tbartley94 merged commit 29ff3ee into NVIDIA:staging_vi_tn Oct 12, 2025
2 checks passed
@folivoramanh folivoramanh deleted the fix_word_range branch October 13, 2025 15:55
folivoramanh added a commit to folivoramanh/NeMo-text-processing that referenced this pull request Oct 13, 2025
@folivoramanh @pre-commit-ci
Fix word range (NVIDIA#334)
9572be7 
* fix range and quote

Signed-off-by: folivoramanh <palasek182@gmail.com>

* fix quote in post process

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix quote and range

Signed-off-by: folivoramanh <palasek182@gmail.com>

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>
folivoramanh added a commit to folivoramanh/NeMo-text-processing that referenced this pull request Oct 13, 2025
@folivoramanh @pre-commit-ci
Fix word range (NVIDIA#334)
eb7d6ed 
* fix range and quote

Signed-off-by: folivoramanh <palasek182@gmail.com>

* fix quote in post process

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix quote and range

Signed-off-by: folivoramanh <palasek182@gmail.com>

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>
mgrafu pushed a commit that referenced this pull request Oct 13, 2025
@folivoramanh @anand-nv @pre-commit-ci
Staging vi tn signed off (#339)
04e34e1 
* Fix Jenkinsfile for CI (#325)

* Fix Jenkinsfile for CI

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Fix requirements for test

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Update paths and docker

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Fix docker name

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Fix click version

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Change path of grammars for sparrowhawk tests

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Update paths in sh_test.sh

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Update paths

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Revert paths

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

---------

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* PR: Add Vietnamese text normalization for cardinal semiotic class (#289)

* Add Vietnamese text normalization for cardinal semiotic class

Signed-off-by: folivoramanh <palasek182@gmail.com>

* Add missing init file

Signed-off-by: folivoramanh <palasek182@gmail.com>

* Fix Cardinal and optimize logic

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Ordinal and Decimal for Vietnamese TN (#290)

* Add Vietnamese text normalization for ordinal and decimal semiotic classes

Signed-off-by: folivoramanh <palasek182@gmail.com>

* update sparrowhawk

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refractor decimal code and docstring

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Vietnamese TN - Fraction (#296)

* Fraction class for Vietnamese TN

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove irrelavant test case

Signed-off-by: folivoramanh <palasek182@gmail.com>

* Remove irrelavant test case

Signed-off-by: folivoramanh <palasek182@gmail.com>

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Date Semiotic Class for Vietnamese TN (#298)

* Date for vietnamese TN

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add roman support and correct copyright header

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change header to current year

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change header time

Signed-off-by: folivoramanh <palasek182@gmail.com>

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Time - semiotic class for Vietnamese TN  (#302)

* Time - semiotic class for Vietnamese TN

Signed-off-by: folivoramanh <palasek182@gmail.com>

* remove irrelevant import and comment

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add comment and refractor pattern

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Change the spaces to NEMO_SPACE for maintenance.

Signed-off-by: folivoramanh <palasek182@gmail.com>

* Change the spaces to NEMO_SPACE for maintenance.

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Change the spaces to NEMO_SPACE for maintenance. - remove quote

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Add Vietnamese TN support for Money and Range semiotic classes (#304)

* Add Vietnamese TN support for Money and Range semiotic classes

- Add money.py tagger and verbalizer for Vietnamese currency handling
- Add range.py tagger for numerical range processing
- Add supporting data files for money (currency, currency_minor, per_unit)
- Add quantity abbreviations and time units data
- Update existing taggers and verbalizers for integration
- Add comprehensive test cases for money and range functionality
- Update tokenize_and_classify to include new semiotic classes

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* modify illogical test cases

Signed-off-by: folivoramanh <palasek182@gmail.com>

* refractor and simplify word and punctuation to avoid hardcoding

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refractor code money range

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Add Vietnamese measure text normalization support (#307)

* Add Vietnamese measure text normalization support

- Added measure tagger and verbalizer for Vietnamese TN
- Updated money tagger and verbalizer to handle per-unit measurements
- Added test cases for measure normalization
- Updated fraction handling for better integration
- Added data files for measurements, prefixes, and per-unit bases

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: folivoramanh <palasek182@gmail.com>

* add test case for range measure

Signed-off-by: folivoramanh <palasek182@gmail.com>

* additional support for cardinal and remove duplicate test case

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refractor cardinal and add test cases

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove duplicate lines in run_eval file

Signed-off-by: folivoramanh <palasek182@gmail.com>

* refractor minor code

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add measure support for unit per unit cases

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Vietnamese MRC 1.0 fix case (#312)

* fix and add cases

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Fix word range (#334)

* fix range and quote

Signed-off-by: folivoramanh <palasek182@gmail.com>

* fix quote in post process

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix quote and range

Signed-off-by: folivoramanh <palasek182@gmail.com>

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Date time itn (#333)

* improve numeric semiotic classes

Signed-off-by: folivoramanh <palasek182@gmail.com>

* Fix Jenkinsfile for CI (#325)

* Fix Jenkinsfile for CI

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Fix requirements for test

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Update paths and docker

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Fix docker name

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Fix click version

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Change path of grammars for sparrowhawk tests

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Update paths in sh_test.sh

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Update paths

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Revert paths

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

---------

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: folivoramanh <palasek182@gmail.com>

* revert old codes

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* revert not inherit

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* improve date time

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix pynini union instead of union operator

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* improve measure, telephone, electronic

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change union operator to pynini union

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Signed-off-by: Anand Joseph <anajoseph@nvidia.com>
Co-authored-by: anand-nv <105917641+anand-nv@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

---------

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: anand-nv <105917641+anand-nv@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
mgrafu pushed a commit that referenced this pull request Oct 16, 2025
@folivoramanh @pre-commit-ci @mgrafu
Fix word range (#334)
6a1bbb7 
* fix range and quote

Signed-off-by: folivoramanh <palasek182@gmail.com>

* fix quote in post process

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix quote and range

Signed-off-by: folivoramanh <palasek182@gmail.com>

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com>
mgrafu pushed a commit that referenced this pull request Oct 16, 2025
@folivoramanh @anand-nv
@pre-commit-ci @mgrafu
Staging vi tn signed off (#339)
74ddac7 
* Fix Jenkinsfile for CI (#325)

* Fix Jenkinsfile for CI

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Fix requirements for test

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Update paths and docker

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Fix docker name

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Fix click version

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Change path of grammars for sparrowhawk tests

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Update paths in sh_test.sh

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Update paths

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Revert paths

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

---------

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* PR: Add Vietnamese text normalization for cardinal semiotic class (#289)

* Add Vietnamese text normalization for cardinal semiotic class

Signed-off-by: folivoramanh <palasek182@gmail.com>

* Add missing init file

Signed-off-by: folivoramanh <palasek182@gmail.com>

* Fix Cardinal and optimize logic

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Ordinal and Decimal for Vietnamese TN (#290)

* Add Vietnamese text normalization for ordinal and decimal semiotic classes

Signed-off-by: folivoramanh <palasek182@gmail.com>

* update sparrowhawk

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refractor decimal code and docstring

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Vietnamese TN - Fraction (#296)

* Fraction class for Vietnamese TN

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove irrelavant test case

Signed-off-by: folivoramanh <palasek182@gmail.com>

* Remove irrelavant test case

Signed-off-by: folivoramanh <palasek182@gmail.com>

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Date Semiotic Class for Vietnamese TN (#298)

* Date for vietnamese TN

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add roman support and correct copyright header

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change header to current year

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change header time

Signed-off-by: folivoramanh <palasek182@gmail.com>

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Time - semiotic class for Vietnamese TN  (#302)

* Time - semiotic class for Vietnamese TN

Signed-off-by: folivoramanh <palasek182@gmail.com>

* remove irrelevant import and comment

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add comment and refractor pattern

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Change the spaces to NEMO_SPACE for maintenance.

Signed-off-by: folivoramanh <palasek182@gmail.com>

* Change the spaces to NEMO_SPACE for maintenance.

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Change the spaces to NEMO_SPACE for maintenance. - remove quote

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Add Vietnamese TN support for Money and Range semiotic classes (#304)

* Add Vietnamese TN support for Money and Range semiotic classes

- Add money.py tagger and verbalizer for Vietnamese currency handling
- Add range.py tagger for numerical range processing
- Add supporting data files for money (currency, currency_minor, per_unit)
- Add quantity abbreviations and time units data
- Update existing taggers and verbalizers for integration
- Add comprehensive test cases for money and range functionality
- Update tokenize_and_classify to include new semiotic classes

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* modify illogical test cases

Signed-off-by: folivoramanh <palasek182@gmail.com>

* refractor and simplify word and punctuation to avoid hardcoding

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refractor code money range

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Add Vietnamese measure text normalization support (#307)

* Add Vietnamese measure text normalization support

- Added measure tagger and verbalizer for Vietnamese TN
- Updated money tagger and verbalizer to handle per-unit measurements
- Added test cases for measure normalization
- Updated fraction handling for better integration
- Added data files for measurements, prefixes, and per-unit bases

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: folivoramanh <palasek182@gmail.com>

* add test case for range measure

Signed-off-by: folivoramanh <palasek182@gmail.com>

* additional support for cardinal and remove duplicate test case

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refractor cardinal and add test cases

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove duplicate lines in run_eval file

Signed-off-by: folivoramanh <palasek182@gmail.com>

* refractor minor code

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add measure support for unit per unit cases

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Vietnamese MRC 1.0 fix case (#312)

* fix and add cases

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Fix word range (#334)

* fix range and quote

Signed-off-by: folivoramanh <palasek182@gmail.com>

* fix quote in post process

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix quote and range

Signed-off-by: folivoramanh <palasek182@gmail.com>

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* Date time itn (#333)

* improve numeric semiotic classes

Signed-off-by: folivoramanh <palasek182@gmail.com>

* Fix Jenkinsfile for CI (#325)

* Fix Jenkinsfile for CI

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Fix requirements for test

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Update paths and docker

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Fix docker name

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Fix click version

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Change path of grammars for sparrowhawk tests

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Update paths in sh_test.sh

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Update paths

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

* Revert paths

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

---------

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: folivoramanh <palasek182@gmail.com>

* revert old codes

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* revert not inherit

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* improve date time

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix pynini union instead of union operator

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* improve measure, telephone, electronic

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change union operator to pynini union

Signed-off-by: folivoramanh <palasek182@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: folivoramanh <palasek182@gmail.com>
Signed-off-by: Anand Joseph <anajoseph@nvidia.com>
Co-authored-by: anand-nv <105917641+anand-nv@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>

---------

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>
Signed-off-by: folivoramanh <palasek182@gmail.com>
Co-authored-by: anand-nv <105917641+anand-nv@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tbartley94 tbartley94 tbartley94 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@folivoramanh @tbartley94