Skip to content

Commit

Permalink
Make non-breaking spaces that are in there more explicit
Browse files Browse the repository at this point in the history
  • Loading branch information
jelmervdl committed Oct 26, 2023
1 parent c57e777 commit 4164186
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion sacremoses/test/test_normalizer.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,6 @@ def test_normalization_pipeline(self):

def test_moses_normalize_with_perl_parity(self):
moses_perl_parity = MosesPunctNormalizer(perl_parity=True)
text = 'from the ‘bad bank’, Northern, wala « dox ci jawwu Les « wagonways » étaient construits'
text = 'from the ‘bad bank’, Northern, wala\u00A0«\u00A0dox ci jawwu Les «\u00A0wagonways\u00A0»\u00A0étaient construits'
expected = '''from the 'bad bank," Northern, wala "dox ci jawwu Les "wagonways" étaient construits'''
assert moses_perl_parity.normalize(text) == expected

0 comments on commit 4164186

Please sign in to comment.