Verbosity

[mjpost]

It seems that the command line always prints a progress bar. I suggest one of the following options:

1. (preferred) it only print if --verbose|-v is specified (silent by default); or
2. that one could suppress this with --quiet|-q

Thanks for a great too, BTW

[alvations]

@mjpost Good idea on the verbosity!

My preference is for --quiet instead of --verbose since tqdm (progress bar) would be print to stderr. And I do get panic attack if I see nothing moving when using the CLI commands on larger datasets =)

But if more people would vot go for --verbose and default silent instead, I'll be glad to make the appropriate changes to #70

P/S: The name is inspired by sacrebleu, thank you for the great tool too! =)

[mjpost]

I think the UNIX convention is to default to silence, but it seems the Moses tools are the opposite. It's your tool, so I think you can decide! I'll be happy as long as I can shut it off.

BTW, did you ever run any regression tests on large data piles to see if sacremoses produces the same results as the Perl tools?

[alvations]

--quiet it is =)

pip install -U sacremoses>=0.034

$ sacremoses --version
sacremoses, version 0.0.34

$ sacremoses tokenize --help
Usage: sacremoses tokenize [OPTIONS]

Options:
  -l, --language TEXT            Use language specific rules when tokenizing
  -j, --processes INTEGER        No. of processes.
  -a, --aggressive-dash-splits   Triggers dash split rules.
  -x, --xml-escape               Escape special characters for XML.
  -p, --protected-patterns TEXT  Specify file with patters to be protected in
                                 tokenisation.
  -c, --custom-nb-prefixes TEXT  Specify a custom non-breaking prefixes file,
                                 add prefixes to the default ones from the
                                 specified language.
  -e, --encoding TEXT            Specify encoding of file.
  -q, --quiet                    Disable progress bar.
  -h, --help                     Show this message and exit.

I haven't run a real regression test on the CLI vs the perl scripts from mosesdecoder. Been on the TODO list #37 though =)

[mjpost]

Awesome thank you!