Language of the Evaluator's Tokenizer is set to English when asked for Dutch or French.

[amobular]

The tokenizer of the Evaluator class, will be set to TokenizerEN when languages from ['nl', 'fr'] are chosen.

See the following code:

deidentify/deidentify/evaluation/evaluator.py

Lines 45 to 56 in 0e455d3

	if language == 'nl':
	from deidentify.tokenizer.tokenizer_ons import TokenizerOns
	self.tokenizer = TokenizerOns(disable=('tagger', 'parser', 'ner'))
	if language == 'fr':
	from deidentify.tokenizer.tokenizer_fr import TokenizerFR
	self.tokenizer = TokenizerFR(disable=('tagger', 'parser', 'ner'))
	if language == 'de':
	from deidentify.tokenizer.tokenizer_de import TokenizerDE
	self.tokenizer = TokenizerDE(disable=('tagger', 'parser', 'ner'))
	else:
	from deidentify.tokenizer.tokenizer_en import TokenizerEN
	self.tokenizer = TokenizerEN(disable=('tagger', 'parser', 'ner'))

Change the if statements to elif and it will work as intended.

Nice project btw :)

[jantrienes]

Great catch! The fix should be simple: replace second and third if with elif. Do you mind submitting a PR?