Contracted tokens in lexers

[LoicGrobol]

So far we pass UD syntactic words to our lexers, so if a tree has e.g. “du”, the lexers will see “de” and “le”, but pretrained embeddings (contextual or not) have been trained seeing surface tokens, so their representations might be somewhat off.
We can hope that it does not have too much impact, especially since we fine-tune everything, but there might be a better way?

Possible strategies:

• Stay as-is: the lexer sees and encodes the syntactic words
• Duplicate: the lexer sees the surface (contracted) token and the syntactic words both get represented by the same corresponding vector
• Duplicate with contraction flag: like duplicate, but a flag signalling that the syntactic words come from a single contracted token is added, for instance by summing them with a trainable vector, or a different trainable vector for every position (or every form? but that would be really impractical)
• ???