: Idiomfier as an NER tagger

[eubinecto]

How?

First of all, let's try this with the baseline approach - just using a simple linear layer approach. Performance is not what matters as of right now.

Could you use BART for this? Yes you could, but.. BART is an auto-regressive model - cannot refer to the future when processing the past. BERT would be a better choice than BART.

To-do's

• delete tokenizer- related fetchers and paths
• change the builders: InputsBuilder
  • we must make sure that the tokens are not split further than it is now..
• explore_ inputs_builder
• change the builders: LabelsBuilder
• explore_labels_builder
• rewrite Idiomifier to learn NER with BERT

[eubinecto]

How should we handle subwords in a token-level classification task?