Add RobertaForTokenClassification and an example checkpoint on Hub

[julien-c]

Use case: perform parts-of-speech tagging in Esperanto

Code to test:

const classifyTokens = await pipeline("token-classification", "julien-c/EsperBERTo-small-pos");
await classifyTokens("Mi estas viro kej estas tago varma.");

Output:

[
  {
    entity: 'PRON',
    score: 0.9999696612358093,
    index: 1,
    word: 'Mi',
    start: null,
    end: null
  },
  {
    entity: 'VERB',
    score: 0.9770894050598145,
    index: 2,
    word: ' estas',
    start: null,
    end: null
  },
  {
    entity: 'ADJ',
    score: 0.7278493642807007,
    index: 3,
    word: ' viro',
    start: null,
    end: null
  },
  {
    entity: 'VERB',
    score: 0.9786341786384583,
    index: 6,
    word: ' estas',
    start: null,
    end: null
  },
  {
    entity: 'NOUN',
    score: 0.8334921002388,
    index: 7,
    word: ' tago',
    start: null,
    end: null
  },
  {
    entity: 'ADJ',
    score: 0.9996510148048401,
    index: 8,
    word: ' varma',
    start: null,
    end: null
  }
]

Consistent with server-side computation as seen on https://huggingface.co/julien-c/EsperBERTo-small-pos

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[xenova]

Perfect! 🚀