Skip to content

Add an XLMRobertaMaskedLM task model #720

@mattdangerw

Description

@mattdangerw

In #653 we added a masked language modeling task for RoBERTa. We can make a similar change for the XLMRoberta model.

  • Update XLMRobertaTokenizer to expect a mask token.
  • Add a XLMRobertaMaskedLMPreprocessor preprocessor layer and tests.
  • Add a XLMRobertaMaskedLM task model and tests.
  • Update keras_nlp/models/__init__.py to export XLMRobertaMaskedLM and XLMRobertaMaskedLMPreprocessor.

Metadata

Metadata

Assignees

Labels

stat:contributions welcomeAdd this label to feature request issues so they are separated out from bug reporting issuestype:featureNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions