Skip to content

We address the task of learning contextualized word, sentence and document representations with a hierarchical language model by stacking Transformer-based encoders on a sentence level and subsequently on a document level and performing masked token prediction.

Notifications You must be signed in to change notification settings

marcomoldovan/hierarchical-language-modeling

Error
Looks like something went wrong!

About

We address the task of learning contextualized word, sentence and document representations with a hierarchical language model by stacking Transformer-based encoders on a sentence level and subsequently on a document level and performing masked token prediction.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published