Skip to content

Commit

Permalink
Update readme
Browse files Browse the repository at this point in the history
PiperOrigin-RevId: 270346317
  • Loading branch information
saberkun authored and tensorflower-gardener committed Sep 20, 2019
1 parent 1345ec9 commit d03ce00
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion official/nlp/xlnet/README.md
Expand Up @@ -2,4 +2,15 @@

The academic paper which describes XLNet in detail and provides full results on
a number of tasks can be found here: https://arxiv.org/abs/1906.08237.
Instructions and user guide will be added soon.

**Instructions and user guide will be added soon.**

XLNet is a generalized autoregressive BERT-like pretraining language model that
enables learning bidirectional contexts by maximizing the expected likelihood
over all permutations of the factorization order. It can learn dependency beyond
a fixed length without disrupting temporal coherence by using segment-level
recurrence mechanism and relative positional encoding scheme introduced in
[Transformer-XL](https://arxiv.org/pdf/1901.02860.pdf). XLNet outperforms BERT
on 20 NLP benchmark tasks and achieves state-of-the-art results on 18 tasks
including question answering, natural language inference, sentiment analysis,
and document ranking.

0 comments on commit d03ce00

Please sign in to comment.