Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PRETRAINED ENCYCLOPEDIA: WEAKLY SUPERVISED KNOWLEDGE-PRETRAINED LANGUAGE MODEL [ICLR, 2020] #38

Open
Sosuke115 opened this issue Nov 18, 2020 · 0 comments
Open

PRETRAINED ENCYCLOPEDIA: WEAKLY SUPERVISED KNOWLEDGE-PRETRAINED LANGUAGE MODEL [ICLR, 2020] #38

Sosuke115 opened this issue Nov 18, 2020 · 0 comments
Assignees
@Sosuke115
Labels
Knowledge Base NLP

Comments

@Sosuke115
Copy link
Owner

ひとことで言うと

エンティティを使って単語を紛らわしいものに入れ替えることで事前学習

論文リンク

https://arxiv.org/pdf/1912.09637.pdf

概要

Screen Shot 2020-11-18 at 9 25 29

entity linking→mentionを同じタイプのentityにランダム置換してネガティブサンプルを作成
元のテキストかネガティブサンプルかの二値分類を行う

Screen Shot 2020-11-18 at 9 28 51

評価方法

fact completionやQAで優位性を確認
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Sosuke115 Sosuke115

Labels
Knowledge Base NLP
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Sosuke115