Multi_Sense_Embedding_Survey (Under construction)

Please commit a related paper for a good survey and discuss (I don't have a plan to publish a survey paper).

Training

Survey

• From Word to Sense Embeddings: A Survey on Vector Representations of Meaning, 2018.
• Word Embeddings: A Survey, 2019.

Supervised (including Knowledge Based)

Concept Hierarchy (e.g. WORDNET, BabelNet)

Post Edit

• Retrofitting Word Vectors to Semantic Lexicons, 2015.
• De-Conflated Semantic Representations, 2016.
• GenSense: A Generalized Sense Retrofitting Model, 2018.

with Word Sense Disambiguation

• SENSEMBED: Learning Sense Embeddings for Word and Relational Similarity, 2015.
• Multi-sense embeddings through a word sense disambiguation process Author links open overlay panel, 2019.
• SensEmBERT: Context-Enhanced Sense Embeddings for Multilingual Word Sense Disambiguation, 2020.

Distill BERT

• Decomposing word embedding with the capsule network, 2020.

Wikipedia

• Sense-aware Semantic Analysis: A Multi-prototype Word Representation Model using Wikipedia, 2015.

Un-Supervised (From Corpora Only)

Parametric

• Multi-prototype vector-space models of word meaning, 2010.
• Improving Word Representations via Global Contextand Multiple Word Prototypes, 2012.
• K-Embeddings: Learning Conceptual Embeddings for Words using Context, 2016.
• Multimodal Word Distributions, 2017.
• Which Evaluations Uncover Sense Representations that Actually Make Sense?, 2020. ※ This paper also proposes the threshold for pruning that can calculate. However, it can use all methods because this is not belonging to Non-Parametric.

with Latent Dirichlet Allocation

• A Mixture Model for Learning Multi-Sense Word Embeddings, 2017.
• Learning Topic-Sensitive Word Representations, 2017.
• Jointly Learning Word Embeddings and Latent Topics, 2017.
• Decoupled Word Embeddings using Latent Topics, 2020.

EM Algorithm

• A Probabilistic Model for Learning Multi-Prototype Word Embeddings, 2014.

Reinforcement Learning

• MUSE: Modularizing Unsupervised Sense Embeddings, 2017.

Non-Parametric (unperfectly, without parametic 'k' about a number of sense which words have)

• Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space, 2014.
  • CODE original github and reimplement?
• Multi Sense Embeddings from Topic Models, 2019.

with Word Similarity Graph

• Making Sense of Word Embeddings, 2016.

non-english

• Modified multi-sense skip-gram using weighted context and x-means 가중 문맥벡터와 X-means 방법을 이용한 변형 다의어스킵그램, 2021.

Non-Parametric (perfectly)

• Do Multi-Sense Embeddings Improve Natural Language Understanding?, 2015.

Others

with Part-of-Speech

• sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddings, 2015.

Other Approaches (Sparse, not Dense)

• NASARI: a Novel Approach to a Semantically-Aware Representation of Items, 2015.
• Word2Sense: Sparse Interpretable Word Embeddings, 2020.
• Conception: Multilingually-Enhanced, Human-Readable Concept Vector Representations, 2020.

Introduction

For Japanese, 言語処理における分散表現学習のフロンティア, 2016.

Evaluation (Including For Word Embedding)

Survey

• A Survey of Word Embeddings Evaluation Methods, 2018.

Semantic Similairy Survey

• An overview of word and sense similarity, 2019.
• Evolution of Semantic Similarity—A Survey, 2020.

Intrinsic (Word Only)

Word Analogy

ACL page https://aclweb.org/aclwiki/Analogy_(State_of_the_art).

Metric

• Dual Embeddings and Metrics for Relational Similarity, 2017.

Word Similarity

ACL page https://aclweb.org/aclwiki/Similarity_(State_of_the_art).

RelatedNess

• SemEval-2017 Task 2: Multilingual and Cross-lingual Semantic Word Similarity
  • Sense identification data: A dataset for lexical semantics (SemEval-2017 Task 2 + with BabelNet ID), 2020.
  • Novel metrics for computing semantic similarity with sense embeddings, 2020.
• MSD-1030: A Well-built Multi-Sense Evaluation Dataset for Sense Representation Models, 2020.

Similarity

• SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation, 2015.
• Multi-SimLex: A Large-Scale Evaluation of Multilingual and Crosslingual Lexical Semantic Similarity, 2020.

Intrinsic & Extrinsic (With Sentence, Slightly related paper is categorized to Application Use)

Similarity

• SCWS Improving Word Representations via Global Contextand Multiple Word Prototypes, 2012. # dupliates
• SemEval-2020 Task 3: Graded Word Similarity in Context, 2020.

Binary Sense Detection (similar to Word Sense Disambiguation)

• WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations, 2019.
• XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization, 2020.
• SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation (MCL-WiC), 2021.
• WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context, 2021.

Related Works

Word Embedding

Survey

• github memory survey https://githubmemory.com/repo/aflyhat/awesome-sentence-embedding, -2019.
• Word Representation, 2020.
• A Comprehensive Survey on Word Representation Models: From Classical to State-Of-The-Art Word Representation Language Models, 2021.

Training

• Efficient Estimation of Word Representations in Vector Space, 2013.
• Distributed Representations of Words and Phrases and their Compositionality, 2013.
• GloVe: Global Vectors for Word Representation, 2014.
• Word Representations via Gaussian Embedding, 2014.
• Poincaré Embeddings for Learning Hierarchical Representations, 2017.
• Enriching Word Vectors with Subword Information, 2017.
• Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings, 2020.

Contextual Word Embedding

survey page https://github.com/tomohideshibata/BERT-related-papers.

• Deep Contextualized Word Representations, 2018.
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2019.

Interpretability

• The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddings, 2020.

Dynamic Word Embedding

Embedding Propery

• Towards Qualitative Word Embeddings Evaluation: Measuring Neighbors Variation, 2018.
• Deconstructing word embedding algorithms, 2020.
• Unsupervised Word Polysemy Quantification with Multiresolution Grids of Contextual Embeddings, 2020.
• Are All Good Word Vector Spaces Isomorphic?, 2020.
• 単語埋め込みによる論理演算, 2021.
• A Cluster-based Approach for Improving Isotropy in Contextual Embedding Space, 2021.
• Monitoring geometrical properties of word embeddings for detecting the emergence of new topics, 2021.

Word Sense Disambiguation

Survey

• Recent Trends in Word Sense Disambiguation: A Survey, 2021.

Knowledge Based

Vector Based

• Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation, 2019.
• With More Contexts Comes Better Performance: Contextualized Sense Embeddings for All-Round Word Sense Disambiguation, 2020.

Word Semantic Change Detection

• SemEval-2020 Task 1:Unsupervised Lexical Semantic Change Detection, 2020.
• Analysing Lexical Semantic Change with Contextualised Word Representations, 2020.

Application Use

• Contextualized Word Representations for Multi-Sense Embedding, 2018.
  • 多義語分散表現の文脈化, 2019.

Brain and Language

• Not all ambiguous words are created equal: An EEG investigation of homonymy and polysemy, 2012.

Open Lexical DataBase

Annotated Corpus

Survey

• A Short Survey on Sense-Annotated Corpora, 2020.

Corpora

semcor

Graphical Knowledge

• BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network, 2012.
• ConceptNet 5.5: An Open Multilingual Graph of General Knowledge, 2016.