Language Models Implement Simple Word2Vec-style Vector Arithmetic

by Jack Merullo, Carsten Eickhoff, and Ellie Pavlick

Paper URL: https://arxiv.org/abs/2305.16130

BIGBench dataset links:

Past Tense (Linguistic Mappings)

• Past tense regular and past tense irregular

Uppercasing (Reasoning About Colored Objects)

• We use the first 200 examples of the colored objects data

Please cite as:

@misc{merullo2023language,
      title={Language Models Implement Simple Word2Vec-style Vector Arithmetic}, 
      author={Jack Merullo and Carsten Eickhoff and Ellie Pavlick},
      year={2023},
      eprint={2305.16130},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}