SurveyPapers4LLM

This repository contains the surveys and analytical papers studying very large language models (LLMs) in various domains (All papers are collected before June 6th, 2023).

Surveying LLMs

1. On the Opportunities and Risks of Foundation Models.

   Rishi Bommasani, et al. [pdf]. 2022.7

2. A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?.

   Chaoning Zhang, Chenshuang Zhang, Sheng Zheng, Yu Qiao, Chenghao Li, Mengchun Zhang, Sumit Kumar Dam, Chu Myaet Thwal, Ye Lin Tun, Le Luang Huy, Donguk kim, Sung-Ho Bae, Lik-Hang Lee, Yang Yang, Heng Tao Shen, In So Kweon, Choong Seon Hong. [pdf]. 2022.7

3. Emergent Abilities of Large Language Models.

   Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, William Fedus. [pdf]. 2022.10

4. Towards Reasoning in Large Language Models: A Survey.

   Jie Huang, Kevin Chen-Chuan Chang. [pdf]. 2022.12

5. ChatGPT Applications, Opportunities, and Threats.

   Aram Bahrini, Mohammadsadra Khamoshifar, Hossein Abbasimehr, Robert J. Riggs, Maryam Esmaeili, Rastin Mastali Majdabadkohne, Morteza Pasehvar. [pdf]. 2023.4

6. Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond.

   Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, Xia Hu. [pdf]. 2023.4

7. A Bibliometric Review of Large Language Models Research from 2017 to 2023.

   Lizhou Fan, Lingyao Li, Zihui Ma, Sanggyu Lee, Huizi Yu, Libby Hemphill. [pdf]. 2023.4

8. Summary of ChatGPT/GPT-4 Research and Perspective Towards the Future of Large Language Models.

   Yiheng Liu, Tianle Han, Siyuan Ma, Jiayue Zhang, Yuanyuan Yang, Jiaming Tian, Hao He, Antong Li, Mengshen He, Zhengliang Liu, Zihao Wu, Dajiang Zhu, Xiang Li, Ning Qiang, Dingang Shen, Tianming Liu, Bao Ge. [pdf]. 2023.4

9. One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Era.

   Chaoning Zhang, Chenshuang Zhang, Chenghao Li, Yu Qiao, Sheng Zheng, Sumit Kumar Dam, Mengchun Zhang, Jung Uk Kim, Seong Tae Kim, Jinwoo Choi, Gyeong-Moon Park, Sung-Ho Bae, Lik-Hang Lee, Pan Hui, In So Kweon, Choong Seon Hong. [pdf]. 2023.4

10. A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT.

    Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S. Yu, Lichao Sun. [pdf]. 2023.4

11. A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT.

    Ce Zhou (1), Qian Li (2), Chen Li (2), Jun Yu (3), Yixin Liu (3), Guangjing Wang (1), Kai Zhang (3), Cheng Ji (2), Qiben Yan (1), Lifang He (3), Hao Peng (2), Jianxin Li (2), Jia Wu (4), Ziwei Liu (5), Pengtao Xie (6), Caiming Xiong (7), Jian Pei (8), Philip S. Yu (9), Lichao Sun (3) ((1) Michigan State University, (2) Beihang University, (3) Lehigh University, (4) Macquarie University, (5) Nanyang Technological University, (6) University of California San Diego, (7) Salesforce AI Research, (8) Duke University, (9) University of Illinois at Chicago). [pdf]. 2023.4

12. Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models.

    Emilio Ferrara. [pdf]. 2023.4

13. A PhD Student's Perspective on Research in NLP in the Era of Very Large Language Models.

    Oana Ignat, Zhijing Jin, Artem Abzaliev, Laura Biester, Santiago Castro, Naihao Deng, Xinyi Gao, Aylin Gunal, Jacky He, Ashkan Kazemi, Muhammad Khalifa, Namho Koh, Andrew Lee, Siyang Liu, Do June Min, Shinka Mori, Joan Nwatu, Veronica Perez-Rosas, Siqi Shen, Zekun Wang, Winston Wu, Rada Mihalcea. [pdf]. 2023.5

14. Large Language Models Humanize Technology.

    Pratyush Kumar [pdf], 2023.5

15. Open-world Story Generation with Structured Knowledge Enhancement: A Comprehensive Survey.

    Yuxin Wang, Jieru Lin, Zhiwei Yu, Wei Hu, Börje F. Karlsson. [pdf]. 2023.5

16. A Survey of Large Language Models.

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, Ji-Rong Wen. [pdf]. 2023.5

17. A Glimpse in ChatGPT Capabilities and its impact for AI research.

    Frank Joublin, Antonello Ceravola, Joerg Deigmoeller, Michael Gienger, Mathias Franzius, Julian Eggert. [pdf]. 2023.5

18. GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models.

    Tyna Eloundou, Sam Manning, Pamela Mishkin, Daniel Rock. [pdf]. 2023.5

Analyzing LLMs

1. Do Prompt-Based Models Really Understand the Meaning of Their Prompts.

   Albert Webson, Ellie Pavlick. [pdf]. 2022.4

2. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?.

   Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, Luke Zettlemoyer. [pdf]. 2022.10

3. Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints.

   Albert Lu, Hongxin Zhang, Yanzhe Zhang, Xuezhi Wang, Diyi Yang. [pdf]. 2023.2

4. Can Large Language Models Be an Alternative to Human Evaluations?.

   Cheng-Han Chiang, Hung-yi Lee. [pdf]. 2023.2

5. Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies.

   Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, Dragomir Radev. [pdf]. 2023.5

6. OPT-R: Exploring the Role of Explanations in Finetuning and Prompting for Reasoning Skills of Large Language Models.

   Badr AlKhamissi, Siddharth Verma, Ping Yu, Zhijing Jin, Asli Celikyilmaz, Mona Diab. [pdf]. 2023.5

7. Observations on LLMs for Telecom Domain: Capabilities and Limitations.

   Sumit Soman, Ranjani H G. [pdf]. 2023.5

8. ChatGPT Is More Likely to Be Perceived as Male Than Female.

   Jared Wong, Jin Kim. [pdf]. 2023.5

9. Re-visiting Automated Topic Model Evaluation with Large Language Models.

   Dominik Stammbach, Vilém Zouhar, Alexander Hoyle, Mrinmaya Sachan, Elliott Ash. [pdf]. 2023.5

10. Large Language Models can be Guided to Evade AI-Generated Text Detection.

    Ning Lu, Shengcai Liu, Rui He, Qi Wang, Ke Tang. [pdf]. 2023.5

11. Prompt-based methods may underestimate large language models' linguistic generalizations.

    Jennifer Hu, Roger Levy. [pdf]. 2023.5

12. How do languages influence each other? Studying cross-lingual data sharing during LLM fine-tuning.

    Rochelle Choenni, Dan Garrette, Ekaterina Shutova. [pdf]. 2023.5

13. Evaluating ChatGPT's Performance for Multilingual and Emoji-based Hate Speech Detection.

    Mithun Das, Saurabh Kumar Pandey, Animesh Mukherjee. [pdf]. 2023.5

14. Don’t Trust GPT When Your Question Is Not In English.

    Xiang Zhang, Senyu Li, Bradley Hauer, Ning Shi, Grzegorz Kondrak. [pdf]. 2023.5

15. Self-contradictory Hallucinations of Large Language Models Evaluation, Detection and Mitigation.

    Niels Mündler, Jingxuan He, Slobodan Jenko, Martin Vechev. [pdf]. 2023.5

16. Emergence of a phonological bias in ChatGPT.

    Juan Manuel Toro. [pdf]. 2023.5

17. ChatGPT an ENFJ, Bard an ISTJ: Empirical Study on Personalities of Large Language Models.

Jen-tse Huang, Wenxuan Wang, Man Ho Lam, Eric John Li, Wenxiang Jiao, Michael R. Lyu. [pdf]. 2023.5

18. Exploring Lottery Prompts for Pre-trained Language Models.

Yulin Chen, Ning Ding, Xiaobin Wang, Shengding Hu, Hai-Tao Zheng, Zhiyuan Liu, Pengjun Xie. [pdf]. 2023.6

19. Is ChatGPT a Good Teacher Coach? Measuring Zero-Shot Performance For Scoring and Providing Actionable Insights on Classroom Instruction.

    Rose E. Wang, Dorottya Demszky. [pdf]. 2023.6

20. Taught by the Internet, Exploring Bias in OpenAIs GPT3.

    Ali Ayaz, Aditya Nawalgaria, Ruilian Yin. [pdf]. 2023.6