Natural Language Processing (NLP) has achieved great progress in the past decade on the basis of neural models, which often make use of large amounts of labeled data to achieve state-of-the-art performance. The dependence on labeled data prevents NLP models from being applied to low-resource settings and languages because of the time, money, and expertise that is often required to label massive amounts of textual data. Consequently, the ability to learn with limited labeled data is crucial for deploying neural systems to real-world NLP applications. Recently, numerous approaches have been explored to alleviate the need for labeled data in NLP such as data augmentation and semi-supervised learning. This tutorial aims to provide a systematic and up-to-date overview of these methods in order to help researchers and practitioners understand the landscape of approaches and the challenges associated with learning from limited labeled data, an emerging topic in the computational linguistics community. We will consider applications to a wide variety of NLP tasks (including text classification, generation, and structured prediction) and will highlight current challenges and future directions.
Diyi Yang, Georgia Institute of Technology
Ankur Parikh, Google Research
Colin Raffel, University of North Carolina Chapel Hill/Hugging Face
9:30am - 1:00pm, Dublin Timezone, May 22nd, 2022
On-site Room: The Liffey A
Virtual Link: See Underline Tutorial 5 for the Zoom link
Edunov, Sergey, Myle Ott, Michael Auli, and David Grangier. "Understanding back-translation at scale." arXiv preprint arXiv:1808.09381 (2018).
Lample, Guillaume, and Alexis Conneau. "Cross-lingual language model pretraining." arXiv preprint arXiv:1901.07291 (2019).
Chen, Jiaao, Zichao Yang, and Diyi Yang. "Mixtext: Linguistically-informed interpolation of hidden space for semi-supervised text classification." arXiv preprint arXiv:2004.12239 (2020).
Morris, John X., Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. "Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp." arXiv preprint arXiv:2005.05909 (2020).
Chau, Ethan C., Lucy H. Lin, and Noah A. Smith. "Parsing with multilingual BERT, a small corpus, and a small treebank." arXiv preprint arXiv:2009.14124 (2020).
Du, Jingfei, Edouard Grave, Beliz Gunel, Vishrav Chaudhary, Onur Celebi, Michael Auli, Ves Stoyanov, and Alexis Conneau. "Self-training improves pre-training for natural language understanding." arXiv preprint arXiv:2010.02194 (2020).
Chen, Jiaao, Derek Tam, Colin Raffel, Mohit Bansal, and Diyi Yang. "An empirical survey of data augmentation for limited data learning in nlp." arXiv preprint arXiv:2106.07499 (2021).