[2024.3] Our basic framework for downstream is based on SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition, Many thanks to this work for providing a strong baseline.
The main code for task specific pre-training is open sourced. Unfortunately, our code for the prototype decomposition part has been cleaned up by others. You can reproduce it based on the detailed description of our paper or contact with me for guidence. We are willing to offer help for reproduction.
This repository contains the open-sourced official implementation of the paper:
A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NER (CIKM 2023 Oral Presentation).
If you find this repo helpful, please cite the following paper:
@misc{dong2023multitask,
title={A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NER},
author={Guanting Dong and Zechen Wang and Jinxu Zhao and Gang Zhao and Daichi Guo and Dayuan Fu and Tingfeng Hui and Chen Zeng and Keqing He and Xuefeng Li and Liwen Wang and Xinyue Cui and Weiran Xu},
year={2023},
eprint={2308.14533},
archivePrefix={arXiv},
primaryClass={cs.CL}
}we propose a Multi-Task Semantic Decomposition Framework via Joint Task-specific Pre-training (MSDP) for few-shot NER. Drawing inspiration from demonstration-based and contrastive learning, we introduce two novel pre-training tasks: Demonstration-based Masked Language Modeling (MLM) and Class Contrastive Discrimination. These tasks effectively incorporate entity boundary information and enhance entity representation in Pre-trained Language Models (PLMs). In the downstream main task, we introduce a multi-task joint optimization framework with the semantic decomposing method, which facilitates the model to integrate two different semantic information for entity classification. Experimental results of two few-shot NER benchmarks demonstrate that MSDP consistently outperforms strong baselines by a large margin. Extensive analyses validate the effectiveness and generalization of MSDP.
