BFDM: Continual Relation Extraction via Bidirectional Feedback and Dynamic Multi-teacher Knowledge Distillation

Continual Relation Extraction (CRE) aims to continuously learn new relations while preventing the forgetting of previously learned relations. However, existing methods still face significant challenges in effectively retaining and utilizing prior knowledge, and their performance is strongly influenced by the degree of task fusion. To address these challenges, we propose a CRE model based on \textbf{B}idirectional \textbf{F}eedback and \textbf{D}ynamic \textbf{M}ulti-teacher knowledge distillation (BFDM). The model significantly enhances the fusion of current and historical tasks through the bidirectional feedback module, allowing new task learning and memory replay learning to relearn using feedback from each other. Additionally, the dynamic multi-teacher knowledge distillation module saves the optimal parameters for each new task into separate teacher models, which are then sequentially distilled into the student model during the memory replay learning phase. This ensures accurate and consistent retention of prior knowledge. With average accuracy of 92.45% and 88.61% on the two benchmark datasets, BFDM demonstrates promising performance, highlighting its potential as a competitive state-of-the-art model. Ablation studies further validated the effectiveness of the BFDM model. Removing any module resulted in varying degrees of decline in average accuracy across both datasets. The maximum drop when removing a single module reached 1.18% and 1.37%, respectively.

---

Environment Setup

Our implementation is based on Python 3.9.21 and PyTorch 2.5.1 (CUDA 12.x). To upgrade to PyTorch 2.5.1, please follow the official installation guide from PyTorch, or execute the following command:

conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.4 -c pytorch -c nvidia

Subsequently, install the remaining dependencies via:

pip install -r requirements.txt

---

Datasets

We evaluate WSMC on two widely-used benchmarks: FewRel and TACRED. Pre-processed splits and task order files are available in the corresponding subdirectories under the data/ folder.

---

Reproducing Experiments

To reproduce the main experimental results, please run:

bash FewRel.sh 0
bash tacred.sh 0

All experiments are conducted on a single NVIDIA RTX 4090 GPU with 24 GB of memory.