Which line is the code of Meta learning in Decomposed meta NER

[dongguanting]

May I ask which line is the related code of the prototype network and MAML? I read your code carefully but do not find them.

[iofu728]

Hi @dongguanting, in fact, the whole logic can be found by analyzing the running script.
You can find MAML logic in forward_meta function, both inner loop and outer loop.
You can also find ProtoNet in the forwad_wuq function https://github.com/microsoft/vert-papers/blob/master/papers/DecomposedMetaNER/modeling.py#L125. We use a packaged nn.embedding class EntityTypes to memorize type embeddings.

[dongguanting]

Hi @iofu728, Thank you for your answer! But I still have another question which bothers me. I find that the model will backward twice during forward_meta function, namely inner update function and outer forward_wuqh. I think it may be related to MAML method, but why split into two processes to backward?

[iofu728]

Hi @dongguanting, this is how MAML does it. You can refer the MAML paper or other tutorials like AAAI21 MetaLearning Tutorial.

In short, for the inner update part, the model fine-tunes specific task i data based on original model parameter $\theta$ (Inner update bp). After an inner update step in the meta-train dataset, the model will store each loss of the meat-test dataset. At the end of each inner update step, the model recovers to the original parameter $\theta$.
The output update bp will calculate after all of the tasks' inner updates. The second-order derivative will make the model pay more attention to the transfer of knowledge between different tasks.

[wjczf123]

Hi, I also have similar question. Is there any parameter to control meta-learning? I want to reproduce the results of 1) Ours w/o MAML.

[iofu728]

Hi @wjczf123, yeah, the code also supports full supervision mode(w/o MAML). You can set the use_supervise argument to True, which will call the forward_supervise function instead of forward_meta.

[wjczf123]

Nice! Thank you very much.