fine-tuning after post-trained

[MrGGLS]

Hi : )

I'm trying to introduce CL into CGEC (Chinese GEC) task. You used post-trained model (trained on non-native learner data) and then fine-tune (trained on native leaner data) with two strategies (NLL & CL) if I remember correctly...

I did the almost same steps use NLL strategy, but the fine-tuned model got a lower score than post-trained model in test-set (their $F_{0.5}$ scores were 5 and 25 respectively)

I think the reason might be the different data distribution, and I wanna know how you can make NLL better than DI (in paper)

I hope I made myself clear.

thanks

[MichaelCaohn]

Hi,

Thank you for the question. I think you might need to re-tune the hyper-parameters to make NLL better than DI. Also, as far as I know, the Chinese GEC needs to go through some segmentation process, which may also affect the performance of NLL.

Ideally, NLL should be better than DI.

Hope this addresses your concern

[MrGGLS]

I got it!!!