I want to know if I can fine-tuning directly on the GPT-GNN model?

[yangxia605]

hi @acbull

that means I must take some categories for pre-training and then using another categories to fine-tuning?
I wonder if i can fine-tuning in the GPT-GNN model directly or not?like fine-tuning in the pre-trained bert,just modify the data to adjust their input format and then fine-tuning for the downstream task.

thanks.

[acbull]

Of course, you can directly pre-train and fine-tune on the same graph. The reason I adopt that in the experiment is to make the task harder and more realistic (in a real setting, we sometimes cannot see the fine-tuning data during pre-training).

[acbull]

Since you mentioned BERT, mostly the pre-training corpus and the fine-tuning corpus of BERT will come from a different domains, but BERT still shows a really impressive generalization ability. This is the reason why we evaluate our model in the three different transfer settings. But of course, you can directly fine-tune on the same graph.

[yangxia605]

I am very grateful for your prompt and patient answer.