How can I do graph regression with graphormer?

[Sangyoon-Bae]

Hi! this is Stella from Seoul National University.
I'd like to ask how can I implement regression task on Graphormer.
I adjusted ogb module for our data, and setted num_class as -1 like other regression datasets.
And I faced problem with editing model dimensions at model.py, line 62 ~ 75.

I think that 512*9+1 is something like vocabulary size, which is calculated by 512 * (number of categories of node features) + 1.
Is my guess right? And you said that it should be greater than the number of the class of all categories in issue #32, and how can I set this number in regression task? maybe number of graphs?

Thank you!

[zhengsx]

Hi, 9 is the number of node features (since there are 9 atom features for sample in pcqm4m dataset), and 512 is the maximum categories for each kind of feature. It's equivalent to seperately define 9 nn.Embedding(512, hidden_dim) for each feature. Hope this could address your question about the atom_encoder.

For another question about regression task, do you mean your label is a real value and the objective of model is to regress it? If so, you don't need to modify anything about the task. But if you mean that your feature is in continous space but not category feature, you could simply replace the nn.Embedding() from atom_encoder to a nn.Linear() or MLP.

[Sangyoon-Bae]

yes graph label is real value and objective of model is to regress it! then my node feature and edge feature are in continuous space. I'll try nn.Linear() ! I have another question, can I st edge feature and node feature as float type?

[zhengsx]

I think you can have a try, by the way, discretize the numerical features then using nn.Embedding() is also commonly used.

[Sangyoon-Bae]

Thanks for kind reply! Sorry for asking lots of question, I'd like to ask that should I change atom_encoder (line 63) , edge_encoder (line 64), out_proj (line 84), and downstream_out_proj (line 86) for regression task like pcqm4m-lsc?

[zhengsx]

If you want to use well-trained Graphormer (pre-trained on PCQM4M), you need to modify and use downstream_out_proj, otherwise you can use out_proj without modification. Whether atom_encoder and edge_encoder are needed to be modifed depends on your features as described in previous reply.

[Sangyoon-Bae]

Thanks a lot!! Your comment really helped me to train my model!!