Add FT-Transformer model
#12
Conversation
| self, | ||
| channels: int, | ||
| # Arguments for Transformer | ||
| num_layers: int = 4, |
There was a problem hiding this comment.
QQ: Does the paper use 4 encoder layers for one layer of FTTransformerConv? This number seems a little bit large to me.
There was a problem hiding this comment.
Good point. I did not check the paper. will set it according to the paper.
There was a problem hiding this comment.
read the original paper, and set num_layers to 3, nhead to 8.
There was a problem hiding this comment.
I see. Another QQ num_layers 3 and nhead 8 is just for one layer of ft transformer or for all the ft transformer?
There was a problem hiding this comment.
full ft transformer.
There was a problem hiding this comment.
That makes sense. I just renamed it into FTTransformer.
There was a problem hiding this comment.
BTW, should we create another folder such as nn/models (like we have in PyG) instead of putting to nn/conv? cc @rusty1s
There was a problem hiding this comment.
I am open. For now, I think we can just treat it as conv and revisit this later.
No description provided.