Is this repo runnable?

[korcys]

There seems some bugs in this repo. For instance, the parameter of nn.TransformerEncoderLayer d_model in line 30 of transformer.py is the number of expected features in the input which must be divisible by num_heads. But in this repo, the d_model is set as NUM_BOX_FEATURES 109, and num_heads is 4. (109 % 4 eq 1)

[Git-oNmE]

I also met the same issue. Have you fixed this problem? Look forward to your reply.

[YichaoLu]

As described in the paper, when num_features is not divisible by num_heads, a fully-connected layer is applied to project num_features into d_model. If you look at the hyperparameter setting for the commit you are referring to, you will find nhead is set to 1 so no fully-connected layer is required. The latest main branch shows the example of adding the fully-connected layer so you can now vary the value of nhead.