Matchmaker - Mismatch Between Model forward() Inputs and Items Generated by Train

[BananaLee]

I've run my implementation of TK for an epoch and the results are worse than completely random weights (i.e. just running it on 10 iterations over 5 epochs, I get ~0.1 MPP@10 whereas running 1 epoch on the full training set gives me 0.01). I assume it has to do with an error in my implementation, so I went into a really deep dive of the matchmaker implementation to see what I'm doing wrong and I'm getting really confused right now.

The input produced by train.py in matchmaker is pos_in += [batch["query_tokens"],batch["doc_pos_tokens"]] followed by output_pos = model.forward(*pos_in, use_fp16 = use_fp16)

Now, every model we are supposed to look at has a forward that takes in four required inputs, none of which are use_fp16. E.g.
def forward(self, query_embeddings: torch.Tensor, document_embeddings: torch.Tensor, query_mask: torch.Tensor, document_mask: torch.Tensor, output_secondary_output: bool = False) -> torch.Tensor:

Even if feeding a tensor argument, I can't see how adding two dictionaries (or even their relevant tensors together) could produce four separate inputs. My pytorch is really rusty as well and unfortunately, none of the pytorch tutorials really help in explaining this.

Given I've already spent 8 hours trying to understand this, I'd appreciate some help on that front as the lecture literally only mentions masking once in passing as though it were a trivial matter.

[sophiaalthammer]

Hello,

to your questions:

• If you want to use lower precision training with fp16 then you can add that as argument to every forward function, if not then just delete it in the train.py. However there is no mentioning of fp16 in the assignment.
• As given in the assignment, the forward function of the tk model should only take as inputs the following: forward(self, query: Dict[str, torch.Tensor], document: Dict[str, torch.Tensor]) -> torch.Tensor. It seems to me that you are not looking at the assignment, but at the implementation solution from the paper. From the given query and document token input you can also derive the query and document mask within the forward function, therefore you only need the two inputs query and document.

Best,
Sophia