run_ot and run_ot_write

[kirefu]

Hi there,

I am trying to reconcile your code with the description in Algorithm 1 of your paper.

In the paper:

entropy, vocab = get_vocab(optimal matrix)
vocabularies.append(entropy,vocab)
Output v∗ from vocabularies satisfying Eq. 3

VOLT/ot_run.py

Line 141 in c9f2e69

scores[iter_number] = Gs-previous_entropy

However, in the code for run_ot, the transport matrix or a vocabulary set for each timestep t is not stored, only the (vocab_size, entropy) pairs are.

Then run_ot_write() takes this optimal vocab size, and recalculates the transport matrix again, and I don't see how this is different from when it was calculated in the for loop with run_ot, surely the same matrix is outputted? I also don't understand how run_ot_write() is doing the same thing as "Output v∗ from vocabularies satisfying Eq. 3" from Algorithm 1, as there are no vocabs being taken into consideration.

Would be very grateful if you could help clarify the above, as I am keen to implement your work :)

[Jingjing-NLP]

Hi. Thanks for you attentions!

Yes! Theses are two equivalent operations. We store all transport matrices at the original version. Due to the large size of transport matrix, it takes a lot of memory usage. Therefore, we slightly change the implementation details. First, at each step, we get the transport matrix and calculate its Eq.3 score. We choose to save the Eq.3 score for each step, rather than transport matrix. Second, after we get the best score, we keep the related step and re-run the ot commands to recover its transport matrix.

Hope this can address your questions. If you have any other questions, please feel free to contact us.

[kirefu]

Thanks for your response. For clarification purposes:

"The inner arg max represents that the target is to find the vocabulary from V_S[t] with the maximum MUV scores. The outer arg max means that the target is to enumerate all timesteps and find the vocabulary with the maximum MUV scores."

Is the inner argmax the sinkhorn algorithm, and the outer argmax the for loop over the timesteps?

Best,
Faheem

[Jingjing-NLP]

Yes, your understanding is right! There are two argmax operations here. The outer argmax means the maximum value over all timesteps.

[kirefu]

Thanks!