Question about the usage of helper in TransformerDecoder

[ha-lins]

Hi~
I want to implement the step-by-step TransformerDecoder with a TrainingHelper(), but I don't know how to call the same forward function as the RNN's, e.g.

outputs, hidden = self.gru(embedded, hidden) # forward for every step

Has it been done in the step method of the class helper ? Hope for your help!

[gpengzhi]

Thank you for your interest in Texar-PyTorch!

The forward function for every step (here) is implemented in TransformerDecoder.

You can take a look at self._inputs_to_outputs, where we have
outputs, state = self._inputs_to_outputs(inputs, state).

[ha-lins]

In fact, I have no idea of how the initialize/step function is called or works. And the step function requires the helper argument. What should I do to build such class of helper? Is there any corresponding example for this?

Thanks!

[huzecong]

Sorry but I'm a bit confused. Is your goal to write (from scratch) a new Helper class, or to use an alternative built-in helper with the decoder?

[ha-lins]

@huzecong Specifically, I want to modify an alternative helper (e.g. TrainingHelper) instead of writing (from scratch). So I may need a corresponding example to help me to understand and then modify.
For example, how to set the embedding_fn in the https://texar-pytorch.readthedocs.io/en/latest/code/modules.html#texar.torch.modules.TrainingHelper ?

[huzecong]

You don't need to do that yourself. helper.initialize is called inside decoder.initialize, where the decoder will pass its own embedding_fn to the helper.

The flow of execution would be:

1. Construct a helper.
2. Call decoder.forward with the constructed helper.
3. decoder.forward calls DecoderBase.dynamic_decode.
4. DecoderBase.dynamic_decode calls decoder.initialize, which in turns calls helper.initialize.
5. DecoderBase.dynamic_decode loops over each time step and calls decoder.step, which in turn calls helper.sample.
6. If the current step is not the final time step, DecoderBase.dynamic_decode calls decoder.next_inputs, which calls helper.next_inputs.
7. DecoderBase.dynamic_decode calls decoder.finalize.

This may not apply to every decoder--helper pair but is a general description of how things work.

[ha-lins]

Thanks for your detailed instruction!