New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multiple setup_rnn
calls
#70
Comments
I think it should actually work fine calling it multiple times. You will just need to call it before every time you want to use the RNN mode. |
Ah, I must've misread the code / the paper -- when we call Edit: I think I get it -- the discrete matrices setup during the |
While I have you, I'm attempting to understand the output of the Sashimi forward pass and how I'd do a simple cross entropy loss on the predictions. If I'm reading the Sashimi backbone right, in the UpPool blocks, we shift the inputs one over to the right (by padding left and removing the last element). Does that imply, if |
Yep, that's right!
Like any autoregressive model, training is done using "teacher forcing" or doing a cross entropy loss on the predictions Note that nothing about this is specific to Sashimi. This setup is the same as the ubiquitous "Language Modeling" (LM) task, and any sequence model can be used as long as causality is enforced (e.g. Transformers with the triangular causal mask). The shifting in the Sashimi UpPool blocks is to enforce causality as you said. |
Sweet, that lines up with my intuition. One last question for you: when trying to generate conditionally, my thought was to convert to RNN, initialize default state, then feed the conditioning sequence through the model to build up state non-auto-regressively (e.g. predict Does this line up with how one conditions S4 based models? |
That's right. This is how the generation script does it. |
Thank you, appreciate the quick response times. |
Hey!
I'm attempting to integrate the Sashimi Backbone into some audio models -- I'd like to train in convolutional mode and run validation inference in RNN mode, but my reading of the code seems to imply that the
setup_step
call isn't repeatable or reversible (#67 seems to imply this as well).In the case that I temporarily want to infer in RNN mode, but then switch back to the convolutional training mode, what's my best option?
The text was updated successfully, but these errors were encountered: