New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Speech] Refactor Examples #14040
[Speech] Refactor Examples #14040
Conversation
Wait until #14026 (comment) is fixed |
@@ -476,6 +476,8 @@ | |||
# Model for Audio Classification mapping | |||
("wav2vec2", "Wav2Vec2ForSequenceClassification"), | |||
("hubert", "HubertForSequenceClassification"), | |||
("sew", "SEWForSequenceClassification"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similar to how all BERT heads (ForQA, ForSequenceClass, ForMC, ...) are added to all BERT-like models for easy comparison and added functionality, all speech models should have the superb heads.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, agreed!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is great! 100% agree that all models should have the SUPERB heads
@@ -476,6 +476,8 @@ | |||
# Model for Audio Classification mapping | |||
("wav2vec2", "Wav2Vec2ForSequenceClassification"), | |||
("hubert", "HubertForSequenceClassification"), | |||
("sew", "SEWForSequenceClassification"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, agreed!
output = (logits,) + outputs[1:] | ||
output = (logits,) + outputs[_HIDDEN_STATES_START_POSITION:] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is way simpler to understand! We should do something like that for BERT & friends too
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is great! Thanks a lot for refactoring this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks very clean now 👍
@@ -1141,8 +1124,8 @@ def forward( | |||
""", | |||
HUBERT_START_DOCSTRING, | |||
) | |||
# Copied from transformers.models.wav2vec2.modeling_wav2vec2.Wav2Vec2ForSequenceClassification with Wav2Vec2->Hubert, wav2vec2->hubert, WAV_2_VEC_2->HUBERT |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great that it works now!
…into refactor_speech_heads
* adapt_examples * up * up * up * up * add auto models * finish
What does this PR do?
This PR adapts all Wav2Vec2-like models to use the same examples and makes sure that
...ForCTC
and...ForSequenceClassification
have exactly the same structure. HuBERT, SEW, SEW-D and soon UniSpeech and others that are based on Wav2Vec2 usually don't have any specific heads, but rather should just follow the Wav2Vec2 head design of CTC and the Superb head design for SequenceClassification (SpeakerVerification, ....). Therefore we should make it as easy as possible to add such heads to new Wav2Vec2 versions.This PR makes sure that a simple
#Copied from ...
command can be used for such heads which should allow us to work faster when adding new speech models while making sure the design is unified and correct.Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.