Add t5 adapter #182
Add t5 adapter #182
Conversation
… is_decoder is also true.
…orConditionalGeneration.
|
@AmirAktify thanks for working on this! My research team was looking to use adapters with T5 and we found this branch you've been working on. Do you think this is in a usable stage? Thanks! |
Hi Deniz. I was running some tests yesterday with sequence classification with T5ForConditionalGeneration, and I ran into some minor errors with the base model. Lemme try and debug those today, but I am hopefully very close. |
@AmirAktify That's great to hear. Thanks for letting me know so quickly. Good luck! |
Done, thank you. |
|
Thanks for finishing this PR btw! I hope it wasn't quite as bad as just doing it from scratch. |
There was a problem hiding this comment.
Looks nice overall :)
Besides the line-specific comments, two general points from the implementation guide document still seem to be missing (from section Testing):
- Insert test_modeling_<model_type> into the list of tested modules in utils/run_tests.py.
- Append <model_type> to the list in check_adapters.py.
|
nice works! |
Followed the pattern of Bart to add adapters to T5. One change is that whereas Bart has separate classes for encoder and decoder, T5 does not. So I am using the
is_decoderfor changes between encoder and decoder classes, such as adding cross_attention adapters and adding invertible adapters.I'm working on some testing.