[Generation] Allow inputs_embeds as an input
#14443
Conversation
tests/test_generation_utils.py
Outdated
| inputs_embeds = model.get_input_embeddings()(input_ids) | ||
|
|
||
| # cannot generate from `inputs_embeds` for decoder only | ||
| with pytest.raises(ValueError): |
There was a problem hiding this comment.
Decoder-only can't generate with inputs_embeds since predicted ids are append to starting ids => so it's assumed that starting ids are input_ids and not inputs_embeds
There was a problem hiding this comment.
With not use self.assertRaises(ValueError) here?
tests/test_generation_utils.py
Outdated
| inputs_embeds = model.get_input_embeddings()(input_ids) | ||
|
|
||
| # cannot generate from `inputs_embeds` for decoder only | ||
| with pytest.raises(ValueError): |
There was a problem hiding this comment.
With not use self.assertRaises(ValueError) here?
| torch_device | ||
| ) | ||
| model.config.eos_token_id = None | ||
| input_ids = tokenizer(article, return_tensors="pt").input_ids.to(torch_device) |
There was a problem hiding this comment.
What's the length of this ?
Without the information it's hard to infer if self.assertEqual(output_sequences.shape, (1, 5)) is actually correct.
(I know it's a encoder-decoder but it gets important in the other decoder only test.)
Maybe as additional guarantee we can force the decoder_input_ids (To prove to reader that some tokens were generated ?)
There was a problem hiding this comment.
Well it comes pretty much only from max_length=5
There was a problem hiding this comment.
added a comment
| model = BartForConditionalGeneration.from_pretrained("hf-internal-testing/tiny-random-bart", max_length=5).to( | ||
| torch_device | ||
| ) | ||
| model.config.eos_token_id = None |
There was a problem hiding this comment.
Why is that necessary for this test? is it so EOS is not produced before max_length?
Maybe add a small comment where this is present ?
There was a problem hiding this comment.
Just to make sure it can't finish before hitting max_length
…ckvonplaten/transformers into fix_inputs_embeds_generate
What does this PR do?
This PR allows
inputs_embedsto be used as an input argument forgenerate(). Fixes: #12218Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.