fix bug in batched_forward_pass#144
Conversation
There was a problem hiding this comment.
Wow thanks a lot for fixing the bug!
I am ok with this fix in the principle that it is a safety checker to make sure all the returned tensors will be on the correct device (regardless where the dataloader will send the device)
Let's run the tests and see!
Wdyt @lvwerra ?
@ArvinZhuang can you run the styling and quality checks? (make style && make quality) Thanks!
| [{"input_ids": q, "attention_mask": torch.ones_like(q)} for q in queries] | ||
| ).to(self.accelerator.device) | ||
|
|
||
| attention_mask = [torch.ones_like(r) for r in responses] |
There was a problem hiding this comment.
Hum why this has been removed? 🤔
There was a problem hiding this comment.
because seems attention_mask variable is never used
There was a problem hiding this comment.
Perfect, can you run the styling checks so that the testing suite will be executed?
|
The documentation is not available anymore as the PR was closed or merged. |
younesbelkada
left a comment
younesbelkada
left a comment
There was a problem hiding this comment.
Thanks for fixing! Could you revert your changes in all the scripts inside examples/ after that we should be good to merge!
Hi |
younesbelkada
left a comment
younesbelkada
left a comment
There was a problem hiding this comment.
Thanks for iterating! 🚀
|
Also I notice this |
|
Had just prepared a PR to fix this 😄 thanks @ArvinZhuang |
@Rebecca-Qian You are welcome :) Im glad to contribute as well |
* fix bug in batched_forward_pass * style and quality * revert examples
5 participants
fix the bug that will cause the devices not match issue in the
batched_forward_passmethod.The reason:
self. data_collatorreturned tensors will be on CPU, thus the laterself.model(**input_kwargs)will give error as model is on GPU.The proposed solution:
do
.to(self.accelerator.device)afterself.data_collatorMy transformers version: 4.26.1