Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BatchFeature: Convert List[np.ndarray] to np.ndarray before converting to pytorch tensors #14306

Merged
merged 7 commits into from
Nov 10, 2021
Merged

BatchFeature: Convert List[np.ndarray] to np.ndarray before converting to pytorch tensors #14306

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
68e3756
update
eladsegal Nov 7, 2021
05fed8b
style fix
eladsegal Nov 7, 2021
1f69c7f
retrigger checks
eladsegal Nov 8, 2021
5e3fbae
check first element
eladsegal Nov 8, 2021
a3cc3fc
fix syntax error
eladsegal Nov 8, 2021
c6f43b5
Update src/transformers/feature_extraction_utils.py
eladsegal Nov 10, 2021
a699a36
remove import
eladsegal Nov 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion src/transformers/feature_extraction_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,11 @@ def convert_to_tensors(self, tensor_type: Optional[Union[str, TensorType]] = Non
raise ImportError("Unable to convert output to PyTorch tensors format, PyTorch is not installed.")
import torch

as_tensor = torch.tensor
def as_tensor(value):
if isinstance(value, (list, tuple)) and len(value) > 0 and isinstance(value[0], np.ndarray):
value = np.array(value)
return torch.tensor(value)

is_tensor = torch.is_tensor
elif tensor_type == TensorType.JAX:
if not is_flax_available():
Expand Down