Couldn't process request when using "automatic-speech-recognition" pipeline on SageMaker #19743
Comments
|
Hi @wildgeece96 . The If so, then it seems the bug comes from somewhere around sagemaker where the numpy array gets converted to a list. I am tentatively against adding support for lists instead of numpy arrays:
That being said there could be workaround probably: Couldn't find better code fast with my google fu, but it's probably doable to create a Wav like buffer with minimal reallocations. |
|
I confirmed inference code like below works from transformers import pipeline
from transformers.pipelines import AutomaticSpeechRecognitionPipeline
import numpy as np
def model_fn(model_dir) -> AutomaticSpeechRecognitionPipeline:
return pipeline(model="facebook/wav2vec2-base-960h")
def predict_fn(data, pipeline):
inputs = data.pop("inputs", data)
parameters = data.pop("parameters", None)
if type(inputs) == list:
inputs = np.array(inputs, dtype=np.float)
print("inputs are: ", inputs)
# pass inputs with all kwargs in data
if parameters is not None:
prediction = pipeline(inputs, **parameters)
else:
prediction = pipeline(inputs)
return prediction |
|
Thanks @Narsil . Actually, in my use case, I deployed wav2vec model on SageMaker, and when I send request via SageMaker SDK seriealizer of SageMaker (like JSONSerializer, NumPySerializer) serialize the input to throw request to the endpoint. I should use JSONSerialization to use SageMaker HuggingFace Inference Toolkit and JSONSeiralizer cannot pass ndarray as it is but convert to list. After reading your comment, the converting logic should be implemented on SageMaker HuggingFace Inference Toolkit because it's specific for SageMaker use case. |
|
Hello @wildgeece96 the |
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
System Info
transformers
torch
sagemaker
Who can help?
@Narsil
@patrickvonplaten
@anton-l
Information
Tasks
examplesfolder (such as GLUE/SQuAD, ...)Reproduction
Run below code on SageMaker.
Returned InternalServerError,
Expected behavior
When I use Transformers on SageMaker, I noticed that Automatic Speech Recognition Pipeline doesn't consider receiving requests when deployed on SageMaker.
When we use SageMaker HuggingFace Inference Toolkit, pipelines will be used for inference.
AutomaticSpeechRecognitionPipeline doesn't accept list as
inputsparameter for__call__method and via API the request body is supposed to be like,{ "inputs": np.array([1., 2., 3., 4.])) }but I cannot pass ndarray via JSON Serializer I can only pass list.
To solve that problem, pipeline should accept list as inputs.
return like
{'text': 'UROUND ME ON YOU E'}The text was updated successfully, but these errors were encountered: