Corner Case - One input feature and prediction

[ChandraLingam]

There is an issue with csv serializer when running prediction on a model that has only one input feature. Serializer or the SageMaker Prediction endpoint is not able to handle single feature models when running predictions. I used CSV for inference format.

I don't have the exact error message that was reported by the serializer; wanted to report this issue.

I trained the data with XGBoost algorithm

Data set consisted of just X, y
X, y
1,1
2,4
3,9
....

[yangaws]

Thanks for using sagemaker!

We will try reproducing the error first. If you have error logs that can be shared, it would be great!

Thanks

[yangaws]

Hi @ChandraLingam ,

Since we have more communications in other issues, I have a better idea of what you tried to do with sagemaker now. Thanks for contributing to us.

But still, for this problem, I need to confirm on many things that will help me reproduce it.

I assume, what you are doing is:

1, Train a xgboost model using sagemaker estimator. The training data is a csv with one column of label and one column of feature.

2, Used the model trained from the previous step to predict on a csv dataset with only one column of feature. And this step gets error.

Correct me if there are inaccuracies above. And if you could provide following things, it would be really helpful:

1, Could you share some codes for how you train and predict on csv data? Especially what do your csv dataset look like? How do you pass it for training and prediction?

2, Could you share some error messages or descriptions? Is that model error, server error, algorithm error, etc?

Thanks

[ChandraLingam]

Training (relevant section):
# content type can be libsvm or csv for XGBoost
training_input_config = sagemaker.session.s3_input(s3_data=s3_training_file_location,content_type="csv")
validation_input_config = sagemaker.session.s3_input(s3_data=s3_validation_file_location,content_type="csv")

# fit
estimator.fit({'train':training_input_config, 'validation':validation_input_config})

# deploy
predictor = estimator.deploy(initial_instance_count=1, instance_type='ml.m4.xlarge',                            endpoint_name = 'my-endpoint')

# predict
arr_test = df.as_matrix(["x"])

type(arr_test)
>>numpy.ndarray

arr_test.dtype
>>dtype('float64')

arr_test[:5]

>>
array([[ 4.43986342],
       [17.41464612],
       [ 4.13438311],
       [18.37221816],
       [ 9.76822378]])


# predict
result = predictor.predict(arr_test[:5])

[ChandraLingam]

Error on client:
---------------------------------------------------------------------------
ModelError                                Traceback (most recent call last)
<ipython-input-26-3c07c053f185> in <module>()
----> 1 result = predictor.predict(arr_test[:5])

~/anaconda3/envs/python3/lib/python3.6/site-packages/sagemaker/predictor.py in predict(self, data)
     84             request_args['Accept'] = self.accept
     85 
---> 86         response = self.sagemaker_session.sagemaker_runtime_client.invoke_endpoint(**request_args)
     87 
     88         response_body = response['Body']

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/client.py in _api_call(self, *args, **kwargs)
    312                     "%s() only accepts keyword arguments." % py_operation_name)
    313             # The "self" in this scope is referring to the BaseClient.
--> 314             return self._make_api_call(operation_name, kwargs)
    315 
    316         _api_call.__name__ = str(py_operation_name)

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/client.py in _make_api_call(self, operation_name, api_params)
    610             error_code = parsed_response.get("Error", {}).get("Code")
    611             error_class = self.exceptions.from_code(error_code)
--> 612             raise error_class(parsed_response, operation_name)
    613         else:
    614             return parsed_response

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (415) from model with message "Loading csv data failed with Exception:
 <class 'xgboost.exceptions.CustomerError'>
 DataFrame.dtypes for data must be int, float or bool.
Did not expect the data types in fields index 1".

[ChandraLingam]

CloudWatch Log has this error:
Exception AttributeError: "'DMatrix' object has no attribute 'handle'" in <bound method DMatrix.__del__ of <xgboost.core.DMatrix object at 0x7fa325d77390>> ignored

[yangaws]

Hi @ChandraLingam ,

I reproduced the modelError by training and inference on single feature csv data. We are working on this. Thanks for your info.

[trungleduc]

Closes as not applicable, the data argument of predictor.predict method does not accept a Numpy array anymore.

    data (object): Input data for which you want the model to provide
        inference. If a serializer was specified when creating the
        Predictor, the result of the serializer is sent as input
        data. Otherwise the data must be sequence of bytes, and the
        predict method then sends the bytes in the request body as is.