s3_input Input Validation for Content Type

[ChandraLingam]

When creating a S3 input configuration, there is no clear documentation on what value needs to sent in content_type parameter. Some documents use "text/csv" convention. Some documents are simply referring as "csv". If there is any mistake, it takes 5-6 minutes for SageMaker to respond back for a simple configuration problem. It will be good if SDK can provide some properties that we can use instead of hard-coding values.

https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-training.html

sagemaker.session.s3_input(s3_data=s3_training_file_location,content_type="csv")

[yangaws]

Thanks for your suggestion.

I have added this to our backlog. And we will work to improve documentation on this part.

I will label this as feature request. Thanks for using sagemaker.

[ChandraLingam]

Thanks! In the meanwhile, could you please clarify how to specify protobuf recordio content type? What text should I use?
sagemaker.session.s3_input(s3_data=s3_training_file_location,content_type="csv")

[yangaws]

Hi @ChandraLingam ,

The content type for training in protobuf recordio is 'application/x-recordio-protobuf'

But as we discussed in other issues, I assume you are doing xgboost. Then protobuf recordio data is not supported. Documentation: https://docs.aws.amazon.com/sagemaker/latest/dg/xgboost.html

So for xgboost, we only support csv and libsvm for both training and inference.

Thanks

[ChandraLingam]

@yangaws - Thank you! Understood about XGBoost. I was planning to use recordio with PCA.

Thanks!
Chandra

[GeorgeMadlis]

Is 'application/x-recordio-protobuf' supported by Tensorflow? I am getting an error: 'Requested unsupported ContentType: Unsupported content-type application/x-recordio-protobuf', when I use transformer.transform(input, content_type = 'application/x-recordio-protobuf')

[elgalu]

Which content types are supported? docs doesn't say much "MIME type of the input data (default: None)." https://sagemaker.readthedocs.io/en/stable/session.html

[mohanasudhan]

@ChandraLingam @elgalu Apologies for the delay in responding to this issue. The following document covers the common data formats supported in training and inference.

However, some containers only support specific data formats. We have introduced ModelBuilder which handles the data formats from the schema (examples). Additionally, you can also pass custom data formats using custom handlers. (refer https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-modelbuilder-creation.html)

[mohanasudhan]

We are closing this issue as we currently lack sufficient information to proceed. Please feel free to reopen the issue if you can provide more details.