Support for CSR matrix with `record_set()`

[harthur]

It's awesome that Estimator.record_set() handles data formatting and uploading to S3. However, in using the FactorizationMachines, the data is a sparse matrix and not a numpy array, e.g. <176304x550182 sparse matrix of type '<class 'numpy.float32'>' with 352608 stored elements in Compressed Sparse Row format>

When trying to use the record_set() API, I get an error:

fm = FactorizationMachines(role=get_execution_role(),
                           train_instance_count=2,
                           train_instance_type='ml.c5.2xlarge',
                           predictor_type='binary_classifier',
                           num_factors=64)

records = fm.record_set(X_train_encoded)
fm.fit(records)

> TypeError: must be real number, not csr_matrix

[laurenyu]

hi @harthur, thanks for bringing this to our attention! Marking this as a feature request - while I can't say how soon we can get to this, we're always reprioritizing our backlog based on customer feedback.

in the meantime, does calling toarray() on your csr_matrix get you past the error message?

[harthur]

@laurenyu I tried toarray() and it runs out of memory right away, since the input to Factorization Machines is very large sparse arrays.

[nadiaya]

Thanks for the update we will keep the feature request open.

[martinRenou]

Thank you for opening an issue and sorry for the late answer. It's true that record_set does not support sparse matrices today.

I am wondering if this other issue could be related in some way.

It seems record_set only uploads the matrix onto s3 then creates a RecordSet pointing to it. Would it work if you uploaded the sparse matrix and created the RecordSet yourself?

[ShwetaSingh801]

Hi @harthur,

Thanks for using SageMaker and taking the time to suggest ways to improve SageMaker Python SDK. We have added your feature request to our backlog of feature requests and may consider putting it into future SDK versions. I will go ahead and close the issue now, please let me know if you have any more feedback. Let me know if you have any other questions.

Best,
Shweta