Potentially silly question: What's the function for writing dgCMatrix to svmLight format?

[zachmayer]

I have a dgCMatrix that I want to send to python. writeMM from the Matrix package is taking forever, so I wanted to try your svmlight writer. Which function should I use?

Thanks! This package looks amazing!

[Laurae2]

My SVMLight converter is not yet implemented in this repo, currently it is separated onto another package (sparsity). Install this package: https://github.com/Laurae2/sparsity

one line install in R:

devtools:::install_github("Laurae2/sparsity")

It requires Rcpp installed beforehand for compilation. If my package complains about Rcpp not being installed, install all the packages (Rcpp) you see on the error printed when trying to install my sparsity package.

Once you installed that package, use:

write.svmlight(<your dgCMatrix>, <your label>, <your filename>)

like this:

#install.packages(c("devtools", "Rcpp", "Matrix"))
devtools:::install_github("Laurae2/sparsity")
library(Matrix)
library(sparsity)
my_mat <- matrix(c(rep(0, 250), rep(1, 250)), nrow = 50) # Creates a matrix with 500 elements of 50 rows and 10 columns
my_mat <- Matrix(my_mat, sparse = TRUE) # Turns to dgCMatrix
my_label <- c(rep(0, 25), rep(1, 25)) # Create label variable (for supervised learning)
write.svmlight(my_mat, my_label, "/home/my_mat.svmlight") # export to SVMLight format for further usage in Python / any other software/programming language

If you have an exotic OS configuration (Solaris, etc.), the installation might not work. So far it works on Windows / Ubuntu 14.xx / Ubuntu 16.xx.

Warning: Columns are zero-based. If you need one-based columns (which should not happen if you use Python, as typically it wants zero-based column - sklearn loads as zero-based with row-compressed format), use the fork from IDEXX found here: https://github.com/IDEXX/sparsity

You can install it by:

devtools:::install_github("IDEXX/sparsity")

If it works: if you can provide your conversion time on your data set + its size it would be great to add to the repository to add it as a benchmark (time to convert + dimensions + file size) - feel free to make a PR if you want to add benchmarks, they are welcome.

If it does not work: provide the error messages.

[zachmayer]

Will it work if I don't have a label?

[Laurae2]

It will not work if you don't provide a label, as the SVMLight format is enforced and requires the data to be in this form:

<target> <feature>:<value> <feature>:<value> ... <feature>:<value> # <info>

You can put a dummy label vector (one solution), or edit the source of code (another solution) to remove the part which adds the label.

numeric(nrow(my_mat)) should fill the label perfectly.

[zachmayer]

I'll use a dummy label. Thanks a lot!

[zachmayer]

That works great. Thank you!