Feature request: Implementing spatially-sparse conv networks in TensorFlow

[jstaker7]

I am inspired by Dr. Ben Graham's recent work regarding spatially-sparse convolutional neural networks:

http://arxiv.org/abs/1409.6070 (particularly section 2.3)
http://www2.warwick.ac.uk/fac/sci/statistics/staff/academic-research/graham/sparse3d.pdf

He has graciously open-sourced his neural network library but I'd like to utilize these same ideas in my models, which are implemented in Tensorflow.

Unfortunately, I'm finding his implementation and citations a little hard to follow, but his description of a "feature matrix" and a "pointer matrix" sounds a little like something that can be implemented using sparse variable updates in Tensorflow (https://www.tensorflow.org/versions/r0.7/api_docs/python/state_ops.html#sparse-variable-updates), but I'm afraid that it might not be so simple and would require building custom kernels to support a new-ish type of convolution. But I don't know enough to say for certain which direction to take.

Any thoughts on how we can bring spatially-sparse convolutions to Tensorflow? Anyone interested in collaborating on implementing this?

[girving]

Yes, this sounds like it requires custom kernels to have reasonable performance, though I don't see the word "pointer" in the paper so I can't confirm.

In terms of collaboration, do you want to post on the discuss@tensorflow list asking if anyone wants to collaborate? It's a good use of the list, and most people outside the team are unlikely to notice a Github issue.

Note that we are doing some work on better sparse ops (e.g., @ebrevdo's https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/sparse_tensor_dense_matmul_op.cc), but I don't think sparse convolutions is on our radar just yet. It would be a good thing to have, and I'd be happy to answer questions if you do end up working on it.

[jstaker7]

Thanks, Geoffrey, that is a great idea, I'll go ahead and post to the email thread. I am definitely interested in implementing this since I am currently working with a sparse dataset (and maybe more in the future), although I don't currently have the skill set to implement it on my own. Thanks for the offer, I'll definitely come back with questions as they come up.

[iurilgit]

hi,
We are interested in the sparse tensor for conv2d as well. Is it supported in the latest version yet?
Thanks!

[evanfeinberg]

Just wondering, did anything ever come of this? It would be a real boon to the community!

[girving]

Not that I know of. Still contributions welcome.

[evanfeinberg]

Thanks! My thesis is a bit far ATM from this sort of core algorithm development but it would be fun to get into this as a side project if I have the chance.