CUDA/GPU Support is Incomplete

[neubig]

Currently, GPU/CUDA support in cnn is incomplete. The most common functions are implemented in CUDA, but some of the less frequently used ones are not. Up until now this led to segmentation faults, but now most unimplemented functions should throw an error when you try to use them.

We're planning on fixing this in a couple ways:

1. In the long term, we're hoping to transition to the Eigen GPU functionality, which will mean that we won't have to re-implement every function on both CPUs and GPUs, and make both basically interchangeable. However, this requires some major changes/rethinking of the way some things are designed, so it may take a while.
2. In the short term, it is possible to add individual GPU implementations of any functions that you want to use. This can generally be done by taking a look at nodes.cc to find a similar function, and modifying nodes.cc, gpu-ops.cc, gpu-ops.h, and functors.h to add the functionality. If you'd like to help with this, please do and send a pull request.

Also, if there are any additional segfaults when using the GPU, please report them here. It will help if you provide an example to reproduce the problem including the exact command you ran and the data that you ran it on.

[neubig]

This is (finally!) almost finished in v2. There are still a few of the more rarely-used functions that don't have CUDA implementations yet, but things are mostly working. If you're interested in working on this, check out the v2 branch, which may be merged into master soon.