MergeLayer

[Qwlouse]

We need a layer that has multiple inputs and just concatenates them along the last feature dimension.
For CPU that one can be omitted, because the NumpyHandler supports slicing the features, but for usage with the PyCudaHandler this is the only way of merging the outputs of two layers.

[flukeskywalker]

@untom Can this work on the GPU at all or should it be removed?

[untom]

it can work... I'll see if I can hack one up today.

[flukeskywalker]

Great! Is this essentially the same issue that holds back the optimized LSTM implementation in LSTMOpt from working with PyCudaHandler?

[Qwlouse]

By using some extra memory this could be accomplished with only a slice-enabled copy_to (no need for add_tt then. So if that is easier we could do that. Actually I haven't even checked if copy_to works on slices...

[untom]

Implemented via f36b57d and f86bb99 (There might be faster ways to implement this on the GPU, so this might be worth revisiting if a profiling run shows that this is really a bottleneck)