Stream usage for backpropagation or softmax usage?

[jyoung3131]

We've been using MKL-DNN to implement a simple CNN, but we had a somewhat basic question about doing the softmax error calculation and backpropagation.

Should we use separate streams for different phases of execution, or do we add backpropagation primitives to our existing forward path stream? It's somewhat unclear from the unit tests in the codebase and there don't seem to be any relevant examples in the Intel caffe branch.

Thanks,
Jeff

[rsdubtso]

Hi Jeff,

In general you'd have to split execution into multiple streams when you have some non-mkl-dnn computations that need to happen in-between two primitives. For example, the loss computations will have to happen after the forward pass, and mkl-dnn currently has no loss primitive. This means that all forward propagation computations will have to be executed in one stream / set of streams, and backward propagation -- in another.

Hope this helps,
Roma