[Speed Compiling]: Reduce NVCC compiling files. #5573
Conversation
qingqing01
force-pushed
the
cmake_speed
branch
from
916a9b9
to
37ee17c
Compare
qingqing01
force-pushed
the
cmake_speed
branch
from
37ee17c
to
5f21709
Compare
qingqing01
changed the title
qingqing01
requested review from
reyoung,
wangkuiyi,
hedaoyuan and
emailweixu
| [self.input_size[0]]] | ||
| self.output_represention = 8 # output feature size | ||
|
|
||
| #class TestSeqProjectCase1(TestSeqProject): |
There was a problem hiding this comment.
Is these codes can be removed?
There was a problem hiding this comment.
Resume these unit testing.
qingqing01
force-pushed
the
cmake_speed
branch
from
e97a5fb
to
e9082bb
Compare
| input.data<T>(), bias.data<T>(), output->data<T>(), in_dims[0], size); | ||
| } | ||
| }; | ||
|
|
There was a problem hiding this comment.
why not use eigen to do it?
There was a problem hiding this comment.
Done. Move RowwiseAdd to paddle/operators/math/math_function and use Eigen for CPU and GPU.
paddle/operators/gru_op.h
Outdated
| math::SetConstant<Place, T> set; | ||
| set(dev_ctx, &ones, static_cast<T>(1)); | ||
| math::gemv<Place, T>(dev_ctx, true, m, n, 1., batch_gate_grad.data<T>(), | ||
| ones.data<T>(), 0., bias_grad->data<T>()); |
There was a problem hiding this comment.
Since this is used at many places, should we make a similar function as RowwiseAdd to do this?
There was a problem hiding this comment.
Done. Add a ColwiseSum functor in paddle/operators/math/math_function.h to compute the gradient of bias.
| void operator()(const platform::DeviceContext& context, | ||
| const framework::Tensor& input, const framework::Tensor& bias, | ||
| framework::Tensor* output); | ||
| }; |
There was a problem hiding this comment.
Why is this at sequence2batch.h?
There was a problem hiding this comment.
At first, this functor is only used in lstm_op and gru_op. Now I move it to paddle/operators/math/math_function.
|
Another possibility is to merge .cc and .cu.cc as a single .cc. What do everyone think? |
|
I agree with this. |
|
Can this PR be merged at first, since it reduces NVCC compile time, and many files will be conflicted with it. How about merge .cc and .cu.cc as a single .cc in next PR? |
Related to #5491
Testing : local machine
Now the PaddlePaddle compiling is much slower, about 30min in GPU server each time. I tested the NVCC and G++ compiling time by operator compiling, the NVCC is slower than G++ more than 1min. And some GPU kernel implementation of operators can be compiled by G++. So I try to change about 25 cu operator files to cu.cc and make them compiled by G++ to speed the compiling time. But I'm not sure whether it is necessary.