Training speed slower than Non-local attention

[noobliang]

I found add a module of Non-local attention which just add a little extra time cost about 0.4s each iter. But if I add a CC-attention take R==1 , the train time each iter about 0.7s, and 1.0s if R==2. It's not like the description in your paper. I dont know why. Can anyone explain it .

[speedinghzl]

@noobliang Thanks for your attention. The inefficient program implementation results in a slower speed than non-local attention. In term of computation cost and memory usage, the CCNet still have advantages mentioned in the paper. Looking forward to more efficient program implementation.

[noobliang]

@noobliang Thanks for your attention. The inefficient program implementation results in a slower speed than non-local attention. In term of computation cost and memory usage, the CCNet still have advantages mentioned in the paper. Looking forward to more efficient program implementation.

Well , thank you reply. I found it almost useless for my segmentation network， it may be the positive samples of the picture too few.