How can I get same performance as in paper when applying GhostModule on ResNet-50?

[twinklecc]

Hi, very thanks for your working, @iamhankai

I'm interested in this GhostModule design, so I substitude the GhostModule in ghostnet for vanilla convolution in ResNet-50. With s=2, paper shows distinguished performance as 75.0% Top1-Acc and 2.2G FLOPs. But I trained this model only to gain a 73.7% Top1-Acc and 2.4G FLOPs. Could you give me relevant training setting file or remind me of some special training details? Thanks again!

[iamhankai]

Please provide your training settings. Let me see what's wrong.

[twinklecc]

Hello, I use mmclassification to train model, and here is my config file. Thanks a lot! 

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "huawei-noah/ghostnet" <notifications@github.com&gt;; 发送时间:&nbsp;2020年10月8日(星期四) 下午5:43 收件人:&nbsp;"huawei-noah/ghostnet"<ghostnet@noreply.github.com&gt;; 抄送:&nbsp;"张晨"<517273836@qq.com&gt;;"Author"<author@noreply.github.com&gt;; 主题:&nbsp;Re: [huawei-noah/ghostnet] How can I get same performance as in paper when applying GhostModule on ResNet-50? (#39) Please provide your training settings. Let me see what's wrong. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[iamhankai]

It seems no attached file here.

The issue may help: #34

Replacing ConvBNReLU with Ghost Module.

[twinklecc]

Sorry, I checked my reply yesterday, there was an attached file named training setting.py. Maybe something wrong with mailing.
Thanks for this referenced issue, I did the same replacing work to get a GhostModule-ResNet50. My model structure ought to be right.
Here I zip and attach config file again, hoping for right delivery. Very thanks!
training setting.zip

[iamhankai]

Your learning rate should be 0.1*{total batch size}/256

[twinklecc]

In my config, samples_per_gpu=64, gpus=8, so the total batch size is computed as 64*8.
MMclassification provides autoscale learning rate function, so initial learning rate is indeed 0.2 (0.1{total_bs}/256).
That's consistent with your instruction. Very thanks!

[iamhankai]

What's the accuracy of the original ResNet-50 with your config file?

In addition, could you show your implemented Ghost-ResNet50?

[twinklecc]

I got about 77.5% Top1-Accuracy of ResNet-50 with my config file. Very thanks!

[huangpan2507]

You mean, use Ghost_module instead of all the conv in the bottleneck ,include conv in downsample and except the first conv in Resnet? So I replace all of the conv except the first conv in Resnet, but The GPU cost increasely almost 10897M( My decive: 1 piece 1080ti GPU, before modify, GPU cost is 7645M ,batchsize= 8 * 4 ),it remind me that GPU is run out of memory, until I modify the batchsize = 1*4, the program can run! when I doesn't replace conv in downsample , it remind me the size are not match.

[iamhankai]

@huangpan2507 If you don't replace conv in downsample , it should NOT remind you the size are not match. You need to check the correctness of your code.

[huangpan2507]

@huangpan2507 If you don't replace conv in downsample , it should NOT remind you the size are not match. You need to check the correctness of your code.

Thanks. Another question is that the cost of GPU memory will become greater if i instead the conv with Ghost_module?