train from scratch on ucf101 dataset

[leviswind]

We try to train i3d model on ucf101 from scratch, but it converges much slower with a final validation accuray around 60%. Can you offer some suggestions on train i3d model without imagenet pretrained model.

[joaoluiscarreira]

Hi, i think 50-60% accuracy is to be expected when training I3D from scratch on RGB in UCF101. If you do the same on flow it should get ~80%. When averaging both we got 88% in the last version of the quo vadis paper. In summary, i think your training setup should be fine. Best, Joao

…

On Wed, Jan 16, 2019 at 3:50 AM leviswind ***@***.***> wrote: We try to train i3d model on ucf101 from scratch, but it converges much slower with a final validation accuray around 60%. Can you offer some suggestions on train i3d model without imagenet pretrained model. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#46>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AO6qaj792doQPF6DLeSuo3yYIm6AKfcOks5vDqGNgaJpZM4aCQxU> .

[leviswind]

How to train i3d with optical-flow with imagenet pretrained model? Can you offer some details of training on UCF101 dataset.

[leviswind]

Also, what's the convergence speed should be when training optical-flow compared with rgb with imagenet pretraining @joaoluiscarreira

[joaoluiscarreira]

The way we did it was that we inflated the weights of the imagenet model into 3D, then trained the model normally from there, without freezing batch norm. I think you can find code online for training the model if you search on google. I tend to remember that the flow model converges faster but this was a long time ago. Best, Joao

…

On Thu, Jan 17, 2019 at 7:36 AM leviswind ***@***.***> wrote: Also, what's the convergence speed should be when training optical-flow compared with rgb with imagenet pretraining — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#46 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AO6qameRG2etXHL7UwUIC5vL1isSGHVHks5vECgCgaJpZM4aCQxU> .

[leviswind]

However, the input channel of the first conv layer is 2 for flow data compared with 3 for rgb. How to deal with the difference? @joaoluiscarreira . I'm really appreciated for your help.

[joaoluiscarreira]

I think we just discarded the weights for one of the input channels in that first layer, before inflating. Best, Joao

…

On Thu, Jan 17, 2019 at 11:55 PM leviswind ***@***.***> wrote: However, the input channel of the first conv layer is 2 for flow data compared with 3 for rgb. How to deal with the difference? @joaoluiscarreira <https://github.com/joaoluiscarreira> . I'm really appreciated for your help. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#46 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AO6qau9xAzRpEZOppvHgaG3JODy5j3pgks5vEQ2KgaJpZM4aCQxU> .

[joaoluiscarreira]

Actually i went back to check what exactly we did and for that particular layer we average the original weights for the 3 input channels then copied it twice -- so the initial weights are the same for both flow input dimensions. But i think it did not make much of a difference compared to the other option. Joao On Fri, Jan 18, 2019 at 9:26 AM João Carreira <joaoluiscarreira@gmail.com> wrote:

…

I think we just discarded the weights for one of the input channels in that first layer, before inflating. Best, Joao On Thu, Jan 17, 2019 at 11:55 PM leviswind ***@***.***> wrote: > However, the input channel of the first conv layer is 2 for flow data > compared with 3 for rgb. How to deal with the difference? > @joaoluiscarreira <https://github.com/joaoluiscarreira> . I'm really > appreciated for your help. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#46 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AO6qau9xAzRpEZOppvHgaG3JODy5j3pgks5vEQ2KgaJpZM4aCQxU> > . >

[leviswind]

We suffered great overfitting when training i3d with optical-flow similar as training rgb data without imagenet pretraining. The test accuracy is only 50%. Have you met such problems? @joaoluiscarreira

[joaoluiscarreira]

As mentioned earlier in the thread, training from scratch on flow got close to 80%. You could try testing with batch statistics to see if there's some batch norm moving average problem.

…

On Sat, Jan 19, 2019 at 5:22 AM leviswind ***@***.***> wrote: We suffered great overfitting when training i3d with optical-flow similar as training rgb data without imagenet pretraining. The test accuracy is only 50%. Have you met such problems? @joaoluiscarreira <https://github.com/joaoluiscarreira> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#46 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AO6qaph7hvuSW91qD5DMfoONVt9O-aA0ks5vEquFgaJpZM4aCQxU> .

[poweryin]

@leviswind,Hi，can you train the i3d on ucf101 successfully? I want to use i3d on ucf101, How can I use i3d model to fine-tune on ucf101? where is the train code?
can you give me some advice.
Thanks
Best wishes