Resizing problem in DAIN?

[meguerreroa]

Hi, i was using your MVIMP DAIN notebook in Google Colab to interpolate a short gif image:
Original
I used a 0.25 time_step (original have 8.33 fps so the interpolated should have 33.32) and i got this:
Result in MVIMP Colab
It looks like a strobing of different sizes frames.

I tried near the same (to 30 fps) in this notebook and i got this, less "strobing" (but still there). Looking up into the notebook i found a "RESIZE HOTFIX" method where the interpolated frames are upscaled slightly (apparently, in the interpolation process, they are generated a bit smaller than originals), also found that there are two interpolation methods, INTER_CUBIC and INTER_LANCZOS4

My question is: Could be there a "universal method" to solve this problem? ? Like an algorithm. Is it because of those two interpolation methods? Or maybe is something i'm doing wrong (codecs?) or a problem with DAIN itself.

Thanks in advance.

[CyFeng16]

I'll check if it's our problem or not.
Keep in touch. : )

[Zotikus1001]

This is a problem with DAIN it self.
Some people try to solve it by resizing the interpolated frames.
The "Colab_DAIN.ipynb" in the DAIN repository comes with said hotfix, it does minimize that problem, but it still isnt perfect.

The DAINApp application fixes this completely, but I don't know how.

[CyFeng16]

@Brokensilence Could you please help us improve it according to the "Colab_DAIN.ipynb" in the DAIN repository, I will assist if needed. 😺

[Zotikus1001]

@CyFeng16 I addded it to inference_dain: https://github.com/BrokenSilence/MVIMP/blob/master/inference_dain.py

But as I said, its not perfect. And I still dont know how to improve it.

[CyFeng16]

Many thanks.

[Zotikus1001]

The dev from DainApp said this to me:

I think it was a pytorch update that change some parameter for rescaling, not sure. Perhaps try to downgrade the pytorch

Maybe give that a try first?
I will test it when I can, maybe saturday.

[CyFeng16]

Some of the advanced features had added into our repo new according to #10 . PTAL.
Looking forward to @Brokensilence 's further movement.

[Zotikus1001]

@CyFeng16 I managed to completely fix this issue.
I did a few things, maybe not all are needed, but it was the only way I could get it to interpolate without errors.

Here is the colab notebook, please take a look: https://github.com/BrokenSilence/MVIMP/blob/master/MVIMP_DAIN_Colab.ipynb

• Downgraded gcc
• Downgraded Torch
• Downgraded Cuda
• Adjusted nvcc_args
• Updated OpenCv

[CyFeng16]

@Brokensilence
Hi, good day!
Do these changes continue after your partially modified DAIN? (Aka #10 )
I'll try and make a PR as long as everything goes well.
Thank you for your continued support, it means a lot.

[Zotikus1001]

@CyFeng16
I didn't quite understand this part: "Do these changes continue after your partially modified DAIN? (Aka #10 )"

Could you rephrase it?
Thank you.

[CyFeng16]

@CyFeng16 I addded it to inference_dain: https://github.com/BrokenSilence/MVIMP/blob/master/inference_dain.py

But as I said, its not perfect. And I still dont know how to improve it.

No problem!
You have already made some modification and I merge it into this PR #10 . Therefore I wonder what you tried at https://github.com/BrokenSilence/MVIMP/blob/master/MVIMP_DAIN_Colab.ipynb has connections with the modification you mentioned before or not.
Sorry for the unclear explanation - -||

[Zotikus1001]

@CyFeng16
Thank you.
It's a completely different solution. So I did remove the frame resizing hotfix from inference_dain: Zotikus1001@a5297bc

That was all. :)

[CyFeng16]

@Brokensilence Thanks in advance. 👍