ResNet18 High FPS Depth Model

[Jokeru18]

If anyone is interested I have trained a new depth model that can reach very high fps on Nvidia GPUs.
The archive just contain the custom files ( including the model) and can be pasted directly over your nunif folder.
The model was trained using Large teacher and with mainly gaming material. Only about 10% movies so I don't recommend it for other content except games.
I'm still working on getting a better model but this one is good enough.

• Solid depth
• Fixed a lot of warping that was present with Distill Any S
• No floating shadows, or other floating dark artefacts.
• Very solid performance
• Faster folded version will follow soon.

Don't forget to install and enable Torch Compile.
And due to normalization for certain content you need to use 0.6-0.8 foreground scale to push the skybox further.

https://drive.google.com/file/d/1esDcAQBk7kZXTHVgvaRilK1XSbhzjBfs/view?usp=sharing

@nagadomi hope it's ok if I post this here.

[nagadomi]

@Jokeru18 If it is needed to use this project and there is no malicious intent, I do not see a problem with having it here.
Regarding the depth model addition, the current integration method may keep conflicting with changes on my side. I plan to make it easier to add custom depth models.

[Jokeru18]

I see there's not much demand for fast Depth model.
So I don't want to loose time pack and sharing an update in case no one wants it. I have a new models, especially ConvNext-Tiny which can provide incredible quality for the size ( much better compared with Resnet 18 or 34) It's not as fast as Resnet but can still provide HIgh FPS inference.
I anyone interested, let me know.
Some comparison with ANY_V2_L

[Jokeru18]

What do you mean by 3D images? Side by side? I don't really think that's a good way to judge a depth model. And this model is intended for Realtime high-FPS depth estimation, not for 3D image or movie conversion, there are a ton of models, many times better for that, already available
I suppose anyone can agree AnyV2-L is a great depth model, you can compare the difference to it. And if you want more detailed summary you can check the attached report.
comparison_summary_best72.md.

[KolaKater]

yes, Side by side

[Jokeru18]

[KolaKater]

Thank you, I just tested a group of images, unluckily the quality is so bad, even the head can be so wrong after converting:

[Jokeru18]

Yes, if you purposelly play with panes and convergence to break a model that could happened. I really don't get what you are doing....and why. I was intrigued and tested the V3.... and for some reason is nothing like what you posted.
And again, the model I posted and the other I'm distilling are intended for Gaming, not for movie, image conversion.

[KolaKater]

why don't you train FlashDepth?

[KolaKater]

Maybe you can check this and add FlashDepth to iw3. FlashDepth is verq quick and has elegant shapes and good depth.
FlashDepth Advanced