IMPORTANT Your shell script breaks all apps using MiDaS

[cooperdk]

Your install.sh script causes every single Python project using your module to break.
It is used by some Automatic1111 extensions and NONE of them work unless manually copying the externals/Next_ViT into the Web UI root folder.

Error loading script: depthmap_for_depth2img.py
Traceback (most recent call last):
File "X:\StableDiffusion\webui\modules\scripts.py", line 195, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "X:\StableDiffusion\webui\modules\script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "X:\StableDiffusion\webui\extensions\depthmap2mask\scripts\depthmap_for_depth2img.py", line 11, in
from repositories.midas.midas.dpt_depth import DPTDepthModel
File "X:\StableDiffusion\webui\repositories\midas\midas\dpt_depth.py", line 5, in
from .blocks import (
File "X:\StableDiffusion\webui\repositories\midas\midas\blocks.py", line 21, in
from .backbones.next_vit import (
File "X:\StableDiffusion\webui\repositories\midas\midas\backbones\next_vit.py", line 8, in
file = open("./externals/Next_ViT/classification/nextvit.py", "r")

When an app uses your module, it will expect the externals/Next_ViT directory at the app root.
You cannot import a module in Python to a fixed location because that will break the dependency, since you're installing it in the wrong location. Your module won't find the module, because the main app installes it to it's own root, not to your module's root.

Anyone using your module will have to manually copy the externals folder to their own app's root folder, which is really unacceptable.

You should clone it to your own project's root using python git and load it by instead of using

file = open("./externals/Next_ViT/classification/nextvit.py", "r")

in midas/backbones/next_vit.py
Use

file = open("../../externals/Next_ViT/classification/nextvit.py", "r")

Also, a shell script to run a git command? Really?? You think nobody uses another operating system?
This is why I say you MUST use Python's own git module.

[juancamilog]

@rbirkl I don't agree witht he tone of the message above, but this also breaks models loading through torch hub.

[JupiterJones]

@cooperdk you sound like you know what's up here. :) Do you have a pull request that would fix the issue? even if it's just a path fix to the actual location of the script?

This is the first time I've had to look under-the-hood on MiDaS - I'm just a new user (via disco) so I don't feel confident in offering a solution myself just yet.

Better still, maybe you can resolve the issue of needing to call the install_next_vit.sh shell script at all, with your suggestion of using pythons git module.

[cooperdk]

I'll take a look at it ASAP.

[hafriedlander]

Oh dear, bit of a mess. The bytedance/Next-ViT repo isn't set up to be used as a module. Specifically, https://github.com/bytedance/Next-ViT/blob/main/classification/nextvit.py imports utils directly rather than relatively.

I've raised bytedance/Next-ViT#11 - if this gets merged, you could then add https://github.com/bytedance/Next-ViT as a git submodule.

(The other option is just to copy nextvit.py and utils.py into the MiDaS project, but there's a license issue to doing that - bytedance/Next-ViT is Apache 2.0, and this project is MIT)

[hafriedlander]

Here is a fork I've made to temporarily fix this issue (not tested yet). https://github.com/hafriedlander/MiDaS. Just for reference. I do not recommend using this fork. I make no guarantees about maintaining it.

[shamnad-sherief]

Go to the file /content/MiDas/midas/backbones/next_vit.py and edit the line

file = open("./externals/Next_ViT/classification/nextvit.py","r")

to

file = open("/content/MiDas/midas/backbones/vit.py","r")

And it is solved

[thias15]

Hi guys. Thanks for pointing out this issue. We will work on a solution ASAP. @hafriedlander thanks for your suggestions. @shamnad-sherief your solution does not exactly solve the issue, it will just run the ViT backbone instead of the NextViT backbone.