feat: Add Depth Anything PreProcessor

[blessedcoolant]

What type of PR is this? (check all applicable)

• Feature

Have you discussed this change with the InvokeAI team?

• Yes

Have you updated all relevant documentation?

• No

Description

• This adds the newly released Depth Anything to InvokeAI. A new node Depth Anything Processor has been added to generate depth maps using this new technique. https://depth-anything.github.io

• All related checkpoints will be downloaded automatically on first boot. The DinoV2 models will be loaded to your torch cache dir and the checkpoints pertaining to Depth Anything will be downloaded to any/annotators/depth_anything.

• Alternatively you can find the checkpoints here and download them to that folder: https://huggingface.co/spaces/LiheYoung/Depth-Anything/tree/main/checkpoints

• This depth map can be used with any depth ControlNet model out there but the folks at DepthAnything have also released a custom fine tuned ControlNet model. From my limited testing, I still prefer the original depth model because this one seems to be producing weird artifacts. Not sure if that is a specific problem to Invoke or just the model itself. I'll test more later. Place these in your controlnet folder like your other ControlNets. You can get that here: https://huggingface.co/spaces/LiheYoung/Depth-Anything/tree/main/checkpoints_controlnet

• Also available in the LinearUI

• DepthAnything has three models large, base and small -- I've defaulted the processor to small but a user can change to the large model if they wish to do so. Small is way faster but obviously somewhat of a lesser quality.

• DepthAnything is now the default processor for depth controlnet models.

Screenshots

Merge Plan

DO NOT MERGE YET. Test it first and I'm sure the model caching can be done better. Coz I don't think I've done that at all. I would appreciate if @brandonrising or @lstein or anyone can take a look at that part of it.

[hipsterusername]

Should we make this the default for depth models?

[blessedcoolant]

Should we make this the default for depth models?

Try and see. I need to experiment with a bit more to see if this is the better alternative. I'll test it a bit more later today and see if it should be the default.

[blessedcoolant]

Added to Linear UI too. Not the default. Can change if people feel its better.

[blessedcoolant]

Added output resolution. Also changed the if statements to match-case because we're using Python 3.10 and above already. If for some reason this needs to be reverted, let me know.

[hipsterusername]

works so good.

we should update to default in a future PR.

[blessedcoolant]

It's already in this PR. Should I undo?

[hipsterusername]

Oh - No I think that's fine. We'll just want to RC.

[hipsterusername]

Does caching still need to be looked at @blessedcoolant ?

[blessedcoolant]

Does caching still need to be looked at @blessedcoolant ?

Think so? One quick glance from them should sort it. I didn't see any glaring issues but maybe @lstein wants to handle the models better and store them in a better location. Right now I put them in the controlnets/annotators folder which is throwing warnings for incompatible models when booting coz they're not recognized. I thought I'll let Lincoln take a jab at that.

[blessedcoolant]

I've moved the models to any/annotators/depth_anything to avoid the model manager trying to parse them as controlnet models on boot which was happening in the previous location

[lstein]

I don't see a problem with the caching per se, but we've generally avoided downloading big models files at unexpected times. Will these models be used frequently enough to make them into "core" models that are downloaded at install time? We do have a way of indicating that one model is dependent on others, which we are using for the relationship of IP Adapters and their encoders, and could use the same syntax to indicate that depth controlnet models require the presence of the Depth Anything models.

[Millu]

Not sure what was going on the first time but works great on macOS!

[hipsterusername]

I don't see a problem with the caching per se, but we've generally avoided downloading big models files at unexpected times. Will these models be used frequently enough to make them into "core" models that are downloaded at install time? We do have a way of indicating that one model is dependent on others, which we are using for the relationship of IP Adapters and their encoders, and could use the same syntax to indicate that depth controlnet models require the presence of the Depth Anything models.

I anticipate we want these to operate similarly with how we're handling midas right now - as a dependency installed early on. I think we can optimize the downloading outside of this PR, but we should make sure to do so.

Thanks @Millu for testing on Mac. Can you make sure we follow up with an issue to track getting this model into an earlier install/config process?

[hipsterusername]

@blessedcoolant can merge after lint issues fixed.