Add Distill Any Depth #36614
Add Distill Any Depth #36614
Conversation
|
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the |
|
cc @qubvel! |
There was a problem hiding this comment.
Hey @keetrap! Thanks for working on the model! As far as I can see from the PR, the model architecture is identical to DepthAnything. In that case, we should be able to load the model with AutoModelForDepthEstimation without adding a new model to transformers. The conversion script for DepthAnything can be adjusted to support this model as well, alternatively, you can upload the conversion script to the model repo on the Hub together with the model weights. Please let me know if anything is not clear or if you need any help.
|
@qubvel Thanks for the review. |
|
Let's update the conversion script, but make sure we do not break the DepthAnything conversion. If it becomes too messy, we can add it as a separate file in the DepthAnything folder. |
keetrap
force-pushed
the
Distill_Any_Depth
branch
from
b527178 to
5362543
Compare
|
@qubvel Added conversion script. |
There was a problem hiding this comment.
Thanks a lot for adding conversion script 🤗. Since it's added as a separate file, it would be great to refactor it to use our new standard for conversion files (see the mllama model conversion script for an example). The general idea is to have a single layers MAPPING dictionary with regex-defined substitutions. I would appreciate it if you could update it so that the new pattern is propagated throughout the repo.
src/transformers/models/depth_anything/convert_distill_any_depth_to_hf.py
Outdated
Show resolved
Hide resolved
…th_to_hf.py Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
|
@qubvel Script updated. |
|
@qubvel Gentle reminder. |
src/transformers/models/depth_anything/convert_distill_any_depth_to_hf.py
Outdated
Show resolved
Hide resolved
…th_to_hf.py Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
|
Thank you! I've opened a PR to add tags to make model more discoverable (please copy them to another checkpoint as well) https://huggingface.co/keetrap/Distill-Any-Depth-Small-hf/discussions/2 Also, can you please create more descriptive model cards? Here is an example of such a model card P.S. no need to make it so descriptive as for DepthPro, but some image + abstract/description + code snippet would be super nice |
I think it should be added in original repo https://huggingface.co/xingyang1/Distill-Any-Depth-Small-hf and https://huggingface.co/xingyang1/Distill-Any-Depth-Large-hf |
|
Oh, there are originally converted models as well! Nice. Would you be so kind as to open PRs for those repos? |
|
PRs opened for model cards. |
This PR add support for Distill Any Depth
Fixes #36499
Pull Request section?
cc: @qubvel