-
Notifications
You must be signed in to change notification settings - Fork 500
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Does anybody achieve the metric depth estimation on a custom dataset successfully? #68
Comments
I did it on the RealEstate10k unfortunately Depth does not exist on the dataset as such so evaluation is not possible but in general if you take an anecdotal look, it's pretty good. |
@1ssb Have you seen results like this with the metric depth outdoor checkpoints? With all of my experimentation it seems like the sky predictions are not good. Although the relative depth predicts the sky and other "background" extremely well! |
Sorry @Denny-kef, but I cannot help you with that. Make sure you are using the outdoor model and not the indoor one.
Skies are always difficult to correctly capture on an absolute scale so I do not think your expectations can be too high for that. Relative scaling of distant backgrounds are also always better than absolute ones in the history of monocular depth estimation.
…On Tue, 6 Feb, 2024, 8:48 am Dennis Loevlie, ***@***.***> wrote:
@1ssb <https://github.com/1ssb> Have you seen results like this with the
metric depth outdoor checkpoints? With all of my experimentation it seems
like the sky predictions are not good. Although the relative depth predicts
the sky and other "background" extremely well!
image.png (view on web)
<https://github.com/LiheYoung/Depth-Anything/assets/121886500/599554a8-ebb2-462c-843d-69c627a9d04a>
—
Reply to this email directly, view it on GitHub
<#68 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AJWHFEDZIZMLRDJWSUSRMI3YSFHSVAVCNFSM6AAAAABCVJF6ACVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRYGE2TIMRVGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Hi @1ssb thanks for getting back to me! I am using the outdoor checkpoints and just wondering if you (or anyone else) has seen similar results with the metric depth predictions? |
That's interesting about the metric vs relative thing. I mean I could mask out the background using the relative depth network or a lightweight segmentation network but it seems like there should be a better way.. |
Oh this is interesting @Denny-kef I took a look at your image and it seems to me it is captured by either a fish eye lens or the image itself is a bit distorted as in to the eye itself it looks like the clouds are much closer than they actually are, this is a very cool thing as well. I am not sure I have an exact answer to this but it might as well be an OOD. |
@Denny-kef Idk if this can help you in your task, but this is my personal interpretation:
That said, I think the features from their frozen encoder are really powerful for metric depth estimation, but I guess it would very hard to use them to produce correct values on an absolute scale for the sky. |
The best solution that I found for the "background" issue with metric depth estimation predictions is this:
Importantly, these operations don't add any significant time to the inference. |
Bottomline: Don't try to predict skies or reflections. |
I was not trying to predict skies but I was trying to remove them from the outputted depth map so they don't show up in the point cloud. But yes do not try to predict the depth of the sky or reflections! |
Hi @loevlie, if you are trying to detect the sky and remove it, you can try our relative depth models. The output value |
Hi @LiheYoung, yes that works very well! Thank you! |
Could you kindly share with me the parameters you adjusted during fine-tuning? I've been encountering poor performance in my experiments with another dataset, and I've been struggling to resolve the issue. The details of the problem are as follows.#172 (comment) |
@Denny-kef |
Hi @1ssb , Are you able to train metric depth estimation on a dataset without depth maps (labels) ? |
Hi, I never train, but test time fine tune using a plug and play method on the RealEstate10k. Kindly remember that Realestate10k is not RGBD dataset, but you can use the triangulation method and use the control points to rescale the predictions directly. It's not very neat but it's the best you can do without retraining from scratch.
Best
Subhransu
…________________________________
From: Shilpa Ullas ***@***.***>
Sent: Thursday, June 27, 2024 6:04:17 PM
To: LiheYoung/Depth-Anything ***@***.***>
Cc: Subhransu Bhattacharjee ***@***.***>; Mention ***@***.***>
Subject: Re: [LiheYoung/Depth-Anything] Does anybody achieve the metric depth estimation on a custom dataset successfully? (Issue #68)
I did it on the RealEstate10k unfortunately Depth does not exist on the dataset as such so evaluation is not possible but in general if you take an anecdotal look, it's pretty good.
Hi @1ssb<https://github.com/1ssb> ,
Are you able to train metric depth estimation on a dataset without depth maps (labels) ?
Could you please share more details about your training trial?
—
Reply to this email directly, view it on GitHub<#68 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AJWHFEH236HROYVD4WEG3DTZJPBQDAVCNFSM6AAAAABCVJF6ACVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJUGA2TCNJQGU>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Hi,
This post is just to discuss how to achieve metric depth estimation on a custom dataset, like I am using SCARED dataset. If anyone successfully fine-tune the model and achieve metric depth estimation, could you tell me which code did you modify?
The text was updated successfully, but these errors were encountered: