-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Questions about the pose optimization (paper and code) #93
Comments
I think the Line 128 in 6c9254c
I've tried using the rasterizer with the original 3DGS paper. My experience is that you cannot optimize the camera pose together with the map... The result map will get more and more blury because the camera is shaking. Although the camera pose only have small refinement, but the original pose already leaves great impact on the map, which is hard to elimate or amend. I think the design of frontend and backend in MonoGS is necessary and smart that it avoids this problem. When tracking camera, the map is fixed. When mapping, you should trust the pose. This is only my personal experience, maybe there are some other ways to optimize camera poses and the map simultaneously. |
Hi thanks for your answer, Have you succeeded in having a pose estimation though ? For now printing the Accordingly to the paper 3.2 section, the gradient is thus provided by the rasterizer jacobian computation and descended by the Adam optimizer in the If my above statements are right, the only thing left to understand for me is how the jacobian gradient is comunicated between the rasterizer and the optimizer. Could you explain me this link ? Many thanks, |
I used some prior pose and tried optimizing the map and refining poses simultaneously, which turns out to be a really bad idea. T^T
yes I think so.
Yes, it is in the paper, but in the code I think it's based on silhouette, see my question here #90 (comment).
Well, I am also not clear about this part... But my undertanding and intuition is that if the rasterizer has position gradient for all GS in one camera, the opposite direction of the mean of all GS gradient projected to the 2D camera plane is the camera gradient. And the optimizer just use this gradient and some learning rate to optimize. |
I think it's defined by the order in which the input tensors are specified when calling Implementation of That's why When calling |
@identxxy I have a question concerning your try at getting a pose estimation. |
I was trying to use the |
Dear authors, comunity,
Is it possible for you clarify my understanding on how the pose optimization works ?
In my understanding the pose is optimized via gradient descent to minimize the jacobian values.
Where the jacobian is derived analytically from the reprojection error (equations 3 to 6 in section 3.2).
viewpoint_camera.cam_rot_delta
andviewpoint_camera.cam_trans_delta
parameters during the rasterization ?I'm trying to reuse your rasterizer with the original 3DGS paper (with GT path) but I can't see the
delta
parameters updating, could you indicate to me how to get a pose estimation out of the rasterizer ?Thanks in advance,
Best regards,
Hugo
The text was updated successfully, but these errors were encountered: