An elegant way to install origin viser instead of modified viser

[zjcs]

I notice that author modify the origin viser to capture the scene click pixel (actually, it is normal pixel). Here is an implemention via the origin viser, which can be used in click_cb function:

        camera = list(self.server.get_clients().values())[0].camera
        W = int(self.resolution_slider.value)
        H = int(W / self.aspect)
        WH = torch.Tensor([W, H])
        fxy = WH/2 / np.tan(camera.fov/2)
        # TODO: fxy from different fovx, fovy?
        dxyz = tf.SO3(camera.wxyz).inverse().as_matrix() @ pointer.ray_direction
        uv = fxy * torch.Tensor(dxyz[:2])/dxyz[2] + WH/2
        click_pos = uv / WH

[zjcs]

By the way, it is a solution to #2

[buaacyw]

Thanks for your contribution!

[zjcs]

@buaacyw sorry to say that the above code has some bug when calucate fov via aspect, here is the right implemention:

    @self.server.on_scene_click
    def _(point):
           for client in self.server.get_clients().values():
                camera = client.camera
                ray_d = torch.Tensor(tf.SO3(camera.wxyz).inverse().as_matrix() @ point.ray_direction)
                # ray_d, the ray in the camera, x->right, y->down, z->far
                xy_norm = ray_d[:2]/ray_d[2]
                
                WH = torch.Tensor([self.resolution_slider.value, self.resolution_slider.value/camera.aspect]).int()
                tan_fov_2 = torch.Tensor([np.tan(camera.fov/2)*camera.aspect, np.tan(camera.fov/2)])
                fxy = WH/2 / tan_fov_2

                uv = (fxy * xy_norm  + WH/2).int()

                click_pos = uv / WH

[jimmychuang95]

@zjcs I modify the webui.py code as you provided, but the bounded box and SAM points still cannot be added into scene, what else should I need to change to let those functions work?