How to obtain aligned 3D object mesh

[dqj5182]

I am trying to re-implement the calculation of Object Chamfer Distance (CDo).

However, I am having a hard time finding the exact part of the code that aligns 3D object mesh with optimized hand mesh scale and translation. More specifically, I am referring to the writings in the paper "Following [11, 26], we first use the optimized hand scale and translation to transform the reconstructed object mesh".

In the code, it seems that only 3D hand is aligned to 3D GT hand with icp.

# Alignment on predictions
    trans = np.array([0, 0, 0])
    scale = np.array([1])
    if not is_hand:
        return verts, faces, trans, scale, pred_mesh
    else:
        if cfg.TEST.chamfer_optim:
            # Load GT mesh
            if cfg.DATASET.testset == 'ObMan':
                gt_mesh = trimesh.load(os.path.join(cfg.data_dir, cfg.DATASET.testset_hand_source, ply_filename_out.split('_')[0] + '.obj'), process=False)
            elif cfg.DATASET.testset == 'DexYCB':
                gt_mesh = trimesh.load(os.path.join(cfg.data_dir, cfg.DATASET.testset_hand_source, '_'.join(ply_filename_out.split('_')[:-1]) + '.obj'), process=False)

            # ICP algorithm for pred_mesh to gt_mesh
            icp_solver = icp_ts(pred_mesh, gt_mesh)
            icp_solver.sample_mesh(30000, 'both')
            icp_solver.run_icp_f(max_iter = 100)
            pred_mesh = icp_solver.get_source_mesh()
            trans, scale = icp_solver.get_trans_scale()
            return verts, faces, trans, scale, pred_mesh
        else:
            return verts, faces, trans, scale, pred_mesh

Where in the code aligns 3D object with the trans and scale obtained from the icp of hand?

[zerchen]

Hi,

Sorry for a late reply. Indeed, I only align the predicted 3D hand with the ground-truth 3D hand and apply the transformation (trans, scale obtained during optimization on the hand) on the object. The logic is that in the quality of the object mesh and the translation between the object and the hand are both important in the 3D hand and object reconstruction, which is also used in [11, 26]. Optionally, you can also apply ICP solely on the object to evaluate the quality of the object mesh.

Best,
Zerui