7. 📐 Evaluation

The evaluation script aligns the reconstruction trajectory to the ground-truth trajectory, optionally refines the point-cloud alignment with ICP, and reports precision/recall and reconstruction-error metrics.

Use the two ground-truth point-cloud sources differently:

• cloud_gt.pcd: dense registered LiDAR map, used for precision and reconstruction-error evaluation.
• cloud_gt_fov/: per-frame LiDAR clouds within the camera field of view, used for recall evaluation.

Common placeholders:

• $GT_FOLDER: sequence ground-truth folder, for example /path/to/r01_gt.
• $RECON_FOLDER: mapping output folder passed as --slam_folder.
• <trajectory>: trajectory output name under recon/.
• <xxx> and <model_name>: suffixes produced by the reconstruction run.

Global Reconstruction Quality

Evaluate a global reconstruction PCD against the dense GT map for precision and reconstruction error:

python3 scripts/eval_utils/compare_pts.py \
  --gt $GT_FOLDER/cloud_gt.pcd \
  --gt-traj $GT_FOLDER/poses_gt.txt \
  --recon $RECON_FOLDER/recon/<trajectory>/pts_global_<xxx>.pcd \
  --recon-traj $RECON_FOLDER/recon/<trajectory>/poses_<model_name>.txt \
  --voxel-size 0.02 --threshold 0.03 \
  --precision --icp --vis-all

Evaluate recall against the field-of-view GT clouds:

python3 scripts/eval_utils/compare_pts.py \
  --gt $GT_FOLDER/cloud_gt_fov \
  --gt-traj $GT_FOLDER/poses_gt.txt \
  --recon $RECON_FOLDER/recon/<trajectory>/pts_global_<xxx>.pcd \
  --recon-traj $RECON_FOLDER/recon/<trajectory>/poses_<model_name>.txt \
  --voxel-size 0.02 --threshold 0.03 \
  --chamfer-threshold 0.1 \
  --recall --icp --vis-all

Chunk-Wise Reconstruction Quality

Chunk-wise evaluation uses the local per-keyframe point clouds in pts_local_<xxx>/. This is useful for measuring local reconstruction quality before global point-cloud accumulation.

Evaluate chunk-wise precision and reconstruction error:

python3 scripts/eval_utils/compare_pts.py \
  --gt $GT_FOLDER/cloud_gt.pcd \
  --gt-traj $GT_FOLDER/poses_gt.txt \
  --recon $RECON_FOLDER/recon/<trajectory>/pts_local_<xxx> \
  --recon-traj $RECON_FOLDER/recon/<trajectory>/poses_<model_name>.txt \
  --voxel-size 0.02 --threshold 0.03 \
  --chunk-m 10.0 \
  --precision --icp

Evaluate chunk-wise recall:

python3 scripts/eval_utils/compare_pts.py \
  --gt $GT_FOLDER/cloud_gt_fov \
  --gt-traj $GT_FOLDER/poses_gt.txt \
  --recon $RECON_FOLDER/recon/<trajectory>/pts_local_<xxx> \
  --recon-traj $RECON_FOLDER/recon/<trajectory>/poses_<model_name>.txt \
  --voxel-size 0.02 --threshold 0.03 \
  --chamfer-threshold 0.1 \
  --chunk-m 10.0 \
  --recall --icp