Query on Info logged using Wandb

[lvmingzhe]

Hello,

Thanks for your great work and I've come across several PSNR metrics that I would like to understand better. Specifically, I'm referring to Tracking/PSNR, Mapping/PSNR, and Final Stats/Average PSNR. Could you please clarify the differences between these metrics? I'm interested in how each is calculated.

Additionally, I've noticed that the Tracking step is showing a value of 0 in my results. Could you provide some insight into why this might be happening? I'm wondering if this is indicative of an issue with my data, a misunderstanding on my part regarding what the Tracking step represents, or if it's a potential bug in the software.

I've already reviewed the documentation available at previous issues but couldn't find the detailed explanations or troubleshooting guidance I was looking for.

Any guidance or references to more detailed resources would be greatly appreciated.

Thank you for your time and help.

---

From wandb summary log:

"Tracking/PSNR": 35.285186767578125,
"Tracking/Depth RMSE": 0.007432922255247831,
"Tracking/Depth L1": 0.007432922255247831,
"Tracking/step": 0,
"Tracking/Latest Pose Error": 0.0,
"Tracking/Latest Relative Pose Error": 0.0,
"Tracking/ATE RMSE": 0.0,
"_timestamp": 1709986693.87413,
"_runtime": 2236.5880041122437,
"_step": 51530,
"Per Iteration Mapping/Loss": 0.043003521859645844,
"Per Iteration Mapping/Image Loss": 0.03194858878850937,
"Per Iteration Mapping/Depth Loss": 0.011054933071136475,
"Per Iteration Mapping/step": 17099,
"Mapping/Number of Gaussians - Pruning": 6095185,
"Mapping/step": 17100,
"Mapping/PSNR": 16.289276123046875,
"Mapping/Depth RMSE": 0.00043166676186956465,
"Mapping/Depth L1": 0.00043166676186956465,
"Per Iteration Tracking/Loss": 5655.00341796875,
"Per Iteration Tracking/Image Loss": 1648.0791015625,
"Per Iteration Tracking/Depth Loss": 4006.92431640625,
"Per Iteration Tracking/step": 17039,
"Mapping/Number of Gaussians": 6099206,
"Final Stats/Average Tracking Iteration Time (ms)": 58.094412564111984,
"Final Stats/Average Tracking Frame Time (s)": 3.4751132212187112,
"Final Stats/Average Mapping Iteration Time (ms)": 69.2872304804841,
"Final Stats/Average Mapping Frame Time (s)": 4.158240836126763,
"Final Stats/step": 1,
"Final Stats/Avg ATE RMSE": 0.02060262257340403,
"Final Stats/Average PSNR": 24.738697052001953,
"Final Stats/Average Depth RMSE": 0.013846458867192268,
"Final Stats/Average Depth L1": 0.013846458867192268,
"Final Stats/Average MS-SSIM": 0.903265655040741,
"Final Stats/Average LPIPS": 0.13082163035869598,

[Nik-V9]

Hi, you can better understand the logging by looking at the report_progress function and the following places of the code:

• SplaTAM/utils/eval_helpers.py

  Line 256 in 7959433

  if wandb_run is not None:

• SplaTAM/scripts/splatam.py

  Line 762 in 7959433

  # Report Final Tracking Progress

• SplaTAM/scripts/splatam.py

  Line 895 in 7959433

  # Report Mapping Progress

• SplaTAM/scripts/splatam.py

  Line 954 in 7959433

  if config['use_wandb']:

Basically this is the short explanation:

• Track/PSNR, etc - Online Evaluation of the current frame after Tracking
• Map/PSNR, etc - Online Evaluation of the current frame after Densification & Mapping
• Final Stats/PSNR, etc - Final Evaluation of SLAM Reconstruction & Tracking

Also I would recommend looking at the plots in Wandb as opposed to these summary metrics.

[lvmingzhe]

Thank you very much for your detailed and patient response. I successfully found the code related to the entire wandb logging process and understood the general flow.
I will follow your advice and focus more on the wandb curves instead of these final metrics.
However, I have a small question that I need to confirm: it seems that there wasn't any code specifically written for adding final metrics to the wandb summary in your code. These results should be generated by the system automatically, right? Because I noticed in the summary document there is content like "Tracking/step": 0, but the tracking shouldn't be 0 steps, usually, it should be a much larger number.
If you could confirm this small issue for me, I would be immensely grateful.