masked (mustard) dataset much slower with main than PR2025

[machenmusik]

On my machine

• using the mustard dataset with masks (see Training with masks results in artefacts outside the mask regions #1498 (comment) ) with PR add transparency carving to nerfacto #2025 had default nerfacto iteration times around 80 to 90 ms
• using same dataset with main (currently d18588d ) has default nerfacto iteration times around 400 ms
  so 4-5x worse.

[machenmusik]

Suggested viewer crop settings for mustard:

Enable
Crop Min -0.15 -0.20 -0.25
Crop Max 0.15 0.20 0.33

[machenmusik]

d4b0437 is 80 to 90ms
3b0f758 is ~400ms

So looks to be a side effect of #1730 for datasets using masks.

[machenmusik]

Removing "mask" from https://github.com/nerfstudio-project/nerfstudio/blob/main/nerfstudio/data/datasets/base_dataset.py#L45 restores full speed for mustard dataset.

Perhaps there should be a parameter to request masks on GPU that would do the above at runtime.

[YifuTao]

I faced similar issue. The training speed drops from 170k/s to 6k/s so more than 20 times slower. Indeed #1730 reduced GPU memory when using mask, but the speed is compromised

[machenmusik]

#2110 should resolve this when the new option is used.