Make colmapDataParser compatible with 360_v2 dataset format #2860

Jing1Ling · 2024-02-01T10:40:33Z

In order to be compatible with the original format of the 360_v2 dataset without reprocessing it, I added an option to colmapDataParser to round up the image size when downscaling. This may change the image height and width in the camera.
But I'm not sure if it will have any negative impact on other parts? If so, please let me know. Thank you!

jb-ye · 2024-02-01T20:19:54Z

nerfstudio/data/dataparsers/colmap_dataparser.py

@@ -56,6 +56,8 @@ class ColmapDataParserConfig(DataParserConfig):
    """How much to scale the camera origins by."""
    downscale_factor: Optional[int] = None
    """How much to downscale images. If not set, images are chosen such that the max dimension is <1600px."""
+    downscale_rounding_mode: Literal["floor", "ceil"] = "floor"


The ceiling mode isn't compatible with the ffmpeg commands.

See here: https://github.com/nerfstudio-project/nerfstudio/blob/main/nerfstudio/data/dataparsers/colmap_dataparser.py#L464-L468

You probably need to change this part and test the auto-downscale implementation.

Hi @jb-ye, I've changed this part. I ran 'ns-train nerfacto --data xxx colmap --downscale_rounding_mode round --downscale_factor 4' and it worked when there only original image folder and colmap folder under the data path. Do you have any other suggestions:)?

sib1 · 2024-02-19T15:59:22Z

Thank for this implementation @Jing1Ling.

I tried using it and discovered that the 360_v2 dataset does not round up when downscaling, but actually rounds to the nearest integer with the half up rule i.e. 7.5 -> 8, 7.7 -> 8, 7.3 -> 7

You can see an example of this behaviour in the Garden scene. The original resolution of 5187 x 3361 gets downscaled by 2x to 2593.5 x 1680.5 -> 2594 x 1681. And it gets downscaled by 4x to 1296.75 x 840.25 -> 1297 x 840.

A solution which works for me is:

self.height = torch.floor(0.5 + (self.height * scaling_factor)).to(torch.int64)
self.width = torch.floor(0.5 + (self.width * scaling_factor)).to(torch.int64)

I haven't tested compatibility with the ffmpeg commands and the auto-downscale implementation.

Jing1Ling · 2024-02-23T04:19:59Z

Thanks for your reminder @sib1. For my implementation, maybe I can add a round mode to deal with the situation you mentioned. I will try to update it in the future.

jb-ye · 2024-04-08T15:17:05Z

nerfstudio/data/dataparsers/colmap_dataparser.py

@@ -488,7 +514,7 @@ def get_fname(parent: Path, filepath: Path) -> Path:
        if self._downscale_factor is None:
            if self.config.downscale_factor is None:
                test_img = Image.open(filepath)
-                h, w = test_img.size
+                w, h = test_img.size


Thanks for catching the error

…ownscaling

commit d9b022b Author: J.Y <132313008+jb-ye@users.noreply.github.com> Date: Fri Apr 19 14:03:55 2024 -0400 Allow alpha transparency training for blender datasets input (nerfstudio-project#3088) Allow alpha transparency training for rgba input Co-authored-by: Justin Kerr <justin.g.kerr@gmail.com> commit eddf2d2 Author: Congrong Xu <50019703+KevinXu02@users.noreply.github.com> Date: Thu Apr 18 17:37:50 2024 -0700 Changes for trainer.py to support the Gradio webui (nerfstudio-project#3046) * changes for trainer to support webui * Update trainer to support webui * format * add a seperated shutdown() function to stop training * typo fix * get rid of _stop_viewer_server() * Update trainer.py * organize import --------- Co-authored-by: Brent Yi <yibrenth@gmail.com> commit 45d8bb7 Author: Justin Kerr <justin.g.kerr@gmail.com> Date: Wed Apr 17 23:54:43 2024 -0700 updates to apply_pca_colormap (nerfstudio-project#3086) * improvements to pca_colormap: allow input pca matrix, optional ignore_zeros arg * typo commit babf577 Author: Patrick Morgan <patrick@patrick-morgan.net> Date: Fri Apr 12 14:23:47 2024 -0700 Fix call to detect ODM reconstruction file (nerfstudio-project#3018) commit a64026f Author: Justin Kerr <justin.g.kerr@gmail.com> Date: Fri Apr 12 12:32:22 2024 -0700 shift principle points by .5 in undistortion (nerfstudio-project#3071) * shift principle points by .5 * fix ordering * add fisheye offset commit c302cc5 Author: Gunjan Chhablani <chhablani.gunjan@gmail.com> Date: Thu Apr 11 16:41:14 2024 -0400 Add pointcloud loading for blender parser (nerfstudio-project#2902) * Add pointcloud loading for blender parser * Fix argument * Fix docstring * Fix metadata in dataparsers * Add changes from review * Avoid changing default behavior * Apply `scale_factor` to loaded points --------- Co-authored-by: Brent Yi <yibrenth@gmail.com> commit 03bdeff Author: hyperzy <50564821+hyperzy@users.noreply.github.com> Date: Thu Apr 11 14:56:37 2024 -0400 Fixed Unexpectedly huge RAM consumption of parallel datamanager (nerfstudio-project#3060) * Reduced the memory usage by caching image after the datamanager process starts commit 4714ae7 Author: J.Y <132313008+jb-ye@users.noreply.github.com> Date: Thu Apr 11 14:37:02 2024 -0400 write eval combined images for diagnostics (nerfstudio-project#3070) commit eba72db Author: Otto Seiskari <otto.seiskari@gmail.com> Date: Thu Apr 11 21:23:17 2024 +0300 Camera pose optimization for Splatfacto (nerfstudio-project#2891) * Add pose optimization to Splatfacto * Disable Splatfacto pose optimization by default * Improve apply_to_camera for Gaussian Splatting pose optimization Do not chain modifications to camera_to_worlds to improve numerical stability and enable L2 rot/trans penalties. * Add separate mean and max rot/trans metrics to camera-opt * Tweak pose optimization hyperparameters Parameters used in the Gaussian Splatting on the Move paper v1 * Unit test fix for new cameara_optimizer training metrics * Adjust splatfacto-big camera pose optimization parameters Same parameters as in normal Splatfacto --------- Co-authored-by: jh-surh <jh.surh@bucketplace.net> commit 2d9bbe5 Author: dberga <dberga@users.noreply.github.com> Date: Thu Apr 11 04:04:33 2024 +0200 fix exporter.py for FullImageDataManager and RandomCamerasDataManager (nerfstudio-project#3065) * fix exporter.py for FullImageDataManager and RandomCamerasDataManager * sort imports * ruff --------- Co-authored-by: Brent Yi <yibrenth@gmail.com> commit c381fc8 Author: Justin Kerr <justin.g.kerr@gmail.com> Date: Wed Apr 10 18:59:20 2024 -0700 Fix custom elements nested under @Property (nerfstudio-project#3066) * patch custom elements bug * Skip `cached_property` * fomrat * comment, type hint --------- Co-authored-by: Brent Yi <yibrenth@gmail.com> commit d1fc2ee Author: Saurav Maheshkar <sauravvmaheshkar@gmail.com> Date: Mon Apr 8 23:46:00 2024 +0100 feat(ci): migrate to `uv` in CI (nerfstudio-project#3038) * feat(ci): migrate to `uv` in CI * feat(ci): drop upgrade strategy * feat(ci): use `uv` in docs workflow * fix(ci/doc): update sphinx build cmd * fix(ci/doc): specify python path * (experimental) simplify * remove venv * Revert line wrap change for doc.yml --------- Co-authored-by: Brent Yi <yibrenth@gmail.com> commit 2b91715 Author: Sahil Jain <sahilja2@andrew.cmu.edu> Date: Mon Apr 8 13:04:20 2024 -0400 Fix config loading for Eyeful Tower dataset (nerfstudio-project#3054) Add test_filenames to transforms.json generated by eyeful_tower.py commit 26804f8 Author: Ling Jing <111032642+Jing1Ling@users.noreply.github.com> Date: Tue Apr 9 00:07:21 2024 +0800 Make colmapDataParser compatible with 360_v2 dataset format (nerfstudio-project#2860) * added an option to colmapdataparser to round up the image size when downscaling * add round mode and update ffmpeg command * [fix] wrong variable order * update format --------- Co-authored-by: Jing <jing1ling@intel.com> commit 911091c Author: Jan-Niklas Dihlmann <j.dihlmann@googlemail.com> Date: Mon Apr 8 16:17:56 2024 +0200 Added SIGNeRF to Nerfstudio methods (nerfstudio-project#3053) * feat: signerf as nerfstudio method * fix: formating issues and indexing

…io-project#2860) * added an option to colmapdataparser to round up the image size when downscaling * add round mode and update ffmpeg command * [fix] wrong variable order * update format --------- Co-authored-by: Jing <jing1ling@intel.com>

jb-ye reviewed Feb 1, 2024

View reviewed changes

Jing1Ling requested a review from jb-ye April 8, 2024 02:34

Jing1Ling mentioned this pull request Apr 8, 2024

add support for nerfstudio SuLvXiangXin/zipnerf-pytorch#98

Merged

jb-ye reviewed Apr 8, 2024

View reviewed changes

jb-ye approved these changes Apr 8, 2024

View reviewed changes

Jing1Ling and others added 4 commits April 8, 2024 11:17

added an option to colmapdataparser to round up the image size when d…

c9e1181

…ownscaling

add round mode and update ffmpeg command

0b54f98

[fix] wrong variable order

42d4ba5

update format

b220a5b

jb-ye force-pushed the colmap-parser-compatibility-360v2 branch from d36e918 to b220a5b Compare April 8, 2024 15:17

jb-ye merged commit 26804f8 into nerfstudio-project:main Apr 8, 2024
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make colmapDataParser compatible with 360_v2 dataset format #2860

Make colmapDataParser compatible with 360_v2 dataset format #2860

Jing1Ling commented Feb 1, 2024

jb-ye Feb 1, 2024

Jing1Ling Apr 3, 2024

sib1 commented Feb 19, 2024 •

edited

Jing1Ling commented Feb 23, 2024

jb-ye Apr 8, 2024

Make colmapDataParser compatible with 360_v2 dataset format #2860

Make colmapDataParser compatible with 360_v2 dataset format #2860

Conversation

Jing1Ling commented Feb 1, 2024

jb-ye Feb 1, 2024

Choose a reason for hiding this comment

Jing1Ling Apr 3, 2024

Choose a reason for hiding this comment

sib1 commented Feb 19, 2024 • edited

Jing1Ling commented Feb 23, 2024

jb-ye Apr 8, 2024

Choose a reason for hiding this comment

sib1 commented Feb 19, 2024 •

edited