You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
got the below error while training the custom data
Skipping global process sync, barrier name: create_tmp_directory:post.checkpoint_1
I0716 20:30:05.510361 140313332638656 utils.py:219] Skipping global process sync, barrier name: Checkpointer:save.checkpoint_1
Traceback (most recent call last):
File "/mnt/sdb1/sumant/multi_nerf/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/mnt/sdb1/sumant/multi_nerf/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/mnt/sdb1/sumant/multi_nerf/multinerf/train.py", line 292, in <module>
app.run(main)
File "/mnt/sdb1/sumant/multi_nerf/lib/python3.9/site-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/mnt/sdb1/sumant/multi_nerf/lib/python3.9/site-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
File "/mnt/sdb1/sumant/multi_nerf/multinerf/train.py", line 222, in main
checkpoints.save_checkpoint(
File "/mnt/sdb1/sumant/multi_nerf/lib/python3.9/site-packages/flax/training/checkpoints.py", line 697, in save_checkpoint
orbax_checkpointer.save(
File "/mnt/sdb1/sumant/multi_nerf/lib/python3.9/site-packages/orbax/checkpoint/checkpointer.py", line 186, in save
self._handler.finalize(tmpdir)
File "/mnt/sdb1/sumant/multi_nerf/lib/python3.9/site-packages/orbax/checkpoint/pytree_checkpoint_handler.py", line 759, in finalize
self._handler_impl.finalize(directory)
File "/mnt/sdb1/sumant/multi_nerf/lib/python3.9/site-packages/orbax/checkpoint/base_pytree_checkpoint_handler.py", line 1064, in finalize
type_handlers.merge_ocdbt_per_process_files(directory)
File "/mnt/sdb1/sumant/multi_nerf/lib/python3.9/site-packages/orbax/checkpoint/type_handlers.py", line 635, in merge_ocdbt_per_process_files
parent_tspec = _get_tensorstore_spec(os.fspath(directory), use_ocdbt=True)
File "/mnt/sdb1/sumant/multi_nerf/lib/python3.9/site-packages/orbax/checkpoint/type_handlers.py", line 745, in _get_tensorstore_spec
raise ValueError(f'Checkpoint path should be absolute. Got {directory}')
ValueError: Checkpoint path should be absolute. Got data_colmap/checkpoints/checkpoint_1.orbax-checkpoint-tmp-0
The text was updated successfully, but these errors were encountered:
got the below error while training the custom data
The text was updated successfully, but these errors were encountered: