Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implicitron:"ValidationError: Unexpected object type: _empty"---video synthesis #1314

Closed
cinamon123 opened this issue Aug 28, 2022 · 16 comments
Assignees
Labels
bug Something isn't working

Comments

@cinamon123
Copy link

🐛 Hello, there is something wrong with video making in implicitron.

INPUT
python pytorch3d/implicitron_trainer/visualize_reconstruction.py exp_dir='pytorch3d/projects/implicitron_trainer/checkpoint' \ visdom_show_preds=True visdom_server=True n_eval_cameras=40 render_size="[64,64]" video_size="[256,256]"

OUTPUT ERROR
raise ValidationError(f"Unexpected object type: {type_str(type_)}") omegaconf.errors.ValidationError: Unexpected object type: _empty full_key: visdom_server object_type=_export_scenes_default_args_
I read many relative py files. visdom_server is defined as str. I have no idea how the error comes.
THANK U

@cinamon123 cinamon123 changed the title ValidationError: Unexpected object type: _empty implicitron:"ValidationError: Unexpected object type: _empty"---video synthesis Sep 2, 2022
@bottler
Copy link
Contributor

bottler commented Sep 5, 2022

I think we need to add enable_get_default_args(export_scenes) after the definition of export_scenes.

@bottler bottler self-assigned this Sep 5, 2022
@bottler bottler added the bug Something isn't working label Sep 5, 2022
@bottler
Copy link
Contributor

bottler commented Sep 5, 2022

There are a few fixes required to visualize_reconstruction and I think you have already made some.

@bottler
Copy link
Contributor

bottler commented Sep 5, 2022

Does it help to change visdom_server to Optional[str]?

@cinamon123
Copy link
Author

cinamon123 commented Sep 6, 2022

Does it help to change visdom_server to Optional[str]?

That did work. Thank you!
But some error in Omegaconf still appeared:

File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 470, in _get_node
   raise ConfigKeyError(f"Missing key {key}")
omegaconf.errors.ConfigAttributeError: Missing key data_source_args
   full_key: data_source_args
   object_type=dict

That seems complex to fix.

There are a few fixes required to visualize_reconstruction and I think you have already made some.

Yes, there is a few bugs in this py file. But I have made only one or two fixes in config.py following the error tips, which is born when reading C in config.py, def enable_get_default_args.
I am just a student. This model is really huge for me. Sometimes I can not always find the root of error. So your help is necessary. Thank you sincerely!!

@bottler
Copy link
Contributor

bottler commented Sep 6, 2022

I am working to fix this all soon. For that error, you need to replace data_source_args with data_source_ImplicitronDataSource_args in one place in visualize_reconstruction.py.

@cinamon123
Copy link
Author

Yes!I forgot that!The name of this args has the same error that I mentioned in #1304、in configs and your markdown file. Thanks! (^ω^)
(It is Ok to fix it later if you need to have a break. It is too late in USA.)

@cinamon123
Copy link
Author

Hello. I have run your fixed files. They killed many bugs, thanks.
But Some error accured in config.py:
raise ValueError(f"{name} has not been registered.")
The {name} print out all lines in expconfig.
ValueError: {'data_source_class_type': 'ImplicitronDataSource', 'model_factory_class_type': 'ImplicitronModelFactory', 'optimizer_factory_class_type': 'ImplicitronOptimizerFactory', 'training_loop_class_type': 'ImplicitronTrainingLoop', 'seed': 42, 'detect_anomaly': False, 'exp_dir': 'llffnerf/ckpts/checkpoints', 'data_source_ImplicitronDataSource_args': {'dataset_map_provider_class_type': 'LlffDatasetMapProvider', 'data_loader_map_provider_class_type': 'SequenceDataLoaderMapProvider', 'dataset_map_provider_LlffDatasetMapProvider_args': {'base_dir': 'llffnerf/llffdata/clst8init', 'object_name': 'clst8init', 'path_manager_factory_class_type': 'PathManagerFactory', 'n_known_frames_for_test': None, 'path_manager_factory_PathManagerFactory_args': {'silence_logs': True}, 'downscale_factor': 4, 'test_on_train': False}, 'data_loader_map_provider_SequenceDataLoaderMapProvider_args': {'batch_size': 1, 'num_workers': 8, 'dataset_length_train': 1000, 'dataset_length_val': 1, 'dataset_length_test': 0, 'train_conditioning_type': 'SAME', 'val_conditioning_type': 'SAME', 'test_conditioning_type': 'KNOWN', 'images_per_seq_options': [], 'sample_consecutive_frames': False, 'consecutive_frames_max_gap': 0, 'consecutive_frames_max_gap_seconds': 0.1}}, 'model_factory_ImplicitronModelFactory_args': {'resume': True, 'model_class_type': 'GenericModel', 'resume_epoch': -1, 'force_resume': False, 'model_GenericModel_args': {'log_vars': ['loss_rgb_psnr_fg', 'loss_rgb_psnr', 'loss_rgb_mse', 'loss_rgb_huber', 'loss_depth_abs', 'loss_depth_abs_fg', 'loss_mask_neg_iou', 'loss_mask_bce', 'loss_mask_beta_prior', 'loss_eikonal', 'loss_density_tv', 'loss_depth_neg_penalty', 'loss_autodecoder_norm', 'loss_prev_stage_rgb_mse', 'loss_prev_stage_rgb_psnr_fg', 'loss_prev_stage_rgb_psnr', 'loss_prev_stage_mask_bce', 'objective', 'epoch', 'sec/it'], 'mask_images': True, 'mask_depths': True, 'render_image_width': 64, 'render_image_height': 64, 'mask_threshold': 0.5, 'output_rasterized_mc': False, 'bg_color': [0.0, 0.0, 0.0], 'num_passes': 2, 'chunk_size_grid': 102400, 'render_features_dimensions': 3, 'tqdm_trigger_threshold': 16, 'n_train_target_views': 1, 'sampling_mode_training': 'mask_sample', 'sampling_mode_evaluation': 'full_grid', 'global_encoder_class_type': None, 'raysampler_class_type': 'AdaptiveRaySampler', 'renderer_class_type': 'MultiPassEmissionAbsorptionRenderer', 'image_feature_extractor_class_type': None, 'view_pooler_enabled': False, 'implicit_function_class_type': 'NeuralRadianceFieldImplicitFunction', 'view_metrics_class_type': 'ViewMetrics', 'regularization_metrics_class_type': 'RegularizationMetrics', 'loss_weights': {'loss_rgb_mse': 1.0, 'loss_prev_stage_rgb_mse': 1.0, 'loss_mask_bce': 1.0, 'loss_prev_stage_mask_bce': 1.0, 'loss_autodecoder_norm': 0.01}, 'raysampler_AdaptiveRaySampler_args': {'n_pts_per_ray_training': 64, 'n_pts_per_ray_evaluation': 64, 'n_rays_per_image_sampled_from_mask': 1024, 'stratified_point_sampling_training': True, 'stratified_point_sampling_evaluation': False, 'scene_extent': 8.0, 'scene_center': [0.0, 0.0, 0.0]}, 'renderer_MultiPassEmissionAbsorptionRenderer_args': {'raymarcher_class_type': 'EmissionAbsorptionRaymarcher', 'n_pts_per_ray_fine_training': 64, 'n_pts_per_ray_fine_evaluation': 64, 'stratified_sampling_coarse_training': True, 'stratified_sampling_coarse_evaluation': False, 'append_coarse_samples_to_fine': True, 'density_noise_std_train': 1.0, 'return_weights': False, 'raymarcher_EmissionAbsorptionRaymarcher_args': {'surface_thickness': 1, 'bg_color': [0.0], 'background_opacity': 10000000000.0, 'density_relu': True, 'blend_output': False}}, 'implicit_function_NeuralRadianceFieldImplicitFunction_args': {'n_harmonic_functions_xyz': 10, 'n_harmonic_functions_dir': 4, 'n_hidden_neurons_dir': 128, 'input_xyz': True, 'xyz_ray_dir_in_camera_coords': False, 'transformer_dim_down_factor': 1.0, 'n_hidden_neurons_xyz': 256, 'n_layers_xyz': 8, 'append_xyz': [5]}, 'view_metrics_ViewMetrics_args': {}, 'regularization_metrics_RegularizationMetrics_args': {}}}, 'optimizer_factory_ImplicitronOptimizerFactory_args': {'betas': [0.9, 0.999], 'breed': 'Adam', 'exponential_lr_step_size': 250, 'gamma': 0.1, 'lr': 0.0005, 'lr_policy': 'MultiStepLR', 'momentum': 0.9, 'multistep_lr_milestones': [], 'weight_decay': 0.0, 'linear_exponential_lr_milestone': 200, 'linear_exponential_start_gamma': 0.1}, 'training_loop_ImplicitronTrainingLoop_args': {'eval_only': False, 'evaluator_class_type': 'ImplicitronEvaluator', 'max_epochs': 1000, 'store_checkpoints': True, 'store_checkpoints_purge': 1, 'test_interval': -1, 'test_when_finished': False, 'validation_interval': 1, 'clip_grad': 0.0, 'metric_print_interval': 5, 'visualize_interval': 0, 'visdom_env': '', 'visdom_port': 8097, 'visdom_server': 'http://127.0.0.1', 'evaluator_ImplicitronEvaluator_args': {'camera_difficulty_bin_breaks': [0.97, 0.98], 'is_multisequence': False}}} has not been registered.
expconfig is produced by nerf itself. Is that possible there is something wrong in llff_nerf.yaml written by myself?
Thank you.

@bottler
Copy link
Contributor

bottler commented Sep 8, 2022

This may be a simple code problem. Have you updated with the fixes from 6e25fe8 (from a few hours ago)? In that I changed Experiment(config) to Experiment(**config) which was a fix for that error. If you have the change but you still see that error, can you paste the call stack where the error occurs?

@cinamon123
Copy link
Author

Great!I solved that error with your help. Thank you.
Now, I am fixing some new bugs:

Traceback (most recent call last):
  File "usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron_trainer/visualize_reconstruction.py", line 153, in <module>
    main(sys.argv)
  File "usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron_trainer/visualize_reconstruction.py", line 149, in main
    visualize_reconstruction(**cfg)
  File "usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron_trainer/visualize_reconstruction.py", line 118, in visualize_reconstruction
    render_flyaround(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron/models/visualization/render_flyaround.py", line 133, in render_flyaround
    seq_idx = list(dataset.sequence_indices_in_order(sequence_name))
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron/dataset/dataset_base.py", line 303, in sequence_indices_in_order
    for _, _, idx in self.sequence_frames_in_order(seq_name):
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron/dataset/dataset_base.py", line 289, in sequence_frames_in_order
    seq_frame_indices = self._seq_to_idx[seq_name]
AttributeError: 'SingleSceneDataset' object has no attribute '_seq_to_idx'

I added this line in database.py in class DatasetBase:
_seq_to_idx: Dict[str, List[int]] = field(init=False)

Then it seems worse:

Traceback (most recent call last):
  File "usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron_trainer/visualize_reconstruction.py", line 26, in <module>
    from pytorch3d.implicitron_trainer.experiment import Experiment
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron_trainer/experiment.py", line 273, in <module>
    cs.store(name="default_config", node=Experiment)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/hydra/core/config_store.py", line 85, in store
    cfg = OmegaConf.structured(node)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 119, in structured
    return OmegaConf.create(obj, parent, flags)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 172, in create
    return OmegaConf._create_impl(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 832, in _create_impl
    return DictConfig(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 113, in __init__
    format_and_raise(node=None, key=key, value=None, cause=ex, msg=str(ex))
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 97, in __init__
    self._set_value(content, flags=flags)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 638, in _set_value
    raise e
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 635, in _set_value
    self._set_value_impl(value, flags)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 665, in _set_value_impl
    data = get_structured_config_data(value, allow_objects=ao)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/_utils.py", line 442, in get_structured_config_data
    return get_dataclass_data(obj, allow_objects=allow_objects)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/_utils.py", line 354, in get_dataclass_data
    value = field.default_factory()  # type: ignore
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron/tools/config.py", line 918, in create
    args = get_default_args(C, _do_not_process=_do_not_process)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron/tools/config.py", line 518, in get_default_args
    out: DictConfig = OmegaConf.structured(C)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 119, in structured
    return OmegaConf.create(obj, parent, flags)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 172, in create
    return OmegaConf._create_impl(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 832, in _create_impl
    return DictConfig(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 113, in __init__
    format_and_raise(node=None, key=key, value=None, cause=ex, msg=str(ex))
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 97, in __init__
    self._set_value(content, flags=flags)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 638, in _set_value
    raise e
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 635, in _set_value
    self._set_value_impl(value, flags)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 665, in _set_value_impl
    data = get_structured_config_data(value, allow_objects=ao)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/_utils.py", line 442, in get_structured_config_data
    return get_dataclass_data(obj, allow_objects=allow_objects)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/_utils.py", line 354, in get_dataclass_data
    value = field.default_factory()  # type: ignore
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron/tools/config.py", line 918, in create
    args = get_default_args(C, _do_not_process=_do_not_process)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron/tools/config.py", line 518, in get_default_args
    out: DictConfig = OmegaConf.structured(C)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 119, in structured
    return OmegaConf.create(obj, parent, flags)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 172, in create
    return OmegaConf._create_impl(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 832, in _create_impl
    return DictConfig(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 113, in __init__
    format_and_raise(node=None, key=key, value=None, cause=ex, msg=str(ex))
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 97, in __init__
    self._set_value(content, flags=flags)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 638, in _set_value
    raise e
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 635, in _set_value
    self._set_value_impl(value, flags)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 665, in _set_value_impl
    data = get_structured_config_data(value, allow_objects=ao)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/_utils.py", line 442, in get_structured_config_data
    return get_dataclass_data(obj, allow_objects=allow_objects)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/_utils.py", line 354, in get_dataclass_data
    value = field.default_factory()  # type: ignore
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron/tools/config.py", line 918, in create
    args = get_default_args(C, _do_not_process=_do_not_process)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron/tools/config.py", line 518, in get_default_args
    out: DictConfig = OmegaConf.structured(C)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 119, in structured
    return OmegaConf.create(obj, parent, flags)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 172, in create
    return OmegaConf._create_impl(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 832, in _create_impl
    return DictConfig(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 113, in __init__
    format_and_raise(node=None, key=key, value=None, cause=ex, msg=str(ex))
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 97, in __init__
    self._set_value(content, flags=flags)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 638, in _set_value
    raise e
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 635, in _set_value
    self._set_value_impl(value, flags)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 665, in _set_value_impl
    data = get_structured_config_data(value, allow_objects=ao)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/_utils.py", line 442, in get_structured_config_data
    return get_dataclass_data(obj, allow_objects=allow_objects)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/_utils.py", line 362, in get_dataclass_data
    d[name] = _maybe_wrap(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 1041, in _maybe_wrap
    return _node_wrap(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 971, in _node_wrap
    node = DictConfig(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 113, in __init__
    format_and_raise(node=None, key=key, value=None, cause=ex, msg=str(ex))
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/dictconfig.py", line 78, in __init__
    metadata=ContainerMetadata(
  File "<string>", line 12, in __init__
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/omegaconf/base.py", line 76, in __post_init__
    assert self.element_type is Any or isinstance(self.element_type, type)
AssertionError

@bottler
Copy link
Contributor

bottler commented Sep 11, 2022

Have you got the whole of the changes in 6e25fe8? The SingleSceneDataset change in that commit should be a proper way to avoid the first error.

What version of OmegaConf do you have by the way? If you /do/ need to diagnose the second error, you can add print(C) before line 518 of config.py to see what the object is which OmegaConf.structured does not like. (I hope to add a handler to add this info to the error.)

@cinamon123
Copy link
Author

Thats my fault. I updated the whole changes. The error disappeared. But video still can not be written in checkpoint/vis, I am trying to fix these. May be my ffmpeg version has something wrong(for the first error).

built with gcc 7.3.0 (crosstool-NG 1.23.0.449-a04d0)
  configuration: --prefix=/usr/local/miniconda3/envs/pytorch3d --cc=/opt/conda/conda-bld/ffmpeg_1597178665428/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc --disable-doc --disable-openssl --enable-avresample --enable-gnutls --enable-hardcoded-tables --enable-libfreetype --enable-libopenh264 --enable-pic --enable-pthreads --enable-shared --disable-static --enable-version3 --enable-zlib --enable-libmp3lame
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
Unrecognized option 'crf'.
Error splitting the argument list: Option not found

The second error :(I am searching for reason now)

 main(sys.argv)
  File "usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron_trainer/visualize_reconstruction.py", line 149, in main
    visualize_reconstruction(**cfg)
  File "usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron_trainer/visualize_reconstruction.py", line 118, in visualize_reconstruction
    render_flyaround(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron/models/visualization/render_flyaround.py", line 197, in render_flyaround
    _generate_prediction_videos(
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron/models/visualization/render_flyaround.py", line 362, in _generate_prediction_videos
    vws[k].get_video()
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/site-packages/pytorch3d/implicitron/tools/video_writer.py", line 159, in get_video
    subprocess.check_call(args)
  File "/usr/local/miniconda3/envs/pytorch3d/lib/python3.8/subprocess.py", line 364, in check_call
 raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ffmpeg', '-r', '20', '-i', '/tmp/tmprwrr9mxl/frame_%08d.png', '-vcodec', 'h264', '-f', 'mp4', '-y', '-crf', '18', '-b', '2000k', '-pix_fmt', 'yuv420p', 'llffnerf/ckpts/checkpoint/vis/video_one_sequence_images_render.mp4']' returned non-zero exit status 1.

This is my omegaconf version:
omegaconf 2.1.2 down by pypi. (In case that the error is gown, handle is not so necessary, right?)
I am sorry to bother you and thankful for all the fixes you did.

@bottler
Copy link
Contributor

bottler commented Sep 12, 2022

Your version of ffmpeg may be an odd one. If you are running in a conda environment, I think you can get a local one in your environment with conda install ffmpeg (but be careful it doesn't propose to remove or change important packages).

@cinamon123
Copy link
Author

Ok, I will try it. By the way, is the new video synthesis module just fit for JsonindexData? I can not get good render images as I got in train_loop.

@bottler
Copy link
Contributor

bottler commented Sep 12, 2022

What do you mean by the "new video synthesis module"? Do you mean evaluate_new_view_synthesis? If so, no, that should be useful for evaluating any implicitron model. If it is giving poor results with a certain dataset, that should mean that the training did not do well.

@cinamon123
Copy link
Author

No, I refer to visualization_reconstruction.py and other modules, which are used to make a video. Are they just fit for JsonindexData?

@cinamon123
Copy link
Author

I think I have got my answer to these questions. I will close this issue.
Thank you sincerely, for your patience and work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants