How to reconstruct the background scene and how to render custom poses?

[zjj421]

Hello, thanks for your work!
I'm new to this direction, I have 2 question:

1. When I have trained the cutom scene model, how to use the trained scene model to reconstruct the background scene(Remove the human in the image), is there some code to do this?
2. After trained the custom scene model and human model, how to render the novel poses or other custom poses. ("python render_reposing.py --scene_dir ./data/custom_data --weights_path ./out/custom_data_human/checkpoint.pth.tar --motion_name=jumpandroll
   " can not get satisfactory result.)Can you teach me what to learn to do this?

[jiangwei221]

1. Yes, you can refer to this code snippet:

   ml-neuman/trainers/vanilla_nerf_trainer.py

   Lines 155 to 165 in 0149d25

   	fine_render, fine_depth = render_utils.render_vanilla(
   	self.coarse_net,
   	dummy_cap,
   	fine_net=self.fine_net,
   	rays_per_batch=self.opt.rays_per_batch,
   	samples_per_ray=self.opt.samples_per_ray,
   	white_bkg=self.opt.white_bkg,
   	near_far_source='bkg',
   	return_depth=True,
   	ablate_nerft=self.opt.ablate_nerft
   	)

   , basically use the render_vanilla function.
2. To render novel poses, you need to manually align the human/motion to the scene. You can refer to: How to change new poses of rendering? #42 and Is there any details of learned human models can be reposed to novel driving motions #26