Problem about loading the torso pretrained models

[sstzal]

Hi, yudong
When I try to load the torso pretrained models for a better initialization. I get an error.

So why the pretrained models donot have the ['network_audattnet_state_dict']. How can I handle this issue?
Thanks!

[sstzal]

Hi, yudong
When I try to load the torso pretrained models for a better initialization. I get an error.

So why the pretrained models donot have the ['network_audattnet_state_dict']. How can I handle this issue?
Thanks!

Should I set the 'with_audatt' to be False?

[YudongGuo]

Hi, yudong
When I try to load the torso pretrained models for a better initialization. I get an error.

So why the pretrained models donot have the ['network_audattnet_state_dict']. How can I handle this issue?
Thanks!

Should I set the 'with_audatt' to be False?

Yep. The reason is that the torso model shares the AudioAttention module with the head model, so there is no need to load it again.

[YudongGuo]

Hi, yudong
When I try to load the torso pretrained models for a better initialization. I get an error.

So why the pretrained models donot have the ['network_audattnet_state_dict']. How can I handle this issue?
Thanks!

Should I set the 'with_audatt' to be False?

Yep. The reason is that the torso model shares the AudioAttention module with the head model, so there is no need to load it again.

From your output, the head model does not contain the audattnet, didn't you include it in the head model training?

[sstzal]

Hi, yudong
When I try to load the torso pretrained models for a better initialization. I get an error.

So why the pretrained models donot have the ['network_audattnet_state_dict']. How can I handle this issue?
Thanks!

Should I set the 'with_audatt' to be False?

Yep. The reason is that the torso model shares the AudioAttention module with the head model, so there is no need to load it again.

From your output, the head model does not contain the audattnet, didn't you include it in the head model training?

I have audattnet in the headnerf. I just mistakenly replaced the ckpt to the pretrained troso model ( add "--ft_path ./AD-NeRF-master/pretrained_models/500000_body.tar" to the command). Now I'm a little confused about how to load the pre-training troso model.

[YudongGuo]

You can just put your trained head model and the pretrained torso model in the torso checkpoint folder (dataset/$id/logs/$id_com by default). You also need to manually adjust the 'global_step' in line 789 to 0.

[sstzal]

I find the torso model has these keys:

and the head model has these:

So when I train the torso nerf model , shoule I load

from troso model and

from head model?

[sstzal]

You can just put your trained head model and the pretrained torso model in the torso checkpoint folder (dataset/$id/logs/$id_com by default). You also need to manually adjust the 'global_step' in line 789 to 0.

I read the code. And I think if i put these two model in the dataset/$id/logs/$id_com. The code will only load one model since ckpt_path = ckpts[-1]. Can you tell me what's wrong with my understanding?

[YudongGuo]

You can just put your trained head model and the pretrained torso model in the torso checkpoint folder (dataset/$id/logs/$id_com by default). You also need to manually adjust the 'global_step' in line 789 to 0.

I read the code. And I think if i put these two model in the dataset/$id/logs/$id_com. The code will only load one model since ckpt_path = ckpts[-1]. Can you tell me what's wrong with my understanding?

In the code here, it only searches the file containing specific substring ('head' or 'torso').

[sstzal]

I get it! Thanks!