refineNet

[Azpril45]

Dear @moberweger

Thanks for sharing your work.
I have read your paper and have some questions about the hand location refinement.
In your paper, it says that you use the ResNet architecture as the refineNet to refine hand location. However, I look into the code(main_myu_posereg_embedding.py) you released,

comref = None # "./eval/NYU_COM_AUGMENT/net_NYU_COM_AUGMENT.pkl"
di = NYUImporter('../data/NYU/', refineNet=comref)

and I realized that you set the refineNet = comref which is None. In my opinion, it means that hand location refinement hasn't been done.
I downloaded the pretrained models and tried to get the net_NYU_COM_AUGMENT.pkl but it seems that it doesn't exsit.

Can you tell me where I can find the net_NYU_COM_AUGMENT.pkl if my opinion was right about the hand location refinement. And It will be appreciated that you can tell me the right answer if I was wrong about the whole refinement things.

Sincerely yours
April

[moberweger]

Hi,
Thanks for using the code.

Yes, you are correct, the refinement is disabled in the code. You should uncomment the string with the location.
Also I updated the pretrained models with the refine networks. Please download again. You can find them here: Pretrained

[anguszxd]

Hello @moberweger . Thanks for your works.
I'm also a bit confused about the refine net here. It seems to be a bit different with your paper. In your paper DeepPrior++, a plain network architecture is designed for refining hand localization, which is demonstrated in Figure 3 in the paper. However, in this code, a ScaleNet is used as comrefNet. If I'm not mistaken, ScaleNet has a multi-scale architecture and appears in your previous work deep-prior, and ScaleNet works during the second stage. Is there a second stage in deep-prior-pp? I think the pose is given out by ResNet directly, and ScaleNet is used before ResNet to refine the hand localization. I'm not sure about that and hope to hear your answer. Thanks a lot!

[anguszxd]

Oh, I also read your previous code for deep-prior. And I find the ScaleNet is also used to refine the hand location. That is to say, in both codes, the joints location given out by PoseNet are not refined by a second stage?

[moberweger]

The ScaleNet is used for improved localization of the hand (comrefNet). The original DeepPrior paper had also a refinements stage for each hand joint location, but the code is not the the repo. As for comrefNet, the DeepPrior++ paper shows a plain architecture, which performs similar to the ScaleNet architecture for the case of hand localization. For the sake of simplicity, the DeepPrior++ repo uses the ScaleNet architecture that was introduced in the original repo.

[xxxxxxxx-dl]

Hi @moberweger , Thank you very much for sharing your work. I also have the question about the refinement. Both deep-prior and deep-prior-pp are used for hand location refinement but hand joint refinement. The code for hand joint refinement is not published in this repo, is that right? Looking forward to your reply! Thank you again for your time.

[moberweger]

@xxxxxxxx-dl
yes you are right. only refinement for localization is in this repo, not for each joint.

[xxxxxxxx-dl]

@moberweger Thanks a lot for your reply!