Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can provide Normal model? #1

Open
bbk40221 opened this issue Jan 29, 2024 · 17 comments
Open

Can provide Normal model? #1

bbk40221 opened this issue Jan 29, 2024 · 17 comments

Comments

@bbk40221
Copy link

In train_Normal/demo_Normal.py, the model path is "/media/lx/4A42-E0B2/code_github/SePIFU_08_15into/checkpoints/model_Normal-Net/Train_Normal_epoch_27_1984",

netG = NormalNet(opt)
        netG.to(cuda)
        # 2023/05/06 we get the 27_1984
        model_path = '/media/lx/4A42-E0B2/code_github/SePIFU_08_15into/checkpoints/model_Normal-Net/Train_Normal_epoch_27_1984'
        netG.load_state_dict(torch.load(model_path, map_location=cuda))
        netG.eval()

This model can't be found in repository, can provide this model?
Thanks.

@lxzlwhy
Copy link
Collaborator

lxzlwhy commented Jan 30, 2024

Hi, because our paper is still under review, we will publish the normal model when the paper is about to be accepted. Please wait for a while. Thank you for your interest in our work.

@FernandoConti
Copy link

Hi, is there an update on the Normal model release? Thank you so much in advance!

@Megidd
Copy link

Megidd commented Apr 29, 2024

Any update on it?

Can we use such tools to create normal maps:

https://github.com/brycedrennan/imaginairy-normal-map

@FernandoConti
Copy link

FernandoConti commented Apr 29, 2024

Hi Megidd, why didn't that codebase work? Just curious :-)

@Megidd
Copy link

Megidd commented Apr 30, 2024

@FernandoConti I have shown below two normal maps:

As can be observed, the SEIF normal model - left one - creates much more fine details:

2024-04-29 06_18_05-Skype - Copy

The right map creates a messed up 3D model eventually, maybe due to the normal map lacking the fine details:

2024-04-29 06_17_28-Skype - Copy

@FernandoConti
Copy link

FernandoConti commented May 1, 2024

Thank you so much for showing me! Yes, much less detail and incorrect normals as well (e.g. cheeks).

Okay, so unfortunately we're not hearing anything from the creators, maybe we can train a new normal model ourselves. We need a library of highres head scans to train a new model, I'm going to research that.

@Megidd
Copy link

Megidd commented May 4, 2024

@FernandoConti That's right.

We believe that it might be possible to just train their normal model in some weeks. Considering the fact that SEIF is apparently using public datasets for their training.

@FernandoConti
Copy link

FernandoConti commented May 4, 2024

@Megidd Yes, I believe that as well. Unfortunately all 3D databases I found need academic approval.

Okay well, please keep me updated on your progress, I will do the same!

@leyuanCv
Copy link
Contributor

leyuanCv commented May 6, 2024

Thank you for your interest in our work. We will upload the trained normal estimation model soon.

@Comfyux
Copy link

Comfyux commented Jun 10, 2024

@leyuanCv Thank you for your response regarding the normal estimation model. We understand that you will release the trained model soon, and we greatly appreciate your efforts in sharing this valuable resource with the community.

If possible, could you please provide an estimated timeline for when we can expect the model release? This information will help us plan and prepare accordingly for integrating the model into our workflow.

We are eager to explore and utilize this advanced normal estimation model, and we look forward to your update. Once again, thank you for your contribution to the field.

@Comfyux
Copy link

Comfyux commented Jun 10, 2024

@FernandoConti Thanks for your interest in training a new normal model. I noticed you mentioned researching to find a library of high-res head scans to train on. If you haven't found a suitable dataset yet, I may be able to provide some training data. I have access to a collection of 3D face scans that could potentially work.

If you would be interested in collaborating and want to discuss further, feel free to reach out to me at qu1989x@gmail.com . I'd be happy to share more details about the data I have available. Combining our efforts, we may be able to make progress on training an improved normal estimation model.

Please let me know if you would like to explore this possibility together. I look forward to hearing from you.

@FernandoConti
Copy link

Hello @Comfyux , thank you for reaching out to me!

The normal estimation model download link has been added to this repo, but it is impossible for us to download it, we cannot access the Baidu network.

Would you maybe be able to help us?

@nvdomidi
Copy link

@FernandoConti

I was able to download the file. I've shared it here for others who don't have access to baidu.

@FernandoConti
Copy link

@nvdomidi Thank you so much!

@Megidd
Copy link

Megidd commented Jun 22, 2024

Approach

We tested by these steps:

  1. Cloned this repo on master branch.
  2. Created a checkpoints folder in the main directory, then pasted the downloaded normal estimate model Train_Normal_epoch_27_1984 into that folder.
  3. Took our custom image and renamed it to 1.png and 1_mask.png and put them into the image folder.
  4. Ran the demo.py and got 1_normal.png.
  5. Pasted 1.png, 1_mask.png and 1_normal.png into SeIF's demo folder.
  6. Ran main_test.py.

Result

Our result is shown below. @FernandoConti, I'm just curious, what do your results look like?

SeIF_result1

@FernandoConti
Copy link

Hi @Megidd, we have similar results, they are pretty bad. Apparently the creators have been cherry picking their results...

@leyuanCv
Copy link
Contributor

Hi @Megidd, we have similar results, they are pretty bad. Apparently the creators have been cherry picking their results...

Could you please share your input images so we can investigate the cause? We have tested up to 25,000 front-facing images without glasses. It is disheartening to receive such feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants