Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The test results seems to be inconsistent with the paper #7

Closed
runningpp opened this issue Jan 9, 2024 · 2 comments
Closed

The test results seems to be inconsistent with the paper #7

runningpp opened this issue Jan 9, 2024 · 2 comments

Comments

@runningpp
Copy link

I tried your code in Ubuntu20.04 cuda 11.8 pytorch2.0.1
I have got the results:

GTA.mp4

but the corresponding results in your paper is much better obviously, as shown in the following:
GTA

I want to know if my result is reasonable。

@River-Zhang
Copy link
Owner

River-Zhang commented Jan 10, 2024

Thanks for your feedback! That's weird. Here are our results showing in Meshlab:
image1
image2

Could you kindly display the command you executed along with the output produced by the code? Additionally, I would appreciate it if you could execute the testing script. This will allow me to ascertain whether the test results on CAPE are within expectation.

The only explanation I can think of currently is that there are discrepancies between our environments. We didn't adopt PyTorch versions beyond 2.0 due to compatibility issues with certain features.

@runningpp
Copy link
Author

Thanks for your feedback! That's weird. Here are our results showing in Meshlab: image1 image2

Could you kindly display the command you executed along with the output produced by the code? Additionally, I would appreciate it if you could execute the testing script. This will allow me to ascertain whether the test results on CAPE are within expectation.

The only explanation I can think of currently is that there are discrepancies between our environments. We didn't adopt PyTorch versions beyond 2.0 due to compatibility issues with certain features.

The command I used:
''
python -m apps.infer -cfg ./configs/GTA.yaml -gpu 0 -in_dir ./examples -out_dir ./results -loop_smpl 100 -loop_cloth 200 -hps_type pixie
"
Note that: I change the GTA.yaml: normal_path: "./data/ckpt/normal.ckpt" to "./data/ckpt/Normal-epoch=19-v5.ckpt", because the normal mode you provided in CKPT is Normal-epoch=19-v5.ckpt.

The test results in CAPE are:

{'cape-easy-NC': 0.03472820296883583,
'cape-easy-chamfer': 0.6505321264266968,
'cape-easy-p2s': 0.6074833273887634,
'cape-hard-NC': 0.04288707673549652,
'cape-hard-chamfer': 0.9164005517959595,
'cape-hard-p2s': 0.8493930101394653}

The I change the the environments to " cuda11.6 pytorch1.13.0", I found the results between the two environments are almost the same. The following is the results on pyotorch 1.13.0:
{'cape-easy-NC': 0.03472774103283882,
'cape-easy-chamfer': 0.6541368961334229,
'cape-easy-p2s': 0.6101600527763367,
'cape-hard-NC': 0.04288462921977043,
'cape-hard-chamfer': 0.9143836498260498,
'cape-hard-p2s': 0.8439348340034485}

Maybe it is not caused by the environments. The results you showed have much better detailed surface, can you tell me how to get your shown results?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants