Using custom parser

[armheb]

Hi, Great work, and thanks for sharing. can I use the PGN_tf parsing model and use the output in this model? currently, I'm using the U^2 net mode to generate the masks.
I'd appreciate it if you could guide me to test this model on custom images.

[minar09]

Hi @armheb , thanks for your kind words. Yes, I think you can use CIHP_PGN parsing for this model.

Since this model is pre-trained with a converted segmentation (labels = 14), to use PGN, you may need to train the model again.
I think, to do that,

1. you will need to update the segmentation/parsing maps channel numbers (labels from 14 to 20).
2. update the input and output channels of each network (both generator and discriminator) for PGN labels accordingly.
3. Break down the model into separate modules according to the paper and train separately (I am guessing that to train end-to-end the whole model, it would require large memories and GPUs. Of course if you can train the whole model end-to-end that might be even better)

I may be wrong here, please let me know how it goes when you do this.
Also, to test on custom images with the current pre-trained model, you need to generate/convert the segmentation into the format used here (14 labels, there is a issue in the original repository where one of the authors provide the link for how to convert the segmentation: switchablenorms/DeepFashion_Try_On#15).

Hope this helps. Thanks and good luck.

[armheb]

Thanks for your detailed explanation, you're right training the model end-to-end is not easy. I just wanted to compare the results of this model on my custom images since some of the results that were shared were not very good.
as mentioned on the link you shared, I'm not sure how to change the labels of the Self-Correction-Human-Parsing outputs to the 11 labels they used for this model, I'd really appreciate if you could help me with this problem. thanks for your time.

[minar09]

Hi @armheb , I am sorry I haven't tried the Self-Correction-Human-Parsing myself. They seem to provide trained models for LIP segmentation, but the ACGPN author mentioned that their label ordering is different. Maybe they trained their own Self-Correction-Human-Parsing. They do provide the ordering in the readme. I hope you can get successful.

[minar09]

On second thought, maybe you can make a script for converting the parsing labels of your custom images, pillow should work. For example, for UpperClothes label do parsing[parsing == 5] = 4 or something like that. You may need to be careful though so that the label orders do not overlap with another.

[armheb]

Thank you so much for the answers, changing the Self-Correction-Human-Parsing labels as you mentioned solved the problem. thanks.

[abc123yuanrui]

Thank you so much for the answers, changing the Self-Correction-Human-Parsing labels as you mentioned solved the problem. thanks.

For Self-Correction-Human-Parsing labels, since ACGPN combined multiple labels (20=>14), their channel 5 might not be purely channel 5 from original 20 labels dataset. You might also need to combine other clothe label like coat (label 7) or Jumpsuits (label 10) from LIP labels. To be more specifically, multiple labels refer to upper cloth in self correction human parsing, and same issues when processing bottom cloth.

[Hanjunzhe]

@abc123yuanrui @minar09 @armheb Your discussion is very useful. I have a question about how to generate a suitable human body analysis diagram and posture diagram, and whether the corresponding code is provided. If so, I would greatly appreciate it