Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stage 1 - training losses - todo contrastive_loss #14

Closed
johndpope opened this issue May 24, 2024 · 5 comments
Closed

Stage 1 - training losses - todo contrastive_loss #14

johndpope opened this issue May 24, 2024 · 5 comments

Comments

@johndpope
Copy link
Owner

https://github.com/johndpope/MegaPortrait-hack/blob/main/train.py#L196

adding any of the others hits a bug with gradients.

@johndpope johndpope changed the title Stage 1 - training losses - only gaze loss is implemented for now. Stage 1 - training losses - todo contrastive_loss May 25, 2024
@Jie-zju
Copy link

Jie-zju commented May 28, 2024

Have you noticed "losses are calculated using only foreground regions" mentioned in Paper? I tried implement the network (include somewhat different from paper ) and train it. However, just use loss on all images for training seems fail, since the training stage is not stable. I think "losses are calculated using only foreground regions" May Be Important ! Please attention on it!

@johndpope
Copy link
Owner Author

thanks for pointing this out. I fix later.

@johndpope
Copy link
Owner Author

johndpope commented May 28, 2024

i draft this - yet to test it out.
https://github.com/johndpope/MegaPortrait-hack/pull/21/files

so much flux - needs stabilising.
fyi @robinchm

I look at the patchgan stuff

@johndpope
Copy link
Owner Author

warped_driving_frame:torch.Size([3, 129, 129])

on my training branch - i implement and fix - but now the warped / cropped image is small....
i can make it bigger - or I have to work out how the network can accept smaller images....
not sure.
https://github.com/johndpope/MegaPortrait-hack/tree/feat/14-training

@johndpope
Copy link
Owner Author

Screenshot from 2024-05-30 12-15-54

some progress.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants