Repository navigation
Releases
Releases · lucmos/anypy
0.1.1
0.1.0
0.0.6
Highlights
🚀 Ensure equal importance of loss penalties in terms of grad norms in a specific layer 🚀
Other weights can then be applied, e.g. to have a loss term that influences the weights 50% more than the other.
# Given two losses
loss_a, loss_b
# Compute the loss_b weight
loss_b_weight = calculate_adaptive_weight(loss_a, loss_b, layer)
loss_b_weighted = loss_b_weight * loss_b
# So that loss_a and loss_b have equal importance in the specified layer grads
loss_a_gradnorm = torch.norm(torch.autograd.grad(loss_a, layer, retain_graph=True)[0])
loss_b_weighted_gradnorm = torch.norm(torch.autograd.grad(loss_b_weighted, layer, retain_graph=True)[0])
assert torch.allclose(loss_a_gradnorm, loss_b_weighted_gradnorm)What's Changed
- Disable pivot sorting in latex df by @lucmos in #12
- Add function to compute adaptive loss weights by @lucmos in #13
Full Changelog: 0.0.5...0.0.6
0.0.5
0.0.4
0.0.3
What's Changed
- add metadata dataset dict by @crisostomi in #2
New Contributors
- @crisostomi made their first contribution in #2
Full Changelog: 0.0.2...0.0.3