v0.18.0

🫐 PCD

This release introduces PCD and PCDWeighting, and fixes several bugs. Thanks a lot to @DaraVaram for contributing PCD, and to @SajalDevX, @ppraneth and @SausageMan99 for the fixes!

We're trying to grow the community and build even more features! To participate, you can join the Discord community!

Changelog

Added

• Added PCD and PCDWeighting from Not All Objectives Are Born Equal: Priority-Constrained Descent for Hierarchical Multi-Objective Optimization (TMLR 2026). The first row of the Jacobian is treated as the primary objective: PCD follows its gradient as closely as possible, subject to each other objective receiving at least a fraction tau of normalized first-order progress. PCD and PCDWeighting are stateful: they normalize the gradients by a bias-corrected moving average of their squared norms.

Fixed

• Fixed ExcessMTL and ExcessMTLWeighting producing nan weights for the rest of the run when a task had a zero gradient at the call that sets its baseline excess risk. The exponentiated gradient update is now computed in log space, so a very large excess risk saturates the weights instead of overflowing.
• Fixed Krum and KrumWeighting sometimes selecting the wrong rows when two rows of the input matrix are almost equal. Rounding errors could make the squared distance between such rows slightly negative, giving a nan distance that was then ignored when computing the scores. Squared distances are now clamped to be non-negative before taking the square root.
• Fixed AlignedMTL and AlignedMTLWeighting ignoring tasks with a much smaller gradient than the others when the input is in float64. The tolerance used to find the rank of the Gramian was always based on the machine epsilon of the default dtype (usually float32) instead of the dtype of the Gramian, so valid small eigenvalues were discarded.
• Fixed Engine.compute_gramian silently ignoring the contribution of a module when one of its forward passes had no associated backward pass (e.g. because its output was detached). It now raises a ValueError instead.
• Fixed jac_to_grad failing with a RuntimeError when the tensors do not all have the same dtype (e.g. a model with some parameters in float32 and others in float64). The aggregated gradient was computed in the promoted dtype and could not be assigned to the .grad field of the tensors of lower precision. Each tensor now gets a .grad of its own dtype, like with torch.autograd.