Configurable Division

By default, HeavyBall's division differs from the industry standard, potentially giving meaningfully different results for otherwise identical optimizer hyperparameters.

You can now set heavyball.utils.default_division_backend to one of

• eps_clamp, HeavyBall's default (x / y.clamp(min=eps))
• eps_add, Standard, used by PyTorch, Optax and others (x / (y + eps))
• atan2, following Adam-Atan2 (atan2(x / scale, y) * scale) - may require heavyball.utils.atan2_scale to be to clamp to a different range of target values
• nan_to_0, resulting in (x/y).nan_to_num(0, 0, 0)