Skip to content

SGD.zero_grad overwrites gradients for non‑trainable parameters #23

Description

@github-actions

File: leanpass/optim.py

The zero_grad method unconditionally creates a NumPy array for param.grad even when the parameter does not require gradients (requires_grad=False). This breaks the convention that non‑trainable tensors have grad=None and can cause downstream code to treat them as trainable. The method should only zero gradients for parameters that have requires_grad=True.

def zero_grad(self):
    for param in self.params:
-        param.grad = np.zeros_like(param.data)
+        if param.requires_grad:
+            param.grad = np.zeros_like(param.data)

Filed automatically by ai-issue-scan.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions