Skip to content

PGOLF-627: add Parameter Golf train-time ReLU^2 graph and runtime support #271

Description

@AtlantisPleb

Parent: #194, #183
Blocked by: none

Roadmap lane: Epic 6, Rust-only single-device trainer closure.

Why this exists:

  • the current public crates/psionic-models/src/parameter_golf.rs baseline MLP uses exact ReLU^2 activation (max(x, 0)^2), not SiLU
  • psionic-ir still does not expose an exact public train-time relu, clamp_min, maximum, or equivalent decomposition that can lower that baseline MLP honestly on the Rust-owned graph path
  • without an exact ReLU^2 answer, #269 cannot finish the full baseline forward-logits graph and seeded backward path honestly

Scope:

  • add one exact public train-time answer for the Parameter Golf ReLU^2 MLP activation on the bounded graph/autodiff/runtime path
  • keep the first answer honest if it is bounded or host-orchestrated rather than fused
  • add targeted parity tests against the existing Parameter Golf reference math or a direct reference implementation

Exit criteria:

  • the public graph/autodiff/runtime stack can execute and differentiate one bounded exact ReLU^2 train-time path needed by the Parameter Golf baseline
  • the landed surface is sufficient to remove ReLU^2 as an implicit blocker for #269 and #194
  • docs or machine-readable truth are updated where the new primitive changes the honest trainer posture

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions