feat: swiglu simple math changes for perf upgrades

[aghilann]

swiglu-backward-hidden2048-float32

M	Old (GB/s)	New (GB/s)	Speedup
1024	419.800	422.132	1.01x
2048	382.291	381.183	1.00x
4096	354.884	354.570	1.00x
8192	326.927	353.063	1.08x
16384	345.202	330.987	0.99x

Avg Speedup: 1.02x

---

swiglu-backward-hidden4096-float32

M	Old (GB/s)	New (GB/s)	Speedup
1024	349.163	343.402	0.98x
2048	342.355	321.585	0.94x
4096	331.145	345.983	1.04x
8192	343.890	342.923	1.00x
16384	333.262	353.621	1.06x

Avg Speedup: 1.00x

---

swiglu-backward-hidden2048-bfloat16

M	Old (GB/s)	New (GB/s)	Speedup
1024	384.610	383.441	1.00x
2048	414.323	417.415	1.01x
4096	380.222	382.472	1.01x
8192	354.887	353.713	1.00x
16384	352.217	353.068	1.00x

Avg Speedup: 1.00x

---

swiglu-backward-hidden4096-bfloat16

M	Old (GB/s)	New (GB/s)	Speedup
1024	369.999	412.272	1.11x
2048	326.090	379.439	1.16x
4096	327.641	352.632	1.08x
8192	332.303	351.517	1.06x
16384	333.074	352.809	1.06x

Avg Speedup: 1.09x

Description

CI Configuration

config:
  build: true
  # valid options are "ops", "benchmark", and "sanity"
  test: ["ops", "benchmark"]

Checklist

• Code formatted and imports sorted via repo specifications (./format.sh)
• Documentation updated (if needed)
• CI configuration reviewed

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[aghilann]

For such a small change, I choose not to move everything into experimental

[hannahli-nv]

/ok to test d18aaac

[hannahli-nv]

/ok to test baa63e7