feat(pi07): add ring-attention sequence parallelism#298

Closed

WilliamYue37 wants to merge 8 commits into

mainfrom

feat/pi07-ring-attention

Commits on May 12, 2026

feat(pi07): add ring-attention sequence parallelism
WilliamYue37
committed

Commits on May 13, 2026

feat(pi07): 2D parallelism — ring sub-groups + DP/ZeRO across rings
WilliamYue37
committed
ring(train): drop per-step batch broadcast; seeded sampler is enough
WilliamYue37
committed
ring(2d): drop DP sub-group; compute dp_rank arithmetically
WilliamYue37
committed
fix(ring): P2POp peer is a global rank, not within-group
WilliamYue37
committed
ring: shrink _NEG_INF to -1e9, add RING_DEBUG_PROBES env
WilliamYue37
committed
ring: more granular debug probes inside backward loop
WilliamYue37
committed

Commits on May 14, 2026

ring: defensive nan_to_num + diagnostic suite
WilliamYue37
committed