Introduce cp-as-ep rule for long context training or strong scaling

[NuojCheng]

Description

The new cp-as-ep sharding rule supports five mesh axes: data, stage, fsdp, context, and expert. All axes follow the default sharding rules, except for the context axis, which applies the Expert Parallelism (EP) rule within the MoE layers.

Compared to the ep-as-cp rule introduced in #3656, this new rule is particularly beneficial when per_device_batch_size × ici_expert_parallelism >> 1. Instead of converting all EP to Context Parallelism (CP)—which can degrade performance—this approach allows us to map only a subset of EP to CP in non-MoE components. For example, when training an MoE model with per_device_batch_size=0.5 where a total EP of 8 is desired, setting EP=4 and CP=2 should yield better performance than simply using EP=8."

Tests

Please describe how you tested this change, and include any instructions and/or
commands to reproduce.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

[codecov]

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/maxtext/layers/moe.py	0.00%	1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!