linear_model: implement complete penalized multinomial logistic suite with CV

[TheHiddenObserver]

Context

Issue #96 is intentionally narrowed to the unpenalized multinomial/softmax estimator so that the public identifiability, shape, weighting, likelihood, formula, inference, and three-backend contracts can stabilize before penalties are exposed.

The penalized surface should then be delivered as one coherent work package rather than adding L2 first and leaving L1/ElasticNet/nonconvex/CV behavior fragmented across several partially compatible implementations.

Roadmap: PR #89. Depends on #96.

Goal

Implement the complete maintained penalized multinomial logistic-regression suite, including direct-fit and CV closure for every supported tunable penalty, across NumPy, CuPy, and Torch.

The issue may be implemented through a small sequence of internal PRs, but it must remain one public capability package and must not close until the full declared matrix is complete. Do not advertise a partial penalized multinomial capability as complete.

Required penalty matrix

The initial complete package should cover, where mathematically and architecturally supported by existing statgpu penalty registries:

• L2;
• L1;
• ElasticNet;
• SCAD;
• MCP;
• adaptive Lasso when the required initialization/weights contract is defined;
• group penalties only if a valid multiclass coefficient-group convention is fixed before implementation.

If adaptive or group penalties are excluded after design review, the issue must record the mathematical or architecture reason, explicit unsupported behavior, tests, documentation, and a separately approved follow-up. L2/L1/ElasticNet/SCAD/MCP are not optional for issue closure.

Direct-fit contract

For each supported penalty, define and test:

• objective normalization and penalty scaling;
• intercept penalization policy;
• identifiability convention inherited from linear_model: design and implement unpenalized multinomial logistic regression phase 1 #96;
• class/sample-weight semantics;
• solver support and solver="auto" routing;
• warm-start and path behavior;
• convergence diagnostics, KKT/proximal or LLA checks, and stopping rules;
• unsupported solver/penalty combinations with precise errors;
• backend-native computation without silent fallback or hidden full-array host transfer.

CV closure

Every tunable direct-fit capability in this issue must ship with its CV path in the same work package:

• alpha/lambda/C path or grid generation;
• ElasticNet mixing-parameter grid where applicable;
• deterministic folds and no leakage;
• sample/class weights;
• fold scoring;
• best-parameter selection;
• tie-breaking policy;
• warm-start/path reuse where valid;
• full-data final refit;
• final-refit inference contract where supported;
• NumPy/CuPy/Torch backend parity;
• failure isolation and transactional state restoration;
• external CV alignment where an authoritative comparator exists.

No penalty may be marked supported by direct fit() while its CV path remains merely planned.

Inference

Define the supported inferential surface separately for:

• L2 penalization;
• sparsity-inducing convex penalties;
• nonconvex penalties.

Strict inference remains the default where inference is offered. Unsupported inferential combinations must be explicitly estimation-only with stable error behavior, tests, and documentation; approximate inference must never be silently substituted.

Formula and sklearn compatibility

• preserve the formula/array semantics fixed in linear_model: design and implement unpenalized multinomial logistic regression phase 1 #96;
• preserve deterministic class ordering and coefficient shapes;
• support cloning, get_params/set_params, pipelines, and grid-search behavior;
• validate fitted-state invalidation and final-CV-refit state;
• keep binary behavior separate unless an explicit compatibility contract is adopted.

Backend contract

• NumPy CPU;
• CuPy CUDA;
• Torch CUDA;
• explicit device="cuda" / device="torch" never silently fall back;
• objective, gradient, prox/LLA, path, scoring, refit, prediction, and supported inference remain backend-native;
• GPU memory lifecycle follows dev/AGENTS.md.

External alignment

Use matched comparisons against the strongest available references, including:

• sklearn multinomial logistic regression and CV for L2/L1/ElasticNet where semantics align;
• statsmodels/R references for unpenalized mapping inherited from linear_model: design and implement unpenalized multinomial logistic regression phase 1 #96;
• glmnet or another authoritative multinomial elastic-net implementation;
• validated numerical invariants and KKT/LLA checks for SCAD/MCP where no direct external API match exists.

Every comparison must state objective normalization, penalty mapping, intercept policy, class ordering, solver, tolerances, weights, and stopping criteria.

Required validation

• penalty registry/export and solver-dispatch matrix;
• objective, gradient, prox/LLA, KKT, and finite-difference checks as applicable;
• direct-fit and CV parity for every supported penalty;
• deterministic folds, no leakage, selection, and transactional refit tests;
• sample/class-weight and row-replication tests where defined;
• formula/array parity;
• clone/pipeline/grid-search/fitted-state tests;
• NumPy/CuPy/Torch parity and maintained physical-GPU validation;
• external coefficient/probability/path/selection comparisons;
• benchmark artifacts for any performance claim;
• synchronized EN/CN docs and all applicable changelogs.

Non-goals

• no sparse-input support before the shared sparse backend contract in backends: define shared sparse-array contract without silent densification #97;
• no one-vs-rest implementation presented as multinomial softmax;
• no silent CPU fallback, hidden densification, or unreported approximate inference;
• no partial issue closure after only L2 or only direct-fit support.

Acceptance criteria

• linear_model: design and implement unpenalized multinomial logistic regression phase 1 #96 unpenalized multinomial contract is merged and stable.
• L2, L1, ElasticNet, SCAD, and MCP direct-fit paths are implemented across NumPy, CuPy, and Torch.
• Every supported tunable penalty has complete CV path, selection, and final refit support.
• Objective and penalty-scale mappings are explicit and externally aligned.
• Solver/penalty compatibility and unsupported combinations are precise and tested.
• Inference is strict and supported where statistically defined, or explicitly estimation-only by penalty family.
• Formula, sklearn compatibility, fitted-state, and backend-locality contracts pass.
• Physical CuPy and Torch validation passes without silent fallback.
• English/Chinese documentation and changelogs are synchronized.
• No penalized multinomial capability remains partially advertised or partially implemented within the declared matrix.