linear_model: design and implement unpenalized multinomial logistic regression phase 1

[TheHiddenObserver]

Context

LogisticRegression currently provides maintained binary logistic/probit behavior across NumPy, CuPy, and Torch. Multiclass multinomial/softmax support remains a central linear-model API gap, but the base statistical contract should be stabilized before any penalized capability is exposed.

Roadmap: PR #89.

Goal

Define the complete public contract for multinomial logistic regression and implement a bounded unpenalized-only Phase-1 estimator covering probability prediction, hard prediction, likelihood diagnostics, and model-based inference.

Penalized multinomial support is intentionally excluded from this issue. The complete penalized surface, including L2/L1/ElasticNet/SCAD/MCP and direct-fit/CV closure, is tracked as one coherent work package in #98.

Design decisions required before implementation

Identifiability

Choose and document one primary convention:

• reference-class coefficients; or
• sum-to-zero coefficients.

The public representation, internal representation, covariance rank, inference degrees of freedom, and external mapping must be explicit. Do not mix conventions across fit, prediction, and summary.

Public shapes

Define:

• coef_ and intercept_ shape;
• class ordering and classes_;
• decision_function shape;
• predict_proba shape;
• binary compatibility behavior if the same class handles two classes;
• covariance and standard-error tensor/matrix layout;
• summary row naming.

Objective and weighting

Document:

• average versus summed negative log-likelihood;
• intercept handling;
• analytic sample_weight semantics;
• class_weight interaction with sample weights;
• row-replication equivalence where applicable;
• information-criterion parameter count.

No regularization parameter, penalty selector, alpha/lambda/C path, or penalized objective belongs to this issue.

Solver matrix

Phase 1 should define and validate:

• solver="auto" routing for the unpenalized objective;
• Newton or trust-region behavior for moderate dimensions;
• L-BFGS/L-BFGS-B behavior;
• stopping criteria and convergence diagnostics;
• warm-start parameter shape and validation where supported;
• numerical stabilization for log-sum-exp and probabilities.

Penalized solvers and penalty dispatch are deferred in full to #98.

Phase-1 public scope

• fit unpenalized multinomial logistic regression;
• decision_function;
• predict_proba;
• integer/class-label predict;
• log-likelihood and convergence diagnostics;
• AIC/BIC under the documented parameter count;
• model-based covariance, standard errors, test statistics, p-values, and confidence intervals where identifiable;
• formula/dataframe and array APIs if current LogisticRegression exposes formulas;
• sklearn cloning, tags, pipeline, set_params, and fitted-state behavior.

Backend contract

• NumPy CPU;
• CuPy CUDA;
• Torch CUDA;
• backend-native objective, gradient, Hessian/Hessian-vector operations, probability prediction, and covariance computation;
• explicit device requests must not silently fall back;
• full GPU arrays must not be copied to NumPy for routine fitting or inference bookkeeping;
• dtype and device normalization must preserve the 0.2.4 public-constructor clone contract.

External alignment

Use matched comparisons against:

• sklearn multinomial LogisticRegression with penalty=None or the closest exact unpenalized configuration;
• statsmodels MNLogit for likelihood and inference;
• R nnet::multinom or an equivalent authoritative baseline for coefficient/prediction mapping.

Alignment must explicitly state:

• reference class or coefficient transformation;
• objective normalization;
• intercept treatment;
• class ordering;
• sample/class weights;
• solver and tolerance;
• parameter count used by AIC/BIC.

Formula and input validation

Cover:

• at least three distinct classes;
• string, integer, and ordered label inputs;
• deterministic class ordering;
• missing classes in validation/prediction data;
• one-class and two-class behavior;
• finite inputs and weights;
• sample-weight shape/device/dtype;
• warm-start class or feature mismatch;
• categorical formula terms and prediction-column alignment.

Non-goals

• no L2, L1, ElasticNet, SCAD, MCP, adaptive, group, or other penalty in Phase 1;
• no multinomial CV in this issue because the public capability is non-tunable;
• no one-vs-rest implementation presented as multinomial softmax;
• no sparse-input support;
• no silent conversion to CPU or binary fallback.

Required validation

• exact softmax/log-sum-exp unit tests;
• gradient and Hessian checks;
• identifiability/rank tests;
• invariance to allowed coefficient reparameterization;
• sample/class-weight tests;
• row-replication tests where defined;
• formula/array parity;
• clone/pipeline/fitted-state tests;
• NumPy/CuPy/Torch parity;
• external coefficient/probability/log-likelihood/inference alignment;
• physical CuPy and Torch GPU validation;
• EN/CN documentation and examples;
• benchmark artifact for any performance claim.

Acceptance criteria

• Identifiability, shapes, class ordering, objective normalization, weighting, and parameter count are documented before final implementation.
• Only the unpenalized multinomial fit/predict/probability path is implemented in this issue.
• Likelihood, convergence, AIC/BIC, and model-based inference are available under the documented convention.
• Public constructors remain sklearn-clone compatible.
• Formula and array paths agree.
• NumPy, CuPy, and Torch pass parity and physical-GPU gates without silent fallback.
• sklearn/statsmodels/R mappings are explicit and tested.
• Any penalty-related arguments or combinations fail clearly and point users to the future linear_model: implement complete penalized multinomial logistic suite with CV #98 capability rather than optimizing an incomplete objective.
• English and Chinese user documentation are synchronized.

Dependency and follow-up

• linear_model: implement complete penalized multinomial logistic suite with CV #98 depends on the merged and stable contracts from this issue.
• linear_model: implement complete penalized multinomial logistic suite with CV #98 owns the complete penalized multinomial suite and all tunable direct-fit/CV closure.
• backends: define shared sparse-array contract without silent densification #97 owns the shared sparse backend contract.