Should we implement SVM grid search & GridSearchCV? (preserved dead code from #398 cleanup)

[Mec-iS]

Context

As part of the dead-code investigation in #398 (Stage 7 of the staged coverage plan #391), the following modules were deleted because they were unreachable (commented out of their parent mod.rs) and had never been enabled:

• src/svm/search/ (mod.rs, svc_params.rs, svr_params.rs)
• src/model_selection/hyper_tuning/ (mod.rs, grid_search.rs)

Before deletion the code is preserved here for reference.

---

Question for maintainers

Should we revive and complete this code, or is it superseded / permanently abandoned?

Options:

• Revive — update SVCSearchParameters to use the current Kernels enum, fix GridSearchCV generic signature, re-enable the modules, add tests.
• Delete permanently — the pattern is not needed; cross_validate + manual parameter iteration is sufficient.

---

Preserved: src/svm/search/mod.rs

//! SVC and Grid Search

/// SVC search parameters
pub mod svc_params;
/// SVC search parameters
pub mod svr_params;

---

Preserved: src/svm/search/svc_params.rs

All code was commented out. Key intended types:

• SVCSearchParameters<TX, TY, X, Y, K> — holds Vec of each SVC hyperparameter
• SVCSearchParametersIterator — yields Cartesian product as SVCParameters

Full commented-out source

// /// SVC grid search parameters
// #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
// #[derive(Debug, Clone)]
// pub struct SVCSearchParameters<
//     TX: Number + RealNumber,
//     TY: Number + Ord,
//     X: Array2<TX>,
//     Y: Array1<TY>,
//     K: Kernel,
// > {
//     #[cfg_attr(feature = "serde", serde(default))]
//     pub epoch: Vec<usize>,
//     #[cfg_attr(feature = "serde", serde(default))]
//     pub c: Vec<TX>,
//     #[cfg_attr(feature = "serde", serde(default))]
//     pub tol: Vec<TX>,
//     #[cfg_attr(feature = "serde", serde(default))]
//     pub kernel: Vec<K>,
//     #[cfg_attr(feature = "serde", serde(default))]
//     m: PhantomData<(X, Y, TY)>,
//     #[cfg_attr(feature = "serde", serde(default))]
//     seed: Vec<Option<u64>>,
// }
//
// pub struct SVCSearchParametersIterator< ... > { ... }
//
// impl IntoIterator for SVCSearchParameters { ... }
// impl Iterator for SVCSearchParametersIterator { ... }
// impl Default for SVCSearchParameters { ... }
//
// #[cfg(test)]
// mod tests {
//     fn search_parameters() { ... }  // NOTE: duplicate test name in original
// }

Known issues before this can compile:

• References LinearKernel {} which no longer exists; must use Kernels::linear()
• Generic K: Kernel on the struct; should probably be replaced with Vec<Kernels>
• Duplicate fn search_parameters test name in original source

---

Preserved: src/svm/search/svr_params.rs

This file was live Rust (not commented out) — it compiled but was never reachable because pub mod search; was commented out in svm/mod.rs. It has tests.

Full source

//! # SVR Grid Search Parameters
//!
//! Provides [`SVRSearchParameters`] and its iterator for exhaustive
//! grid search over SVR hyperparameters.

#[cfg(feature = "serde")]
use serde::{Deserialize, Serialize};

use crate::linalg::basic::arrays::Array2;
use crate::numbers::basenum::Number;
use crate::numbers::floatnum::FloatNumber;
use crate::numbers::realnum::RealNumber;
use crate::svm::{Kernels, svr};
use std::marker::PhantomData;

#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Debug, Clone)]
pub struct SVRSearchParameters<T: Number + RealNumber, M: Array2<T>> {
    pub eps: Vec<T>,
    pub c: Vec<T>,
    pub tol: Vec<T>,
    pub kernel: Vec<Kernels>,
    pub m: PhantomData<M>,
}

pub struct SVRSearchParametersIterator<T: Number + RealNumber, M: Array2<T>> {
    svr_search_parameters: SVRSearchParameters<T, M>,
    current_eps: usize,
    current_c: usize,
    current_tol: usize,
    current_kernel: usize,
}

impl<T: Number + FloatNumber + RealNumber, M: Array2<T>> IntoIterator
    for SVRSearchParameters<T, M>
{
    type Item = svr::SVRParameters<T>;
    type IntoIter = SVRSearchParametersIterator<T, M>;
    fn into_iter(self) -> Self::IntoIter { ... }
}

impl<T: Number + FloatNumber + RealNumber, M: Array2<T>> Iterator
    for SVRSearchParametersIterator<T, M>
{
    type Item = svr::SVRParameters<T>;
    fn next(&mut self) -> Option<Self::Item> { ... }  // Cartesian product iterator
}

impl<T: Number + FloatNumber + RealNumber, M: Array2<T>> Default for SVRSearchParameters<T, M> {
    fn default() -> Self { ... }
}

#[cfg(test)]
mod tests {
    // test_default_parameters, test_single_grid_iteration,
    // test_cartesian_grid_iteration, test_empty_grid, test_kernel_enum_variants
}

Status: compiles against current API. Could be revived with minimal effort.

---

Preserved: src/model_selection/hyper_tuning/mod.rs

mod grid_search;
pub use grid_search::{GridSearchCV, GridSearchCVParameters};

---

Preserved: src/model_selection/hyper_tuning/grid_search.rs

Full source

// TODO: missing documentation

use crate::{
    api::{Predictor, SupervisedEstimator},
    error::{Failed, FailedError},
    linalg::basic::arrays::{Array1, Array2},
    numbers::basenum::Number,
    numbers::realnum::RealNumber,
};
use crate::model_selection::{cross_validate, BaseKFold, CrossValidationResult};

#[derive(Debug)]
pub struct GridSearchCVParameters<T, M, C, I, E, F, K, S> { ... }

impl<...> GridSearchCVParameters<...> {
    pub fn new(parameters_search: I, estimator: F, score: S, cv: K) -> Self { ... }
}

#[derive(Debug)]
pub struct GridSearchCV<T: RealNumber, M: Array2<T>, C: Clone, E: Predictor<M, M::RowVector>> {
    _phantom: PhantomData<(T, M)>,
    predictor: E,
    pub cross_validation_result: CrossValidationResult<T>,
    pub best_parameter: C,
}

impl<...> GridSearchCV<...> {
    pub fn fit(x: &M, y: &M::RowVector, gs_parameters: ...) -> Result<Self, Failed> {
        // iterates parameter_search, calls cross_validate for each,
        // keeps best by mean_test_score, refits on full data
    }
    pub fn cv_results(&self) -> &CrossValidationResult<T> { ... }
    pub fn best_parameters(&self) -> &C { ... }
    pub fn predict(&self, x: &M) -> Result<M::RowVector, Failed> { ... }
}

// Also implements SupervisedEstimator and Predictor for GridSearchCV.

#[cfg(test)]
mod tests {
    // test_grid_search: uses LogisticRegressionSearchParameters + KFold(5)
    // NOTE: references crate::linalg::naive::dense_matrix::DenseMatrix
    //       and LogisticRegressionSearchParameters — both need path/API verification
}

Known issues before this can compile:

• CrossValidationResult<T> is generic in the preserved code but CrossValidationResult in current model_selection is not generic (uses f64 scores directly) — signature mismatch
• Test references crate::linalg::naive::dense_matrix::DenseMatrix (old path)
• Test references LogisticRegressionSearchParameters — check if still present

---

References

• Tracking: Improve test coverage significantly (staged plan) #391
• Investigation issue: Stage 7: Dead-code investigation — svm/search & model_selection/hyper_tuning (tracking #391) #398