Feature selection is an important task in any machine learning pipeline. Notably, there are various methods to perform feature selection with each method possessing their own advantages and drawbacks especially in terms of their suitability to different datasets.
The most common methods for feature selections are:
• Filter Methods: where feature selection is derived from statistical tests for the correlation between features. Including the performance of both parametric and non-parametric tests.
• Wrapper method: is a search problem, which is computationally expensive, as this method consists of trying a subset of features and training the model using that subset through a trial-and-error removal and insertion of features.
• Embedded method: feature selection part is integrated as part of the learning algorithm. A set of features is selected, the model learns and its performance is evaluated, and the process is repeated with another set of features until the best result is achieved.
This statistical series focuses primarily on filter methods with specific reference to both parametric and non-parametric tests.