diff --git a/LICENSE.md b/LICENSE.md index 06d0d8166..ff727ede5 100644 --- a/LICENSE.md +++ b/LICENSE.md @@ -1,6 +1,6 @@ BSD 3-Clause License -Copyright (c) 2018-2023 The Feature-engine developers. +Copyright (c) 2018-2024 The Feature-engine developers. All rights reserved. Redistribution and use in source and binary forms, with or without diff --git a/docs/contribute/contribute_other.rst b/docs/contribute/contribute_other.rst index 2b65aa1f2..27fc5de87 100644 --- a/docs/contribute/contribute_other.rst +++ b/docs/contribute/contribute_other.rst @@ -46,8 +46,8 @@ Feature-engine. :target: https://github.com/sponsors/solegalli Currently, Sole and our contributors dedicate their free time voluntarily -to advancing the project. You can help us reach a funding milestone, so that we can -gather on a group of 2-3 contributors who will commit regular hours each week to enhance +to advancing the project. You can help us reach a funding milestone so that we can +gather a group of 2-3 contributors who will commit regular hours each week to enhance documentation and expand Feature-engine's functionality at a faster pace. `Your contribution `_ will play a vital role in propelling Feature-engine to new heights, ensuring it remains a valuable resource for diff --git a/docs/images/sponsors/how-did-you-discover.png b/docs/images/sponsors/how-did-you-discover.png new file mode 100644 index 000000000..d5b18113c Binary files /dev/null and b/docs/images/sponsors/how-did-you-discover.png differ diff --git a/docs/index.rst b/docs/index.rst index 5b75b5913..653b81d4d 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -7,7 +7,7 @@ Feature-engine ============== A Python library for Feature Engineering and Selection -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------------------------------------ .. figure:: images/logo/FeatureEngine.png :align: center @@ -15,31 +15,34 @@ A Python library for Feature Engineering and Selection **Feature-engine rocks!** Feature-engine is a Python library with multiple transformers to engineer and select -features to use in machine learning models. Feature-engine preserves Scikit-learn +features for machine learning models. Feature-engine adopts Scikit-learn functionality with methods `fit()` and `transform()` to learn parameters from and then transform the data. Feature-engine includes transformers for: - Missing data imputation -- Categorical encoding -- Discretisation +- Encoding of categorical features +- Discretization - Outlier capping or removal -- Variable transformation -- Variable creation -- Variable selection +- Feature transformation +- Creation of new features +- Feature selection - Datetime features - Time series - Preprocessing -Feature-engine allows you to select the variables you want to transform **within** each +We normally apply different feature engineering processes to different feature subsets. With sklearn, we restrict the +feature engineering techniques to a certain group of variables by using an auxiliary class: the `ColumnTransformer`. +This class also results in a change in the name of the variables after the transformation. + +Feature-engine, instead, allows you to select the variables you want to transform **within** each transformer. This way, different engineering procedures can be easily applied to -different feature subsets. +different feature subsets without the need for additional transformers or changes in the feature names. -Feature-engine transformers can be assembled within the Scikit-learn pipeline, -therefore making it possible to save and deploy one single object (.pkl) with the -entire machine learning pipeline. Check :ref:`**Quick Start** ` for an -example. +Feature-engine transformers are fully compatible with scikit-learn. That means that you can assemble Feature-engine +transformers within a Scikit-learn pipeline, or use them in a grid or random search for hyperparameters. +Check :ref:`**Quick Start** ` for an example. Pst! How did you find us? ------------------------- @@ -47,33 +50,49 @@ Pst! How did you find us? We want to share Feature-engine with more people. It'd help us loads if you tell us how you discovered us. -Then we'd know what we are doing right and which channels to use to share the love. +We'd know what we are doing right and which channels we should use to share the love. + +.. figure:: images/sponsors/how-did-you-discover.png + :align: center + :target: https://docs.google.com/forms/d/e/1FAIpQLSfxvgnJvuvPf2XgosakhXo5VNQafqRrjNXkoW5qDWqnuxZNSQ/viewform?usp=sf_link Please share your story by answering 1 quick question -`at this link `_ -. 😃 +`at this link `_ 😃 + +What is feature engineering? +---------------------------- + +Feature engineering is the process of using domain knowledge and statistical tools to create features fit for use with +machine learning algorithms. The raw data that we normally gather as part of our business activities is rarely fit to +train machine learning models. Instead, data scientists spend a large part of their time on data analysis, preprocessing, +and feature engineering. + +Pandas is a common library for data preprocessing and feature engineering. It supports pretty much every method that is +commonly used to transform raw data. However, pandas is not compatible with sklearn out of the box and is also not able +to learn and store the feature engineering parameters. + +Feature-engine’s transformers wrap pandas functionality around an API that exposes `fit` and `transform` methods to learn +and store parameters from data and then use these parameters to transform the variables. Like this, Feature-engine makes +the awesome functionality available in pandas fully compatible with Scikit-Learn. + What is unique about Feature-engine? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------------------ The following characteristics make Feature-engine unique: - Feature-engine contains the most exhaustive collection of feature engineering transformations. - Feature-engine can transform a specific group of variables in the dataframe. -- Feature-engine returns dataframes, hence suitable for data exploration and model deployment. +- Feature-engine returns dataframes, hence suitable for data analysis and model deployment. - Feature-engine is compatible with the Scikit-learn pipeline, Grid and Random search and cross validation. - Feature-engine automatically recognizes numerical, categorical and datetime variables. - Feature-engine alerts you if a transformation is not possible, e.g., if applying logarithm to negative variables or divisions by 0. -If you want to know more about what makes Feature-engine unique, check this -`article `_. - Installation -~~~~~~~~~~~~ +------------ -Feature-engine is a Python 3 package and works well with 3.7 or later. Earlier versions -are not compatible with the latest versions of Python numerical computing libraries. +Feature-engine is a Python 3 package and works well with 3.9 or later. The simplest way to install Feature-engine is from PyPI with pip: @@ -102,25 +121,28 @@ If you’re using Anaconda, you can install the $ conda install -c conda-forge feature_engine -Feature-engine features in the following resources -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Feature-engine features in the following tutorials +-------------------------------------------------- - `Feature Engineering for Machine Learning `_, Online Course. - `Feature Selection for Machine Learning `_, Online Course. -- `Feature Engineering for Time Series Forecasting `_, Online Course. +- `Feature Engineering for Time Series Forecasting `_, Online Course. - `Python Feature Engineering Cookbook `_, book. - `Feature Selection in Machine Learning with Python `_, book. -More learning resources in the :ref:`**Learning Resources** `. +More learning resources in :ref:`**Learning Resources** `. Feature-engine's Transformers -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +----------------------------- Feature-engine hosts the following groups of transformers: Missing Data Imputation: Imputers ---------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Missing data imputation consists in replacing missing values in categorical data and numerical variables with estimates +of those nan values or arbitrary data points. Feature-engine supports the following missing data imputation methods: - :doc:`api_doc/imputation/MeanMedianImputer`: replaces missing data in numerical variables by the mean or median - :doc:`api_doc/imputation/ArbitraryNumberImputer`: replaces missing data in numerical variables by an arbitrary number @@ -131,7 +153,14 @@ Missing Data Imputation: Imputers - :doc:`api_doc/imputation/DropMissingData`: removes observations (rows) containing missing values from dataframe Categorical Encoders: Encoders ------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Categorical encoding is the process of replacing categorical values by numerical values. Most machine learning models, and +in particular, those supported by scikit-learn, don't accept strings as inputs. Hence, we need to convert these strings +into numbers that can be interpeted by these models. + +There are various categorical encoding techniques, including one hot encoding, ordinal encoding and target encoding. +Feature-engine supports the following methods: - :doc:`api_doc/encoding/OneHotEncoder`: performs one hot encoding, optional: of popular categories - :doc:`api_doc/encoding/CountFrequencyEncoder`: replaces categories by the observation count or percentage @@ -142,8 +171,12 @@ Categorical Encoders: Encoders - :doc:`api_doc/encoding/RareLabelEncoder`: groups infrequent categories - :doc:`api_doc/encoding/StringSimilarityEncoder`: encodes categories based on string similarity -Variable Discretisation: Discretisers -------------------------------------- +Variable Discretization: Discretizers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Discretization, or binning, consists in sorting numerical features into discrete intervals. The most commonly used methods +are equal-width and equal-frequency discretization. Feature-engine supports these and more advanced methods, like +discretization with decision trees: - :doc:`api_doc/discretisation/ArbitraryDiscretiser`: sorts variable into intervals defined by the user - :doc:`api_doc/discretisation/EqualFrequencyDiscretiser`: sorts variable into equal frequency intervals @@ -152,14 +185,21 @@ Variable Discretisation: Discretisers - :doc:`api_doc/discretisation/GeometricWidthDiscretiser`: sorts variable into geometrical intervals Outlier Capping or Removal --------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Outliers are values that are very different with respect to the distribution observed by the variable. Some machine-learning +models and statistical tests are sensitive to outliers. In some cases, we may want to remove outliers or replace them with +permitted values. - :doc:`api_doc/outliers/ArbitraryOutlierCapper`: caps maximum and minimum values at user defined values - :doc:`api_doc/outliers/Winsorizer`: caps maximum or minimum values using statistical parameters - :doc:`api_doc/outliers/OutlierTrimmer`: removes outliers from the dataset Numerical Transformation: Transformers --------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We normally use variance stabilizing transformations to make the data meet the assumptions of certain statistical tests, +like anova, and machine learning models, like linear regression. Feature-engine supports the following transformations: - :doc:`api_doc/transformation/LogTransformer`: performs logarithmic transformation of numerical variables - :doc:`api_doc/transformation/LogCpTransformer`: performs logarithmic transformation after adding a constant value @@ -170,20 +210,28 @@ Numerical Transformation: Transformers - :doc:`api_doc/transformation/ArcsinTransformer`: performs arcsin transformation of numerical variables Feature Creation: ------------------ +~~~~~~~~~~~~~~~~~ + +Feature-engine allows you to create new features by combining them mathematically or transforming them with mathematical functions: - :doc:`api_doc/creation/MathFeatures`: creates new variables by combining features with mathematical operations - :doc:`api_doc/creation/RelativeFeatures`: combines variables with reference features - :doc:`api_doc/creation/CyclicalFeatures`: creates variables using sine and cosine, suitable for cyclical features Datetime: ---------- +~~~~~~~~~ + +Data scientists rarely use datetime features in their original representation with machine learning models. Instead, we +extract many new features from the date and time parts of the datetime variable: - :doc:`api_doc/datetime/DatetimeFeatures`: extract features from datetime variables - :doc:`api_doc/datetime/DatetimeSubtraction`: computes subtractions between datetime variables Feature Selection: ------------------- +~~~~~~~~~~~~~~~~~~ + +Simpler models are easier to interpret, deploy, and maintain. Feature-engine expands the feature selection functionality +existing in other libraries like sklearn and MLXtend, with additional methods: - :doc:`api_doc/selection/DropFeatures`: drops an arbitrary subset of variables from a dataframe - :doc:`api_doc/selection/DropConstantFeatures`: drops constant and quasi-constant variables from a dataframe @@ -200,26 +248,41 @@ Feature Selection: - :doc:`api_doc/selection/ProbeFeatureSelection`: selects features whose importance is greater than those of random variables Forecasting: ------------- +~~~~~~~~~~~~ + +To address forecasting as a regression by using traditional machine learning algorithms, we first need to transform the +time series into a table of static fetaures. We can do this through lags and windows combined with aggregations over past data: - :doc:`api_doc/timeseries/forecasting/LagFeatures`: extract lag features - :doc:`api_doc/timeseries/forecasting/WindowFeatures`: create window features - :doc:`api_doc/timeseries/forecasting/ExpandingWindowFeatures`: create expanding window features Preprocessing: --------------- +~~~~~~~~~~~~~~ + +When transforming variables and doing data cleaning, we usually change the variables data types (dtype in pandas). These +can cause problems further down the pipeline. To tackle this head on, Feature-engine has transformers to ensure the data +types and variable names match. - :doc:`api_doc/preprocessing/MatchCategories`: ensures categorical variables are of type 'category' - :doc:`api_doc/preprocessing/MatchVariables`: ensures that columns in test set match those in train set Scikit-learn Wrapper: ---------------------- +~~~~~~~~~~~~~~~~~~~~~ + +An alternative to scikit-learn's `ColumnTransformer`: - :doc:`api_doc/wrappers/Wrapper`: applies Scikit-learn transformers to a selected subset of features +Feature scaling +--------------- + +Scikit-learn offers a comprehensive array of tools to apply data normalization, standardization, and min-max scaling, among +other processes, so we felt that there was no need to bring that functionality to Feature-engine. If you want to apply these +procedures to a subset of the variables only, check out the :doc:`api_doc/wrappers/Wrapper`: Getting Help -~~~~~~~~~~~~ +------------ Can't get something to work? Here are places where you can find help. @@ -232,7 +295,7 @@ Can't get something to work? Here are places where you can find help. Contributing -~~~~~~~~~~~~ +------------ Interested in contributing to Feature-engine? That is great news! @@ -249,7 +312,7 @@ For more details on how to contribute check the contributing page. Click on the :ref:`**Contribute** ` guide. Sponsor us -~~~~~~~~~~ +---------- `Empower Sole `_, the main developer of Feature-engine, to assemble a team of paid contributors to accelerate the development of @@ -260,8 +323,8 @@ Feature-engine. :target: https://github.com/sponsors/solegalli Currently, Sole and our contributors dedicate their free time voluntarily -to advancing the project. You can help us reach a funding milestone, so that we can -gather on a group of 2-3 contributors who will commit regular hours each week to enhance +to advancing the project. You can help us reach a funding milestone so that we can +gather a group of 2-3 contributors who will commit regular hours each week to enhance documentation and expand Feature-engine's functionality at a faster pace. `Your contribution `_ will play a vital role in @@ -272,7 +335,7 @@ If you don't have a Github account, you can also `sponsor us here `_ is an open source BSD 3-Clause. @@ -283,7 +346,7 @@ The `issues `_ and Table of Contents -~~~~~~~~~~~~~~~~~ +----------------- .. toctree:: :maxdepth: 2