update front page of documentation

LICENSE.md

@@ -1,6 +1,6 @@
 BSD 3-Clause License
 
-Copyright (c) 2018-2023 The Feature-engine developers.
+Copyright (c) 2018-2024 The Feature-engine developers.
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

docs/contribute/contribute_other.rst

@@ -46,8 +46,8 @@ Feature-engine.
    :target: https://github.com/sponsors/solegalli
 
 Currently, Sole and our contributors dedicate their free time voluntarily
-to advancing the project. You can help us reach a funding milestone, so that we can
-gather on a group of 2-3 contributors who will commit regular hours each week to enhance
+to advancing the project. You can help us reach a funding milestone so that we can
+gather a group of 2-3 contributors who will commit regular hours each week to enhance
 documentation and expand Feature-engine's functionality at a faster pace.
 `Your contribution <https://github.com/sponsors/solegalli>`_ will play a vital role in
 propelling Feature-engine to new heights, ensuring it remains a valuable resource for

docs/index.rst

@@ -7,73 +7,92 @@ Feature-engine
 ==============
 
 A Python library for Feature Engineering and Selection
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+------------------------------------------------------
 
 .. figure::  images/logo/FeatureEngine.png
    :align:   center
 
    **Feature-engine rocks!**
 
 Feature-engine is a Python library with multiple transformers to engineer and select
-features to use in machine learning models. Feature-engine preserves Scikit-learn
+features for machine learning models. Feature-engine adopts Scikit-learn
 functionality with methods `fit()` and `transform()` to learn parameters from and then
 transform the data.
 
 Feature-engine includes transformers for:
 
 - Missing data imputation
-- Categorical encoding
-- Discretisation
+- Encoding of categorical features
+- Discretization
 - Outlier capping or removal
-- Variable transformation
-- Variable creation
-- Variable selection
+- Feature transformation
+- Creation of new features
+- Feature selection
 - Datetime features
 - Time series
 - Preprocessing
 
-Feature-engine allows you to select the variables you want to transform **within** each
+We normally apply different feature engineering processes to different feature subsets. With sklearn, we restrict the
+feature engineering techniques to a certain group of variables by using an auxiliary class: the `ColumnTransformer`.
+This class also results in a change in the name of the variables after the transformation.
+
+Feature-engine, instead, allows you to select the variables you want to transform **within** each
 transformer. This way, different engineering procedures can be easily applied to
-different feature subsets.
+different feature subsets without the need for additional transformers or changes in the feature names.
 
-Feature-engine transformers can be assembled within the Scikit-learn pipeline,
-therefore making it possible to save and deploy one single object (.pkl) with the
-entire machine learning pipeline. Check :ref:`**Quick Start** <quick_start>` for an
-example.
+Feature-engine transformers are fully compatible with scikit-learn. That means that you can assemble Feature-engine
+transformers within a Scikit-learn pipeline, or use them in a grid or random search for hyperparameters.
+Check :ref:`**Quick Start** <quick_start>` for an example.
 
 Pst! How did you find us?
 -------------------------
 
 We want to share Feature-engine with more people. It'd help us loads if you tell us
 how you discovered us.
 
-Then we'd know what we are doing right and which channels to use to share the love.
+We'd know what we are doing right and which channels we should use to share the love.
+
+.. figure::  images/sponsors/how-did-you-discover.png
+   :align:   center
+   :target: https://docs.google.com/forms/d/e/1FAIpQLSfxvgnJvuvPf2XgosakhXo5VNQafqRrjNXkoW5qDWqnuxZNSQ/viewform?usp=sf_link
 
 Please share your story by answering 1 quick question
-`at this link <https://docs.google.com/forms/d/e/1FAIpQLSfxvgnJvuvPf2XgosakhXo5VNQafqRrjNXkoW5qDWqnuxZNSQ/viewform?usp=sf_link>`_
-. 😃
+`at this link <https://docs.google.com/forms/d/e/1FAIpQLSfxvgnJvuvPf2XgosakhXo5VNQafqRrjNXkoW5qDWqnuxZNSQ/viewform?usp=sf_link>`_ 😃
+
+What is feature engineering?
+----------------------------
+
+Feature engineering is the process of using domain knowledge and statistical tools to create features fit for use with
+machine learning algorithms. The raw data that we normally gather as part of our business activities is rarely fit to
+train machine learning models. Instead, data scientists spend a large part of their time on data analysis, preprocessing,
+and feature engineering.
+
+Pandas is a common library for data preprocessing and feature engineering. It supports pretty much every method that is
+commonly used to transform raw data. However, pandas is not compatible with sklearn out of the box and is also not able
+to learn and store the feature engineering parameters.
+
+Feature-engine’s transformers wrap pandas functionality around an API that exposes `fit` and `transform` methods to learn
+and store parameters from data and then use these parameters to transform the variables. Like this, Feature-engine makes
+the awesome functionality available in pandas fully compatible with Scikit-Learn.
+
 
 What is unique about Feature-engine?
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+------------------------------------
 
 The following characteristics make Feature-engine unique:
 
 - Feature-engine contains the most exhaustive collection of feature engineering transformations.
 - Feature-engine can transform a specific group of variables in the dataframe.
-- Feature-engine returns dataframes, hence suitable for data exploration and model deployment.
+- Feature-engine returns dataframes, hence suitable for data analysis and model deployment.
 - Feature-engine is compatible with the Scikit-learn pipeline, Grid and Random search and cross validation.
 - Feature-engine automatically recognizes numerical, categorical and datetime variables.
 - Feature-engine alerts you if a transformation is not possible, e.g., if applying logarithm to negative variables or divisions by 0.
 
-If you want to know more about what makes Feature-engine unique, check this
-`article <https://trainindata.medium.com/feature-engine-a-new-open-source-python-package-for-feature-engineering-29a0ab88ea7c>`_.
-
 
 Installation
-~~~~~~~~~~~~
+------------
 
-Feature-engine is a Python 3 package and works well with 3.7 or later. Earlier versions
-are not compatible with the latest versions of Python numerical computing libraries.
+Feature-engine is a Python 3 package and works well with 3.9 or later.
 
 The simplest way to install Feature-engine is from PyPI with pip:
 
@@ -102,25 +121,28 @@ If you’re using Anaconda, you can install the
     $ conda install -c conda-forge feature_engine
 
 
-Feature-engine features in the following resources
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Feature-engine features in the following tutorials
+--------------------------------------------------
 
 - `Feature Engineering for Machine Learning <https://www.trainindata.com/p/feature-engineering-for-machine-learning>`_, Online Course.
 - `Feature Selection for Machine Learning <https://www.trainindata.com/p/feature-selection-for-machine-learning>`_, Online Course.
-- `Feature Engineering for Time Series Forecasting <https://www.www.trainindata.com/p/feature-engineering-for-forecasting>`_, Online Course.
+- `Feature Engineering for Time Series Forecasting <https://www.trainindata.com/p/feature-engineering-for-forecasting>`_, Online Course.
 - `Python Feature Engineering Cookbook <https://packt.link/0ewSo>`_, book.
 - `Feature Selection in Machine Learning with Python <https://leanpub.com/feature-selection-in-machine-learning>`_, book.
 
-More learning resources in the :ref:`**Learning Resources** <learning_resources>`.
+More learning resources in :ref:`**Learning Resources** <learning_resources>`.
 
 
 Feature-engine's Transformers
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+-----------------------------
 
 Feature-engine hosts the following groups of transformers:
 
 Missing Data Imputation: Imputers
----------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Missing data imputation consists in replacing missing values in categorical data and numerical variables with estimates
+of those nan values or arbitrary data points. Feature-engine supports the following missing data imputation methods:
 
 - :doc:`api_doc/imputation/MeanMedianImputer`: replaces missing data in numerical variables by the mean or median
 - :doc:`api_doc/imputation/ArbitraryNumberImputer`: replaces missing data in numerical variables by an arbitrary number
@@ -131,7 +153,14 @@ Missing Data Imputation: Imputers
 - :doc:`api_doc/imputation/DropMissingData`: removes observations (rows) containing missing values from dataframe
 
 Categorical Encoders: Encoders
-------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Categorical encoding is the process of replacing categorical values by numerical values. Most machine learning models, and
+in particular, those supported by scikit-learn, don't accept strings as inputs. Hence, we need to convert these strings
+into numbers that can be interpeted by these models.
+
+There are various categorical encoding techniques, including one hot encoding, ordinal encoding and target encoding.
+Feature-engine supports the following methods:
 
 - :doc:`api_doc/encoding/OneHotEncoder`: performs one hot encoding, optional: of popular categories
 - :doc:`api_doc/encoding/CountFrequencyEncoder`: replaces categories by the observation count or percentage
@@ -142,8 +171,12 @@ Categorical Encoders: Encoders
 - :doc:`api_doc/encoding/RareLabelEncoder`: groups infrequent categories
 - :doc:`api_doc/encoding/StringSimilarityEncoder`: encodes categories based on string similarity
 
-Variable Discretisation: Discretisers
--------------------------------------
+Variable Discretization: Discretizers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Discretization, or binning, consists in sorting numerical features into discrete intervals. The most commonly used methods
+are equal-width and equal-frequency discretization. Feature-engine supports these and more advanced methods, like
+discretization with decision trees:
 
 - :doc:`api_doc/discretisation/ArbitraryDiscretiser`: sorts variable into intervals defined by the user
 - :doc:`api_doc/discretisation/EqualFrequencyDiscretiser`: sorts variable into equal frequency intervals
@@ -152,14 +185,21 @@ Variable Discretisation: Discretisers
 - :doc:`api_doc/discretisation/GeometricWidthDiscretiser`: sorts variable into geometrical intervals
 
 Outlier Capping or Removal
---------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Outliers are values that are very different with respect to the distribution observed by the variable. Some machine-learning
+models and statistical tests are sensitive to outliers. In some cases, we may want to remove outliers or replace them with
+permitted values.
 
 -  :doc:`api_doc/outliers/ArbitraryOutlierCapper`: caps maximum and minimum values at user defined values
 -  :doc:`api_doc/outliers/Winsorizer`: caps maximum or minimum values using statistical parameters
 -  :doc:`api_doc/outliers/OutlierTrimmer`: removes outliers from the dataset
 
 Numerical Transformation: Transformers
---------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We normally use variance stabilizing transformations to make the data meet the assumptions of certain statistical tests,
+like anova, and machine learning models, like linear regression. Feature-engine supports the following transformations:
 
 - :doc:`api_doc/transformation/LogTransformer`: performs logarithmic transformation of numerical variables
 - :doc:`api_doc/transformation/LogCpTransformer`: performs logarithmic transformation after adding a constant value
@@ -170,20 +210,28 @@ Numerical Transformation: Transformers
 - :doc:`api_doc/transformation/ArcsinTransformer`: performs arcsin transformation of numerical variables
 
 Feature Creation:
------------------
+~~~~~~~~~~~~~~~~~
+
+Feature-engine allows you to create new features by combining them mathematically or transforming them with mathematical functions:
 
 -  :doc:`api_doc/creation/MathFeatures`: creates new variables by combining features with mathematical operations
 -  :doc:`api_doc/creation/RelativeFeatures`: combines variables with reference features
 -  :doc:`api_doc/creation/CyclicalFeatures`: creates variables using sine and cosine, suitable for cyclical features
 
 Datetime:
----------
+~~~~~~~~~
+
+Data scientists rarely use datetime features in their original representation with machine learning models. Instead, we
+extract many new features from the date and time parts of the datetime variable:
 
 - :doc:`api_doc/datetime/DatetimeFeatures`: extract features from datetime variables
 - :doc:`api_doc/datetime/DatetimeSubtraction`: computes subtractions between datetime variables
 
 Feature Selection:
-------------------
+~~~~~~~~~~~~~~~~~~
+
+Simpler models are easier to interpret, deploy, and maintain. Feature-engine expands the feature selection functionality
+existing in other libraries like sklearn and MLXtend, with additional methods:
 
 - :doc:`api_doc/selection/DropFeatures`: drops an arbitrary subset of variables from a dataframe
 - :doc:`api_doc/selection/DropConstantFeatures`: drops constant and quasi-constant variables from a dataframe
@@ -200,26 +248,41 @@ Feature Selection:
 - :doc:`api_doc/selection/ProbeFeatureSelection`: selects features whose importance is greater than those of random variables
 
 Forecasting:
-------------
+~~~~~~~~~~~~
+
+To address forecasting as a regression by using traditional machine learning algorithms, we first need to transform the
+time series into a table of static fetaures. We can do this through lags and windows combined with aggregations over past data:
 
 - :doc:`api_doc/timeseries/forecasting/LagFeatures`: extract lag features
 - :doc:`api_doc/timeseries/forecasting/WindowFeatures`: create window features
 - :doc:`api_doc/timeseries/forecasting/ExpandingWindowFeatures`: create expanding window features
 
 Preprocessing:
---------------
+~~~~~~~~~~~~~~
+
+When transforming variables and doing data cleaning, we usually change the variables data types (dtype in pandas). These
+can cause problems further down the pipeline. To tackle this head on, Feature-engine has transformers to ensure the data
+types and variable names match.
 
 - :doc:`api_doc/preprocessing/MatchCategories`: ensures categorical variables are of type 'category'
 - :doc:`api_doc/preprocessing/MatchVariables`: ensures that columns in test set match those in train set
 
 Scikit-learn Wrapper:
----------------------
+~~~~~~~~~~~~~~~~~~~~~
+
+An alternative to scikit-learn's `ColumnTransformer`:
 
 -  :doc:`api_doc/wrappers/Wrapper`: applies Scikit-learn transformers to a selected subset of features
 
+Feature scaling
+---------------
+
+Scikit-learn offers a comprehensive array of tools to apply data normalization, standardization, and min-max scaling, among
+other processes, so we felt that there was no need to bring that functionality to Feature-engine. If you want to apply these
+procedures to a subset of the variables only, check out the :doc:`api_doc/wrappers/Wrapper`:
 
 Getting Help
-~~~~~~~~~~~~
+------------
 
 Can't get something to work? Here are places where you can find help.
 
@@ -232,7 +295,7 @@ Can't get something to work? Here are places where you can find help.
 
 
 Contributing
-~~~~~~~~~~~~
+------------
 
 Interested in contributing to Feature-engine? That is great news!
 
@@ -249,7 +312,7 @@ For more details on how to contribute check the contributing page. Click on the
 :ref:`**Contribute** <contribute>` guide.
 
 Sponsor us
-~~~~~~~~~~
+----------
 
 `Empower Sole <https://github.com/sponsors/solegalli>`_, the main developer of
 Feature-engine, to assemble a team of paid contributors to accelerate the development of
@@ -260,8 +323,8 @@ Feature-engine.
    :target: https://github.com/sponsors/solegalli
 
 Currently, Sole and our contributors dedicate their free time voluntarily
-to advancing the project. You can help us reach a funding milestone, so that we can
-gather on a group of 2-3 contributors who will commit regular hours each week to enhance
+to advancing the project. You can help us reach a funding milestone so that we can
+gather a group of 2-3 contributors who will commit regular hours each week to enhance
 documentation and expand Feature-engine's functionality at a faster pace.
 
 `Your contribution <https://github.com/sponsors/solegalli>`_ will play a vital role in
@@ -272,7 +335,7 @@ If you don't have a Github account, you can also `sponsor us here <https://buyme
 
 
 Open Source
-~~~~~~~~~~~
+-----------
 
 Feature-engine's `license <https://github.com/feature-engine/feature_engine/blob/master/LICENSE.md>`_
 is an open source BSD 3-Clause.
@@ -283,7 +346,7 @@ The `issues <https://github.com/feature-engine/feature_engine/issues/>`_ and
 
 
 Table of Contents
-~~~~~~~~~~~~~~~~~
+-----------------
 
 .. toctree::
    :maxdepth: 2