[MRG] create TimeSeriesLagTrasnformer: new feature by Morgan-Sell · Pull Request #360 · feature-engine/feature_engine

Morgan-Sell · 2022-01-19T01:54:02Z

closes #342
closes #370

Notes from PR #342

This would be a transformer that lags features (pandas.shift())

The transformer would lag all numerical features or those entered by the user, and automatically concatenate them to the df.
Option to drop the original variables.

To think about: can we make multiple lags within one transformer? lag 1hr, lag2hrs etc, or would it be better to pile up 3 transformers instead?

The previous could be very easily done with a Pipeline, so maybe no need to overcomplicate the class.

# Move forward 24 hrs.
tmp = data[variables].shift(freq='24H')

# Rename the variables.
tmp.columns = [v + '_lag_24' for v in variables]

# Add the features to the original data.
data = data.merge(tmp, left_index=True, right_index=True, how='left')

Morgan-Sell · 2022-01-19T02:26:55Z

Hola @solegalli,

A few random questions/thoughts:

Should the transformer confirm there is a time series for one of the dataframe columns? If so, do we ask the user to state the time-series column when instantiating the transformer? Do we autodetect the interval measurement used in the time series (I need to brainstorm how to do so.) or ask the user for the measurement's denomination?
Does the user state the lag's time interval - e.g., minute, hour, and day - when instantiating the class?
If the user claims the lag's time interval, I'm assuming we need to check that it's compatible with the stated time series. How should we define compatibility? Let's imagine that the time series is denominated in 15-minute intervals and the user selects the lag to be one hour, which is equal to four observations. Do we consider this to be compatible? Or, does "compatibility" require the denominations of both the time lag interval and time-series interval to be equivalent? Maybe it's a version 1 and 2.
How do you envision multiple lags being implemented? I'm imagining a world where a user would instantiate a Pipeline with multiple TimeSeriesLagTransformer classes. Is that what you're referring to when you mentioned, The previous could be very easily done with a Pipeline, so maybe no need to overcomplicate the class.
Should we take a look at the statsmodel package for suggestions?

Lmk what I'm missing! Gracias!

solegalli · 2022-01-19T12:41:24Z

Hola @solegalli,

A few random questions/thoughts:

1. Should the transformer confirm there is a time series for one of the dataframe columns? If so, do we ask the user to state the time-series column when instantiating the transformer? Do we autodetect the interval measurement used in the time series (I need to brainstorm how to do so.) or ask the user for the measurement's denomination?

2. Does the user state the lag's time interval - e.g., minute, hour, and day - when instantiating the class?

3. If the user claims the lag's time interval, I'm assuming we need to check that it's compatible with the stated time series. How should we define compatibility? Let's imagine that the time series is denominated in 15-minute intervals and the user selects the lag to be one hour, which is equal to four observations. Do we consider this to be compatible? Or, does "compatibility" require the denominations of both the time lag interval and time-series interval to be equivalent? Maybe it's a version 1 and 2.

4. How do you envision multiple lags being implemented? I'm imagining a world where a user would instantiate a Pipeline with multiple TimeSeriesLagTransformer classes. Is that what you're referring to when you mentioned, `The previous could be very easily done with a Pipeline, so maybe no need to overcomplicate the class.`

5. Should we take a look at the statsmodel package for suggestions?

Lmk what I'm missing! Gracias!

pandas shift works with the index. It has a param, freq to detect the frequency interval. Our transformer would take pandas shift params as well, and with that, the user would be able to regulate how to move their features. I would not re-invent the wheel :p
the user would instantiate pandas shift arguments, which we would bring forward to the init method of our transformer
it is handled by pandas shift.
in the param freq, if instead of a string, the user passes a list, for example ['1H', '2H', '3H'], we could loop over this list to concatenate these 3 shifts. I am still not convinced this adds value. Let's start by implementing a transformer with just 1 shift, and then we see.
Not at this stage.

I think a good starting point would be to read pandas shift docs and play around with that and a time series dataset with a time index or an integer index to understand how it works

Thank you!

solegalli · 2022-01-21T14:43:49Z

@Morgan-Sell github alerted me of a question, but I can't find it. Did you delete it?

Morgan-Sell · 2022-01-22T03:03:14Z

Hi @solegalli,

Yes, I wrote a question then answered it myself. I must be learning ;)

I'm still plugging away.

Disfruta el finde!

Morgan-Sell · 2022-01-22T19:30:44Z

Hola @solegalli,

There seems to be a bug in the test_check_estimator_selectors.py. I'm not familiar with the feature-selection transformers. Do you have an idea of the cause of the bug?

Gracias!

…eep_original_during_installation()

solegalli · 2022-02-08T03:19:09Z

+                    f"input_features must be a list. Got {input_features} instead."
+                )
+            # Create just indicated lag features.
+            input_features_ = input_features


Add a control to check that the user entered variable belongs to self.variables_

Otherwise, this function will output names for lag features that were not created. For example if user passes a string with categorical variables.

@Morgan-Sell I discovered a bug here, so I will probably make another PR to your repo tomorrow :/

solegalli · 2022-02-08T03:36:01Z

@KishManani

I think we've got the main functionality now. Would you like to have a final look?

KishManani · 2022-02-09T21:19:17Z

+The :class:`LagFeatures` adds lag features to the dataframe. A lag feature is a feature
+with information about a prior time step.
+
+In forecasting, past values of the variable we want to forecast, are likely to be


Suggest change from:

In forecasting, past values of the variable we want to forecast, are likely to be predictive.

to

When forecasting the future values of a variable the past values of that variable are likely to be predictive.

Reason: The change is grammatically correct (I think the comma usage in the first sentence in the original is incorrect) and clearer imo.

KishManani · 2022-02-09T21:30:43Z

+            _check_contains_na(X, self.variables_)
+            _check_contains_inf(X, self.variables_)
+
+        # if freq is not None, it overrides periods.


Is the change to ensure that if a list is passed to both freq and periods that this method raises error happening in a different PR?

We are not raising an error. The logic should just not use both and ensures that if freq is given, either list or string, it will override periods. And we mention that in the docstrings above.

solegalli · 2022-02-13T13:05:47Z

@KishManani is the review done?

I've got from you the change in the wording in the docs, and the question on the error, which we are not raising.

From my side, I need to add a variable check in the get_feature_names_out.

Is that all @KishManani or do you need more time?

Thank you!

KishManani · 2022-02-13T17:10:39Z

@solegalli

A couple of small comments:

If you pass a series such as this:

LagFeatures(variables=['y'], freq=['1MS', '2MS', '3MS']).fit_transform(df['y'])

it will raise a TypeError because df['y'] is a pandas Series rather than a Dataframe. Do you want to be able to support passing a Series like this and returning a dataframe of lags?

The current implementation only works when there is a unique time stamp for an index. If you had two observations with the same timestamp the current implementation would throw an error:

InvalidIndexError: Reindexing only valid with uniquely valued Index objects because of the method being used to join the dataframes tmp = pd.concat(df_ls, axis=1). An alternative and probably safer mechanism here to re-join is to use .merge() and specify the timestamp column as the index to join on. Whilst this would allow the generation of lag features whilst having a non-unique time index, it would be odd for the user to have multiple observations for the same timestamp for a univariate forecasting problem. So you could potentially leave it as it is. But you would need the suggested change of using .merge() for my next comment which may be out of scope for this PR.

Even with the fix proposed in number 2) the existing implementation does not cater for the use case where you have multiple time series in the same y column that looks like this for example:

ds	y	country
2000-01-01 00:00:00	0.0191932	UK
2000-01-02 00:00:00	0.301575	UK
2000-01-03 00:00:00	0.660174	UK
2000-01-04 00:00:00	0.290078	UK
2000-01-05 00:00:00	0.618015	UK
2000-01-06 00:00:00	0.428769	UK
2000-01-03 00:00:00	0.135474	France
2000-01-04 00:00:00	0.298282	France
2000-01-05 00:00:00	0.569965	France
2000-01-06 00:00:00	0.590873	France
2000-01-07 00:00:00	0.574325	France
2000-01-08 00:00:00	0.653201	France
2000-01-09 00:00:00	0.652103	France
2000-01-10 00:00:00	0.431418	France

Catering for this kind of example would require significant changes (e.g., including the concept of a column(s) the defines the different time series - in this example it is Country).

I would like to work on a version which extends the existing approach to multiple time series after the current version is merged. Would this be okay @solegalli?

solegalli · 2022-02-16T16:34:33Z

Hi @KishManani

Thank you for the comments. My thoughts and I how I addressed them:

Good point.

Sklearn does not allow the use of pandas series with their transformers. You always need to transform a series to a dataframe to use their classes. So I think we should do the same.

To lag a pandas Series using LagFeatures, users only need to add .to_frame() to the series and that resolves the issue.

I added an example in the user_guide.

I could not reproduce the error. Here is my code:

import pandas as pd
from feature_engine.timeseries.forecasting import LagFeatures

# create a dataframe
X = {"ambient_temp": [31.31, 31.51, 32.15, 32.39, 32.62, 32.5, 32.52, 32.68],
     "module_temp": [49.18, 49.84, 52.35, 50.63, 49.61, 47.01, 46.67, 47.52],
     "irradiation": [0.51, 0.79, 0.65, 0.76, 0.42, 0.49, 0.57, 0.56],
     "color": ["green"] * 4 + ["blue"] * 4,
     }

X = pd.DataFrame(X)
X.index = pd.date_range("2020-05-15 12:00:00", periods=8, freq="15min")

# Add 2 rows with same timestamp value, but different column values

tmp = X.head(2).copy()

tmp.iloc[0] = [1,1,1,'blue']

Xd = pd.concat([X, tmp], axis=0)

# Add lag features
lag_f = LagFeatures(freq="30min")

X_tr = lag_f.fit_transform(Xd)

# the output

                     ambient_temp  module_temp  irradiation  color  \
2020-05-15 12:00:00         31.31        49.18         0.51  green   
2020-05-15 12:00:00          1.00         1.00         1.00   blue   
2020-05-15 12:15:00         31.51        49.84         0.79  green   
2020-05-15 12:15:00         31.51        49.84         0.79  green   
2020-05-15 12:30:00         32.15        52.35         0.65  green   
2020-05-15 12:30:00         32.15        52.35         0.65  green   
2020-05-15 12:45:00         32.39        50.63         0.76  green   
2020-05-15 12:45:00         32.39        50.63         0.76  green   
2020-05-15 13:00:00         32.62        49.61         0.42   blue   
2020-05-15 13:15:00         32.50        47.01         0.49   blue   
2020-05-15 13:30:00         32.52        46.67         0.57   blue   
2020-05-15 13:45:00         32.68        47.52         0.56   blue   

                     ambient_temp_lag_30min  module_temp_lag_30min  \
2020-05-15 12:00:00                     NaN                    NaN   
2020-05-15 12:00:00                     NaN                    NaN   
2020-05-15 12:15:00                     NaN                    NaN   
2020-05-15 12:15:00                     NaN                    NaN   
2020-05-15 12:30:00                   31.31                  49.18   
2020-05-15 12:30:00                    1.00                   1.00   
2020-05-15 12:45:00                   31.51                  49.84   
2020-05-15 12:45:00                   31.51                  49.84   
2020-05-15 13:00:00                   32.15                  52.35   
2020-05-15 13:15:00                   32.39                  50.63   
2020-05-15 13:30:00                   32.62                  49.61   
2020-05-15 13:45:00                   32.50                  47.01   

                     irradiation_lag_30min  
2020-05-15 12:00:00                    NaN  
2020-05-15 12:00:00                    NaN  
2020-05-15 12:15:00                    NaN  
2020-05-15 12:15:00                    NaN  
2020-05-15 12:30:00                   0.51  
2020-05-15 12:30:00                   1.00  
2020-05-15 12:45:00                   0.79  
2020-05-15 12:45:00                   0.79  
2020-05-15 13:00:00                   0.65  
2020-05-15 13:15:00                   0.76  
2020-05-15 13:30:00                   0.42  
2020-05-15 13:45:00                   0.49

The output is funny, because if you have 2 rows with the same timestamp and different values, when lagging, it duplicates following rows, to add the lag of both. But I think this is something that the user should take care of, and not us.

Would you agree? or does it happen often and then we should address it?

If this is not what you meant, could you add a bit of code to raise the error you are mentioning please?

We do use merge on index in the class though. Which concat you are referring to? Those within the loops when we have series instead of values in freq or period?

What is the result of the procedure in the df with multiple ts per y?

I ask this, because, sklearn, and therefore us, aim to output arrays or dfs that are ready to be consumed by ML models.

If we have multiple ts and we lag the features in the same df, then we can't really pass that df to a model. We still need to divide the data into the relevant bits to train the models separately. Am I getting this right?

Bottom line, I need to better understand the output of the transformation and how it fits with training a model afterwards to understand how a potential new class could work. Do you have some code to show the desired output? Or we can chat next Tuesday?

If you have some code, feel free to create a new issue with the new transformer request and the suggested code implementation. Just a snippet to show what the class needs to do or what the output should be.

In summary:

Made a new PR here which addresses the following:

adds error catch to get_feature_names_out
adds docstrings to work with pandas series
changes docstring in user guide as per Kishan's suggestion
reorganises index in api and user guide

I think, with these changes we would be happy to merge and close this class.

Unless I got 2 wrong. Let me know @KishManani

Thank you @Morgan-Sell and @KishManani for the great work! This class rocks :p

solegalli · 2022-02-16T18:50:55Z

Actually, the double rows appear because of our merge!!

In fact, I just noticed that pandas will shift rows only forwards. It does not really re-order the time stamp before moving the rows, even if we pass a frequency in minutes. I didn't notice before, because I always re-order my df for safety before using shift(). But, I am not sure everybody would do so?

This means, that new labels will be created if the row with the required frequency for the shift does not exist, and when we merge, we will create a very fun dataframe.

See for example this:

import pandas as pd
from feature_engine.timeseries.forecasting import LagFeatures

# create a dataframe
X = {"ambient_temp": [31.31, 31.51, 32.15, 32.39, 32.62, 32.5, 32.52, 32.68],
     "module_temp": [49.18, 49.84, 52.35, 50.63, 49.61, 47.01, 46.67, 47.52],
     "irradiation": [0.51, 0.79, 0.65, 0.76, 0.42, 0.49, 0.57, 0.56],
     "color": ["green"] * 4 + ["blue"] * 4,
     }

X = pd.DataFrame(X)
X.index = pd.date_range("2020-05-15 12:00:00", periods=8, freq="15min")

# Add 2 rows with same timestamp value, but different column values

tmp = X.head(2).copy()

tmp.iloc[0] = [1,1,1,'blue']

Xd = pd.concat([X, tmp], axis=0)

tmp = Xd.sample(len(Xd))

tmp


tmp.shift(freq="15min")

                     ambient_temp  module_temp  irradiation  color
2020-05-15 13:00:00         32.62        49.61         0.42   blue
2020-05-15 12:15:00         31.51        49.84         0.79  green
2020-05-15 13:15:00         32.50        47.01         0.49   blue
2020-05-15 12:45:00         32.39        50.63         0.76  green
2020-05-15 13:45:00         32.68        47.52         0.56   blue
2020-05-15 12:00:00         31.31        49.18         0.51  green
2020-05-15 12:30:00         32.15        52.35         0.65  green
2020-05-15 13:30:00         32.52        46.67         0.57   blue
2020-05-15 12:00:00          1.00         1.00         1.00   blue
2020-05-15 12:15:00         31.51        49.84         0.79  green

tmp.shift(freq="15min")

                     ambient_temp  module_temp  irradiation  color
2020-05-15 13:15:00         32.62        49.61         0.42   blue
2020-05-15 12:30:00         31.51        49.84         0.79  green
2020-05-15 13:30:00         32.50        47.01         0.49   blue
2020-05-15 13:00:00         32.39        50.63         0.76  green
2020-05-15 14:00:00         32.68        47.52         0.56   blue
2020-05-15 12:15:00         31.31        49.18         0.51  green
2020-05-15 12:45:00         32.15        52.35         0.65  green
2020-05-15 13:45:00         32.52        46.67         0.57   blue
2020-05-15 12:15:00          1.00         1.00         1.00   blue
2020-05-15 12:30:00         31.51        49.84         0.79  green

To decide:

Should we enforce unique index values?
And also, should we re-order in case the user forgets?
Or should we inform about the behaviour in the docstrings?

My inclination:

yes enforce
reorder when user passes freq but not period
add a note in the user guide.

@KishManani what's your view?

solegalli · 2022-02-17T01:47:31Z

@Morgan-Sell @KishManani

I went on and enforced dataframes to have indexes with unique values and no NaN to be compatible with this transformer.
Otherwise we duplicate rows when we merge.

I also added an argument in the init, to give the option to the user to sort the index or not before lagging the features.

I then added tests and updated the docstrings.

Are we good to go?

Morgan-Sell · 2022-02-17T04:28:16Z

@Morgan-Sell @KishManani

Wow! You two have been having a lot of fun w/o me :(

I like the idea of requiring the user to provide unique time intervals. I worked in renewable energy and dealt with a lot of time-series data (I wasn't using Python) both energy production and wholesale electricity markets.

I would like to know if the dataset was comprised of duplicate time intervals. I cannot envision a scenario in which I would've developed a model/anlayis using duplicate time intervals. In the case of energy production, the time-interval duplication would inform me that there may be something wrong w/ the SCADA - the system that collects energy generation and other relevant data.

final changes to class

solegalli · 2022-02-18T00:53:13Z

+                "with this transformer."
+            )
+
+        if X.index.isnull().sum() > 0:


this check needs to come before the check for unique values.

Morgan-Sell · 2022-02-19T00:24:00Z

@solegalli @KishManani

Is the PR completed?

solegalli · 2022-02-22T01:26:47Z

Oh, yes it is, big time!

Thank you guys, merging now.

@Morgan-Sell @KishManani

…erformance, estimators and more (#372) * Added `get_feature_names` API to encoders + added checks for feature name output on tests * Corrected type hinting for `input_features` parameter * Simplify `get_feature_names` - store input features on fit - use transformed array to compare output of `get_feature_names` in tests * Update base transformer with new method + update tests as well * adds functionality to transformation module * adds functionality to discretization module * adds tests to estimator checks * adds func to base num transformer * fixes minor wordning * fixes minor wordning * fixes minor wordning * update discretisation transformers * update discretisation transformers * fixes all * updates discretisers * removes test from tree disc * update base categorical * updates tree encoder * updates encoders and tests * updates tests encoders * add tests ohe get names out * adds tests for errors * minor adjustment transformers * removes unnecessary docstrings workaround * update imputers and tests * test missing indicator get f names out * updates outliers * updates match variables * adds get_f_name_out to selectors * adds test get feature out selectors * adds tests for selectors * uncomments tests * aligns code line * blacks single feat perf * starts changes to sklearn wrapper * expands sklearn wrapper * expands test on allowed transformers * expands sklearn wrapper * finishes sklearn wrapper * fixes codestyle * rebases main after merging #360 * remove todo from time series * remove unwanted notebook * add get feature names to datetime transformer * fixes getfeatnames bug * add additional test datetime * adds first attempt in get feat names out creation * remove type hint from base transformer attribute * deprecates creation transformers * deprecates creators * reorganises common checks * fixes style * fixes typehint issues * refactors tags in encoders, adds comment in discreatiser base * removed 2 tests for encoders, they are now in general tests * edits estimator_checks docstrings and fixes minor bugs * changes wording in cyclicalfeatures adds fixme in creation init * remove get_feature_out func from deprecated creation transformers * add common checks to creation transformers * adds common tests to creation and datetime, sorts style issues * changes wording of estimatorchecks * changes wording in docstring base_creation * adds doc files for new creation classes * changes wording in mathfeatures * reorders df in datetime and changes wording in comments * reformulates get_feat_names_out missing indicator * creates abstraction of features_names_in * removed unused tag in matchvariables * removes duplicated df check from recursive selectors * creates abstraction for featurenamesin in selectors * changes wording sklearn wrapper * updates common tests wrapper * updates tests cyclicalfeatures * fixes style issues * updates relative features logic * updates cyclical features user guide * adds get feature names out demo * creates docs for new creation modules * updates readme and remaining links * adds link to example jupyter notebooks repo * adds all methods in docstrings * adds whats new * fixes error in select by target mean performance * fixes name contributor * fixes whitespace issue * updates sklearn version requirement * removes support for python 3.6 * Fixes CV split bug in SelectByShuffling (#384) * Draft fix computing performance * lay-out * Remove assignment of y to pandas series * Enforce y has iloc attribute in shuffle feature selection * Remove unused argument in the test Co-authored-by: Gilles Verbockhaven <gilles.verbockhaven@ing.com> * replaces imputation loop by dictionary within fillna (#391) * replaces imputation loop by dictionary within fillna * renamed private method * replaces np.where by pd.isna() in missing indicators * modified mode imputation to remove loop * updates wording in base imputer * removes redundant df copy * adds test for double mode error * reformats transform method of categorical imputer * adds whats new in this pr * improves select by target mean functionality (#390) * add new folder * create TargetMeanPredictor class and its outline * built more of TargetMeanPredictor class framework * add 3 init params * expand fit() method * add discretisers to fit() method * add 'numeric_var_startegy' param and cleaned up init() * create new init params * identify variable types * instatiate encoder and discretisers in fit() * instatiate and fit encoder and discretiser * add init params and check * create disc_mean_dict to store means for the bins of each numerical variable * add checks in predict() * create test_prediction directory and files * start creating first check * create df_pred() in conftest.py * create prediction init file and expand test_target_mean_predictor_fit() * fix bugs * fix bugs * create df_pred() to test TestMeanPredictor * bug: KeyError:None when slicing df_pred even though all variables exists w/in df. sucessfully printed sliced df using column names * resolve bug in fit(). code pass initial part of test_target_mean_predictor_fit * add df checks and bins to the discretisers * add fit params tests * add test for fit params * add test for fit params * create code for predict(). outstanding items to be discussed. * add functionality in fit() if self.variables is None and rearrange 2 lines of code in fit() * create _make_categorical_pipeline() * create _make_numerical_pipeline() * create _make_combine_pipeline() * incorporate pipeline methods into fit() * incorporate pipeline methods into fit() * edit bins check in init method * delete ignore_format param * delete ignore_format param * remove variables check in the beginning of fit() * clean fit() code * start refactoring predict() * refactor fit() and predict() * update MeanEncoder instantiations * update fit params test * create conftest_prediction and move df_pred() from conftest to conftest_prediction * fix docstring * refactor file * complete predict() * create df_pred_small() in conftest * refactor pipeline code * create test_target_mean_predictor_transformation() * start creating r2_score and clean code * create mean_accuracy_score() * edit df_pred_small() * refactor code * create test_r2_score_calculation_with_equal_distance() * refactor code * add binary-label feature to df_pred and df_pred_small * fix styler errors" * fix style errors * fix style errors * fix style errors * fix style errors * add 'regression=False' to DecisionTreeEncoder() in test_check_estimator_encoders.py * coalesce r_squared_score() and mean_accuracy_score() to create score() * clean code * add 'Height_cm' feature to dataframes * add test_predictor_with_all_numerical_variables() * clean code * add new tests * create test_error_if_df_contains_na_in_fit() and test_error_if_df_contains_na_in_transform() * create test_error_when_x_is_not_a_dataframe() * fix styler errors * add dataframe check * fix test code * create BaseTargetMeanPredictor class * add fit() and supporting methods to BaseTargetMeanPredictor class * add init() and predict() to TargetMeanRegressor * expand BaseTargetMeanEstimator docstring * expand docstrings * clean code * create TargetMeanClassifier class * revise precition __init__.py * create predict_proba for TargetMeanClassifier * change test_target_mean_prediction.py to test_target_mean_regressor.py * clean code on test_target_mean_prediction.py * resolve errors returned from test_target_mean_regressor.py * edit TargetMeanClassifier init * create TargetMeanClassifier class * fix error * changes wording init * refactors base perdictor, expands variable detection * refactors target mean regressor * finish refactoring code * final edits of docstrings * add numpy array check for y_pred in TargetMeanRegressor * add 2d numpy array check for 'prob' in TargetMeanClassifier * edit warning string * add 2-column numpy array check for 'log_prob' in TargetMeanClassifier * add numpy check to TargetMeanClassifier predict() * create test_raises_error_when_wrong_input_params() * create test_default_params() * delete test_incorrect_strategy_during_instantiation() * delete test_incorrect_bin_value_during_instantiation() * clean code in test_target_mean_regressor.py * create test_raises_error_when_not_fitting_a_df() using mark.parametrize * create test_raises_error_when_not_transforming_a_df() and clean test code * refactor code * clean text * create test_target_mean_classifier.py and add 2 tests * create test_attributes_upon_fitting() * refactor test_attributes_upon_fitting() for TargetMeanRegressor * fix error * fix bug on test files * fix regressors test errors except for Pipeline attribute. Cause is unclear given the results are identical. * add typehint to _find_categorical_and_numerical_variables() * fix _find_categorical_and_numerical_variables() * fix test_attributes_upon_fitting() error for regressor * fix test_attributes_upon_fitting() error for classifier * fix clsfr predict() * create test_classifier_prediction_results_with_all_numerical_variables() * edit df_pred_small * create test_classifier_results_with_all_categorical_variables() * create two clsfr tests * revise rgrsr test_raises_error_when_df_has_nan() * create clsfr test_error_if_df_contains_na_in_transform() * create clsfr test_raises_error_when_not_fitting_a_df() * fix style errors * create 2 tests for test_variable_manipulation * add 1 test and refactor code * create df_enc_categorical_and_numeric * create test_find_cat_and_num_vars_df_contains_num_and_cat() for test_variable_manipulation * create test for when user passes None, df contains numerical for test_variable_manipulation * create test for when user passes None, df contains categorical for test_variable_manipulation * create test for user passes empty list, function raises error for test_variable_manipulation * create 3 tests for test_variable_manipulation * fix bug in _find_categorical_and_numerical_variables() * try to debut test_error_find_cat_and_num_vars_datetime_var() * rename unit tests * fix style error * consolidating test__find_categorical_and_numeric. can a df be passed to @pytest.mark.parameterize? * consolidating test__find_categorical_and_numeric. can a df be passed to @pytest.mark.parameterize? * fix dataframe checks in BaseTargetMeanEstimator * revise dataframe checks. try sklearn check_X_y cause upstream errors b/c fcn returns numpy arrays which don't have dtypes * clean code for BaseTargetMeanEstimator and TargetMeanClassifier * change df_enc_categorical_and_numeric to df_vartypes * delete df_enc_categorical_and_numeric * expand test_classifier_results_with_all_categorical_variables() * update test_classifier_results_with_all_numerical_variables() and fix test_attributes_upon_fitting() * update test_regression_score_calculation_with_equal_frequency * create test_regressor_with_two_variables() * create 2 regressor tests * add 1 regressor test * fix TargetMeanClassifier fit() * create 2 tests for TargetMeanClassifier * refactor classifier test code * change 2 unit test names * updates encoding tests * updates variable manipulation * improves function to select num and cat vars * resets tests variable manipulation * finishes tests new var selection method * fixes codestyle in var manipulation files * renames folder and updates base predictor * reformats basepredictor * updates target mean classifier * updates target mean regressor * small fix * finishes general checks for all prediction classes * adds tests for classifier * first draft tests * finishes tests predictors * fixes bug * removed notebook * split predict method in transform and predict * deprecates pipeline attr and replaces by encoding dicst * refactors assignment * last touches to predictor classes * finishes target selection and tests * fix typos * updates user guide of select by target mean Co-authored-by: sana <sana@fraugster.com> Co-authored-by: Morgan-Sell <morganpsell@gmail.com> * edits to cyclical features user guide * updates and expands whats new * changes wording in mathfeatures * changes wording in relative features * finishes adding changes to whatsnew Co-authored-by: Alejandro Giacometti <alejandro.giacometti@gmail.com> Co-authored-by: gverbock <32060943+gverbock@users.noreply.github.com> Co-authored-by: Gilles Verbockhaven <gilles.verbockhaven@ing.com> Co-authored-by: sana <sana@fraugster.com> Co-authored-by: Morgan-Sell <morganpsell@gmail.com>

initial commit for TimeSeriesLagTrasnformer class

a97996e

add init method and docstring template

fba5358

Morgan-Sell added 3 commits January 20, 2022 15:16

expand init() method

b65bc66

expand init() and wrote some of the method's docstring

007a2df

create check for dateoffset/freq

24cc5d0

Morgan-Sell added 7 commits January 21, 2022 18:09

add fill_value to init()

3aa4d67

fix init()

dd0f338

create rename_columns()

055960a

add variables to init() and fix rename_columns()

85efa1f

edit rename_variables()

b1a6062

create transform()

3afaa5b

delete lag_time()

90beacd

Morgan-Sell added 6 commits January 22, 2022 10:33

fix bug

c06d5f4

create test_time_series dir and time-series conftest

9ceeddb

create test_time_lag_period_shift()

8414fc6

fix bug in test

4eec364

fix test_check_estimator_encoders bug

678e4c7

create __init__.py in forecasting directory

5b5dad3

Morgan-Sell added 7 commits January 22, 2022 11:55

create test_time_lag_frequency_shift_and_ignore_original_data()

d02ec3c

create test_time_lag_fill_value()

702a928

create test_incorrect_periods_during_installation()

32eb0b7

create test_incorrect_axis_during_installation() and test_incorrect_k…

0471db4

…eep_original_during_installation()

create test_error_when_df_in_transform_is_not_a_dataframe()

10a48cb

fix bug in init()

2dc4776

add datetime object check

aadf760

solegalli reviewed Feb 8, 2022

View reviewed changes

KishManani reviewed Feb 9, 2022

View reviewed changes

solegalli added 4 commits February 16, 2022 12:29

adds variable catch in get feature names out

9da3d08

fixes docstring user guide as per kishans

450b408

adds example to work with series

196e8db

reorders index in docs

a75bc50

solegalli added 2 commits February 16, 2022 22:34

adds functionality to check index

6ca98fe

updates docstring

6b000bd

solegalli added 2 commits February 16, 2022 22:49

fixes documentation link

b0b1aaf

updates docs

9dd3fc2

Merge pull request #8 from feature-engine/lag_features_final

46ac643

final changes to class

solegalli reviewed Feb 18, 2022

View reviewed changes

solegalli added 3 commits February 17, 2022 22:01

reorders checks for nan and unique values in index

98347e2

remove unnecessary comment from test

7015cb4

removes whitespace

a0f896d

solegalli changed the title ~~create TimeSeriesLagTrasnformer: new feature~~ [MRG] create TimeSeriesLagTrasnformer: new feature Feb 18, 2022

solegalli mentioned this pull request Feb 18, 2022

adds get_feature_names_out() method and feature_names_in_ attribute #372

Merged

15 tasks

solegalli merged commit c5b542b into feature-engine:main Feb 22, 2022

solegalli added a commit that referenced this pull request Feb 22, 2022

rebases main after merging #360

92d1be9

Uh oh!

Conversation

Morgan-Sell commented Jan 19, 2022 • edited by solegalli Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Morgan-Sell commented Jan 19, 2022

Uh oh!

solegalli commented Jan 19, 2022

Uh oh!

solegalli commented Jan 21, 2022

Uh oh!

Morgan-Sell commented Jan 22, 2022

Uh oh!

Morgan-Sell commented Jan 22, 2022

Uh oh!

solegalli Feb 8, 2022

Choose a reason for hiding this comment

Uh oh!

solegalli Feb 8, 2022

Choose a reason for hiding this comment

Uh oh!

solegalli commented Feb 8, 2022

Uh oh!

KishManani Feb 9, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KishManani Feb 9, 2022

Choose a reason for hiding this comment

Uh oh!

solegalli Feb 10, 2022

Choose a reason for hiding this comment

Uh oh!

solegalli commented Feb 13, 2022

Uh oh!

KishManani commented Feb 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

solegalli commented Feb 16, 2022

Uh oh!

solegalli commented Feb 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

solegalli commented Feb 17, 2022

Uh oh!

Morgan-Sell commented Feb 17, 2022

Uh oh!

solegalli Feb 18, 2022

Choose a reason for hiding this comment

Uh oh!

Morgan-Sell commented Feb 19, 2022

Uh oh!

solegalli commented Feb 22, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Morgan-Sell commented Jan 19, 2022 •

edited by solegalli

Loading

KishManani Feb 9, 2022 •

edited

Loading

KishManani commented Feb 13, 2022 •

edited

Loading

solegalli commented Feb 16, 2022 •

edited

Loading