[BUG] Detrender returns incorrect result when using inverse_transform

[KishManani]

Describe the bug
The inverse_transform is not returned correctly when passing a Series type which has a different name from the DataFrame or Series used during fit. Instead two columns which are null are returned. Another condition to trigger this bug appears to be when the user supplies data with a different time index during fit and inverse_transform.

To Reproduce

from sktime.datasets import load_airline
from sktime.transformations.series.detrend import Detrender
df = load_airline().rename("y").to_frame()

detrender = Detrender()
# Fit on some time period
detrender.fit(df.loc[:"1949-03", "y"])
# De-trend on a different time period
y_detrended = detrender.transform(df.loc["1949-04":, "y"])
# Inverse transform (i.e., add the trend back)
detrender.inverse_transform(y_detrended)

This behaviour only incurs when the input is a Series. If I were to wrap "y" above in a list, ["y"], so that a DataFrame is instead passed to fit and transform, then we do not get this bug.

Expected behavior
A single column with the trend added back resulting in the same data as the original dataframe.

Versions

System:
python: 3.8.7 (default, May 6 2021, 21:53:45) [Clang 12.0.0 (clang-1200.0.32.29)]
executable: /Users/kishan_manani/.pyenv/versions/3.8.7/envs/udemy-ts/bin/python3.8
machine: macOS-11.6.6-x86_64-i386-64bit

Python dependencies:
pip: 22.3.1
setuptools: 49.2.1
sklearn: 1.2.0
sktime: 0.15.1
statsmodels: 0.13.2
numpy: 1.21.6
scipy: 1.8.1
pandas: 1.5.2
matplotlib: 3.6.2
joblib: 1.2.0
numba: 0.56.4
pmdarima: 1.8.5
tsfresh: 0.19.0

[fkiraly]

Confirmed on python 3.10, windows.

I think it is unclear whether there is expected behaviour - i.e., whether a series of different name should be, by default treated as a different column.

The "logical root case" imo is that the name gets lost after transform - is it?

[fkiraly]

Also, slightly confused - I thought we test for all forecasters that they preserve the name attribute for pd.Series - this has been a source of problems previously:
#3323

[fkiraly]

hm, reason is that convert_MvS_to_UvS_as_Series, in line 89, deletes the name of any pd.Series output after conversion brom pd.DataFrame.
That's interesting, and I vaguely remember this had to be added due to some conversion breakage.

Let's see what happens if I remove this...
#4150
(fixes the problem in this issue, but may have other side effects, e.g., when converting pd.DataFrame to pd.Series and back)

[fkiraly]

Another condition for trigger this bug appears to be when the user supplies data with a different time index during fit and inverse_transform.

Can you give an example for this? This should not happen, as the Detrender is tested indirectly via pipelines.

[fkiraly]

Another attempted fix is to remove the name at the start and potentially add it back at the end, so internal usage never sees the pd.Series name attr:
#4155

[fkiraly]

You've opened a can of worms @KishManani.
Looks like the name attribute is inconsistently handled, the problem is that tests did not cover this properly (but I thought they were).

See here for an explanation why it looked like it was covered but wasn't, and a test that adds this coverage:
#4157

See here for various related fixes:
#4150
#4155
#4161

if you have time, would be interested to hear which approach looks more promising.

And, whether there will be unintended side effects of the fix...

[KishManani]

Can you give an example for this? This should not happen, as the Detrender is tested indirectly via pipelines.

I'm unable to reproduce the error so this is likely a mistake on my end.

[fkiraly]

I think the combination of PR merged together in #4150 now fixes this.