Add optional inverse transform in historical forecast #2267

alicjakrzeminska · 2024-03-05T08:33:34Z

Edit:

Checklist before merging this PR:

Mentioned all issues that this PR fixes or addresses.
Summarized the updates of this PR under Summary.
Added an entry under Unreleased in the Changelog.

Fixes #2260.

Summary

InvertibleDataTransformer now supports parallelized inverse transformation for series being a list of lists of TimeSeries (Sequence[Sequence[TimeSeries]]). This series type represents for example the output from historical_forecasts() when using multiple series.

VascoSch92 · 2024-03-05T08:47:26Z

darts/models/forecasting/forecasting_model.py

+
+        """
+
+        if not forecasts_list_transformed:


i don't get this point. If forecasts_list_transformed is empty you are doing nothing right?

Can you check that before calling the method?

The method is private, so it should be used within this class.

Make it sense? :-)

VascoSch92 · 2024-03-05T08:49:22Z

darts/models/forecasting/forecasting_model.py

+            return forecasts_list_transformed
+
+        if isinstance(forecasts_list_transformed[0], TimeSeries):
+            forecasts_list_inversely_transformed = scaler.inverse_transform(


Suggested change

forecasts_list_inversely_transformed = scaler.inverse_transform(

return scaler.inverse_transform(

in this way you don't have to go down in the method to check what is returned in this case :-)

VascoSch92 · 2024-03-05T08:51:14Z

darts/models/forecasting/forecasting_model.py

+        else:
+            fill_value = TimeSeries.from_values(np.empty(1))
+
+            forecasts_list_inversely_transformed = list(


at this stage here the list is not inversely transformed. Perhaps another name for this variable.

VascoSch92 · 2024-03-05T08:55:08Z

darts/models/forecasting/forecasting_model.py

+        """Applies inverse transform on the list (of lists) of forecasts.
+        Uses the scaler trained previously on the same set of series.


Suggested change

"""Applies inverse transform on the list (of lists) of forecasts.

Uses the scaler trained previously on the same set of series.

"""Applies inverse transform on the list (of lists) of forecasts,

using the scaler trained previously on the same set of series.

VascoSch92 · 2024-03-05T08:56:24Z

darts/models/forecasting/forecasting_model.py

+        scaler
+            Optionally, the scaler trained on the same `series`, performs inverse transform on the generated forecasts.


Suggested change

scaler

Optionally, the scaler trained on the same `series`, performs inverse transform on the generated forecasts.

scaler

Optionally, the scaler trained on the same `series` is used to perform an inverse transform on the generated forecasts.

codecov-commenter · 2024-03-05T09:04:27Z

Codecov Report

Attention: Patch coverage is 85.29412% with 10 lines in your changes are missing coverage. Please review.

Project coverage is 93.94%. Comparing base (d764bc4) to head (ff80a7e).

Files	Patch %	Lines
...taprocessing/transformers/base_data_transformer.py	71.42%	8 Missing ⚠️
...ocessing/transformers/fittable_data_transformer.py	94.11%	1 Missing ⚠️
...essing/transformers/invertible_data_transformer.py	95.65%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2267      +/-   ##
==========================================
- Coverage   94.00%   93.94%   -0.07%     
==========================================
  Files         136      136              
  Lines       13648    13673      +25     
==========================================
+ Hits        12830    12845      +15     
- Misses        818      828      +10

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

dennisbader

Thanks for this Draft PR @alicjakrzeminska. I would suggest to move the logic to darts.dataprocessing.transformers.invertible_data_transformer.InvertibleDataTrasnformer. In my opinion this can generally be applied to any InvertibleDataTransfomer. What do you think?

VascoSch92 · 2024-03-08T12:57:37Z

darts/dataprocessing/transformers/invertible_data_transformer.py

+        * [A1, B1, C1] - list of series,
+        * [[A1, A2, A3], [B1, B2], [C1, C2, C3]] - list of lists of series,


Suggested change

* [A1, B1, C1] - list of series,

* [[A1, A2, A3], [B1, B2], [C1, C2, C3]] - list of lists of series,

- [A1, B1, C1] - list of series,

- [[A1, A2, A3], [B1, B2], [C1, C2, C3]] - list of lists of series,

VascoSch92 · 2024-03-08T12:59:55Z

darts/dataprocessing/transformers/invertible_data_transformer.py

+        Allows for different lengths of the lists.
+        Allows for missing series the list.


Suggested change

Allows for different lengths of the lists.

Allows for missing series the list.

Supports lists of varying lengths and missing elements within the list/s.

alicjakrzeminska · 2024-03-08T14:04:17Z

@dennisbader, I moved the function to InvertibleDataTransfomer . Please, take a look, and if you find it ok, I'll add some tests.
@VascoSch92, thank you for your feedback, I applied your suggestions. :)

alicjakrzeminska · 2024-03-14T16:02:32Z

@dennisbader, I integrated the functionality with inverse_transform method of InvertibleDataTransfomer as you suggested. Please, have a look, and I'll write the docstrings and tests if it's ok for you.

VascoSch92 · 2024-03-16T07:55:02Z

darts/dataprocessing/transformers/invertible_data_transformer.py

+                for series_ in series_list:
+                    input_series.append(series_)
+                    data.append(series_)


Suggested change

for series_ in series_list:

input_series.append(series_)

data.append(series_)

input_series.extend(series_list)

data.extend(series_list

dennisbader

Really nice, thanks a lot for this great PR @alicjakrzeminska 🚀

Add optional inverse transform in historical forecast

25b2a85

VascoSch92 reviewed Mar 5, 2024

View reviewed changes

dennisbader requested changes Mar 5, 2024

View reviewed changes

alicjakrzeminska added 3 commits March 8, 2024 10:53

Update variables names and docstrings

c44d16a

Move the inverse transform to InvertibleDataTransformer

1693181

Fix single element list

e9112cd

VascoSch92 reviewed Mar 8, 2024

View reviewed changes

Update docstrings

3671bf2

Move the inverse transform of list of lists to inverse_transform method

370ada2

dennisbader added 2 commits March 15, 2024 11:58

Merge branch 'master' into feat/scaler-inverse-transform-ts-list

200da0a

make invertible transformers act on list of lists of series

e34f15d

dennisbader marked this pull request as ready for review March 15, 2024 18:06

dennisbader requested a review from madtoinou as a code owner March 15, 2024 18:06

VascoSch92 reviewed Mar 16, 2024

View reviewed changes

dennisbader added 2 commits March 16, 2024 11:04

add tests

ef21431

update changelog

ff80a7e

dennisbader approved these changes Mar 16, 2024

View reviewed changes

dennisbader merged commit 91c7087 into unit8co:master Mar 16, 2024
9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add optional inverse transform in historical forecast #2267

Add optional inverse transform in historical forecast #2267

alicjakrzeminska commented Mar 5, 2024 •

edited by dennisbader

Loading

VascoSch92 Mar 5, 2024

VascoSch92 Mar 5, 2024

VascoSch92 Mar 5, 2024

VascoSch92 Mar 5, 2024

VascoSch92 Mar 5, 2024

codecov-commenter commented Mar 5, 2024 •

edited

Loading

dennisbader left a comment

VascoSch92 Mar 8, 2024

VascoSch92 Mar 8, 2024

alicjakrzeminska commented Mar 8, 2024

alicjakrzeminska commented Mar 14, 2024

VascoSch92 Mar 16, 2024

dennisbader left a comment

	forecasts_list_inversely_transformed = scaler.inverse_transform(
	return scaler.inverse_transform(

		"""Applies inverse transform on the list (of lists) of forecasts.
		Uses the scaler trained previously on the same set of series.

		scaler
		Optionally, the scaler trained on the same `series`, performs inverse transform on the generated forecasts.

		* [A1, B1, C1] - list of series,
		* [[A1, A2, A3], [B1, B2], [C1, C2, C3]] - list of lists of series,

		Allows for different lengths of the lists.
		Allows for missing series the list.

	Allows for different lengths of the lists.
	Allows for missing series the list.
	Supports lists of varying lengths and missing elements within the list/s.

Add optional inverse transform in historical forecast #2267

Add optional inverse transform in historical forecast #2267

Conversation

alicjakrzeminska commented Mar 5, 2024 • edited by dennisbader Loading

Summary

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Mar 5, 2024 • edited Loading

Codecov Report

dennisbader left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alicjakrzeminska commented Mar 8, 2024

alicjakrzeminska commented Mar 14, 2024

Choose a reason for hiding this comment

dennisbader left a comment

Choose a reason for hiding this comment

alicjakrzeminska commented Mar 5, 2024 •

edited by dennisbader

Loading

codecov-commenter commented Mar 5, 2024 •

edited

Loading