Skip to content

Add Per-segment metric visualization #658

Merged
merged 5 commits into from
Apr 29, 2022
Merged

Add Per-segment metric visualization #658

merged 5 commits into from
Apr 29, 2022

Conversation

Mr-Geekman
Copy link
Contributor

@Mr-Geekman Mr-Geekman commented Apr 22, 2022

IMPORTANT: Please do not create a Pull Request without creating an issue first.

Before submitting (must do checklist)

  • Did you read the contribution guide?
  • Did you update the docs? We use Numpy format for all the methods and classes.
  • Did you write any new necessary tests?
  • Did you update the CHANGELOG?

Type of Change

  • Examples / docs / tutorials / contributors update
  • Bug fix (non-breaking change which fixes an issue)
  • Improvement (non-breaking change which improves an existing feature)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Proposed Changes

Look #545.

Related Issue

#545.

Closing issues

Closes #545.

@Mr-Geekman Mr-Geekman added the enhancement New feature or request label Apr 22, 2022
@Mr-Geekman Mr-Geekman self-assigned this Apr 22, 2022
@Mr-Geekman
Copy link
Contributor Author

Example script:

import matplotlib.pyplot as plt
import pandas as pd

from etna.analysis import plot_metric_per_segment
from etna.datasets import TSDataset
from etna.metrics import MAE
from etna.metrics import SMAPE
from etna.models import ProphetModel
from etna.pipeline import Pipeline


def main():
    df = pd.read_csv("examples/data/example_dataset.csv", parse_dates=["timestamp"])
    df_wide = TSDataset.to_dataset(df)
    ts = TSDataset(df=df_wide, freq="D")

    pipeline = Pipeline(model=ProphetModel(), horizon=10)
    metrics_df, _, _ = pipeline.backtest(ts=ts, n_folds=3, metrics=[MAE(), SMAPE()])

    plot_metric_per_segment(metrics_df=metrics_df, metric_name="MAE", per_fold_aggregation_mode="mean")
    plt.savefig("metric_per_segment_plot")

    pipeline = Pipeline(model=ProphetModel(), horizon=10)
    metrics_df, _, _ = pipeline.backtest(ts=ts, n_folds=3, metrics=[MAE(), SMAPE()])

    plot_metric_per_segment(
        metrics_df=metrics_df, metric_name="MAE", per_fold_aggregation_mode="mean", ascending=True, top_k=2
    )
    plt.savefig("metric_per_segment_plot_limited")


if __name__ == "__main__":
    main()

metric_per_segment_plot:
metric_per_segment_plot

metric_per_segment_plot_limited:
metric_per_segment_plot_limited

@github-actions
Copy link

github-actions bot commented Apr 22, 2022

🚀 Deployed on https://deploy-preview-658--etna-docs.netlify.app

@github-actions github-actions bot temporarily deployed to pull request April 22, 2022 14:08 Inactive
@codecov-commenter
Copy link

codecov-commenter commented Apr 22, 2022

Codecov Report

Merging #658 (eac489a) into master (29a730d) will decrease coverage by 0.31%.
The diff coverage is 26.47%.

@@            Coverage Diff             @@
##           master     #658      +/-   ##
==========================================
- Coverage   84.45%   84.14%   -0.32%     
==========================================
  Files         119      119              
  Lines        6322     6356      +34     
==========================================
+ Hits         5339     5348       +9     
- Misses        983     1008      +25     
Impacted Files Coverage Δ
etna/analysis/plotters.py 21.16% <24.24%> (+0.21%) ⬆️
etna/analysis/__init__.py 100.00% <100.00%> (ø)

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@martins0n martins0n requested a review from iKintosh April 25, 2022 07:56
Copy link
Contributor

@iKintosh iKintosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seem I have found a bug =)

if self.value == "mean":
return np.nanmean
elif self.value == "median":
return self.nanmedian
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure it is self.nanmedian and not np.nanmedian?

@github-actions github-actions bot temporarily deployed to pull request April 29, 2022 07:45 Inactive
@Mr-Geekman Mr-Geekman merged commit b48ef0d into master Apr 29, 2022
@Mr-Geekman Mr-Geekman deleted the issue-545 branch April 29, 2022 10:25
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Per-segment metric visualization
3 participants