Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Chronos notebook #10

Merged
merged 1 commit into from
Mar 27, 2024
Merged

Fix Chronos notebook #10

merged 1 commit into from
Mar 27, 2024

Conversation

abdulfatir
Copy link

Hi!

Thank you for trying Chronos out! It looks like the predictions from Chronos are misaligned when you compute the metrics. This PR fixes this issue. Furthermore, there were multiple inefficiencies in the inference loop for Chronos:

  • Pandas DataFrame was being filtered in a loop.
  • Predictions were being generated one time series at a time. In this notebook I have used a batch size of 12, which should fit on a typical consumer GPU, even for the large model.

Here are the final numbers after these fixes:

metric MLP NBEATS Chronos-Tiny Chronos-Large
0 mae 566.162569 551.727285 582.312115 563.345977
1 smape 0.064215 0.062675 0.065570 0.063084
2 Time 23.294084 49.143765 19.080808 138.417686

In case you're wondering what was the bug in the previous version: the index in forecast_df was incorrect after sorting. You can fix the previous version by simply changing:

forecast_df = forecast_df.drop(columns=['SortKey'])

to

forecast_df = forecast_df.drop(columns=['SortKey']).reset_index(drop=True)

Based on these results, I kindly ask you to modify your code and blog post. Furthermore, I would also request you to post an update on your social media profiles. Please let me know if you need any help from our side running Chronos! Thank you again!

@marcopeix marcopeix merged commit 92dad7d into marcopeix:master Mar 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants