[ENH] umbrella issue - foundation models and pre-trained models

[fkiraly]

Collecting links to some issues on foundation models, pre-trained models, and related interface discussions.

specific foundation models and marketplaces

• hugging face pretrained models [ENH] Loader for pretrained transformer models from hugging face #5790
• chronos [ENH] Amazon chronos forecaster interface #6119
• LagLlama [ENH] Interfacing LagLLama #6142
• MOIRAI [ENH] Adapter for Uni2TS, MOIRAI #6161
• TimesFM [ENH] Add support for TimesFM #6408
• momentfm [ENH] momentfm foundation models #6542

Note: most of these are for forecasting, some are multi-purpose. Adapatation to other tasks like TSC, or TSC-specific models should also be investigated.

open questions

• how to design interface for already fitted models, and fine-tuning: [ENH] design issue - mapping weight handling and fine-tuning for FM on forecaster interface #6580
  • also see [ENH] FittedForecaster for using serialized/fitted forecasters #2903
• relation to the update interface point
• "pairing weights with network architectures" - constructor or other interface point?
• relation to global modelling interface, [ENH] Support Global Time Series Forecasting #4651, as global models allow to train and pre-train
• multi-task or polymorphic models, mapping on interfaces

[fkiraly]

Very relevant for API design questions and layer architecture: the fastAI API and desigs, see https://docs.fast.ai/tutorial.vision.html

• inteface for fine tuning
• high-level abstraction layer

fastai also has a huggingface integration: https://docs.fast.ai/huggingface.html - the first good integation with consistent/unified API that I have seen so far.