AI4TS

Awesome resources focus on the application of cutting-edge AI technologies for time-series analysis (**AI4TS **). They delve into advanced topics such as self-supervised learning (SSL), Graph Neural Networks for Time Series (GNN4TS), Large Language Models for Time Series (LLM4TS), Diffusion models, Mixture-of-Experts (MoE) architectures and Mamba models, Kolmogorov Arnold Networks (KAN) among others. These resources span various domains, including healthcare, finance, and traffic, offering a comprehensive view of the field. In addition, they feature top-notch tutorials, courses, and workshops from prestigious conferences, hosted by globally renowned scholars and research teams. Whether you're a professional, data scientist, or researcher, these tools and techniques can significantly enhance your time-series data analysis capabilities, providing a clear roadmap for your studies.

LLM4TS

• Forecasting
  • PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting arxiv 20 Sep 2022 PISA)
  • Temporal Data Meets LLM -- Explainable Financial Time Series Forecasting arxiv 19 Jun 2023
  • Time-LLM: Time Series Forecasting by Reprogramming Large Language Models arxiv 3 Oct 2023
  • TimeGPT-1 arxiv 2023 5 Oct 2023
  • TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting arxiv 8 Oct 2023
  • Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domain arxiv 2023 8 Oct 2023
  • Large Language Models Are Zero-Shot Time Series Forecasters NeurIPS 2023 11 Oct 2023 llmtime
  • Lag-Llama: Towards Foundation Models for Time Series Forecasting arxiv 12 Oct 2023 Lag-Llama
  • iTransformer: Inverted Transformers Are Effective for Time Series Forecasting arxiv 13 Oct 2023 [iTransformer](GitHub - thuml/iTransformer: This is the official implementation for "iTransformer: Inverted Transformers Are Effective for Time Series Forecasting".)
  • Harnessing LLMs for Temporal Data - A Study on Explainable Financial Time Series Forecasting 2023.emnlp-industry
  • LLM4TS: Two-Stage Fine-Tuning for Time-Series Forecasting with Pre-Trained LLMs arxiv16 Aug 2023
  • UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting 15 Oct 2023
  • AutoTimes: Autoregressive Time Series Forecasters via Large Language Models 4 Feb 2024
  • Unified Training of Universal Time Series Forecasting Transformers 4 Feb 2024 SalesforceAIResearch/uni2ts
  • LSTPrompt: Large Language Models as Zero-Shot Time Series Forecasters by Long-Short-Term Prompting 25 Feb 2024 LSTPrompt
  • Multi-Patch Prediction: Adapting LLMs for Time Series Representation Learning 7 Feb 2024
  • Chronos: Learning the Language of Time Series 12 Mar 2024 chronos
  • S^2 IP-LLM: Semantic Space Informed Prompt Learning with LLM for Time Series Forecasting 9 Mar 2024
  • CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning 12 Mar 2024 CALF
• Anomaly Detection
  • Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection 26 Jan 2024 AnomalyLLM
  • Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review 15 Feb 2024
  • Large language models can be zero-shot anomaly detectors for time series? 23 May 2024
• Imputation
  • GATGPT: A Pre-trained Large Language Model with Graph Attention Network for Spatiotemporal Imputation 24 Nov 2023
• Spatio-temporal prediction
  • Spatio-Temporal Graph Learning with Large Language Model 20 Sept 2023
  • How Can Large Language Models Understand Spatial-Temporal Data? 25 Jan 2024
  • UrbanGPT: Spatio-Temporal Large Language Models 25 Feb 2024
  • TPLLM: A Traffic Prediction Framework Based on Pretrained Large Language Models 4 Mar 2024
  • Spatial-Temporal Large Language Model for Traffic Prediction 18 Jan 2024
• One Fits all
  • One Fits All:Power General Time Series Analysis by Pretrained LM arxiv 23 Feb 2023 NeurIPS2023-One-Fits-All no-officail reproduction
  • TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series arxiv16 Aug 2023 TEST
  • Timer: Transformers for Time Series Analysis at Scale 4 Feb 2024
  • UniTS: Building a Unified Time Series Model 29 Feb 2024 UniTS
  • MOMENT: A Family of Open Time-series Foundation Models 6 Feb 2024 MOMENT
  • TSLANet: Rethinking Transformers for Time Series Representation Learning ICML2024 TSLANet
  • RWKV-TS: Beyond Traditional Recurrent Neural Network for Time Series Tasks 17 Jan 2024 RWKV-TS
  • UniCL: A Universal Contrastive Learning Framework for Large Time Series Models 17 May 2024

Multimodal large models include TS

• Meta-Transformer: A Unified Framework for Multimodal Learning 20 Jul 2023 Meta-Transformer
• UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video,Point Cloud, Time-Series and Image Recognition 27 Nov 2023 UniRepLKNet
• ViT-Lens-2: Gateway to Omni-modal Intelligence 27 Nov 2023 ViT-Lens-2

Patch && Tokenizers methods

• Patches Are All You Need? ICLR 24 Jan 2022 convmixer
• A Time Series is Worth 64 Words: Long-term Forecasting with Transformers ICLR 2023 27 Nov 2022 PatchTST
• PatchMixer: A Patch-Mixing Architecture for Long-Term Time Series Forecasting arxiv 1 Oct 2023 PatchMixer Chinese blog
• Learning to Embed Time Series Patches Independently NeurIPS Workshop on Self-Supervised Learning: Theory and Practice, 2023 pits
• The first step is the hardest: Pitfalls of Representing and Tokenizing Temporal Data for Large Language Models arxiv 12 Sep 2023
• What Makes for Good Visual Tokenizers for Large Language Models? 20 May 2023 GVT
• SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models ICLR2024 SpeechTokenizer
• Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecasting ICLR2024
• From Similarity to Superiority: Channel Clustering for Time Series Forecasting 31 Mar 2024

GNN

• Graph-Aware Contrasting for Multivariate Time-Series Classification AAAI2024 TSGAC

MLPer

• TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting KDD2023PatchTSMixer
• A Multi-Scale Decomposition MLP-Mixer for Time Series Analysis VLDB2024 zshhans/MSD-Mixer
• Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series 8 Jan 2024
• U-Mixer: An Unet-Mixer Architecture with Stationarity Correction for Time Series Forecasting AAAI2024 U-Mixer
• LightTS: Lightweight Time Series Classification with Adaptive Ensemble Distillation—Extended Version SIGMOD 2023
• An Analysis of Linear Time Series Forecasting Models ICML2024

Mixture-of-Experts (MoE)

• Mixture-of-Linear-Experts for Long-term Time Series Forecasting 11 Dec 2023
• Prompt-based Domain Discrimination for Multi-source Time Series Domain Adaptation 19 Dec 2023
• A Review of Sparse Expert Models in Deep Learning 4 Sep 2022
• MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts 8 Jan 2024
• ST-MoE: Spatio-Temporal Mixture of Experts for Multivariate Time Series Forecasting 2023ISKE
• Tutorial for Mixture of Expert (MoE) Forecasting Model — Merlion 1.1.0 documentation (salesforce.com)

Mamba

• Is Mamba Effective for Time Series Forecasting? 17 Mar 2024 wzhwzhwzh0921/S-D-Mamba
• TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting 14 Mar 2024 Atik-Ahamed/TimeMachine
• STG-Mamba: Spatial-Temporal Graph Learning via Selective State Space Model 19 Mar 2024
• SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time series 22 Mar 2024 badripatro/simba
• MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selection 29 Mar 2024
• Traj-LLM: A New Exploration for Empowering Trajectory Prediction with Pre-trained Large Language Models 8 May 2024
• Time-SSM: Simplifying and Unifying State Space Models for Time Series Forecasting 25 May 2024

KAN

• KAN: Kolmogorov-Arnold Networks 30 Apr 2024 pykan
• TKAN: Temporal Kolmogorov-Arnold Networks arxiv12 May 2024 TKAN
• Kolmogorov-Arnold Networks (KANs) for Time Series Analysis 14 May 2024
• Feature-Based Time Series Classification with Kolmogorov–Arnold Networks Simple-KAN-4-Time-Series

Multiple instance learning

• TimeMIL: Advancing Multivariate Time Series Classification via a Time-aware Multiple Instance Learning ICML2024 xiwenc1/TimeMIL

NeXt (Classic networks make a comeback)

• A ConvNet for the 2020s CVPR2022 ConvNext
• ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders 2 Jan 2023 ConvNeXt-V2
• ModernTCN：A Modern Pure Convolution Structure for General Time Series Analysis ICLR2024 luodhhh/ModernTCN
• InceptionNeXt: When Inception Meets ConvNeXt CVPR2024 InceptionNeXt
• RWKV: Reinventing RNNs for the Transformer Era 22 May 2023
• RWKV-TS: Beyond Traditional Recurrent Neural Network for Time Series Tasks arxiv17 Jan 2024 RWKV-TS
• Efficient and Effective Time-Series Forecasting with Spiking Neural Networks ICML2024
• xLSTM: Extended Long Short-Term Memory 7 May 2024 xLSTM

Biosignal dataset

• Neuro-GPT: Developing A Foundation Model for EEG arxiv 7 Nov 2023
• Brant: Foundation Model for Intracranial Neural Signal NeurIPS23
• PPi: Pretraining Brain Signal Model for Patient-independent Seizure Detection NeurIPS23
• Large-scale training of foundation models for wearable biosignals submit ICLR 2024
• BIOT: Cross-data Biosignal Learning in the Wild NeurIPS23 BIOT
• Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI submit ICLR 2024
• Practical intelligent diagnostic algorithm for wearable 12-lead ECG via self-supervised learning on large-scale dataset Nature Communications 2023
• Large AI Models in Health Informatics: Applications, Challenges, and the Future IEEE Journal of Biomedical and Health Informatics Awesome-Healthcare-Foundation-Models
• Data science opportunities of large language models for neuroscience and biomedicine Neuron
• BioSignal Copilot: Leveraging the power of LLMs in drafting reports for biomedical signals July 06, 2023
• Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Data 12 Jan 2024
• Self-supervised Learning for Electroencephalogram: A Systematic Survey 9 Jan 2024
• Learning Topology-Agnostic EEG Representations with Geometry-Aware Modeling NeurIPS 2023 MMM
• A Survey of Large Language Models in Medicine: Progress, Application, and Challenge 9 Nov 2023 AI-in-Health/MedLLMsPracticalGuide
• EEG-GPT: Exploring Capabilities of Large Language Models for EEG Classification and Interpretation 31 Jan 2024
• A Survey on Multimodal Wearable Sensor-based Human Action Recognition 14 Apr 2024
• Unveiling Thoughts: A Review of Advancements in EEG Brain Signal Decoding into Text 26 Apr 2024
• AI for Biomedicine in the Era of Large Language Models 23 Mar 2024

Foundation for other domains

• Foundation Models for Weather and Climate Data Understanding: A Comprehensive Survey 5 Dec 2023
• Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models usail-hkust/Awesome-Urban-Foundation-Models

Multimodal TS

• Biosignal
  • Frozen Language Model Helps ECG Zero-Shot Learning 22 Mar 2023
  • Learning Transferable Time Series Classifier with Cross-Domain Pre-training from Language Model 19 Mar 2024 CrossTimeNet
  • Zero-Shot ECG Diagnosis with Large Language Models and Retrieval-Augmented Generation ML4H2023
  • Large Language Models with Retrieval-Augmented Generation for Zero-Shot Disease Phenotyping 11 Dec 2023
  • Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge Enhancement 11 Mar 2024
  • Electrocardiogram Instruction Tuning for Report Generation 7 Mar 2024
  • Multimodal Pretraining of Medical Time Series and Notes PMLR2023 kingrc15/multimodal-clinical-pretraining
  • SleepFM: Multi-modal Representation Learning for Sleep across ECG, EEG and Respiratory Signals AAAI 2024 Spring Symposium Series Clinical FMs
• Financial
  • FinGPT: Open-Source Financial Large Language Models arxiv 9 Jun 2023 AI4Finance-Foundation/FinNLP
  • FinVis-GPT: A Multimodal Large Language Model for Financial Chart Analysis (FinLLM 2023)@IJCAI 2023
  • Insight Miner: A Time Series Analysis Dataset for Cross-Domain Alignment with Natural Language [NeurIPS2023-AI4Science Poster](https://openreview.net/forum?id=E1khscdUdH
• Other fields
  • GPT4MTS: Prompt-Based Large Language Model for Multimodal Time-Series Forecasting AAAI2024
  • Advancing Time Series Classification with Multimodal Language Modeling 19 Mar 2024 Mingyue-Cheng/InstructTime
  • Frequency-Aware Masked Autoencoders for Multimodal Pretraining on Biosignals 12 Sep 2023
  • FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Space NeurIPS'23 30 Oct 2023 focal
  • Multimodal Adaptive Emotion Transformer with Flexible Modality Inputs on A Novel Dataset with Continuous Labels ACMMM 27 October 2023
  • Improving day-ahead Solar Irradiance Time Series Forecasting by Leveraging Spatio-Temporal Context 1 Jun 2023 CrossViVit

Survey4TS

• Representation learning(self-supervised learning && Semi-supervised learning&&Supervised learning)

• Self-supervised Contrastive Representation Learning for Semi-supervised Time-Series Classification TPAMI 13 Aug 2022 CA-TCC

• Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey ACM Computing Surveys, 2023

• Label-efficient Time Series Representation Learning: A Review 13 Feb 2023

• Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects 16 Jun 2023 SSL4TS

• Unsupervised Representation Learning for Time Series: A Review 3 Aug 2023 ULTS

• Self-Supervised Learning for Time Series: Contrastive or Generative? AI4TS workshop at IJCAI 2023 SSL_Comparison

• Self-Supervised Contrastive Learning for Medical Time Series: A Systematic Review Sensors in 2023 Contrastive-Learning-in-Medical-Time-Series-Survey

• Applications of Self-Supervised Learning to Biomedical Signals: where are we now post date 2023-04-11

• What Constitutes Good Contrastive Learning in Time-Series Forecasting? last revised 13 Aug 2023

• A review of self-supervised learning methods in the field of ECG 2023

• Universal Time-Series Representation Learning: A Survey 8 Jan 2024 itouchz/awesome-deep-time-series-representations

• Deep Learning for Trajectory Data Management and Mining: A Survey and Beyond 21 Mar 2024 yoshall/Awesome-Trajectory-Computing

• Scaling-laws for Large Time-series Models 22 May 2024

• GNN4TS

• A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection 7 Jul 2023 KimMeen/Awesome-GNN4TS

• Generative models

  • General

    • A Survey of Generative Techniques for Spatial-Temporal Data Mining 15 May 2024

  • Diffusion4TS

    • Diffusion models for time-series applications: a survey 1 May 2023
    • A Survey on Diffusion Models for Time Series and Spatio-Temporal Data 29 Apr 2024 yyysjz1997/Awesome-TimeSeries-SpatioTemporal-Diffusion-Model

• LLM4TS

• Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook arxiv 16 Oct 2023 Awesome-TimeSeries-SpatioTemporal-LM-LLM

• Large Language Models for Time Series: A Survey 2 Feb 2024 xiyuanzh/awesome-llm-time-series

• Position Paper:What Can Large Language Models Tell Us about Time Series Analysis 5 Feb 2024

• Empowering Time Series Analysis with Large Language Models: A Survey 5 Feb 2024

• Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilities 16 Feb 2024

• A Survey of Time Series Foundation Models: Generalizing Time Series Representation with Large Language Model 3 May 2024 start2020/Awesome-TimeSeries-LLM-FM

• Large Language Models for Mobility in Transportation Systems: A Survey on Forecasting Tasks 3 May 2024

• Foundation && Pre-Trained models

• A Survey on Time-Series Pre-Trained Models 18 May 2023 time-series-ptms

• Toward a Foundation Model for Time Series Data 21 Oct 2023 code

• A Review for Pre-Trained Transformer-Based Time Series Forecasting Models ITMS2023

• A Survey of Deep Learning and Foundation Models for Time Series Forecasting 25 Jan 2024

• Foundation Models for Time Series Analysis: A Tutorial and Survey 21 Mar 2024

• Heterogeneous Contrastive Learning for Foundation Models and Beyond 30 Mar 2024

• Application

• Deep Learning for Multivariate Time Series Imputation: A Survey 6 Feb 2024 WenjieDu/Awesome_Imputation

• Chinese

  • 任利强,贾舒宜,王海鹏,等.基于深度学习的时间序列分类研究综述[J/OL].电子与信息学报:1-23[2024-05-25].http://kns.cnki.net/kcms/detail/11.4494.TN.20240109.0749.008.html.
  • 毛远宏,孙琛琛,徐鲁豫,等.基于深度学习的时间序列预测方法综述[J].微电子学与计算机,2023,40(04):8-17.DOI:10.19304/J.ISSN1000-7180.2022.0725.
  • 梁宏涛,刘硕,杜军威,等.深度学习应用于时序预测研究综述[J].计算机科学与探索,2023,17(06):1285-1300.

Workshop

• MulTiSA 2024 | MultiTISA 2024 in conjunction with ICDE'24

Project

• ddz16/TSFpaper
• Time_Series_Instruct
• qingsongedu/Awesome-TimeSeries-AIOps-LM-LLM
• LLM for Time Series
• Time Series AI Papers
• Multivariate Time Series Transformer Framework
• xiyuanzh/time-series-papers
• vincentlux/Awesome-Multimodal-LLM
• yyysjz1997/Awesome-TimeSeries-SpatioTemporal-Diffusion-Model
• SitaoLuan/LLM4Graph
• willxxy/awesome-mmps
• mintisan/awesome-kan: A comprehensive collection of KAN(Kolmogorov-Arnold Network)-related resources, including libraries, projects, tutorials, papers, and more, for researchers and developers in the Kolmogorov-Arnold Network field. (github.com)
• Foundation-Models
  • uncbiag/Awesome-Foundation-Models
  • UbiquitousLearning/Efficient_Foundation_Model_Survey
  • zhanghm1995/Forge_VFM4AD
  • Yangyi-Chen/Multimodal-AND-Large-Language-Models
  • MM-LLMs: Recent Advances in MultiModal Large Language Models
  • Learning on Multimodal Graphs: A Survey
  • The (R)Evolution of Multimodal Large Language Models: A Survey
  • A Practical Guide for Medical Large Language Models
  • Instruction Tuning for Large Language Models: A Survey xiaoya-li/Instruction-Tuning-Survey
  • [Visual Instruction Tuning towards General-Purpose Multimodal Model: A Survey](https://arxiv.org/ abs/2312.16602)
  • Brain-Conditional Multimodal Synthesis: A Survey and Taxonomy

Self-supervised learning tools

• Westlake-AI/openmixup: CAIRI Supervised, Semi- and Self-Supervised Visual Representation Learning Toolbox and Benchmark (github.com)
• open-mmlab/mmselfsup: OpenMMLab Self-Supervised Learning Toolbox and Benchmark (github.com)
• lightly-ai/lightly: A python library for self-supervised learning on images. (github.com)

Blog list

• Anomaly Detection in Time Series using ChatGPT
• Change Point Detection in Time Series using ChatGPT
• Huggingface 镜像站

Laboratory

• Deep Learning for Mobile Health Lab
• Mingsheng Long - Tsinghua University
• Yang Yang - Zhejiang University
• https://yuxuanliang.com/publications/
• 北京交通大学教师名录 (bjtu.edu.cn)
• 国内外高校和企业里时空数据挖掘相关的团队有哪些？ - 知乎 (zhihu.com)
• Xiyuan Zhang | Home
• Mingyue Cheng's HomePage (mingyue-cheng.github.io)

Course

• Time Series Analysis from nan jing university
• Time-Series Analysis from intel company
• 北京大学数学科学学院李东风2024年春季学期课程主页 (pku.edu.cn)

Others