[ICML 2026] QuITE: Query-Based Irregular Time Series Embedding

Official PyTorch implementation of QuITE: Query-Based Irregular Time Series Embedding (ICML 2026).

A plug-and-play input-embedding module that lets any standard MTS backbone — PatchTST, PatchMixer, TMix, iTransformer, S-Mamba, TimeXer — handle Irregular Multivariate Time Series (IMTS) without architectural changes or artificial value generation.

📄 Paper (OpenReview) · 💻 Code · 📜 Citation

---

🔥 News

• 2026-05-01 — QuITE is accepted by ICML 2026 🎉

---

🧐 Overview

Irregular Multivariate Time Series (IMTS) are common in healthcare, industrial monitoring, and climatology, yet they break the uniform-sampling assumption baked into standard MTS embeddings. We address this at the input-embedding stage.

• QuITE (paper §4) — a plug-and-play module. A small set of learnable query tokens aggregates irregular observations through a single masked self-attention layer.
• QuITE++ (paper §5) — a hierarchical extension: query-based patch embedding → patch-level self-attention → variable-level self-attention → cross-attention decoder over future-time queries.

Across 7 benchmarks × 6 MTS backbones, plugging QuITE in yields average relative gains of up to 54.7 % in forecasting and up to 15.8 % in classification; QuITE++ achieves the best performance on 20 / 24 forecasting settings (paper Tables 2-4).

Figure 1 (a). Effectiveness of QuITE on forecasting — averaged over all datasets.

Figure 1 (b). Effectiveness of QuITE on classification — averaged over all datasets.

---

🧠 Model

Figure 2. Overall framework of QuITE. Learnable query tokens aggregate irregular observations through a single self-attention layer.

Figure 3. Overall architecture of QuITE++. A hierarchical encoder models intra-variable patch-level (B) and inter-variable (C) interactions via learnable query tokens.

---

📊 Datasets

We use 4 forecasting + 3 classification benchmarks, following t-PatchGNN for forecasting and Raindrop for classification preprocessing. Place all data under ../data/ (sibling of this repository).

Forecasting	# Samples	# Vars	Missing		Classification	# Samples	# Vars	# Classes	Missing
Human Activity	5,400	12	75.0 %		P19	38,803	34	2	94.9 %
USHCN	26,736	5	77.9 %		P12	11,988	36	2	88.4 %
PhysioNet	12,000	36	88.4 %		PAM	5,333	17	8	60.0 %
MIMIC-III	23,457	96	96.7 %

• PhysioNet / Human Activity — auto-downloaded by the code.
• USHCN — use the preprocessed small_chunked_sporadic.csv from GRU-ODE-Bayes.
• MIMIC-III — request raw data via PhysioNet (credentialed), then run the Neural Flows preprocessing.
• P12 / P19 / PAM — use the Raindrop-processed splits (P19, P12).

---

🛠 Installation

Tested on Python 3.10+ and PyTorch 2.0+.

git clone https://github.com/Meaningfull9502/QuITE.git
cd QuITE
pip install -r requirements.txt
pip install mamba-ssm  # only required for the S-Mamba backbone

---

⚡ Quick Start

# QuITE + iTransformer on PhysioNet (forecasting, 24 → 24)
python train_forecasting.py --dataset physionet --history 24 \
    --patch_size 6 --stride 6 --hid_dim 64 --nhead 4 --nlayer 3 \
    --batch_size 64 --lr 1e-3 --seed 1 --gpu 0 \
    --irr_emb --model itransformer --mode quite

# QuITE++ on the same setting
python train_quite_plus.py --dataset physionet --history 24 \
    --patch_size 6 --stride 6 --hid_dim 64 --nhead 4 --nlayer 2 \
    --batch_size 64 --lr 1e-3 --seed 1 --gpu 0

# QuITE + PatchTST on P19 (classification)
python train_classification.py --dataset P19 \
    --patch_size 3.75 --stride 3.75 --hid_dim 64 --nhead 2 --nlayer 3 \
    --batch_size 64 --lr 1e-3 --epoch 1000 --gpu 0 \
    --irr_emb --model patchtst --mode quite

Per-dataset hyperparameters follow paper Table C.1 and are encoded in jobs/*.sh. Run python <script>.py --help for the full argument list.

---

🔁 Reproducing Paper Results

bash jobs/run_forecasting.sh     # Table 2  — 6 backbones × 12 (dataset, horizon) × 5 seeds
bash jobs/run_quite_plus.sh      # Table 4  — QuITE++ on 12 settings × 5 seeds
bash jobs/run_classification.sh  # Table 3  — 6 backbones × 3 datasets

All runs use Adam, lr=1e-3, patience=50, seeds {1..5}, MSE loss (forecasting), CE loss (classification) — paper §6.1.

---

🧱 Supported Backbones

QuITE-equipped models are standardized to --hid_dim 64 and --nhead 4 (forecasting) / --nhead 2 (classification). The per-backbone --nlayer follows paper Appendix B.1 and is set automatically by the batch scripts.

Family	--model	Token	--nlayer
Patch	patchtst	per-patch (Transformer)	3
Patch	patchmixer	per-patch (CNN, single-layer)	1
Patch	tmix	per-patch (MLP, TSMixer-style)	2
Variate	itransformer	per-variable (inverted Transformer)	3
Variate	s_mamba	per-variable (bidirectional Mamba)	2
Hybrid	timexer	per-patch + per-variable exogenous	3

QuITE++. Tuned per dataset via grid search over --hid_dim ∈ {32, 64}, --nlayer ∈ {1, 2, 3}, --nhead ∈ {1, 2, 4, 8} (paper §6.1).

---

🎛 Embedding Modes (--mode)

--mode	Meaning	Paper	Pair with --irr_emb?
quite	QuITE — query-based irregular embedding (main method)	Eq. 5-13	✅
mean	Mean Pooling baseline	Table 5	✅
mtand	mTAND attention baseline	Table 5	✅
add	value + time embedding	Table 5	❌
concat	value ‖ time embedding	Table 5	❌
False	vanilla backbone embedding (no time conditioning)	—	❌

---

📜 Citation

@inproceedings{lim2026quite,
  title     = {Qu{ITE}: Query-based Irregular Time-series Embedding},
  author    = {Lim, JungHoon},
  booktitle = {Forty-third International Conference on Machine Learning},
  year      = {2026},
  url       = {https://openreview.net/forum?id=ILQGHFvEoo}
}

---

🙏 Acknowledgements

Built on top of t-PatchGNN, Raindrop, Time-Series-Library, S-D-Mamba, Hi-Patch, and PyOmniTS.