Decoupled Local/Global Time-Series Representations on FreshRetailNet-50K

This repository adapts the algorithmic structure from Decoupling Local and Global Representations of Time Series to FreshRetailNet-50K.

The model keeps the paper's core data structure:

• X in R^{d x T}: one store-product time series.
• non-overlapping windows X_t in R^{d x delta}.
• local latent sequence Z_l = [z_0, ..., z_t], one vector per window.
• global latent vector z_g, one vector per store-product series.
• mask channel for censored or unavailable observations.
• GP prior over local latents and standard normal prior over global latents.
• counterfactual regularization by decoding (Z_l, z_g*), re-encoding the generated sample globally, and penalizing likelihood preference for the original z_g.

FreshRetailNet mapping:

• sample identity: store_id, product_id
• hourly sales: hours_sale
• stockout/censoring mask: hours_stock_status == 1
• covariates: discount, holiday/activity flags, precipitation, temperature, humidity, wind, hour-of-day and day-of-week sin/cos features
• subgroup evaluation target: city_id by default
• forecasting target: held-out final forecast_windows days

Setup

uv sync

Prepare Data

Download the Hugging Face dataset to local parquet files:

uv run python scripts/prepare_freshretailnet.py --out data/freshretailnet

You can also place train.parquet and eval.parquet under data/freshretailnet.

For a quick smoke experiment, set max_train_series and max_eval_series in configs/freshretailnet.json to small values such as 256 and 64.

Preprocessed tensors are cached under data/freshretailnet/cache when use_cache is enabled, so the slow row-to-series conversion is skipped on later runs with the same config.

Train

uv run decoupled-ts train --config configs/freshretailnet.json

The best checkpoint is written to runs/freshretailnet_glr/best.pt.

The training config uses device: "auto" to prefer CUDA, then Apple MPS, then CPU. CUDA runs use AMP and TF32 where available. Data loading uses worker processes and prefetching; tune batch_size and num_workers in configs/freshretailnet.json for your machine.

Evaluate Paper-Style Experiments

uv run decoupled-ts evaluate --checkpoint runs/freshretailnet_glr/best.pt

This runs FreshRetailNet-adapted versions of the paper's three representation evaluations:

• downstream prediction from learned local/global representations
• subgroup identification from global representations
• multi-window forecasting using GP conditional local latent prediction

Metrics are written to runs/freshretailnet_glr/metrics.json.

Retail Multi-Grain Experiments

The retail-specific extension keeps the original static/dynamic idea, but splits the local representation into day-level and hour-level factors:

• z_global: store-product or store-category baseline demand.
• z_day: date-level movement such as weekday, holiday, weather, promotion, or event effects.
• z_hour: repeated intra-day demand shape such as morning, lunch, evening, or late-night peaks.
• z_interaction: optional day x hour interaction such as promotion-day lunch peaks.

Run a quick smoke test:

uv run decoupled-ts retail-experiment --config configs/retail_multigrain_smoke.json

Run the full synthetic ablation suite:

uv run decoupled-ts retail-experiment --config configs/retail_multigrain.json

Run on FreshRetailNet after preparing local parquet files:

uv run decoupled-ts retail-experiment --config configs/retail_multigrain_freshretailnet.json

To isolate the incremental claim from the original paper-style decomposition, compare the two-component global + local model against the four-factor global + day + hour + interaction split without using the residual-target setup. These Exp-26 configs also enable a paper-style counterfactual global regularizer: decode a history after replacing z_global, re-encode the generated history globally, and penalize cases where the generated history is closer to the original global latent than to the counterfactual one.

uv run decoupled-ts retail-experiment --config configs/2-Exp-26_global_local_to_four_factor_smoke.json
uv run decoupled-ts retail-experiment --config configs/2-Exp-26_global_local_to_four_factor_synthetic.json
uv run decoupled-ts retail-experiment --config configs/2-Exp-26_global_local_to_four_factor_freshretailnet.json

The key variants are:

• paper_global_local: the original-style two-component split, with one global latent and one local latent per day-hour cell.
• four_factor_global_day_hour_interaction: the proposed four-factor split, with separate global, day, hour, and day-hour interaction latents.

The runner trains these variants from the config:

• baseline_flatten_mlp
• global_only
• global_day
• global_hour
• global_day_hour
• global_day_hour_interaction

Outputs are collected under train.output_dir:

• root run.log: overall experiment log
• root summary.json and summary.csv: final WAPE/MAE/RMSE/Bias comparison
• per-variant run.log: training log
• per-variant history.jsonl: epoch losses
• per-variant metrics.json: test metrics
• per-variant z_global.npy, z_day.npy, z_hour.npy: latent arrays for probes and visualization

Residual Diagnostics

To validate the residual-centered hypothesis before training representation models, run:

uv run decoupled-ts residual-diagnostics --config configs/2-Exp-1_residual_diagnostics_smoke.json
uv run decoupled-ts residual-diagnostics --config configs/2-Exp-1_residual_diagnostics_synthetic.json
uv run decoupled-ts residual-diagnostics --config configs/2-Exp-1_residual_diagnostics_freshretailnet.json

This compares baseline components such as recent same-hour means, then analyzes structure in r = y - b by hour, weekday, and subgroup. Outputs are written under analysis.output_dir, including baseline_metrics.json, summary.json, and residual CSV heatmaps.

To train residual representation models and evaluate b + r_hat correction:

uv run decoupled-ts residual-experiment --config configs/2-Exp-2_to_6_residual_smoke.json
uv run decoupled-ts residual-experiment --config configs/2-Exp-2_to_6_residual_synthetic.json
uv run decoupled-ts residual-experiment --config configs/2-Exp-2_to_6_residual_freshretailnet.json

To compare residual models with and without counterfactual-style latent swap regularization:

uv run decoupled-ts residual-experiment --config configs/2-Exp-7_swap_regularization_smoke.json
uv run decoupled-ts residual-experiment --config configs/2-Exp-7_swap_regularization_synthetic.json
uv run decoupled-ts residual-experiment --config configs/2-Exp-7_swap_regularization_freshretailnet.json

To validate the hypothesis under a synthetic dataset with explicit residual global/day/hour/interaction structure:

uv run decoupled-ts residual-diagnostics --config configs/2-Exp-8_structured_residual_diagnostics_smoke.json
uv run decoupled-ts residual-experiment --config configs/2-Exp-8_structured_residual_smoke.json
uv run decoupled-ts residual-diagnostics --config configs/2-Exp-8_structured_residual_diagnostics_synthetic.json
uv run decoupled-ts residual-experiment --config configs/2-Exp-8_structured_residual_synthetic.json

To compare the two main interaction variants across multiple seeds and a FreshRetailNet residual-structured subset:

uv run decoupled-ts residual-sweep --config configs/2-Exp-9_multiseed_structured_residual_smoke.json
uv run decoupled-ts residual-sweep --config configs/2-Exp-9_multiseed_structured_residual_synthetic.json
uv run decoupled-ts residual-sweep --config configs/2-Exp-9_freshretailnet_subset_interaction.json

To test residual components with explicit global/day/hour/interaction heads:

uv run decoupled-ts residual-experiment --config configs/2-Exp-11_output_decomposition_smoke.json
uv run decoupled-ts residual-sweep --config configs/2-Exp-11_output_decomposition_synthetic.json
uv run decoupled-ts residual-sweep --config configs/2-Exp-11_output_decomposition_freshretailnet.json

To run follow-up sweeps for subset selection, bias control, synthetic difficulty, and final paper-style tables:

uv run decoupled-ts residual-sweep --config configs/2-Exp-12_freshretailnet_subset_conditions.json
uv run decoupled-ts residual-sweep --config configs/2-Exp-13_freshretailnet_bias_control.json
uv run decoupled-ts residual-sweep --config configs/2-Exp-14_synthetic_difficulty_sweep.json
uv run decoupled-ts residual-sweep --config configs/2-Exp-15_final_synthetic.json
uv run decoupled-ts residual-sweep --config configs/2-Exp-15_final_freshretailnet.json

The sweep runner writes per-seed runs plus all_results.csv, aggregate.csv, and summary.json under sweep.output_dir.

To evaluate factor-structured subsets with positive probes, leakage probes, and targeted latent ablations:

uv run decoupled-ts residual-sweep --config configs/2-Exp-10_factor_subsets_smoke.json
uv run decoupled-ts residual-sweep --config configs/2-Exp-10_factor_subsets_freshretailnet.json