LSSP v1.0.0

LLM-Serving Scheduler Portability Benchmark (LSSP) — v1.0.0

Research artifact for How Portable Are LLM-Serving Scheduler Rankings
Across Workloads, Operating Regions, and Metrics?

LSSP studies whether comparative scheduler rankings remain stable across
workload sources, operating regions, evaluation metrics, and SLO definitions.

Scientific scope

The primary simulator campaign evaluates 13 scheduling policies
(11 PRIMARY and 2 STYLE-APPROXIMATION) over 120 workload windows from
three independently released workload sources and six calibrated operating
regions. This yields 9,360 unique simulator configurations, each executed
in two deterministic verification passes (18,720 executions total).
The workload window, not the verification repetition, is the inferential unit.

The primary cross-source analysis found Kendall rank correlations of
approximately 0.55–1.00. Of 990 primary pairwise portability comparisons,
36 (3.6%) were supported practical reversals; all 36 involved the same policy
pair and BurstGPT on one side.

Cross-metric and SLO-definition analyses are post-campaign sensitivity
extensions rather than primary preregistered evidence.

The synthetic-to-real study (RQ3) is an inconclusive pilot and does not support
a transfer conclusion.

Selected-case real-system validation

RQ6 evaluates slai_faithful and vllm_faithful on vLLM 0.27.1 using
40 independent windows from each workload source (240 physical executions
in total).

The simulator-predicted Azure/BurstGPT ranking reversal did not reproduce on
the physical system: both real-system conditions favored vllm_faithful.
The stable-control comparison retained qualitative cross-source consistency.
This is selected-case evidence about simulator-to-hardware fidelity, not an
estimate of hardware reversal prevalence.

See:
docs/RQ6_PUBLIC_RESULT_PROVENANCE.md

Public artifact structure

GitHub
contains the simulator, scheduler implementations, manifests, reproduction
infrastructure, manuscript source, and documentation.

Hugging Face
contains the public v1.0.0 analysis-ready/reduced derived artifacts:

https://huggingface.co/datasets/SoroushVahidi/llm-serving-scheduler-portability

The Hugging Face release does not contain the complete exploded
18,720-execution simulator matrix or all raw 240 RQ6 execution records.

Raw third-party workload traces are not redistributed by this project and must
be obtained from their original sources subject to their respective terms.

Archival record

Zenodo DOI:

https://doi.org/10.5281/zenodo.22306798

The v1.0.0 tag and Zenodo record preserve the archived v1.0.0 snapshot.
The repository's current main branch may contain later manuscript and
documentation corrections; the canonical scientific results represented by
v1.0.0 remain unchanged.

Reproduction

Start with:

git clone https://github.com/SoroushVahidi/llm-serving-scheduler-robustness-benchmark
cd llm-serving-scheduler-robustness-benchmark
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
pytest
./scripts/artifact/run_toy_reproduction.sh

See docs/ARTIFACT_EVALUATION_GUIDE.md
for the full reviewer-facing path, including how to independently regenerate
the paper's tables/figures from the released dataset.

Citation

See CITATION.cff.