SpectraBench Initial Release

SpectraBench: Adaptive Scheduling System for Efficient LLM Evaluation

SpectraBench is an open-source benchmarking framework designed to optimize the evaluation process of Large Language Models (LLMs) through intelligent scheduling and resource management. It supports a wide range of models and benchmark tasks and adapts to different workloads via a multi-stage scheduling system.

Key Features

• Adaptive execution scheduling with support for heuristic, hybrid, and machine learning-based strategies
• Modular architecture for evaluating diverse LLMs on multiple benchmarks
• Flexible YAML-based configuration for models and tasks
• GPU-aware memory management to reduce runtime errors
• Visualization tools for performance, scheduling behavior, and thermal patterns
• Reproducible experiment pipeline with academic publication support

Repository Contents

• code/ – Core scheduling and orchestration logic
• analysis/ – Performance analysis and figure generation modules
• configs/ – Example configuration files for models and tasks
• scripts/ – Shell scripts for experiment execution
• figures/ – Auto-generated publication-quality graphs

License

This software is released under the Apache License 2.0 and is intended for reproducible research, academic publications, and LLM system development.

For usage instructions, examples, and citation information, please visit the GitHub repository:
🔗 https://github.com/gwleee/SpectraBench