Repository navigation
What's Changed
basal benchmarknow skips whole requests whose full prompts exceed the selected model's context limit, allowing mini to run the remaining supported workload instead of stopping during planning.- Filtering uses actual prompt validation, so shorter requests in the same document-size class remain eligible. It applies to embedded workloads and
--requestsreplay files. Other planning errors still stop the benchmark; an empty workload after filtering produces an explicit error. - JSON reports include source and skipped request counts plus skipped IDs, classes and error details. Console summaries show skipped counts. Timing and throughput statistics cover only retained requests.
- Documented filtering and the need to use a common supported request set when comparing models.
Validation
Local formatting, workspace Clippy, Rustdoc and dependency checks passed. GPU inference and a full benchmark run were not performed for this fix; static checks do not establish runtime performance or model parity.
Full Changelog: v0.1.8...v0.1.9