Repository navigation
v0.13.0
A minor bump: meta.json identity moves to schema 4, where inference
digests what the model is asked, including its context limit, and no longer
how it is served; inference_identity takes context_limit instead of
recipe. RESULT_SCHEMA_VERSION stays 1, so 0.12.x result directories
resume, but a run recorded at a context window that sieval run now raises is
refused on resume. Refusals get their own exit codes and a machine-readable
stderr line, and a task can declare the Python packages it needs.
Migrating from 0.12.x
meta.jsonidentityis schema 4 (#233). A reader ofmeta.jsonmust accept schemas 1 to 4.inferencedrops the engine and serving recipe and adds the context limit; eachruns[]entry recordsservinginstead, never compared. A schema 1 to 3 directory resumes once by the digest it was written with, so its engine and recipe must still match, then is rewritten as schema 4.inference_identitytakescontext_limitinstead ofrecipe;TaskRunner,MultiTaskRunner.add_taskandtask_resume_plantakecontext_limit(#233).- New root-model keys:
context_limit(an endpoint's window),weights(compared on resume instead of the name when both runs state it),serving(an endpoint's statedengine_version/topology/hardware; refused on a modelsieval runlaunches) (#233), andcontext(native/max/scaling) (#236). - Serving groups (#236): a task needing more than the recipe window raises it up to
native, or, up tomax, runs on its own scaled engine in serving group<model>@<window>. Groups run one after another;--dry-runlists them underserving, and--serving-group KEYruns one. - Breaking for resume: a run recorded at a window
sieval runnow raises is refused, since the window is part of the inference identity and a resume never reruns inference. Start it fresh, or pin the recorded window withinfer.overrides(context_length/max_model_len) (#236). - Refusals exit 3 (resume), 4 (hardware) or 5 (environment), not 1, and write one
SIEVAL_REFUSAL {json}line per refusal to stderr (#230, #235). A scheduler that treated every non-zero exit alike now sees these codes; the line'svis 2. extraat both placements is refused (models.<m>.extraandmodels.<m>.args.extra, or both in an inlinegrader/extractorconfig). Keep one (#228). A 0.12.0 result directory written from a top-levelextraran without it, so its resume is refused; start it fresh.
Added
- Refusals exit with their own code.
sieval run,evalandinfer startexit 3 on a refused resume and 4 on refused hardware (no GPU for an engine, or not the declared GPUs), and print oneSIEVAL_REFUSAL {json}line to stderr; both used to exit 1 like a crash.--dry-run, and a batch failing in more than one way, still exit 1. Annvidia-smitimeout now fails instead of reading as "no GPU".StageResumeErrorandResumeVersionErrorgaintask; GPU refusals raiseLaunchGpuError(aRuntimeError). See Exit codes (#230, closes #229). - A task declares the Python packages it needs, and a run without them is refused.
Task.python_requires = {import name: PEP 508 requirement}on the class, or apython_requires_for(context)classmethod when the need depends on the task'sargs.sieval runandsieval evalcheck it before any engine starts or dataset loads: a package is met when its import name is found (not imported) and, if pinned, the installed distribution satisfies the pin; extras are rejected. A missing package is anenvironmentrefusal: exit 5, oneSIEVAL_REFUSALline per task withkind: "environment",reason: "python_requires".--dry-runreports it as a newdependenciescheck,task list/showas apython-requiresentry undermissing, andTaskMeta.python_requiresis inmeta/index.json.sieval infer startdoes not check it. TheSIEVAL_REFUSALline'svmoves to 2; theresumeandhardwarelines are otherwise unchanged. It is outside every identity, so resumes are unaffected. No built-in task declares one yet (#235, closes #234). - How the model is served no longer refuses a resume.
meta.jsonidentitymoves to schema 4:inferencedigests what the model is asked, now including the context limit, and drops the engine and serving recipe. Eachruns[]entry recordsservinginstead, never compared, so a run on 1 GPU resumes on 8 and a launched engine's run resumes on an equivalent endpoint. New root-model keys:context_limit(an endpoint's window; a launched engine's comes from its plan),weights(compared on resume instead of the name when both runs state it) andserving(engine_version/topology/hardwareof an endpoint, recorded inruns[].serving.stated). Readers ofmeta.jsonmust accept schemas 1 to 4. A schema 1 to 3 directory resumes once by the digest it was written with, so its engine and recipe must still match, then is rewritten as schema 4.inference_identitytakescontext_limitinstead ofrecipe;TaskRunner,MultiTaskRunner.add_taskandtask_resume_plantakecontext_limit(#233, closes #232). sieval runsizes serving to what the tasks need. A task's need isTaskRequirements.min_context(RULER declares one) or its effectivemax_tokens, overridable with a task'smin_context; a root model's newcontext:block statesnative/max/scaling. The recipe window is raised up tonative; a need up tomaxgets its own scaled engine in serving group<model>@<window>, run after the default group; anything else, or a need beyond a window pinned byinfer.overridesorcontext_limit, is refused before launch.--dry-runlists groups underserving,--serving-group KEYruns one, andruns[].serving.grouprecords it.sieval infer start <yaml>serves the same raised window. Breaking for resume: a run recorded at a window now raised is refused (inference is never rerun); start fresh or pin it withinfer.overrides. An SGLanggenmodel with nodialect:now defaults tosglang_legacywhen a task scores input, instead of refusing (#236, closes #231).
Fixed
- A model's top-level
extrareaches the model.models.<m>.extrapassed--dry-runand was then dropped at binding, soModel.extrawas{}and the inference identity omitted it. It now binds likeargs.extraon every dialect and on derived models;args.extraidentities are unchanged. A 0.12.0 result directory written from a top-levelextraran without it, so its resume is refused as an inference change; start it fresh (#228, closes #227). - Breaking:
extraat both placements is refused before any engine starts, on named models and inlinegrader/extractorconfigs. Inline configs used to accept both and silently useargs.extra. Keep one (#228).