Panel Training

Panel Training

Train on a local dataset, evaluate measured results, export receipt-verified artifacts and attach a compatible low-rank adapter to an instance. Use Training at /rust/train for native training and Lexification at /rust/lexification for topology-bound search. All jobs and their recorded evidence remain visible after reloading; a successful form submission is not proof of completed work.

Fine-tune a LoRA, evaluate it and deploy

1. Open Training (/rust/train) and select Base GGUF from model library. Use Panel Models to import a valid GGUF first if none appears.
2. Open Browse or upload a dataset, place a file in the approved Models or Uploads directory, and supply Dataset file path (under Models or Uploads). Select Dataset format to match the file.
3. Set Training mode to LoRA, then configure LoRA rank, LoRA alpha, LoRA dropout (0..1) and LoRA targets (comma separated: q,k,v,o,gate,up,down). Full training does not use these adapter settings.
4. Set Optimizer steps, Batch size, Sequence length, Learning rate, Checkpoint every N steps, Evaluation fraction (0 to 0.5) and Seed. To enforce a quality requirement at deployment, enter Target evaluation loss (optional). A held-out fraction greater than zero is required for later evaluation.
5. Choose Precision (reported by lexi-train) and Executor (reported by lexi-train). Only worker-advertised options are offered; if the worker is unavailable, the page explains why no new run can start. Optionally select Resume verified managed checkpoint (only an identical job configuration is accepted).
6. Check the authorization and resource-cost confirmation, then select Create and start job. Record the run ID. Under Recent jobs inspect the state, progress steps, losses, errors, checkpoint artifacts and native receipt. Use Refresh training progress to update the displayed status. If needed, Create STOP file requests a cooperative stop, not an immediate process kill.
7. Once a run is actually finished and shows verified evidence, select Run native held-out evaluation. Measured source chooses either the base model or an existing bound checkpoint; the displayed held-out loss and hardware qualification come only from the native worker. A base-model evaluation must not be misread as an adapter evaluation.
8. Select Export a verified copy and choose Format only if the final artifact type exists. Export verifies the artifact against the durable receipt and records an output digest; the original stays in place.
9. Select Deploy to Lexi Engine instance and, if available, set Adapter scale (positive, max 100). Check the restart confirmation, then select Deploy GGUF LoRA. This is allowed only for a matching, finished LoRA run with a verifiable GGUF artifact and any configured quality requirement met. Deployment restarts the instance and may interrupt requests. Attached Lexi Engine adapters shows the recorded adapter and live status.
10. Use Detach and restart when finished. Confirm the action; this removes the selected adapter and restarts the instance. Do not confuse an attached-adapter record with uninterrupted inference.

The matching guided task is Fine-tune a LoRA, evaluate it and deploy at /rust/training/guides/lora-deploy (also /rust/guides/training.lora-deploy). The guide requires an explicit verified managed checkpoint for its trained-adapter evaluation, uses registered routes, reports each JSON result, and waits for confirmed completion rather than assuming it. It provides a real stop reversal for a newly queued job and a detach reversal for a deployed adapter. If a browser session closes, open Training to inspect durable runs; the guide's input handoff is not a separate persistence system.

Lexify a model

1. Open Lexification (/rust/lexification) and inspect the configured instance models and any eligibility reasons. Choose an eligible Instance; the saved instance configuration supplies the model, not a text field from the browser.
2. Set Budget (minutes, 1 to 360). Optionally check Compare with installed engines using the LX15-08 reference suite, which asks for reference evidence only when supported and measured.
3. Confirm that the search can restart inference, then select Start Lexification. This starts a local, topology-bound candidate search; it does not deploy a candidate automatically.
4. Use the refresh link to view Live job state, current step, observations, recent jobs and measured candidate screening. Stop Lexification requests a cooperative stop and restoration after the bounded operation. A queued or running job is not a completed measurement.
5. Inspect Lexification receipt for the exact candidate, device topology, hashes, quality and comparison metrics. The UI labels unmeasured values rather than generating estimates. Export verified receipt JSON downloads the recorded evidence.
6. When the native receipt is complete and its current hash binding verifies, select Apply this Lexified candidate and choose Target (save instance settings or create a known-good profile). Confirm the action. The privileged handler checks the hashes again. Applying saved settings does not automatically restart a running instance.
7. To restore pre-apply settings, select Roll back saved settings and confirm; restart separately if needed. Use Panel Instances to manage the instance.

The guided task Lexify a model on its instance is at /rust/training/guides/lexify-model (also /rust/guides/training.lexify-model). It checks the job's recorded state = completed and requires digest-verified receipts (has_receipts) before allowing you to treat an outcome as verified. It can request the registered cooperative stop when a newly started search is still active.

For a joined fleet, Per-node Lexification status on /rust/fleet shows the member's reported topology, latest receipt and active configuration. Run Lexification there queues a signed request for an online permitted member. It does not copy another member's settings or measurement evidence.

Native worker operations

The Train page uses the installed executable rather than a separate browser trainer. The installed CLI accepts the following forms:

Operation	Native invocation	Operator-facing result
Run	lexi-train run --job PATH	Managed job, step events, checkpoints and receipt
Candidate search	lexi-train lexify --job PATH	Independent candidate-search training job and recorded evidence
Evaluate	lexi-train evaluate --job PATH [--checkpoint DIR]	Held-out loss and source/executor/precision fields
Matrix	lexi-train matrix	Worker-reported operations and qualifications, shown on /rust/training/matrix
Export	`lexi-train export --run DIR --format gguf-lora	safetensors --out PATH`
Capabilities	lexi-train --capabilities-json	Exact executor, precision and adapter availability

Native training matrix executes the installed worker's matrix command under a bounded output and runtime limit. An unavailable worker or unrecognized report produces an explicit error, not a fabricated feature list. Matrix and capability reports are not hardware performance benchmarks.

Routes and safeguards

Method and route	Purpose
GET /api/train/runs	List managed training history
GET /api/train/run?id=ID	Read one run's native progress, losses and artifacts
POST /api/train/start	Create and start a validated worker job
POST /api/train/stop	Create the durable cooperative stop marker
POST /api/train/lexify	Start a separate native candidate search from a finished run
POST /api/train/evaluate	Perform held-out evaluation on a finished run
POST /api/train/export	Copy a supported, receipt-verified artifact
GET /api/training/matrix	Read the native worker's operation matrix
GET /api/training/attachments	List recorded adapter bindings and their live status
POST /api/training/deploy	Attach verified LoRA to a matching instance with a restart
POST /api/training/detach	Detach one adapter with a restart
GET /api/training/lexification/status?run_id=ID	Read local search state, observations and receipt IDs
POST /rust/lexification/run	Local measured search through the existing native handler
POST /rust/lexification/stop	Cooperative search stop through the existing handler
GET /api/lexification/export	Download a recorded Lexification receipt
POST /api/optimize/apply / POST /api/optimize/rollback	Existing guarded candidate apply and rollback operations
GET /api/fleet / POST /api/fleet/action	Fleet observation and signed remote action

Training API calls and guided steps require an authenticated administrator. Native forms also require an administrator for mutations. The native training pages, D3 controls, Lexification actions and fleet-triggered search share the same role policy. A scoped access token must authorize the exact requested route; having training permission for one endpoint does not authorize another step or its reversal. Viewers can inspect permitted progress but do not receive privileged job forms. All input paths and identities are rechecked by the existing owner, and export/deploy operations refuse missing or inconsistent receipt evidence. Role restrictions, request authentication and cross-site protection remain in the shared server pipeline. If the native worker is absent, do not interpret a saved model or old receipt as evidence that a new training job has started.

Training controls and help text

The following control names and tooltip sentences are the registered text; the same help markers appear beside native controls. The classic Fit links use selectors to receive the same guidance without editing the classic page.

Control	Help text	Page
Training	Inspect managed training jobs, receipts, progress and worker availability.	/rust/train
Installed native trainer capabilities	Shows exactly the executors, precisions and training modes reported by the installed worker.	/rust/train
Base GGUF from model library	Select a validated local GGUF as the unchanged starting point for training.	/rust/train
Dataset file path (under Models or Uploads)	Use a locally managed dataset file; training refuses paths outside the approved directories.	/rust/train
Dataset format	Choose text or pre-tokenized data to match the selected dataset file.	/rust/train
Training mode	Choose a low-rank adapter or full training only when the installed worker supports it.	/rust/train
LoRA rank, alpha, dropout and targets	Set adapter capacity, scaling, regularization and target layers; these apply only in LoRA mode.	/rust/train
Optimizer steps, batch, sequence length and learning rate	Set the number of updates, samples per update, context length and learning rate before starting.	/rust/train
Executor (reported by lexi-train)	Use an execution device reported by this worker; a missing device is never substituted silently.	/rust/train
Precision (reported by lexi-train)	Choose a numerical precision explicitly advertised by the installed worker.	/rust/train
Checkpoint every N steps	Save resumable checkpoints at this interval so interrupted jobs can be recovered.	/rust/train
Evaluation fraction and target loss	Reserve a held-out dataset fraction and optionally require a measured loss threshold before deployment.	/rust/train
Resume verified managed checkpoint	Resume only from a bound checkpoint with matching inputs; total steps may be extended.	/rust/train
Seed	Set the reproducible random seed recorded with the native job configuration.	/rust/train
Create and start job	Validate the model, dataset and worker options before starting a resource-consuming native run.	/rust/train
Refresh training progress	Reload durable step counts, losses, receipts, errors and verified artifact state.	/rust/train
Create STOP file	Request a cooperative stop; the worker checkpoints after the current step when it is alive.	/rust/train
Start new Lexification experiment	Clone a finished job into a separate native candidate search without changing the original run.	/rust/train
Run native held-out evaluation	Measure held-out loss against the chosen bound checkpoint or base model; no score is invented.	/rust/train
Export a verified copy	Copy a supported final artifact after the worker validates its receipt and recorded content digest.	/rust/train
Deploy GGUF LoRA	Attach a verified compatible adapter and restart the target instance; active requests may be interrupted.	/rust/train
Detach and restart	Remove a selected adapter and restart its instance; existing requests may be interrupted.	/rust/train
Native training matrix	Read the worker's reported operation matrix without inferring device or quality qualifications.	/rust/training/matrix
Lexification	Inspect topology-bound search jobs, progress and receipt evidence without applying unverified results.	/rust/lexification
Start Lexification	Search candidate settings on this instance; a restart may interrupt active requests.	/rust/lexification
Budget (minutes, 1 to 360)	Limit the duration of the local measured parameter search to the chosen number of minutes.	/rust/lexification
Compare with installed engines	Optionally record reference comparisons when an installed reference is available for measurement.	/rust/lexification
Stop Lexification	Stop after the current bounded request and restore the instance's original settings.	/rust/lexification
Export verified receipt JSON	Download a recorded topology-bound receipt and its exact measured evidence.	/rust/lexification
Apply this Lexified candidate	Recheck receipt hashes before saving qualified instance settings or a profile; restart separately.	/rust/lexification
Roll back saved settings	Restore saved settings from before candidate application; restart the instance separately.	/rust/lexification
Per-node Lexification status	Show each member's own topology, receipt and active configuration without copying settings between nodes.	/rust/fleet
Run Lexification there	Request a signed, permitted search on one member after verifying its reported topology identity.	/rust/fleet
Guided training tasks	Follow guarded steps for training, evaluation, deployment and local parameter search.	/rust/training/guides

For related workflows see Panel Fit Lab, Panel Models, Panel Instances and Panel Fleet.