Repository navigation
Train/Test/Validate Split in TUI? #6264
|
New discussion as to avoid necroposting in #2555. Is there a way to set up the three way data split so Or is it kind of just always a "roll your own chart" situation? I figure there might be a way to do it integrated with burn to avoid any variance in how stats are calculated (and to get pleasant visuals, perhaps). |
Replies: 1 comment 1 reply
|
Yes, burn supports this. Train and validation come from The flow is:
Sketch, based on the mnist example: let result = training.launch(Learner::new(model, optim, lr_scheduler));
let mut renderer = EvaluatorBuilder::new(ARTIFACT_DIR)
.renderer(result.renderer)
.metrics((AccuracyMetric::new(), LossMetric::new()))
.summary()
.build(result.model.clone())
.eval("test", dataloader_test)
.renderer;
renderer.manual_close();The example does this at burn/examples/mnist/src/training.rs Lines 131 to 138 in 5856e07 eval_all with several named test variants). eval and eval_all are defined at burn/crates/burn-train/src/evaluator/base.rs Lines 38 to 60 in 5856e07 The test set is only evaluated once, after training. It does not appear as a curve per epoch, since the evaluator runs on the final model. Burn does not do the three-way split of your data for you. You split the dataset yourself (for example with a (I looked this up in the code with help from an AI assistant; the links are to the lines I checked.) |
Yes, burn supports this. Train and validation come from
SupervisedTraining, and the test split goes throughEvaluatorBuilder. The metrics are computed by the same metric types, so you get the same accuracy and loss calculation on all three splits.The flow is:
SupervisedTraining::new(...)and register metrics with.metrics((AccuracyMetric::new(), LossMetric::new())). Those are reported per epoch as Train and Valid in the TUI.launchreturns a result that includes the trainedmodeland therenderer.EvaluatorBuilderwith the same metrics, give it that renderer so the test results show up in the same TUI, and calleval(one test set) oreval…