LBANN has numerous callbacks that can be used to collect data about an experiment, such as scalars, metrics, weights, memory usage, images, etc. The documentation of many of these is pending; see the list of Available
Callbacks<available-callbacks>
for the documented ones.
The callbacks are set to execute at various times, and can be used to display images according to either a boolean output or their global sample index.
For a complete listing of callbacks and details about their functionality, please see Available
Callbacks<available-callbacks>
.
Callbacks are used by adding them to the python front end with the appropriate arguments and passing them as a list into the model. For example, the callbacks timer, print_statistics, and save model could be included with the following:
timer = lbann.CallbackTimer()
print_stats = lbann.CallbackPrintStatistics(
batch_interval=5)
save_model = lbann.CallbackSaveModel(
dir=".",
disable_save_after_training=True)
callbacks = [timer,
print_stats,
save_model]
model = lbann.Model(num_epochs,
layers,
objective_function,
metrics,
callbacks)
callback {
timer {
}
print_statistics {
batch_interval: 5
}
save_model {
dir: "."
disable_save_after_training: true
}
}
Export Onnx <callbacks/export_onnx> Summarize images <callbacks/summarize_images> Print progress bar <callback/progress_bar>