Callbacks
LBANN has numerous callbacks that can be used to collect data about an experiment, such as scalars, metrics, weights, memory usage, images, etc. The documentation of many of these is pending; see the list of Available Callbacks for the documented ones.
The callbacks are set to execute at various times, and can be used to display images according to either a boolean output or their global sample index.
For a complete listing of callbacks and details about their functionality, please see Available Callbacks.
Using Callbacks
Callbacks are used by adding them to the python front end with the appropriate arguments and passing them as a list into the model. For example, the callbacks timer, print_statistics, and save model could be included with the following:
Python Front End
timer = lbann.CallbackTimer()
print_stats = lbann.CallbackPrintStatistics(
batch_interval=5)
save_model = lbann.CallbackSaveModel(
dir=".",
disable_save_after_training=True)
callbacks = [timer,
print_stats,
save_model]
model = lbann.Model(num_epochs,
layers,
objective_function,
metrics,
callbacks)
Profobuf (Advanced)
callback {
timer {
}
print_statistics {
batch_interval: 5
}
save_model {
dir: "."
disable_save_after_training: true
}
}