Classes |
Public Member Functions |
Protected Member Functions |
Private Member Functions |
Private Attributes |
List of all members
lbann::callback::minibatch_schedule Class Reference
#include <variable_minibatch.hpp>
Inheritance diagram for lbann::callback::minibatch_schedule:
Collaboration diagram for lbann::callback::minibatch_schedule:
Classes | |
| struct | minibatch_step |
| Represents a step in a schedule of mini-batch sizes. More... | |
Public Member Functions | |
| minibatch_schedule (size_t starting_mbsize, std::vector< minibatch_step > steps) | |
| minibatch_schedule (const minibatch_schedule &)=default | |
| minibatch_schedule & | operator= (const minibatch_schedule &)=delete |
| minibatch_schedule * | copy () const override |
| std::string | name () const override |
| Return this callback's name. More... | |
 Public Member Functions inherited from lbann::callback::variable_minibatch | |
| variable_minibatch (size_t starting_mbsize) | |
| variable_minibatch (const variable_minibatch &)=default | |
| variable_minibatch & | operator= (const variable_minibatch &)=default |
| void | on_train_begin (model *m) override |
| Set the initial mini-batch size. More... | |
| void | on_epoch_end (model *m) override |
| Potentially change the mini-batch size. More... | |
| Public Member Functions inherited from lbann::callback_base | |
| callback_base (int batch_interval=1) | |
| Initialize a callback with an optional batch interval. More... | |
| callback_base (const callback_base &)=default | |
| virtual | ~callback_base ()=default |
| virtual void | setup (trainer *t) |
| Called once to set up the callback on the trainer. More... | |
| virtual void | setup (model *m) |
| Called once to set up the callback on the model (after all layers are set up). More... | |
| virtual void | on_setup_end (model *m) |
| Called at the end of setup. More... | |
| virtual void | on_train_end (model *m) |
| Called at the end of training. More... | |
| virtual void | on_phase_end (model *m) |
| Called at the end of every phase (multiple epochs) in a layer-wise model training. More... | |
| virtual void | on_epoch_begin (model *m) |
| Called at the beginning of each epoch. More... | |
| virtual void | on_batch_begin (model *m) |
| Called at the beginning of a (mini-)batch. More... | |
| virtual void | on_batch_end (model *m) |
| Called immediately after the end of a (mini-)batch. More... | |
| virtual void | on_test_begin (model *m) |
| Called at the beginning of testing. More... | |
| virtual void | on_test_end (model *m) |
| Called immediately after the end of testing. More... | |
| virtual void | on_validation_begin (model *m) |
| Called at the beginning of validation. More... | |
| virtual void | on_validation_end (model *m) |
| Called immediately after the end of validation. More... | |
| virtual void | on_forward_prop_begin (model *m) |
| Called when a model begins forward propagation. More... | |
| virtual void | on_forward_prop_begin (model *m, Layer *l) |
| Called when a layer begins forward propagation. More... | |
| virtual void | on_forward_prop_end (model *m) |
| Called when a model ends forward propagation. More... | |
| virtual void | on_forward_prop_end (model *m, Layer *l) |
| Called when a layer ends forward propagation. More... | |
| virtual void | on_backward_prop_begin (model *m) |
| Called when a model begins backward propagation. More... | |
| virtual void | on_backward_prop_begin (model *m, Layer *l) |
| Called when a layer begins backward propagation. More... | |
| virtual void | on_backward_prop_end (model *m) |
| Called when a model ends backward propagation. More... | |
| virtual void | on_backward_prop_end (model *m, Layer *l) |
| Called when a layer ends backward propagation. More... | |
| virtual void | on_optimize_begin (model *m) |
| Called when a model begins optimization. More... | |
| virtual void | on_optimize_begin (model *m, weights *w) |
| Called when weights begins optimization. More... | |
| virtual void | on_optimize_end (model *m) |
| Called when a model ends optimization. More... | |
| virtual void | on_optimize_end (model *m, weights *w) |
| Called when weights ends optimization. More... | |
| virtual void | on_batch_evaluate_begin (model *m) |
| Called at the beginning of a (mini-)batch evaluation (validation / testing). More... | |
| virtual void | on_batch_evaluate_end (model *m) |
| Called at the end of a (mini-)batch evaluation (validation / testing). More... | |
| virtual void | on_evaluate_forward_prop_begin (model *m) |
| Called when a model begins forward propagation for evaluation (validation / testing). More... | |
| virtual void | on_evaluate_forward_prop_begin (model *m, Layer *l) |
| Called when a layer begins forward propagation for evaluation (validation / testing). More... | |
| virtual void | on_evaluate_forward_prop_end (model *m) |
| Called when a model ends forward propagation for evaluation (validation / testing). More... | |
| virtual void | on_evaluate_forward_prop_end (model *m, Layer *l) |
| Called when a layer ends forward propagation for evaluation (validation / testing). More... | |
| int | get_batch_interval () const |
| Return the batch interval. More... | |
| virtual description | get_description () const |
| Human-readable description. More... | |
| template<class Archive > | |
| void | serialize (Archive &ar) |
| Store state to archive for checkpoint and restart. More... | |
| void | write_proto (lbann_data::Callback &proto) const |
| Write a protobuf description of the callback. More... | |
Protected Member Functions | |
| bool | schedule (model *m, size_t &new_mbsize, float &new_lr, size_t &ramp_time) override |
| Protected Member Functions inherited from lbann::callback::variable_minibatch | |
| void | change_learning_rate (model *m, float new_lr) const |
| Change the learning rate of every layer in m to new_lr. More... | |
| float | get_current_learning_rate (model *m) const |
| Get the current learning rate (assumes every layer has the same one). More... | |
| Protected Member Functions inherited from lbann::callback_base | |
| std::string | get_multi_trainer_path (const model &m, const std::string &root_dir) |
| Build a standard directory hierarchy including trainer ID. More... | |
| std::string | get_multi_trainer_ec_model_path (const model &m, const std::string &root_dir) |
| Build a standard directory hierachy including trainer, execution context, and model information (in that order). More... | |
| std::string | get_multi_trainer_model_path (const model &m, const std::string &root_dir) |
| Build a standard directory hierachy including trainer, model information in that order. More... | |
| callback_base & | operator= (const callback_base &)=default |
| Copy-assignment operator. More... | |
Private Member Functions | |
| void | write_specific_proto (lbann_data::Callback &proto) const final |
Private Attributes | |
| std::vector< minibatch_step > | m_steps |
| Steps in the mini-batch schedule, stored in reverse sorted order. More... | |
Additional Inherited Members | |
| Protected Attributes inherited from lbann::callback::variable_minibatch | |
| size_t | m_starting_mbsize |
| Initial mini-batch size. More... | |
| size_t | m_current_mini_batch_size |
| size_t | m_ramp_count = 0 |
| Current number of epochs left to ramp the learning rate. More... | |
| float | m_lr_incr = 0.0f |
| Amount to increment the learning rate by when ramping. More... | |
| Protected Attributes inherited from lbann::callback_base | |
| int | m_batch_interval |
| Batch methods should once every this many steps. More... | |
Detailed Description
Definition at line 123 of file variable_minibatch.hpp.
Constructor & Destructor Documentation
◆ minibatch_schedule() [1/2]
| lbann::callback::minibatch_schedule::minibatch_schedule | ( | size_t | starting_mbsize, |
| std::vector< minibatch_step > | steps | ||
| ) |
◆ minibatch_schedule() [2/2]
|
default |
Member Function Documentation
◆ copy()
|
inlineoverridevirtual |
Implements lbann::callback_base.
Definition at line 145 of file variable_minibatch.hpp.
◆ name()
|
inlineoverridevirtual |
Return this callback's name.
Implements lbann::callback_base.
Definition at line 149 of file variable_minibatch.hpp.
Here is the call graph for this function:
◆ operator=()
|
delete |
◆ schedule()
|
overrideprotectedvirtual |
Implemented by child classes to provide the mini-batch/learning schedule. This is called at the end of every training epoch. If it returns false, no changes are made from the currently established schedule. If this returns true, the mini-batch size will be changed accordingly. If the mini-batch size is larger than the model's maximum mini-batch size, a warning is printed and the maximum mini-batch size is used. If new_lr also non-zero, the learning rate will be changed to new_lr, with a linear ramp time. (If ramp_time is 0, it is changed immediately.) Note changing the learning rate while in a ramp may lead to unexpected behavior; also be aware of interactions with other learning rate schedules.
Implements lbann::callback::variable_minibatch.
◆ write_specific_proto()
|
finalprivatevirtual |
Add callback specific data to prototext
Implements lbann::callback_base.
Member Data Documentation
◆ m_steps
|
private |
Steps in the mini-batch schedule, stored in reverse sorted order.
Definition at line 162 of file variable_minibatch.hpp.
The documentation for this class was generated from the following file:
