Public Member Functions |
Private Member Functions |
Private Attributes |
Friends |
List of all members
lbann::callback::mixup Class Reference
#include <mixup.hpp>
Inheritance diagram for lbann::callback::mixup:
Collaboration diagram for lbann::callback::mixup:
Public Member Functions | |
| mixup (std::unordered_set< std::string > layers, float alpha) | |
| mixup * | copy () const override |
| std::string | name () const override |
| Return this callback's name. More... | |
| void | on_forward_prop_end (model *m, Layer *l) override |
| Called when a layer ends forward propagation. More... | |
Serialization | |
| template<class Archive > | |
| void | serialize (Archive &ar) |
| Store state to archive for checkpoint and restart. More... | |
 Public Member Functions inherited from lbann::callback_base | |
| callback_base (int batch_interval=1) | |
| Initialize a callback with an optional batch interval. More... | |
| callback_base (const callback_base &)=default | |
| virtual | ~callback_base ()=default |
| virtual void | setup (trainer *t) |
| Called once to set up the callback on the trainer. More... | |
| virtual void | setup (model *m) |
| Called once to set up the callback on the model (after all layers are set up). More... | |
| virtual void | on_setup_end (model *m) |
| Called at the end of setup. More... | |
| virtual void | on_train_begin (model *m) |
| Called at the beginning of training. More... | |
| virtual void | on_train_end (model *m) |
| Called at the end of training. More... | |
| virtual void | on_phase_end (model *m) |
| Called at the end of every phase (multiple epochs) in a layer-wise model training. More... | |
| virtual void | on_epoch_begin (model *m) |
| Called at the beginning of each epoch. More... | |
| virtual void | on_epoch_end (model *m) |
| Called immediate after the end of each epoch. More... | |
| virtual void | on_batch_begin (model *m) |
| Called at the beginning of a (mini-)batch. More... | |
| virtual void | on_batch_end (model *m) |
| Called immediately after the end of a (mini-)batch. More... | |
| virtual void | on_test_begin (model *m) |
| Called at the beginning of testing. More... | |
| virtual void | on_test_end (model *m) |
| Called immediately after the end of testing. More... | |
| virtual void | on_validation_begin (model *m) |
| Called at the beginning of validation. More... | |
| virtual void | on_validation_end (model *m) |
| Called immediately after the end of validation. More... | |
| virtual void | on_forward_prop_begin (model *m) |
| Called when a model begins forward propagation. More... | |
| virtual void | on_forward_prop_begin (model *m, Layer *l) |
| Called when a layer begins forward propagation. More... | |
| virtual void | on_forward_prop_end (model *m) |
| Called when a model ends forward propagation. More... | |
| virtual void | on_backward_prop_begin (model *m) |
| Called when a model begins backward propagation. More... | |
| virtual void | on_backward_prop_begin (model *m, Layer *l) |
| Called when a layer begins backward propagation. More... | |
| virtual void | on_backward_prop_end (model *m) |
| Called when a model ends backward propagation. More... | |
| virtual void | on_backward_prop_end (model *m, Layer *l) |
| Called when a layer ends backward propagation. More... | |
| virtual void | on_optimize_begin (model *m) |
| Called when a model begins optimization. More... | |
| virtual void | on_optimize_begin (model *m, weights *w) |
| Called when weights begins optimization. More... | |
| virtual void | on_optimize_end (model *m) |
| Called when a model ends optimization. More... | |
| virtual void | on_optimize_end (model *m, weights *w) |
| Called when weights ends optimization. More... | |
| virtual void | on_batch_evaluate_begin (model *m) |
| Called at the beginning of a (mini-)batch evaluation (validation / testing). More... | |
| virtual void | on_batch_evaluate_end (model *m) |
| Called at the end of a (mini-)batch evaluation (validation / testing). More... | |
| virtual void | on_evaluate_forward_prop_begin (model *m) |
| Called when a model begins forward propagation for evaluation (validation / testing). More... | |
| virtual void | on_evaluate_forward_prop_begin (model *m, Layer *l) |
| Called when a layer begins forward propagation for evaluation (validation / testing). More... | |
| virtual void | on_evaluate_forward_prop_end (model *m) |
| Called when a model ends forward propagation for evaluation (validation / testing). More... | |
| virtual void | on_evaluate_forward_prop_end (model *m, Layer *l) |
| Called when a layer ends forward propagation for evaluation (validation / testing). More... | |
| int | get_batch_interval () const |
| Return the batch interval. More... | |
| virtual description | get_description () const |
| Human-readable description. More... | |
| template<class Archive > | |
| void | serialize (Archive &ar) |
| Store state to archive for checkpoint and restart. More... | |
| void | write_proto (lbann_data::Callback &proto) const |
| Write a protobuf description of the callback. More... | |
Private Member Functions | |
| void | write_specific_proto (lbann_data::Callback &proto) const final |
| mixup () | |
Private Attributes | |
| std::unordered_set< std::string > | m_layers |
| float | m_alpha |
Friends | |
| class | cereal::access |
Additional Inherited Members | |
| Protected Member Functions inherited from lbann::callback_base | |
| std::string | get_multi_trainer_path (const model &m, const std::string &root_dir) |
| Build a standard directory hierarchy including trainer ID. More... | |
| std::string | get_multi_trainer_ec_model_path (const model &m, const std::string &root_dir) |
| Build a standard directory hierachy including trainer, execution context, and model information (in that order). More... | |
| std::string | get_multi_trainer_model_path (const model &m, const std::string &root_dir) |
| Build a standard directory hierachy including trainer, model information in that order. More... | |
| callback_base & | operator= (const callback_base &)=default |
| Copy-assignment operator. More... | |
| Protected Attributes inherited from lbann::callback_base | |
| int | m_batch_interval |
| Batch methods should once every this many steps. More... | |
Detailed Description
Apply mixup to named input layers.
See:
Zhang, H. et al. "mixup: Beyond Empirical Risk Minimization." ICLR, 2018.
This implementation does mixup within a single batch, per the recommendation within the paper.
This approach may create duplicate images, and so uses
lambda = max(lambda, 1 - lambda)
for the mixing value.
This recommendation comes from https://docs.fast.ai/callbacks.mixup.html
The recommended default alpha (from the paper) is 0.4.
Constructor & Destructor Documentation
◆ mixup() [1/2]
| lbann::callback::mixup::mixup | ( | std::unordered_set< std::string > | layers, |
| float | alpha | ||
| ) |
Apply mixup to layers named in layers with mixup parameter alpha.
◆ mixup() [2/2]
|
private |
Here is the caller graph for this function:
Member Function Documentation
◆ copy()
|
inlineoverridevirtual |
Implements lbann::callback_base.
Definition at line 64 of file mixup.hpp.
Here is the call graph for this function:
◆ name()
|
inlineoverridevirtual |
Return this callback's name.
Implements lbann::callback_base.
Definition at line 65 of file mixup.hpp.
Here is the call graph for this function:
◆ on_forward_prop_end()
Called when a layer ends forward propagation.
Reimplemented from lbann::callback_base.
Here is the caller graph for this function:
◆ serialize()
template<class Archive >
| void lbann::callback::mixup::serialize | ( | Archive & | ar | ) |
Store state to archive for checkpoint and restart.
Here is the caller graph for this function:
◆ write_specific_proto()
|
finalprivatevirtual |
Add callback specific data to prototext
Implements lbann::callback_base.
Here is the caller graph for this function:
Friends And Related Function Documentation
◆ cereal::access
Member Data Documentation
◆ m_alpha
|
private |
◆ m_layers
|
private |
The documentation for this class was generated from the following file:
