#include <variable_minibatch.hpp>

Inheritance diagram for lbann::callback::variable_minibatch:

Collaboration diagram for lbann::callback::variable_minibatch:

Public Member Functions
	variable_minibatch (size_t starting_mbsize)

	variable_minibatch (const variable_minibatch &)=default

variable_minibatch &	operator= (const variable_minibatch &)=default

void	on_train_begin (model *m) override
	Set the initial mini-batch size. More...

void	on_epoch_end (model *m) override
	Potentially change the mini-batch size. More...

Public Member Functions inherited from lbann::callback_base
	callback_base (int batch_interval=1)
	Initialize a callback with an optional batch interval. More...

	callback_base (const callback_base &)=default

virtual	~callback_base ()=default

virtual callback_base *	copy () const =0

virtual void	setup (trainer *t)
	Called once to set up the callback on the trainer. More...

virtual void	setup (model *m)
	Called once to set up the callback on the model (after all layers are set up). More...

virtual void	on_setup_end (model *m)
	Called at the end of setup. More...

virtual void	on_train_end (model *m)
	Called at the end of training. More...

virtual void	on_phase_end (model *m)
	Called at the end of every phase (multiple epochs) in a layer-wise model training. More...

virtual void	on_epoch_begin (model *m)
	Called at the beginning of each epoch. More...

virtual void	on_batch_begin (model *m)
	Called at the beginning of a (mini-)batch. More...

virtual void	on_batch_end (model *m)
	Called immediately after the end of a (mini-)batch. More...

virtual void	on_test_begin (model *m)
	Called at the beginning of testing. More...

virtual void	on_test_end (model *m)
	Called immediately after the end of testing. More...

virtual void	on_validation_begin (model *m)
	Called at the beginning of validation. More...

virtual void	on_validation_end (model *m)
	Called immediately after the end of validation. More...

virtual void	on_forward_prop_begin (model *m)
	Called when a model begins forward propagation. More...

virtual void	on_forward_prop_begin (model m, Layer l)
	Called when a layer begins forward propagation. More...

virtual void	on_forward_prop_end (model *m)
	Called when a model ends forward propagation. More...

virtual void	on_forward_prop_end (model m, Layer l)
	Called when a layer ends forward propagation. More...

virtual void	on_backward_prop_begin (model *m)
	Called when a model begins backward propagation. More...

virtual void	on_backward_prop_begin (model m, Layer l)
	Called when a layer begins backward propagation. More...

virtual void	on_backward_prop_end (model *m)
	Called when a model ends backward propagation. More...

virtual void	on_backward_prop_end (model m, Layer l)
	Called when a layer ends backward propagation. More...

virtual void	on_optimize_begin (model *m)
	Called when a model begins optimization. More...

virtual void	on_optimize_begin (model m, weights w)
	Called when weights begins optimization. More...

virtual void	on_optimize_end (model *m)
	Called when a model ends optimization. More...

virtual void	on_optimize_end (model m, weights w)
	Called when weights ends optimization. More...

virtual void	on_batch_evaluate_begin (model *m)
	Called at the beginning of a (mini-)batch evaluation (validation / testing). More...

virtual void	on_batch_evaluate_end (model *m)
	Called at the end of a (mini-)batch evaluation (validation / testing). More...

virtual void	on_evaluate_forward_prop_begin (model *m)
	Called when a model begins forward propagation for evaluation (validation / testing). More...

virtual void	on_evaluate_forward_prop_begin (model m, Layer l)
	Called when a layer begins forward propagation for evaluation (validation / testing). More...

virtual void	on_evaluate_forward_prop_end (model *m)
	Called when a model ends forward propagation for evaluation (validation / testing). More...

virtual void	on_evaluate_forward_prop_end (model m, Layer l)
	Called when a layer ends forward propagation for evaluation (validation / testing). More...

int	get_batch_interval () const
	Return the batch interval. More...

virtual std::string	name () const =0
	Return this callback's name. More...

virtual description	get_description () const
	Human-readable description. More...

template<class Archive >
void	serialize (Archive &ar)
	Store state to archive for checkpoint and restart. More...

void	write_proto (lbann_data::Callback &proto) const
	Write a protobuf description of the callback. More...

Protected Member Functions
virtual bool	schedule (model *m, size_t &new_mbsize, float &new_lr, size_t &ramp_time)=0

void	change_learning_rate (model *m, float new_lr) const
	Change the learning rate of every layer in m to new_lr. More...

float	get_current_learning_rate (model *m) const
	Get the current learning rate (assumes every layer has the same one). More...

Protected Member Functions inherited from lbann::callback_base
std::string	get_multi_trainer_path (const model &m, const std::string &root_dir)
	Build a standard directory hierarchy including trainer ID. More...

std::string	get_multi_trainer_ec_model_path (const model &m, const std::string &root_dir)
	Build a standard directory hierachy including trainer, execution context, and model information (in that order). More...

std::string	get_multi_trainer_model_path (const model &m, const std::string &root_dir)
	Build a standard directory hierachy including trainer, model information in that order. More...

callback_base &	operator= (const callback_base &)=default
	Copy-assignment operator. More...

virtual void	write_specific_proto (lbann_data::Callback &proto) const =0
	Add callback specific data to prototext. More...

Protected Attributes
size_t	m_starting_mbsize
	Initial mini-batch size. More...

size_t	m_current_mini_batch_size

size_t	m_ramp_count = 0
	Current number of epochs left to ramp the learning rate. More...

float	m_lr_incr = 0.0f
	Amount to increment the learning rate by when ramping. More...

Protected Attributes inherited from lbann::callback_base
int	m_batch_interval
	Batch methods should once every this many steps. More...

Detailed Description

Support changing the mini-batch size on different schedules. Implementations should override implement the abstract methods to define concrete schedules.

Definition at line 42 of file variable_minibatch.hpp.

Constructor & Destructor Documentation

◆ variable_minibatch() [1/2]

lbann::callback::variable_minibatch::variable_minibatch ( size_t starting_mbsize )

◆ variable_minibatch() [2/2]

lbann::callback::variable_minibatch::variable_minibatch ( const variable_minibatch & )

default

Member Function Documentation

◆ change_learning_rate()

void lbann::callback::variable_minibatch::change_learning_rate	(	model *	m,
		float	new_lr
	)		const

protected

Change the learning rate of every layer in m to new_lr.

◆ get_current_learning_rate()

float lbann::callback::variable_minibatch::get_current_learning_rate ( model * m ) const

protected

Get the current learning rate (assumes every layer has the same one).

◆ on_epoch_end()

void lbann::callback::variable_minibatch::on_epoch_end ( model * m )

overridevirtual

Potentially change the mini-batch size.

Reimplemented from lbann::callback_base.

◆ on_train_begin()

void lbann::callback::variable_minibatch::on_train_begin ( model * m )

overridevirtual

Set the initial mini-batch size.

Reimplemented from lbann::callback_base.

◆ operator=()

variable_minibatch& lbann::callback::variable_minibatch::operator= ( const variable_minibatch & )

default

Here is the caller graph for this function:

◆ schedule()

virtual bool lbann::callback::variable_minibatch::schedule	(	model *	m,
		size_t &	new_mbsize,
		float &	new_lr,
		size_t &	ramp_time
	)

protectedpure virtual

Implemented by child classes to provide the mini-batch/learning schedule. This is called at the end of every training epoch. If it returns false, no changes are made from the currently established schedule. If this returns true, the mini-batch size will be changed accordingly. If the mini-batch size is larger than the model's maximum mini-batch size, a warning is printed and the maximum mini-batch size is used. If new_lr also non-zero, the learning rate will be changed to new_lr, with a linear ramp time. (If ramp_time is 0, it is changed immediately.) Note changing the learning rate while in a ramp may lead to unexpected behavior; also be aware of interactions with other learning rate schedules.

Implemented in lbann::callback::minibatch_schedule, and lbann::callback::step_minibatch.

Here is the caller graph for this function:

Member Data Documentation

◆ m_current_mini_batch_size

size_t lbann::callback::variable_minibatch::m_current_mini_batch_size

protected

The current mini-batch size for this epoch. This is kept separately from the model's get_current_mini_batch_size() method, as calling that in on_epoch_end returns the size of the last mini- batch, not the "base" mini-batch.

Definition at line 82 of file variable_minibatch.hpp.

◆ m_lr_incr

float lbann::callback::variable_minibatch::m_lr_incr = 0.0f

protected

Amount to increment the learning rate by when ramping.

Definition at line 86 of file variable_minibatch.hpp.

◆ m_ramp_count

size_t lbann::callback::variable_minibatch::m_ramp_count = 0

protected

Current number of epochs left to ramp the learning rate.

Definition at line 84 of file variable_minibatch.hpp.

◆ m_starting_mbsize

size_t lbann::callback::variable_minibatch::m_starting_mbsize

protected

Initial mini-batch size.

Definition at line 75 of file variable_minibatch.hpp.

The documentation for this class was generated from the following file:

variable_minibatch.hpp

Public Member Functions

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ variable_minibatch() [1/2]

◆ variable_minibatch() [2/2]

Member Function Documentation

◆ change_learning_rate()

◆ get_current_learning_rate()

◆ on_epoch_end()

◆ on_train_begin()

◆ operator=()

◆ schedule()

Member Data Documentation

◆ m_current_mini_batch_size

◆ m_lr_incr

◆ m_ramp_count

◆ m_starting_mbsize