#include <variable_minibatch.hpp>

Inheritance diagram for lbann::callback::step_minibatch:

Collaboration diagram for lbann::callback::step_minibatch:

Public Member Functions
	step_minibatch (size_t starting_mbsize, size_t step, size_t ramp_time=0)

	step_minibatch (const step_minibatch &)=default

step_minibatch &	operator= (const step_minibatch &)=delete

step_minibatch *	copy () const override

std::string	name () const override
	Return this callback's name. More...

Public Member Functions inherited from lbann::callback::variable_minibatch
	variable_minibatch (size_t starting_mbsize)

	variable_minibatch (const variable_minibatch &)=default

variable_minibatch &	operator= (const variable_minibatch &)=default

void	on_train_begin (model *m) override
	Set the initial mini-batch size. More...

void	on_epoch_end (model *m) override
	Potentially change the mini-batch size. More...

Public Member Functions inherited from lbann::callback_base
	callback_base (int batch_interval=1)
	Initialize a callback with an optional batch interval. More...

	callback_base (const callback_base &)=default

virtual	~callback_base ()=default

virtual void	setup (trainer *t)
	Called once to set up the callback on the trainer. More...

virtual void	setup (model *m)
	Called once to set up the callback on the model (after all layers are set up). More...

virtual void	on_setup_end (model *m)
	Called at the end of setup. More...

virtual void	on_train_end (model *m)
	Called at the end of training. More...

virtual void	on_phase_end (model *m)
	Called at the end of every phase (multiple epochs) in a layer-wise model training. More...

virtual void	on_epoch_begin (model *m)
	Called at the beginning of each epoch. More...

virtual void	on_batch_begin (model *m)
	Called at the beginning of a (mini-)batch. More...

virtual void	on_batch_end (model *m)
	Called immediately after the end of a (mini-)batch. More...

virtual void	on_test_begin (model *m)
	Called at the beginning of testing. More...

virtual void	on_test_end (model *m)
	Called immediately after the end of testing. More...

virtual void	on_validation_begin (model *m)
	Called at the beginning of validation. More...

virtual void	on_validation_end (model *m)
	Called immediately after the end of validation. More...

virtual void	on_forward_prop_begin (model *m)
	Called when a model begins forward propagation. More...

virtual void	on_forward_prop_begin (model m, Layer l)
	Called when a layer begins forward propagation. More...

virtual void	on_forward_prop_end (model *m)
	Called when a model ends forward propagation. More...

virtual void	on_forward_prop_end (model m, Layer l)
	Called when a layer ends forward propagation. More...

virtual void	on_backward_prop_begin (model *m)
	Called when a model begins backward propagation. More...

virtual void	on_backward_prop_begin (model m, Layer l)
	Called when a layer begins backward propagation. More...

virtual void	on_backward_prop_end (model *m)
	Called when a model ends backward propagation. More...

virtual void	on_backward_prop_end (model m, Layer l)
	Called when a layer ends backward propagation. More...

virtual void	on_optimize_begin (model *m)
	Called when a model begins optimization. More...

virtual void	on_optimize_begin (model m, weights w)
	Called when weights begins optimization. More...

virtual void	on_optimize_end (model *m)
	Called when a model ends optimization. More...

virtual void	on_optimize_end (model m, weights w)
	Called when weights ends optimization. More...

virtual void	on_batch_evaluate_begin (model *m)
	Called at the beginning of a (mini-)batch evaluation (validation / testing). More...

virtual void	on_batch_evaluate_end (model *m)
	Called at the end of a (mini-)batch evaluation (validation / testing). More...

virtual void	on_evaluate_forward_prop_begin (model *m)
	Called when a model begins forward propagation for evaluation (validation / testing). More...

virtual void	on_evaluate_forward_prop_begin (model m, Layer l)
	Called when a layer begins forward propagation for evaluation (validation / testing). More...

virtual void	on_evaluate_forward_prop_end (model *m)
	Called when a model ends forward propagation for evaluation (validation / testing). More...

virtual void	on_evaluate_forward_prop_end (model m, Layer l)
	Called when a layer ends forward propagation for evaluation (validation / testing). More...

int	get_batch_interval () const
	Return the batch interval. More...

virtual description	get_description () const
	Human-readable description. More...

template<class Archive >
void	serialize (Archive &ar)
	Store state to archive for checkpoint and restart. More...

void	write_proto (lbann_data::Callback &proto) const
	Write a protobuf description of the callback. More...

Protected Member Functions
bool	schedule (model *m, size_t &new_mbsize, float &new_lr, size_t &ramp_time) override

Protected Member Functions inherited from lbann::callback::variable_minibatch
void	change_learning_rate (model *m, float new_lr) const
	Change the learning rate of every layer in m to new_lr. More...

float	get_current_learning_rate (model *m) const
	Get the current learning rate (assumes every layer has the same one). More...

Protected Member Functions inherited from lbann::callback_base
std::string	get_multi_trainer_path (const model &m, const std::string &root_dir)
	Build a standard directory hierarchy including trainer ID. More...

std::string	get_multi_trainer_ec_model_path (const model &m, const std::string &root_dir)
	Build a standard directory hierachy including trainer, execution context, and model information (in that order). More...

std::string	get_multi_trainer_model_path (const model &m, const std::string &root_dir)
	Build a standard directory hierachy including trainer, model information in that order. More...

callback_base &	operator= (const callback_base &)=default
	Copy-assignment operator. More...

Private Member Functions
void	write_specific_proto (lbann_data::Callback &proto) const final

Private Attributes
size_t	m_step
	Number of epochs between mini-batch size increases. More...

size_t	m_ramp_time
	Number of steps to ramp the learning rate over. More...

Additional Inherited Members
Protected Attributes inherited from lbann::callback::variable_minibatch
size_t	m_starting_mbsize
	Initial mini-batch size. More...

size_t	m_current_mini_batch_size

size_t	m_ramp_count = 0
	Current number of epochs left to ramp the learning rate. More...

float	m_lr_incr = 0.0f
	Amount to increment the learning rate by when ramping. More...

Protected Attributes inherited from lbann::callback_base
int	m_batch_interval
	Batch methods should once every this many steps. More...

Detailed Description

Double the mini-batch size every set number of epochs. Also doubles the learning rate.

Definition at line 93 of file variable_minibatch.hpp.

Constructor & Destructor Documentation

◆ step_minibatch() [1/2]

lbann::callback::step_minibatch::step_minibatch	(	size_t	starting_mbsize,
		size_t	step,
		size_t	ramp_time = `0`
	)

◆ step_minibatch() [2/2]

lbann::callback::step_minibatch::step_minibatch ( const step_minibatch & )

default

Member Function Documentation

◆ copy()

step_minibatch* lbann::callback::step_minibatch::copy ( ) const

inlineoverridevirtual

Implements lbann::callback_base.

Definition at line 99 of file variable_minibatch.hpp.

◆ name()

std::string lbann::callback::step_minibatch::name ( ) const

inlineoverridevirtual

Return this callback's name.

Implements lbann::callback_base.

Definition at line 100 of file variable_minibatch.hpp.

Here is the call graph for this function:

◆ operator=()

step_minibatch& lbann::callback::step_minibatch::operator= ( const step_minibatch & )

delete

◆ schedule()

bool lbann::callback::step_minibatch::schedule	(	model *	m,
		size_t &	new_mbsize,
		float &	new_lr,
		size_t &	ramp_time
	)

overrideprotectedvirtual

Implemented by child classes to provide the mini-batch/learning schedule. This is called at the end of every training epoch. If it returns false, no changes are made from the currently established schedule. If this returns true, the mini-batch size will be changed accordingly. If the mini-batch size is larger than the model's maximum mini-batch size, a warning is printed and the maximum mini-batch size is used. If new_lr also non-zero, the learning rate will be changed to new_lr, with a linear ramp time. (If ramp_time is 0, it is changed immediately.) Note changing the learning rate while in a ramp may lead to unexpected behavior; also be aware of interactions with other learning rate schedules.

Implements lbann::callback::variable_minibatch.

◆ write_specific_proto()

void lbann::callback::step_minibatch::write_specific_proto ( lbann_data::Callback & proto ) const

finalprivatevirtual

Add callback specific data to prototext

Implements lbann::callback_base.

Member Data Documentation

◆ m_ramp_time

size_t lbann::callback::step_minibatch::m_ramp_time

private

Number of steps to ramp the learning rate over.

Definition at line 115 of file variable_minibatch.hpp.

◆ m_step

size_t lbann::callback::step_minibatch::m_step

private

Number of epochs between mini-batch size increases.

Definition at line 113 of file variable_minibatch.hpp.

The documentation for this class was generated from the following file:

variable_minibatch.hpp

Public Member Functions

Protected Member Functions

Private Member Functions

Private Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ step_minibatch() [1/2]

◆ step_minibatch() [2/2]

Member Function Documentation

◆ copy()

◆ name()

◆ operator=()

◆ schedule()

◆ write_specific_proto()

Member Data Documentation

◆ m_ramp_time

◆ m_step