#include <sync_layers.hpp>

Inheritance diagram for lbann::callback::sync_layers:

Collaboration diagram for lbann::callback::sync_layers:

Public Member Functions
	sync_layers (bool sync_gpus=true, bool sync_mpi=true, bool only_input=false)

	sync_layers (const sync_layers &)=default

sync_layers &	operator= (const sync_layers &)=default

sync_layers *	copy () const override

std::string	name () const override
	Return this callback's name. More...

void	on_forward_prop_end (model m, Layer l) override
	Called when a layer ends forward propagation. More...

void	on_backward_prop_end (model m, Layer l) override
	Called when a layer ends backward propagation. More...

Serialization
template<class Archive >
void	serialize (Archive &ar)
	Store state to archive for checkpoint and restart. More...

Public Member Functions inherited from lbann::callback_base
	callback_base (int batch_interval=1)
	Initialize a callback with an optional batch interval. More...

	callback_base (const callback_base &)=default

virtual	~callback_base ()=default

virtual void	setup (trainer *t)
	Called once to set up the callback on the trainer. More...

virtual void	setup (model *m)
	Called once to set up the callback on the model (after all layers are set up). More...

virtual void	on_setup_end (model *m)
	Called at the end of setup. More...

virtual void	on_train_begin (model *m)
	Called at the beginning of training. More...

virtual void	on_train_end (model *m)
	Called at the end of training. More...

virtual void	on_phase_end (model *m)
	Called at the end of every phase (multiple epochs) in a layer-wise model training. More...

virtual void	on_epoch_begin (model *m)
	Called at the beginning of each epoch. More...

virtual void	on_epoch_end (model *m)
	Called immediate after the end of each epoch. More...

virtual void	on_batch_begin (model *m)
	Called at the beginning of a (mini-)batch. More...

virtual void	on_batch_end (model *m)
	Called immediately after the end of a (mini-)batch. More...

virtual void	on_test_begin (model *m)
	Called at the beginning of testing. More...

virtual void	on_test_end (model *m)
	Called immediately after the end of testing. More...

virtual void	on_validation_begin (model *m)
	Called at the beginning of validation. More...

virtual void	on_validation_end (model *m)
	Called immediately after the end of validation. More...

virtual void	on_forward_prop_begin (model *m)
	Called when a model begins forward propagation. More...

virtual void	on_forward_prop_begin (model m, Layer l)
	Called when a layer begins forward propagation. More...

virtual void	on_forward_prop_end (model *m)
	Called when a model ends forward propagation. More...

virtual void	on_backward_prop_begin (model *m)
	Called when a model begins backward propagation. More...

virtual void	on_backward_prop_begin (model m, Layer l)
	Called when a layer begins backward propagation. More...

virtual void	on_backward_prop_end (model *m)
	Called when a model ends backward propagation. More...

virtual void	on_optimize_begin (model *m)
	Called when a model begins optimization. More...

virtual void	on_optimize_begin (model m, weights w)
	Called when weights begins optimization. More...

virtual void	on_optimize_end (model *m)
	Called when a model ends optimization. More...

virtual void	on_optimize_end (model m, weights w)
	Called when weights ends optimization. More...

virtual void	on_batch_evaluate_begin (model *m)
	Called at the beginning of a (mini-)batch evaluation (validation / testing). More...

virtual void	on_batch_evaluate_end (model *m)
	Called at the end of a (mini-)batch evaluation (validation / testing). More...

virtual void	on_evaluate_forward_prop_begin (model *m)
	Called when a model begins forward propagation for evaluation (validation / testing). More...

virtual void	on_evaluate_forward_prop_begin (model m, Layer l)
	Called when a layer begins forward propagation for evaluation (validation / testing). More...

virtual void	on_evaluate_forward_prop_end (model *m)
	Called when a model ends forward propagation for evaluation (validation / testing). More...

virtual void	on_evaluate_forward_prop_end (model m, Layer l)
	Called when a layer ends forward propagation for evaluation (validation / testing). More...

int	get_batch_interval () const
	Return the batch interval. More...

virtual description	get_description () const
	Human-readable description. More...

template<class Archive >
void	serialize (Archive &ar)
	Store state to archive for checkpoint and restart. More...

void	write_proto (lbann_data::Callback &proto) const
	Write a protobuf description of the callback. More...

Protected Member Functions
void	write_specific_proto (lbann_data::Callback &proto) const final

virtual void	do_sync (Layer *l)

Protected Member Functions inherited from lbann::callback_base
std::string	get_multi_trainer_path (const model &m, const std::string &root_dir)
	Build a standard directory hierarchy including trainer ID. More...

std::string	get_multi_trainer_ec_model_path (const model &m, const std::string &root_dir)
	Build a standard directory hierachy including trainer, execution context, and model information (in that order). More...

std::string	get_multi_trainer_model_path (const model &m, const std::string &root_dir)
	Build a standard directory hierachy including trainer, model information in that order. More...

callback_base &	operator= (const callback_base &)=default
	Copy-assignment operator. More...

Protected Attributes
bool	m_sync_gpus

bool	m_sync_mpi

bool	m_only_input

Protected Attributes inherited from lbann::callback_base
int	m_batch_interval
	Batch methods should once every this many steps. More...

Detailed Description

Synchronize layers after forward and backward prop. Additionally updates layer timing information to account for this. Note that this callback should come before the summarizer callback to report time correctly (otherwise it will be shifted by one mini-batch).

Definition at line 42 of file sync_layers.hpp.

Constructor & Destructor Documentation

◆ sync_layers() [1/2]

lbann::callback::sync_layers::sync_layers	(	bool	sync_gpus = `true`,
		bool	sync_mpi = `true`,
		bool	only_input = `false`
	)

inline

Parameters

sync_gpus	The GPU stream will be synchronized.
sync_mpi	A global barrier will synchronize processes.
only_input	The only synchronization will be after the input layer in forward prop.

Definition at line 51 of file sync_layers.hpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ sync_layers() [2/2]

lbann::callback::sync_layers::sync_layers ( const sync_layers & )

default

Member Function Documentation

◆ copy()

sync_layers* lbann::callback::sync_layers::copy ( ) const

inlineoverridevirtual

Implements lbann::callback_base.

Definition at line 61 of file sync_layers.hpp.

Here is the call graph for this function:

◆ do_sync()

virtual void lbann::callback::sync_layers::do_sync ( Layer * l )

protectedvirtual

◆ name()

std::string lbann::callback::sync_layers::name ( ) const

inlineoverridevirtual

Return this callback's name.

Implements lbann::callback_base.

Definition at line 62 of file sync_layers.hpp.

Here is the call graph for this function:

◆ on_backward_prop_end()

void lbann::callback::sync_layers::on_backward_prop_end	(	model *	m,
		Layer *	l
	)

overridevirtual

Called when a layer ends backward propagation.

Reimplemented from lbann::callback_base.

Here is the caller graph for this function:

◆ on_forward_prop_end()

void lbann::callback::sync_layers::on_forward_prop_end	(	model *	m,
		Layer *	l
	)

overridevirtual

Called when a layer ends forward propagation.

Reimplemented from lbann::callback_base.

Here is the caller graph for this function:

◆ operator=()

sync_layers& lbann::callback::sync_layers::operator= ( const sync_layers & )

default

Here is the caller graph for this function:

◆ serialize()

template<class Archive >

void lbann::callback::sync_layers::serialize ( Archive & ar )

Store state to archive for checkpoint and restart.

Here is the caller graph for this function:

◆ write_specific_proto()

void lbann::callback::sync_layers::write_specific_proto ( lbann_data::Callback & proto ) const

finalprotectedvirtual

Add callback specific data to prototext

Implements lbann::callback_base.

Here is the caller graph for this function:

Member Data Documentation

◆ m_only_input

bool lbann::callback::sync_layers::m_only_input

protected

Whether to only synchronize after the input layer.

Definition at line 88 of file sync_layers.hpp.

◆ m_sync_gpus

bool lbann::callback::sync_layers::m_sync_gpus

protected

Whether to synchronize GPUs.

Definition at line 84 of file sync_layers.hpp.

◆ m_sync_mpi

bool lbann::callback::sync_layers::m_sync_mpi

protected

Whether to do a global synchronization.

Definition at line 86 of file sync_layers.hpp.

The documentation for this class was generated from the following file:

sync_layers.hpp

Public Member Functions

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ sync_layers() [1/2]

◆ sync_layers() [2/2]

Member Function Documentation

◆ copy()

◆ do_sync()

◆ name()

◆ on_backward_prop_end()

◆ on_forward_prop_end()

◆ operator=()

◆ serialize()

◆ write_specific_proto()

Member Data Documentation

◆ m_only_input

◆ m_sync_gpus

◆ m_sync_mpi