Neural network tensor operation. More...

#include <operator.hpp>

Inheritance diagram for lbann::Operator< InputT, OutputT, D >:

Collaboration diagram for lbann::Operator< InputT, OutputT, D >:

Public Types
Public Types
using	InputTensorType = utils::DistTensorView< InputT, D >

using	OutputTensorType = utils::DistTensorView< OutputT, D >

using	ConstInputTensorType = utils::ConstDistTensorView< InputT, D >

using	ConstOutputTensorType = utils::ConstDistTensorView< OutputT, D >

Public Member Functions
	Operator ()=default
	Constructor. More...

virtual	~Operator ()=default
	Destructor. More...

virtual std::string	get_type () const =0
	Get the operator type's name. More...

virtual int	get_backprop_requirements () const
	Returns the necessary tensors for computing backpropagation for this operator. More...

Description	get_description () const override
	Get the description of the operator. More...

void	write_proto (google::protobuf::Message &msg) const
	Write a protobuf description of the operator. More...

Serialization
template<typename ArchiveT >
void	serialize (ArchiveT &ar)

Computational interface
virtual void	fp_compute (std::vector< ConstInputTensorType > const &inputs, std::vector< OutputTensorType > const &outputs) const =0
	Apply operator's forward operation. More...

virtual void	bp_compute (std::vector< ConstInputTensorType > const &inputs, std::vector< ConstOutputTensorType > const &gradient_wrt_outputs, std::vector< InputTensorType > const &gradient_wrt_inputs) const
	Compute operator's "backward" operation. More...

Public Member Functions inherited from lbann::Cloneable< T, Base >
std::unique_ptr< T >	clone () const
	Return an exception-safe, memory-safe copy of this object. More...

Public Member Functions inherited from lbann::Describable
virtual	~Describable ()=default

Public Member Functions inherited from lbann::ProtobufSerializable
virtual	~ProtobufSerializable ()=default

void	write_proto (google::protobuf::Message &proto) const
	Write the object to a protobuf message. More...

Protected Member Functions
	Operator (Operator &&other) noexcept=default

Operator &	operator= (Operator &&other) noexcept=default

	Operator (Operator const &other)=default

Operator &	operator= (Operator const &other)=default

Private Member Functions
virtual void	set_proto_params (lbann_data::Operator &) const =0
	Fill the concrete operator parameters. More...

virtual void	do_fill_description (Description &) const =0
	Concrete operator description. More...

Detailed Description

template<typename InputT, typename OutputT, El::Device D>
class lbann::Operator< InputT, OutputT, D >

Neural network tensor operation.

An operator defines a mathematical function that that supports both forward and possibly backward operations. In the forward direction, it takes a vector of input tensors and produces a vector of output tensors. In the backward direction they implement the differentiation of the forward operation, applying the function to the operator's forward inputs and gradient with respect to the outputs, to compute the gradient with respect to the input.

Operators act as curried functions, they can have state that is defined during construction but do not hold internal state. A operator should also be able to take objective function gradients w.r.t. the outputs ("previous error signals") and compute the objective function gradients w.r.t. the inputs ("error signals"). This allows the model to perform automatic differentiation.

Operators are specified for unique input and output data types.

Definition at line 85 of file operator.hpp.

Member Typedef Documentation

◆ ConstInputTensorType

template<typename InputT , typename OutputT , El::Device D>

using lbann::Operator< InputT, OutputT, D >::ConstInputTensorType = utils::ConstDistTensorView<InputT, D>

Definition at line 94 of file operator.hpp.

◆ ConstOutputTensorType

template<typename InputT , typename OutputT , El::Device D>

using lbann::Operator< InputT, OutputT, D >::ConstOutputTensorType = utils::ConstDistTensorView<OutputT, D>

Definition at line 95 of file operator.hpp.

◆ InputTensorType

template<typename InputT , typename OutputT , El::Device D>

using lbann::Operator< InputT, OutputT, D >::InputTensorType = utils::DistTensorView<InputT, D>

Definition at line 92 of file operator.hpp.

◆ OutputTensorType

template<typename InputT , typename OutputT , El::Device D>

using lbann::Operator< InputT, OutputT, D >::OutputTensorType = utils::DistTensorView<OutputT, D>

Definition at line 93 of file operator.hpp.

Constructor & Destructor Documentation

◆ Operator() [1/3]

template<typename InputT , typename OutputT , El::Device D>

lbann::Operator< InputT, OutputT, D >::Operator ( )

default

Constructor.

Here is the caller graph for this function:

◆ ~Operator()

template<typename InputT , typename OutputT , El::Device D>

virtual lbann::Operator< InputT, OutputT, D >::~Operator ( )

virtualdefault

Destructor.

◆ Operator() [2/3]

template<typename InputT , typename OutputT , El::Device D>

lbann::Operator< InputT, OutputT, D >::Operator ( Operator< InputT, OutputT, D > && other )

protecteddefaultnoexcept

◆ Operator() [3/3]

template<typename InputT , typename OutputT , El::Device D>

lbann::Operator< InputT, OutputT, D >::Operator ( Operator< InputT, OutputT, D > const & other )

protecteddefault

Member Function Documentation

◆ bp_compute()

template<typename InputT , typename OutputT , El::Device D>

void lbann::Operator< InputT, OutputT, D >::bp_compute	(	std::vector< ConstInputTensorType > const &	inputs,
		std::vector< ConstOutputTensorType > const &	gradient_wrt_outputs,
		std::vector< InputTensorType > const &	gradient_wrt_inputs
	)		const

virtual

Compute operator's "backward" operation.

Given the inputs and gradient w.r.t. output tensors, the gradient w.r.t. input tensors are populated with the computed values.

Definition at line 209 of file operator.hpp.

Here is the caller graph for this function:

◆ do_fill_description()

template<typename InputT , typename OutputT , El::Device D>

virtual void lbann::Operator< InputT, OutputT, D >::do_fill_description ( Description & ) const

privatepure virtual

Concrete operator description.

Here is the caller graph for this function:

◆ fp_compute()

template<typename InputT , typename OutputT , El::Device D>

virtual void lbann::Operator< InputT, OutputT, D >::fp_compute	(	std::vector< ConstInputTensorType > const &	inputs,
		std::vector< OutputTensorType > const &	outputs
	)		const

pure virtual

Apply operator's forward operation.

Given the input tensors, the output tensors are populated with computed values.

Here is the caller graph for this function:

◆ get_backprop_requirements()

template<typename InputT , typename OutputT , El::Device D>

virtual int lbann::Operator< InputT, OutputT, D >::get_backprop_requirements ( ) const

inlinevirtual

Returns the necessary tensors for computing backpropagation for this operator.

Definition at line 121 of file operator.hpp.

Here is the call graph for this function:

◆ get_description()

template<typename InputT , typename OutputT , El::Device D>

Description lbann::Operator< InputT, OutputT, D >::get_description ( ) const

overridevirtual

Get the description of the operator.

Implements lbann::Describable.

Definition at line 193 of file operator.hpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ get_type()

template<typename InputT , typename OutputT , El::Device D>

virtual std::string lbann::Operator< InputT, OutputT, D >::get_type ( ) const

pure virtual

Get the operator type's name.

A operator type name should be brief, human-readable description of the operator's mathematical operation.

Here is the caller graph for this function:

◆ operator=() [1/2]

template<typename InputT , typename OutputT , El::Device D>

Operator& lbann::Operator< InputT, OutputT, D >::operator= ( Operator< InputT, OutputT, D > && other )

protecteddefaultnoexcept

Here is the caller graph for this function:

◆ operator=() [2/2]

template<typename InputT , typename OutputT , El::Device D>

Operator& lbann::Operator< InputT, OutputT, D >::operator= ( Operator< InputT, OutputT, D > const & other )

protecteddefault

◆ serialize()

template<typename InputT , typename OutputT , El::Device D>

template<typename ArchiveT >

void lbann::Operator< InputT, OutputT, D >::serialize ( ArchiveT & ar )

Definition at line 217 of file operator.hpp.

Here is the caller graph for this function:

◆ set_proto_params()

template<typename InputT , typename OutputT , El::Device D>

virtual void lbann::Operator< InputT, OutputT, D >::set_proto_params ( lbann_data::Operator< InputT, OutputT, D > & ) const

privatepure virtual

Fill the concrete operator parameters.

Here is the caller graph for this function:

◆ write_proto()

template<typename InputT , typename OutputT , El::Device D>

void lbann::Operator< InputT, OutputT, D >::write_proto ( google::protobuf::Message & msg ) const

Write a protobuf description of the operator.

This should be able to be passed back to LBANN to exactly recreate this operator from protobuf.

Definition at line 179 of file operator.hpp.

Here is the call graph for this function:

Here is the caller graph for this function:

The documentation for this class was generated from the following file:

operator.hpp

Public Types

Public Member Functions

Protected Member Functions

Private Member Functions

Detailed Description

template<typename InputT, typename OutputT, El::Device D> class lbann::Operator< InputT, OutputT, D >

Member Typedef Documentation

◆ ConstInputTensorType

◆ ConstOutputTensorType

◆ InputTensorType

◆ OutputTensorType

Constructor & Destructor Documentation

◆ Operator() [1/3]

◆ ~Operator()

◆ Operator() [2/3]

◆ Operator() [3/3]

Member Function Documentation

◆ bp_compute()

◆ do_fill_description()

◆ fp_compute()

◆ get_backprop_requirements()

◆ get_description()

◆ get_type()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ serialize()

◆ set_proto_params()

◆ write_proto()

template<typename InputT, typename OutputT, El::Device D>
class lbann::Operator< InputT, OutputT, D >