lbann::variance_scaling_initializer< TensorDataType > Class Template Reference

Generalization of "Xavier" initialization. More...

#include <variance_scaling_initializers.hpp>

Inheritance diagram for lbann::variance_scaling_initializer< TensorDataType >:

Collaboration diagram for lbann::variance_scaling_initializer< TensorDataType >:

Public Types
Public Types
using	AbsDistMatrixType = El::AbstractDistMatrix< TensorDataType >
	The tensor type expected in this object. More...

Public Member Functions
	variance_scaling_initializer (probability_distribution dist)
description	get_description () const override
void	fill (AbsDistMatrixType &matrix) override
void	set_fan_in (El::Int fan_in)
void	set_fan_out (El::Int fan_out)
probability_distribution	get_prob_dist () const noexcept
Public Member Functions inherited from lbann::Cloneable< HasAbstractFunction< variance_scaling_initializer< TensorDataType > >, data_type_weights_initializer< TensorDataType > >
std::unique_ptr< HasAbstractFunction< variance_scaling_initializer< TensorDataType > > >	clone () const
	Return an exception-safe, memory-safe copy of this object. More...

Private Member Functions
virtual TensorDataType	get_variance (El::Int fan_in, El::Int fan_out)=0

Private Attributes
probability_distribution	m_prob_dist
El::Int	m_fan_in
El::Int	m_fan_out

Detailed Description

template<typename TensorDataType>
class lbann::variance_scaling_initializer< TensorDataType >

Generalization of "Xavier" initialization.

Weights values are randomly sampled from a probability distribution with a variance determined by a "fan-in" and a "fan-out" parameter.

Weights with variance scaling initialization are only compatible with layers that set fan-in and fan-out parameters, e.g. the convolution and fully-connected layers.

Definition at line 47 of file variance_scaling_initializers.hpp.

Member Typedef Documentation

AbsDistMatrixType

template<typename TensorDataType >

using lbann::variance_scaling_initializer< TensorDataType >::AbsDistMatrixType = El::AbstractDistMatrix<TensorDataType>

The tensor type expected in this object.

Definition at line 57 of file variance_scaling_initializers.hpp.

Constructor & Destructor Documentation

variance_scaling_initializer()

template<typename TensorDataType >

lbann::variance_scaling_initializer< TensorDataType >::variance_scaling_initializer

(

probability_distribution

dist

)

Member Function Documentation

fill()

template<typename TensorDataType >

void lbann::variance_scaling_initializer< TensorDataType >::fill ( AbsDistMatrixType &  matrix )

override

get_description()

template<typename TensorDataType >

description lbann::variance_scaling_initializer< TensorDataType >::get_description ( ) const

override

get_prob_dist()

template<typename TensorDataType >

probability_distribution lbann::variance_scaling_initializer< TensorDataType >::get_prob_dist ( ) const

inlinenoexcept

Get probability distribution.

Definition at line 72 of file variance_scaling_initializers.hpp.

Here is the call graph for this function:

get_variance()

template<typename TensorDataType >

virtual TensorDataType lbann::variance_scaling_initializer< TensorDataType >::get_variance ( El::Int  fan_in, El::Int  fan_out  )

privatepure virtual

Get probability distribution variance.

Here is the caller graph for this function:

set_fan_in()

template<typename TensorDataType >

void lbann::variance_scaling_initializer< TensorDataType >::set_fan_in ( El::Int  fan_in )

inline

Set fan-in parameter.

Definition at line 67 of file variance_scaling_initializers.hpp.

Here is the caller graph for this function:

set_fan_out()

template<typename TensorDataType >

void lbann::variance_scaling_initializer< TensorDataType >::set_fan_out ( El::Int  fan_out )

inline

Set fan-out parameter.

Definition at line 69 of file variance_scaling_initializers.hpp.

Here is the caller graph for this function:

Member Data Documentation

m_fan_in

template<typename TensorDataType >

El::Int lbann::variance_scaling_initializer< TensorDataType >::m_fan_in

private

Fan-in parameter.

Definition at line 85 of file variance_scaling_initializers.hpp.

m_fan_out

template<typename TensorDataType >

El::Int lbann::variance_scaling_initializer< TensorDataType >::m_fan_out

private

Fan-out parameter.

Definition at line 87 of file variance_scaling_initializers.hpp.

m_prob_dist

template<typename TensorDataType >

probability_distribution lbann::variance_scaling_initializer< TensorDataType >::m_prob_dist

private

Probability distribution.

Definition at line 83 of file variance_scaling_initializers.hpp.

---

The documentation for this class was generated from the following file:

• variance_scaling_initializers.hpp