LBANN  0.103.0
LivermoreBigArtificialNeuralNetworkToolkit
Public Member Functions | Public Attributes | List of all members
lbann::data_buffer< TensorDataType > Class Template Reference

#include <io_data_buffer.hpp>

Public Types

Public Types
using AbsDistMatrixType = El::AbstractDistMatrix< TensorDataType >
 The tensor type expected in this object. More...
 

Public Member Functions

 data_buffer (lbann_comm *comm)
 
 data_buffer (const data_buffer &other)
 
data_buffer & operator= (const data_buffer &other)
 
data_buffer * copy () const
 
template<class Archive >
void serialize (Archive &ar)
 
void initialize_buffer_for_data_field (data_field_type const data_field, lbann_comm *comm)
 Create a data parallel distributed matrix to hold the input data for the field. More...
 
void set_fetch_data_in_background (bool flag)
 
bool is_data_fetched_in_background () const
 
const El::Matrix< El::Int > * get_sample_indices_fetched_per_mb () const
 
El::Matrix< El::Int > * get_sample_indices_fetched_per_mb ()
 
int num_samples_ready ()
 
void set_data_fetch_future (std::future< void > future)
 
std::future< void > get_data_fetch_future ()
 

Public Attributes

int m_num_samples_fetched
 
std::map< data_field_type, std::unique_ptr< AbsDistMatrixType > > m_input_buffers
 
std::atomic< bool > m_fetch_data_in_background
 
std::future< void > m_data_fetch_future
 
El::Matrix< El::Int > m_indices_fetched_per_mb
 1-D Matrix of which indices were fetched in this mini-batch More...
 

Detailed Description

template<typename TensorDataType>
class lbann::data_buffer< TensorDataType >

Definition at line 35 of file io_data_buffer.hpp.

Member Typedef Documentation

◆ AbsDistMatrixType

template<typename TensorDataType>
using lbann::data_buffer< TensorDataType >::AbsDistMatrixType = El::AbstractDistMatrix<TensorDataType>

The tensor type expected in this object.

Definition at line 42 of file io_data_buffer.hpp.

Constructor & Destructor Documentation

◆ data_buffer() [1/2]

template<typename TensorDataType>
lbann::data_buffer< TensorDataType >::data_buffer ( lbann_comm *  comm)
inline

Definition at line 56 of file io_data_buffer.hpp.

Here is the caller graph for this function:

◆ data_buffer() [2/2]

template<typename TensorDataType>
lbann::data_buffer< TensorDataType >::data_buffer ( const data_buffer< TensorDataType > &  other)
inline

Definition at line 62 of file io_data_buffer.hpp.

Member Function Documentation

◆ copy()

template<typename TensorDataType>
data_buffer* lbann::data_buffer< TensorDataType >::copy ( ) const
inline

Definition at line 83 of file io_data_buffer.hpp.

Here is the call graph for this function:

◆ get_data_fetch_future()

template<typename TensorDataType>
std::future<void> lbann::data_buffer< TensorDataType >::get_data_fetch_future ( )
inline

Definition at line 123 of file io_data_buffer.hpp.

◆ get_sample_indices_fetched_per_mb() [1/2]

template<typename TensorDataType>
const El::Matrix<El::Int>* lbann::data_buffer< TensorDataType >::get_sample_indices_fetched_per_mb ( ) const
inline

Return the sample indices fetched in the current mini-batch.

Definition at line 107 of file io_data_buffer.hpp.

◆ get_sample_indices_fetched_per_mb() [2/2]

template<typename TensorDataType>
El::Matrix<El::Int>* lbann::data_buffer< TensorDataType >::get_sample_indices_fetched_per_mb ( )
inline

Definition at line 111 of file io_data_buffer.hpp.

◆ initialize_buffer_for_data_field()

template<typename TensorDataType >
void lbann::data_buffer< TensorDataType >::initialize_buffer_for_data_field ( data_field_type const  data_field,
lbann_comm *  comm 
)

Create a data parallel distributed matrix to hold the input data for the field.

Definition at line 45 of file io_data_buffer_impl.hpp.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ is_data_fetched_in_background()

template<typename TensorDataType>
bool lbann::data_buffer< TensorDataType >::is_data_fetched_in_background ( ) const
inline

Definition at line 99 of file io_data_buffer.hpp.

◆ num_samples_ready()

template<typename TensorDataType>
int lbann::data_buffer< TensorDataType >::num_samples_ready ( )
inline

Definition at line 116 of file io_data_buffer.hpp.

◆ operator=()

template<typename TensorDataType>
data_buffer& lbann::data_buffer< TensorDataType >::operator= ( const data_buffer< TensorDataType > &  other)
inline

Definition at line 72 of file io_data_buffer.hpp.

◆ serialize()

template<typename TensorDataType >
template<class Archive >
void lbann::data_buffer< TensorDataType >::serialize ( Archive &  ar)

Archive for checkpoint and restart

Definition at line 36 of file io_data_buffer_impl.hpp.

Here is the caller graph for this function:

◆ set_data_fetch_future()

template<typename TensorDataType>
void lbann::data_buffer< TensorDataType >::set_data_fetch_future ( std::future< void >  future)
inline

Definition at line 118 of file io_data_buffer.hpp.

◆ set_fetch_data_in_background()

template<typename TensorDataType>
void lbann::data_buffer< TensorDataType >::set_fetch_data_in_background ( bool  flag)
inline

Definition at line 94 of file io_data_buffer.hpp.

Member Data Documentation

◆ m_data_fetch_future

template<typename TensorDataType>
std::future<void> lbann::data_buffer< TensorDataType >::m_data_fetch_future

Definition at line 52 of file io_data_buffer.hpp.

◆ m_fetch_data_in_background

template<typename TensorDataType>
std::atomic<bool> lbann::data_buffer< TensorDataType >::m_fetch_data_in_background

Definition at line 51 of file io_data_buffer.hpp.

◆ m_indices_fetched_per_mb

template<typename TensorDataType>
El::Matrix<El::Int> lbann::data_buffer< TensorDataType >::m_indices_fetched_per_mb

1-D Matrix of which indices were fetched in this mini-batch

Definition at line 54 of file io_data_buffer.hpp.

◆ m_input_buffers

template<typename TensorDataType>
std::map<data_field_type, std::unique_ptr<AbsDistMatrixType> > lbann::data_buffer< TensorDataType >::m_input_buffers

Distributed matrix used to stage local data to layer output

Definition at line 50 of file io_data_buffer.hpp.

◆ m_num_samples_fetched

template<typename TensorDataType>
int lbann::data_buffer< TensorDataType >::m_num_samples_fetched

Number of samples in the current mini-batch

Definition at line 48 of file io_data_buffer.hpp.

The documentation for this class was generated from the following files: