LBANN
latest

Getting Started

  • Quick Start
    • What can LBANN do?
    • Installing LBANN
    • Test LBANN Install
    • Basic Usage
    • PyTorch to LBANN
      • Loading Data
      • Building a Model
      • Setup Model Training
      • Run the Experiment
  • Building LBANN
    • Download
    • Building with Spack
      • Setup Spack (One-time setup)
      • Building & Installing LBANN as a user at a known HPC Center
      • Building & Installing LBANN as a user (with maximum flexiblity)
      • Building & Installing LBANN as a developer
    • Debugging some common Spack related issues
      • Using Python packages with LBANN
    • Advanced build methods
      • Building LBANN on OS X
        • Setup Homebrew
        • Building & Installing LBANN
      • Building LBANN with CMake
        • Dependencies
        • LBANN CMake options
        • Controlling dependency resolution
        • Building JAG utilities
        • Example CMake invocation
      • Building LBANN in containers
        • Singularity
        • Docker
      • Specific information for Livermore Computing (LC) systems
        • The build_lbann_lc.sh script
        • Pre-installed Binary Packages
  • Running LBANN
    • Sanity check with a simple test using LeNet
    • Anatomy of an LBANN experiment
      • Parallelism
      • Model components
      • Data readers
    • Python frontend
      • Setup
      • Basic usage
      • A simple example
      • Useful submodules
        • lbann.modules
        • lbann.models
        • lbann.proto
        • lbann.onnx
    • Protobuf frontend (advanced)
    • Setup of LBANN for manual CMake build (advanced)

Python Front-End

  • Callbacks
    • Using Callbacks
      • Python Front End
      • Profobuf (Advanced)
    • Available Callbacks
      • Export Onnx
        • Execution Points
        • Callback Arguments
        • Example Using Export ONNX Callback (Python Front-End)
      • Summarize images
        • Execution Points
        • Callback Arguments (Python Front-End)
        • Examples Using Summarize Images Callback
  • Data Transformation Pipeline
    • General Transformations
      • Normalize
      • Sample Normalize
      • Scale
      • Scale and Translate
    • Image Transformations
      • Adjust Brightness
      • Adjust Contrast
      • Adjust Saturation
      • Center Crop
      • Change Image Tensor Layout
      • Color Jitter
      • Colorize
      • Cutout
      • Grayscale
      • Horizontal Flip
      • Normalize To LBANN Layout
      • Random Affine
      • Random Crop
      • Random Resized Crop
      • Random Resized Crop With Fixed Aspect Ratio
      • Resize
      • Resized Center Crop
      • To LBANN Layout
      • Vertical Flip
  • Execution Algorithms
    • Batched Inference
    • Training Algorithms
    • Python Front-end API Documentation
      • lbann.TrainingAlgorithm interface
        • TrainingAlgorithm
        • __init__()
        • export_proto()
        • do_export_proto()
    • Supported algorithms
      • Batched inference
        • Python Front-end Example
        • Python Front-end API Documentation
      • Batched first-order optimization
        • Python Front-end Example
        • Python Front-end API Documentation
      • LTFB
        • Python Front-end Example
        • Truncation Selection Exchange (TSE) Variant of LTFB
        • Python Front-end API Documentation
      • KFAC
        • Python Front-end API Documentation
  • Hyperparameter Tuning
    • Simple Example

LBANN Layers

  • Layers
    • Using Layers
      • Python Front End
    • Common Layer Arguments
    • Advanced Layer Options
    • Deprecated Layer Options
    • LBANN Layers List
      • I/O Layers
        • Input
      • Operator Layer
      • Transform Layers
        • BatchwiseReduceSum
        • IdentityZero
        • Bernoulli
        • Concatenation
        • Constant
        • Crop
        • Cross_Grid_Sum
        • Cross_Grid_Sum_Slice
        • Dummy
        • Evaluation
        • Gather
        • Gaussian
        • Hadamard
        • InTopK
        • Pooling
        • Reduction
        • Reshape
        • Scatter
        • Slice
        • Sort
        • Split
        • StopGradient
        • Sum
        • TensorPermute
        • Tessellate
        • Uniform
        • Unpooling
        • WeightedSum
        • WeightsLayer
        • CategoricalRandom (Deprecated)
        • DiscreteRandom (Deprecated)
      • Learning Layers
        • ChannelwiseFullyConnected
        • ChannelwiseScaleBias
        • Convolution
        • Deconvolution
        • Embedding
        • EntrywiseScaleBias
        • FullyConnected
        • GRU
      • Loss Layers
        • CategoricalAccuracy
        • CrossEntropy
        • L1Norm
        • L2Norm2
        • MeanAbsoluteError
        • MeanSquaredError
        • TopKCategoricalAccuracy
      • Math Layers
        • DFTAbs
        • MatMul
      • Regularization Layers
        • BatchNormalization
        • Dropout
        • EntrywiseBatchNormalization
        • InstanceNorm
        • LayerNorm
        • LocalResponseNormalization
        • SeluDropout
      • Activation Layers
        • Elu
        • Identity
        • LeakyRelu
        • LogSoftmax
        • Relu
        • Softmax
      • Image Layers
        • BilinearResize
        • CompositeImageTransformation
        • Rotation
      • Miscellaneous Layers
        • Argmax
        • Argmin
        • ChannelwiseMean
        • ChannelwiseSoftmax
        • Covariance
        • DistEmbedding
        • External
        • MiniBatchIndex
        • MiniBatchSize
        • OneHot
        • RowwiseWeightsNorms
        • UniformHash
        • Variance

LBANN Operators

  • Operators
    • Operator
    • Abs
    • Acosh
    • Acos
    • Add
    • AddConstant
    • Asin
    • Asinh
    • Atan
    • Atanh
    • BinaryCrossEntropy
    • BooleanAccuracy
    • BooleanFalseNegative
    • BooleanFalsePositive
    • Ceil
    • Clamp
    • ConstantSubtract
    • Cos
    • Cosh
    • Divide
    • Equal
    • EqualConstant
    • Erf
    • ErfInv
    • Exp
    • Expm1
    • Floor
    • Gelu (GELU tanh approximation)
    • Greater
    • GreaterConstant
    • GreaterEqual
    • GreaterEqualConstant
    • Less
    • LessConstant
    • LessEqual
    • LessEqualConstant
    • Log
    • Log1p
    • LogSigmoid
    • LogSoftmax
    • LogicalAnd
    • LogicalNot
    • LogicalOr
    • LogicalXor
    • Max
    • MaxConstant
    • Min
    • MinConstant
    • Mod
    • Multiply
    • Negative
    • NotEqual
    • NotEqualConstant
    • Pow
    • Reciprocal
    • Round
    • Rsqrt
    • SafeDivide
    • SafeReciprocal
    • Scale
    • Select
    • Selu
    • Sigmoid
    • SigmoidBinaryCrossEntropy
    • Sign
    • Sin
    • Sinh
    • Softplus
    • Softsign
    • Sqrt
    • Square
    • SquareDifference
    • Subtract
    • SubtractConstant
    • Tan
    • Tanh

Data Ingestion

  • Data Ingestion
    • Legacy Data Readers
    • “New” Data Readers
    • Iterative algorithms and data ingestion
  • Sample Lists
    • Example of CONDUIT_HDF5_* sample lists
  • HDF5 Data Reader
    • HDF5 Schema files
    • Metadata Directives
    • Larger Example
  • Generating Sample lists and Schema Files for HDF5 data sets
    • Sample IDs and samples: the hdf5 file
    • Running the utility
      • Editing the YAML file
      • Data type coercion
    • Relationships between HDF5, YAML, and sample list files

Publications

  • Papers, Presentations, and Posters
    • Channel and filter parallelism for large-scale CNN training
    • Parallelizing Training of Deep Generative Models on Massive Scientific Datasets
    • Scalable Topological Data Analysis and Visualization for Evaluating Data-Driven Models in Scientific Applications
    • Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism
    • Aluminum: An Asynchronous, GPU-Aware Communication Library Optimized for Large-Scale Training of Deep Neural Networks on HPC Systems
    • Towards Scalable Parallel Training of Deep Neural Networks
    • Communication Quantization for Data-Parallel Training of Deep Neural Networks
    • Towards Scalable Parallel Training of Deep Neural Networks

Developer Documentation

  • LBANN Software Architecture and Class Overview
    • Trainers (i.e. execution environment)
    • Execution Context
    • Termination Criteria (Pending)
    • Training Algorithms
    • Model
  • LBANN API
  • LBANN Style Guide
    • In-Source Documentation
      • Documentation of Functions
      • Documentation of Classes
  • LBANN CI
    • Plan Configuration
    • Writing Your Own Tests
    • Running Tests Yourself
    • Running Tests From The Command Line
    • Helpful Files
  • LBANN Documentation Building
    • Adding Documentation Outside Code
    • Making The Build Work
LBANN
  • »
  • Index
  • Edit on GitHub

Index

_ | B | D | E | K | L | M | R | T

_

  • __init__()
    • (LTFB.StoppingCriteria method)
    • (MutationStrategy method)

B

  • BatchedIterativeOptimizer (built-in class)
  • BatchedIterativeOptimizer.StoppingCriteria (built-in class)

D

  • do_export_proto()
    • (BatchedIterativeOptimizer method)
    • (KFAC method)

E

  • export_proto()
    • (BatchedIterativeOptimizer.StoppingCriteria method)
    • (LTFB.StoppingCriteria method)
    • (MetricStrategy method)
    • (MutationStrategy method)
    • (RandomPairwiseExchange method)
    • (RandomPairwiseExchange.ExchangeStrategy method)

K

  • KFAC (built-in class)

L

  • LTFB (built-in class)
  • LTFB.StoppingCriteria (built-in class)

M

  • MetaLearningStrategy (built-in class)
  • MetricStrategy (built-in class)
  • MutationStrategy (built-in class)

R

  • RandomPairwiseExchange (built-in class)
  • RandomPairwiseExchange.ExchangeStrategy (built-in class)
  • RandomPairwiseExchange.MetricsStrategy (built-in class)
  • RegularizedEvolution (built-in class)
  • RegularizedEvolution.MetricStrategy (built-in class)

T

  • TrainingAlgorithm (built-in class)

© Copyright 2014-2021, LLNL. Revision 4ddc6c42.

Built with Sphinx using a theme provided by Read the Docs.