Operators
An operator defines a mathematical function that that supports both forward and possibly backward operations. In the forward direction, it takes a vector of input tensors and produces a vector of output tensors. In the backward direction they implement the differentiation of the forward operation, applying the function to the operator’s forward inputs and gradient with respect to the outputs, to compute the gradient with respect to the input.
Operators only store immutable state; they do not have learnable parameters. An operator should also be able to take objective function gradients w.r.t. the outputs (“previous error signals”) and compute the objective function gradients w.r.t. the inputs (“error signals”). This allows the model to perform automatic differentiation.
Operators are specified for unique input and output data types.
Operator |
Description |
---|---|
Base class for LBANN operators |
|
Apply the Abs operator entrywise |
|
Apply the Acosh operator entrywise |
|
Apply the Acos operator entrywise |
|
Apply the Add operator entrywise |
|
Add a constant to each input value (x+c) |
|
Apply the Asin operator entrywise |
|
Apply the Asinh operator entrywise |
|
Apply the Atan operator entrywise |
|
Apply the Atanh operator entrywise |
|
Apply the BinaryCrossEntropy operator entrywise |
|
Apply the BooleanAccuracy operator entrywise |
|
Apply the BooleanFalseNegative operator entrywise |
|
Apply the BooleanFalsePositive operator entrywise |
|
Apply the Ceil operator entrywise |
|
Constrain all values in a tensor within a range |
|
Subtract each input value from a constant (c-x) |
|
Apply the Cos operator entrywise |
|
Apply the Cosh operator entrywise |
|
Apply the Divide operator entrywise |
|
Apply the Equal operator entrywise |
|
Test each value for equality with a constant (x==c) |
|
Apply the Erf operator entrywise |
|
Apply the ErfInv operator entrywise |
|
Apply the Exp operator entrywise |
|
Apply the Expm1 operator entrywise |
|
Apply the Floor operator entrywise |
|
Apply the Greater operator entrywise |
|
Test each value for ‘greater-than’ with a constant (x>c) |
|
Apply the GreaterEqual operator entrywise |
|
Test each value for ‘greater-than-or-equal-to’ with a constant (x>=c) |
|
Apply the Less operator entrywise |
|
Test each value for ‘less-than’ with a constant (x<c) |
|
Apply the LessEqual operator entrywise |
|
Test each value for ‘less-than-or-equal-to’ with a constant (x<=c) |
|
Apply the Log operator entrywise |
|
Apply the Log1p operator entrywise |
|
Apply the LogSigmoid operator entrywise |
|
Apply the LogSoftmax operator entrywise |
|
Apply the LogicalAnd operator entrywise |
|
Apply the LogicalNot operator entrywise |
|
Apply the LogicalOr operator entrywise |
|
Apply the LogicalXor operator entrywise |
|
Apply the Max operator entrywise |
|
Apply the MaxConstant operator entrywise |
|
Apply the Min operator entrywise |
|
Apply the MinConstant operator entrywise |
|
Apply the Mod operator entrywise |
|
Apply the Multiply operator entrywise |
|
Apply the Log Negative entrywise |
|
Apply the NotEqual operator entrywise |
|
Test each value for inequality with a constant (x!=c) |
|
Apply the Pow operator entrywise |
|
Apply the Reciprocal operator entrywise |
|
Apply the Round operator entrywise |
|
Apply the Rsqrt operator entrywise |
|
Apply the SafeDivide operator entrywise |
|
Apply the SafeReciprocal operator entrywise |
|
Scale each input value by a constant value (c*x) |
|
Apply the Selu operator entrywise |
|
Apply the Sigmoid operator entrywise |
|
Apply the SigmoidBinaryCrossEntropy operator entrywise. |
|
Apply the Sign operator entrywise |
|
Apply the Sin operator entrywise |
|
Apply the Sinh operator entrywise |
|
Apply the Softplus operator entrywise |
|
Apply the Softsign operator entrywise |
|
Apply the Sqrt operator entrywise |
|
Apply the Square operator entrywise |
|
Apply the SquareDifference operator entrywise |
|
Apply the Subtract operator entrywise |
|
Apply the SubtractConstant operator entrywise |
|
Apply the Tan operator entrywise |
|
Apply the Tanh operator entrywise |
Operator
Operator is the base class for LBANN operators
Arguments:
- input_type
(
lbann.DataType
) The type expected as input- output_type
(
lbann.DataType
) The type expected as output- device
(
lbann.device_allocation
) The device allocation
Methods:
- export_proto()
Get a protobuf representation of this object
- do_export_proto()
Get a protobuf representation of this object
Must be implemented in derived classes
Abs
Perform entrywise absolute value on the input tensor.
Acosh
Apply the inverse hyperbolic cosine entrywise.
Acos
Apply the inverse cosine function entrywise.
Add
Perform entrywise addition on two input tensors.
AddConstant
Add a constant to each input value.
Arguments:
- constant
(
double
) The constant to be added
Asin
Apply the inverse sine function entrywise.
Asinh
Apply the hyperbolic inverse sine function entrywise.
Atan
Apply the inverse tangent function entrywise.
Atanh
Apply the hyperbolic inverse tangent function entrywise.
BinaryCrossEntropy
Apply the BinaryCrossEntropy operator entrywise.
Compare each predicted probability to actual class value, either 0 or 1. Calculate the score that penalizes the probabilities based on the distance from the expected value.
BooleanAccuracy
Apply the BooleanAccuracy operator entrywise.
Applies the function:
BooleanFalseNegative
Apply the BooleanFalseNegative operator entrywise.
BooleanFalsePositive
Apply the BooleanFalsePositive operator entrywise.
Ceil
Apply the ceiling function to an input tensor entrywise.
Clamp
Constrain all values in a tensor within a range
Arguments:
- min
(
double
) Minimum value in range- max
(
double
) Maximum value in range
ConstantSubtract
Subtract each input value from a constant.
Arguments:
- constant
(
double
) The constant to subtract from
Cos
Compute the cosine of the input tensor entrywise.
Cosh
Compute the hyperbolic cosine of the input tensor entrywise.
Divide
Perform entrywise division on two input tensors.
Equal
Perform entrywise logical equal on two input tensors.
EqualConstant
Perform entrywise logical equal on input tensor and a constant.
Arguments:
- constant
(
double
) The constant used for comparison
Erf
Compute the error function of the inpute tensor entrywise.
ErfInv
Compute the inverse error function entrywise.
Exp
Calculate the exponential of the input tensor entrywise.
Expm1
Calculate the exponential minus one of the input tensor entrywise.
Floor
Apply the floor function to the input tensor entrywise.
Greater
Perform entrywise logical ‘greater’ on two input tensors.
GreaterConstant
Perform entrywise logical ‘greater-than’ on input tensor and a constant.
Arguments:
- constant
(
double
) The constant to be used for comparison
GreaterEqual
Perform entrywise logical ‘greater-or-equal’ on two input tensors.
GreaterEqualConstant
Perform entrywise logical ‘greater-or-equal’ on input tensor and a constant.
Arguments:
- constant
(
double
) The constant to be used for comparison
Less
Perform entrywise logical ‘less-than’ on two input tensors.
LessConstant
Perform entrywise logical ‘less-than’ on input tensor and a constant.
Arguments:
- constant
(
double
) The constant to be used for comparison
LessEqual
Perform entrywise logical ‘less-equal’ on two input tensors.
LessEqualConstant
Perform entrywise logical ‘less-or-equal’ on input tensor and a constant.
Arguments:
- constant
(
double
) The constant to be used for comparison
Log
Calculate the log of the input tensor entrywise.
Log1p
Calculate the log of one plus the input tensor entrywise.
LogSigmoid
Calculate the log of the output from the sigmoid function entrywise.
LogSoftmax
Calculate the log of the softmax function entrywise.
LogicalAnd
Perform entrywise logical ‘and’ on two input tensors.
LogicalNot
Perform entrywise logical ‘not’ on two input tensors.
LogicalOr
Perform entrywise logical ‘or’ on two input tensors.
LogicalXor
Perform entrywise logical ‘xor’ on two input tensors.
Max
Perform entrywise max of input tensors.
MaxConstant
Perform entrywise max of input tensor against a constant.
Min
Perform entrywise min of input tensors.
MinConstant
Perform entrywise min of input tensor against a constant.
Mod
Perform entrywise modulus on two input tensors.
Multiply
Perform entrywise multiplication on input tensors.
Negative
Produce output tensor with flipped sign.
NotEqual
Perform entrywise logical ‘not-equal’ on two input tensors.
NotEqualConstant
Perform entrywise logical ‘not-equal’ on input tensor and a constant.
Arguments:
- constant
(
double
) The constant to be used for comparison
Pow
Perform entrywise exponent using one input tensor as the base and a second input tensor as the exponent.
Reciprocal
Perform entrywise reciprocal function on input tensor.
Round
Round input tensor values to the nearest integer entrywise.
Rsqrt
Compute reciprocal of square-root of values in the input tensor entrywise.
SafeDivide
FIXME: Is this right?
Perform entrywise division on two input tensors. Return zero if the divisor is zero.
SafeReciprocal
FIXME: Is this right?
Perform entrywise reciprocal function on input tensor. Return zero if the input value is zero.
Scale
Scale each input value by a constant.
Arguments:
- constant
(
double
) The constant to scale by
Selu
Apply scaled exponential linear unit function to input tensor entrywise.
Sigmoid
Apply the sigmoid function to the input tensor entrywise.
SigmoidBinaryCrossEntropy
FIXME: Better description of this?
Apply the SigmoidBinaryCrossEntropy operator entrywise.
Sign
FIXME: Is this output right?
Compute the sign of the imput tensor entrywise. If input > 0, output 1. if input < 0, output -1. if input == 0, output 0.
Sin
Calculate entrywise sine of the input tensor.
Sinh
Calculate entrywise hyperbolic sine of the input tensor.
Softplus
Calculate the softplus of the input tensor entrywise.
Softsign
Calculate the softsign of the input tensor entrywise.
Sqrt
Compute square root of input tensor values entrywise.
Square
Compute square of input tensor values entrywise.
SquareDifference
FIXME: Better description?
Apply the SquareDifference operator entrywise
Subtract
Perform entrywise subtraction on two input tensors.
SubtractConstant
Subtract a constant from from the input tensor entrywise.
Arguments:
- constant
(
double
) The constant to subtract
Tan
Apply the tangent function entrywise.
Tanh
Apply the hyperbolic tangent function entrywise.