Kronecker-Factored Approximate Curvature (KFAC)

The Kronecker-Factored Approximate Curvature (KFAC) algorithm is an approach that uses a second-order optimization method. It shows similar capabilies to first-order methods, but converges faster and can handle larger mini-batches. See Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks.

Python Front-end API Documentation

class KFAC(TrainingAlgorithm)

Kronecker-Factored Approximate Curvature algorithm.

Applies second-order information to improve the quality of gradients in SGD-like optimizers.

__init__(name: str, first_order_optimizer:

BatchedIterativeOptimizer, **kfac_args)

Construct a new KFAC algorithm.

Parameters

name (string) – A user-defined name to identify this object in logs.
first_order_optimizer (BatchedIterativeOptimizer) – The SGD-like algorithm to apply.
**kfac_args – See KFAC message for a list of kwargs.

do_export_proto()

Get a protobuf representation of this object.