Summarize Images Callback

The purpose of this callback is to output images into an event file at the end of each epoch, according to the specified intervals. The images in the event file are displayed using Tensorboard. This callback could be used, for example, to display categorized images or images generated by an autoencoder or by a GAN.

The method of selecting images, and the layers from which images are displayed, can be controlled via the selection_strategy argument to the callback. Images that match some boolean value may be selected with CategoricalAccuracyStrategy. A canonical example of this would be to output images that are classified incorrectly by a classification network. Alternatively, a fixed number of images can be displayed using TrackSampleIDsStrategy. This may be used, for example, to visualize the progress in training a GAN or an autoencoder.

Execution Points

• After each testing/validation minibatch

Callback Arguments (Python Front-End)

• selection_strategy: The image selection strategy. Currently supported options are:

  • TrackSampleIDsStrategy

  • CategoricalAccuracyStrategy

• image_source_layer_name: The name of the layer from which images will be pulled. A Python Front-End layer’s name can be accessed via the name attribute. This may be the input layer, if the true image is requested, or it may be any layer that outputs a valid image tensor. This means it must be either a 2-D tensor (greyscale image) or a 3-D tensor with the channel dimension equal to 1 or 3 (greyscale or RGB, respectively).

• epoch_interval: Epoch frequency to output images. The default value is 1; that is, perform the output every epoch.

Examples Using Summarize Images Callback

Python Front-End

Track Sample IDs Strategy

Note

There is currenly no built-in way to print the original images using a single callback instance. As a work-around, if the original image is desired, add a second instance of the CallbackSummarizeImages with the image_source_layer_name field set to the input layer’s name and the epoch_interval field set to be larger than the total number of epochs you expect to run (so it will only output from epoch 0 and never again).

# Set up image selection strategy
img_strategy = lbann.TrackSampleIDsStrategy(
                 input_layer_name="input",
                 num_tracked_images=10)

# Pass parameters to callback
summarize_images = lbann.CallbackSummarizeImages(
                     selection_strategy=img_strategy,
                     image_source_layer_name="reconstruction",
                     epoch_interval=5)

# Optional- Output original image from input layer once using
#           a high epoch interval
summarize_input_layer = lbann.CallbackSummarizeImages(
                          selection_strategy=img_strategy,
                          image_source_layer_name="input",
                          epoch_interval=10000)

Categorical Accuracy Strategy

# Set up categorical accuracy layer
accuracy = lbann.CategoricalAccuracy(prediction_scores, labels)

# Set up image selection criteria
match_type = lbann.CategoricalAccuracyStrategy.MatchType

# Set up image selection strategy
img_strategy = lbann.CategoricalAccuracyStrategy(
                 cat_accuracy_layer_name=accuracy.name,
                 match_type.NOMATCH,
                 num_images=10)

# Pass parameters to callback
summarize_images = lbann.CallbackSummarizeImages(
                     selection_strategy=img_strategy,
                     image_source_layer_name=images.name,
                     epoch_interval=5)

Profotext (Advanced)

Track Sample IDs

callback {
  summarize_images {
    selection_strategy {
      track_sample_ids {
        input_layer_name: "input"
        num_tracked_images: 10
      }
      image_source_layer_name: "reconstruction"
      epoch_interval: 1
    }
  }
}

Categorical Accuracy Strategy

# Set up categorical accuracy layer
layer {
 parents: "prob"
 parents: "label"
 name: "accuracy"
 data_layout: "data_parallel"
 categorical_accuracy {}
}

# Set up callback
callback {
  summarize_images {
    selection_strategy {
      categorical_accuracy {
        cat_accuracy_layer_name: "accuracy"
        num_images: 10
      }
      image_source_layer_name: "images"
      epoch_interval: 1
      img_format: ".jpg"
    }
  }
}