Papers, Presentations, and Posters

Publications about or related to using LBANN:

• Sam Ade Jacobs, Tim Moon, Kevin McLoughlin, Derek Jones, David Hysom, Dong H. Ahn, John Gyllenhaal, Pythagoras Watson, Felice C. Lightstone, Jonathan E. Allen, Ian Karlin, Brian Van Essen. “Enabling Rapid COVID-19 Small Molecule Drug Design Through Scalable Deep Learning of Generative Models”, to appear as finalist for the 2020 Gordon Bell Special Prize in the The International Journal of High Performance Computing Applications (IJHPCA), Nov. 2020.

• Arpan Jain, Tim Moon, Tom Benson, Hari Subramoni, Sam Adé Jacobs, Dhabaleswar K Panda, Brian Van Essen. “SUPER: SUb-Graph Parallelism for TransformERs”, in submission to 35th IEEE International Parallel & Distributed Processing Symposium (IPDPS), May 17-21, 2021.

• Yosuke Oyama, Naoya Maruyama, Nikoli Dryden, Erin McCarthy, Peter Harrington, Jan Balewski, Satoshi Matsuoka, Peter Nugent, Brian Van Essen. “The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism”, under review for Special Session on Parallel and Distributed Computing Techniques for AI, ML and DL in Transactions on Parallel and Distributed Systems, July 2020.

  • arXiv.org/abs/2007.12856

• Nikoli Dryden, Naoya Maruyama, Tom Benson, Tim Moon, Marc Snir, Brian Van Essen. “Channel and Filter Parallelism for Large-Scale CNN Training”, in SC ‘19: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, November 2019 Article No. 10, Pages 1-20, DOI: 10.1145/3295500.3356207.

  • BibTex

• Sam Ade Jacobs, Brian Van Essen, David Hysom, Jae Seung Yeom, Tim Moon, Rushil Anirudh, Jayaraman Thiagaranjan, Shusen Liu, Jim Gaffney, Peer-Timo Bremer, Tom Benson, Peter Robinson, and Luc Peterson, Brian Spears, “Parallelizing Training of Deep Generative Models on Massive Scientific Datasets”, in *2019 IEEE International Conference on Cluster Computing (CLUSTER)*, Albuquerque, NM, USA, 2019, pp. 1-10, DOI: 10.1109/CLUSTER.2019.8891012.

  • BibTex

  • arXiv.org/abs/1910.02270

• Shusen Liu, Di Wang, Dan Maljovec, Rushil Anirudh, Jayaraman J. Thiagarajan, Sam Ade Jacobs, Brian C. Van Essen, David Hysom, Jae-Seung Yeom, Jim Gaffney, Luc Peterson, Peter B. Robinson, Harsh Bhatia, Valerio Pascucci, Brian K. Spears, Peer-Timo Bremer. “Scalable Topological Data Analysis and Visualization for Evaluating Data-Driven Models in Scientific Applications”, in IEEE Transactions on Visualization and Computer Graphics, 2019.

  • BibTex

  • arXiv.org/abs/1907.08325

• Nikoli Dryden, Naoya Maruyama, Tom Benson, Tim Moon, Marc Snir, Brian Van Essen. “Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism”, in *Proceedings of IEEE International Parallel & Distributed Processing Symposium*, 2019.

  • BibTex

  • arXiv.org/abs/1903.06681

• Nikoli Dryden, Naoya Maruyama, Tim Moon, Tom Benson, Andy Yoo, Marc Snir, Brian Van Essen. “Aluminum: An Asynchronous, GPU-Aware Communication Library Optimized for Large-Scale Training of Deep Neural Networks on HPC Systems”, in *Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments (MLHPC ‘18)*, Nov. 2018. DOI: 10.1109/MLHPC.2018.8638639

  • BibTex

• Sam Ade Jacobs, Nikoli Dryden, Roger Pearce, and Brian Van Essen. “Towards Scalable Parallel Training of Deep Neural Networks”, in *Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments (MLHPC ‘17)*, pages 1-8, Nov. 2017. DOI: 10.1145/3146347.3146353

  • BibTex

• Nikoli Dryden, Sam Ade Jacobs, Tim Moon, Brian Van Essen. “Communication quantization for data-parallel training of deep neural networks”, in *Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments (MLHPC ‘16)*, pages 1-8, Nov. 2016. DOI: 10.1109/MLHPC.2016.004

  • BibTex

• Brian Van Essen, Hyojin Kim, Roger Pearce, Kofi Boakye, Barry Chen. “LBANN: Livermore Big Artificial Neural Network HPC Toolkit”, in *Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments (MLHPC ‘15)*, pages 5:1-6, Nov. 2015. DOI: 10.1145/2834892.2834897

  • BibTex

Presentations highlighting LBANN and its impact on science applications:

Note

Presentations and links to be added

Posters about LBANN and its core algorthms and features:

Note

Posters and links to be added

• Nikoli Dryden, Naoya Maruyama, Tom Benson, Tim Moon, Marc Snir, Brian Van Essen. “Scalable CNN Training on Large-Scale HPC Systems” in Proceedings of the Workshop on Systems for ML and Open Source Software at NeurIPS 2018, December 7, 2018. abs

  • Systems for ML 2018