Exploring Network Efficiency of Distributed Model-Replicated Deep Neural Network Systems

Roy H Campbell, University of Illinois at Urbana-Champaign

Usage Details

In recent years, there has been a substantial growth of interest in deep neural networks usually referred to as Deep Learning. Deep Learning applications are computationally expensive and have to be distributed. In this proposal, we want to systematically look at two different distribution patterns and make an informed understanding of the performance difference between each distribution pattern for various applications, models, and infrastructures. The result of this study can greatly benefit future large-scale Deep Learning projects, especially those which use Blue Waters as the platform.