Neural Network Reduction for Efficient Execution on Edge Devices

Abstract

As the size of neural networks increase, the resources needed to support their execution also increase. This presents a barrier for creating neural networks that can be trained and executed within resource limited embedded systems. To reduce the resources needed to execute neural networks, weight reduction is often the first target. A network that has been significantly pruned can be executed on-chip, that is, in low SWaP hardware. But, this does not enable either training or pruning in embedded hardware which first requires a full-sized network to fit within the restricted resources. We introduce two methods of network reduction that allows neural networks to be grown and trained within edge devices, Artificial Neurogenesis and Synaptic Input Consolidation.