RTL design and analysis of Softmax Layer in Deep Neural Networks

Deep neural networks (DNNs) are widely used in modern machine learning systems in the big data era for their superior accuracy. These artificial neural networks suffer from high computational complexity. The structure of DNN layers vary depending on the nature of training and inference tasks. Softmax Layer is a critical layer in DNNs and is usually used as the output layer in multi-category classification tasks. Softmax layer involves exponentiation and division, thereby resulting in high computational complexity and long critical paths. This report focuses on frontend implementation of an efficient microarchitecture of Softmax layer, which tries to address some of the problems associated with a simple, direct implementation. Techniques like pipelining are employed to boost the performance of the complex datapath logic. Error analysis of the hardware is performed with software results from MATLAB. Synthesis of the RTL code is performed on Xilinx Artix-7 FPGA, resulting in a clock frequency of 274.3 MHz.