Browsing by Subject "Deep neural network"

Now showing 1 - 2 of 2

Estimating the minimum bit-width precision for stable deep neural networks utilizing numerical linear algebra
(2019-06-20) Maheshwari, Naman; Kulkarni, Jaydeep P.
Understanding the bit-width precision is critical in compact representation of a Deep Neural Network (DNN) model with minimal degradation in the inference accuracy. While DNNs are resilient to small errors and noise as pointed out by many prior sources, there is a need to develop a generic mathematical framework for evaluating a given DNN’s sensitivity to input bit-width precision. In this work, we derive a bit-width precision estimator which incorporates the sensitivity of DNN inference accuracy to round-off errors, noise, or other perturbations in inputs. We use the tools of numerical linear algebra, particularly stability analysis, to establish the general bounds that can be imposed on the precision. Random perturbations and ‘worst-case’ perturbations, via adversarial attacks, are applied to determine the tightness of the proposed estimator. The experimental results on AlexNet and VGG-19 showed that minimum 11 bits of input bit-width precision is required for these networks to remain stable. The proposed bit-width precision estimator can enable compact yet highly accurate DNN implementations
Towards power-efficient and intelligent wireless communication systems
(2023-06-15) Cho, Yunseong; Evans, Brian L. (Brian Lawrence), 1965-; Andrews, Jeffrey G; Hanasusanto, Grani A; Kim, Hyeji; Mokhtari, Aryan
With the growing demand for higher date rates and more reliable service capabilities, wireless communication systems continue to grow in popularity and importance. In order to enable higher data rate via broader bandwidth, millimeter wave (mmWave) systems are deployed for modern and future communication systems. Due to the high transmission loss of the mmWave frequency bands, a massive number of antennas are employed to focus transmitted power in narrow radio frequency (RF) beams. However, associating one RF chain with two high-resolution data converters for each antenna element would consume a prohibitively large amount of power. Furthermore, challenging service requirements can be handled by machine learning techniques in a variety of application spaces. The goal of this dissertation is to propose communication systems that are not only reliable and high-performing, but also power-efficient as well as intelligent. Two possible ways to alleviate the huge power consumption problem are 1) low-resolution data converters, and 2) hybrid analog-digital beamforming architectures since the former tries to reduce the power consumption of each individual RF chain and the latter directly scales down the number of RF chains. Additionally, intelligent communication systems that can adapt to changing network conditions and user requirements are crucial for ensuring reliable and efficient communication. In either case, these solutions introduce severe non-convexity and non-linearity to the entire system. In this regard, I propose new solutions that can respond to future communication systems requiring a fundamental re-design of current communication systems based on a power-efficient and intelligent framework. First, I investigate a coordinated multipoint (CoMP) beamforming and power control problem for base stations (BSs) with a massive number of antenna arrays under coarse quantization by low-resolution analog-to-digital converters (ADCs) and digital-to-analog converters (DACs). I first formulate total power minimization problems of both uplink (UL) and downlink (DL) systems subject to signal-to-quantization-plue-interference-and-noise ratio (SQINR) constraints. I then show strong duality for the UL and DL problems under the coarse quantization condition when channel reciprocity holds with time-division duplexing (TDD) assumption. Leveraging the duality, I propose a framework that is directed toward a twofold aim: to discover the optimal transmit powers in UL by developing iterative algorithm in a distributed manner and to obtain the optimal precoder in DL as a scaled instance of UL combiner. Under homogeneous transmit power and SQINR constraints per cell, I further derive a deterministic solution for the UL CoMP problem by analyzing the lower bound of the SQINR. Lastly, I extend the derived result to wideband orthogonal frequency-division multiplexing (OFDM) systems to optimize transmit power and beamformer for all subcarriers. Simulation results validate the theoretical results and proposed algorithms in terms of total transmit power, duality gap, and convergence. Second, I aim to find the DL beamformer that minimizes the maximum power on transmit antenna array of each BS under received SQINR constraints while minimizing per-antenna transmit power for a more realistic deployment. I first formulate formulating the quantized DL OFDM antenna power minimax problem and deriving its associated dual problem. With proving strong duality, I use the associated UL dual solution to compute the DL beamformer. Subsequently, the DL beamformer is used in updating the covariance matrix of the uplink noise signals. The series of processes builds an efficient algorithm to find a numerical solution. Simulations validate the proposed algorithm in terms of the maximum antenna transmit power and peak-to-average-power ratio. Third, I propose a learning-based maximum likelihood detection framework with an acceptable learning length for uplink massive multiple-input-multiple-output (MIMO) systems with one-bit ADCs. The learning-based detection only requires counting the occurrences of the quantized outputs at each antenna. The learning in the high signal-to-noise ratio (SNR) regime, however, needs excessive training to estimate the extremely small likelihood probabilities. To address this drawback, I utilize a dithering signal to artificially decrease the SNR and then remove the impact of the dithering noise via post processing. I evolve the technique by developing an adaptive dither-and-learning method that updates the dithering power according the patterns observed in the quantized dithered signals. Lastly, the computed likelihood probabilities are utilized in deriving log-likelihood ratio to enable state-of-the-art channel coding schemes. I compare the uncoded and coded detection performance of the proposed algorithm with other learning-based frameworks and show that the proposed algorithm shows the performance closest to optimal performance. Fourth, I propose a deep reinforcement learning (DRL)-based solution for joint hybrid beamforming (HB) and power control problems when multiple massive MIMO BSs are communicating with multiple users in the uplink mmWave band. The HB method requires both digital and analog beamformers, with the latter using discrete phase shifters to project high-dimensional antenna ports to low-dimensional logical ports and scale down the number of RF chains. However, this results in non-convexity, making the problem difficult to solve using existing algorithms. In multicell uplink communication systems, I aim to jointly design the HB at each BS and transmit power control of the associated users while ensuring that the received signal-to-interference-and-noise ratio (SINR) constraints are satisfied. Considering the use of the DRL-based approach and the primal problem, I formulate the RL basics. To handle the combination of discrete and continuous inputs, I use the DDPG RL algorithm, which outputs a valid action that maps to the design factors. In particular, I aim to control each phase shifter individually by introducing an intermediate vector and applying a differentiable argmax function to estimate the phase angle index. The proposed method is evaluated through simulation results based on the achieved SINR. The four contributions could make a worthwhile enhancement to the development of power-efficient and intelligent wireless communication systems by meeting the communication needs of modern society while minimizing energy consumption and maximizing the use of available resources.