Redundancy and robustness in deep learning with applications to digital communications and magnetic resonance imaging

Compressed representations are a fundamental building block in signal processing algorithms, whether the downstream applications involve data storage, reconstruction from undersampled measurements, or sampling from structured high-dimensional distributions. This dissertation proposes the use of compression - in conjunction with modern deep learning tools - to achieve robust downstream performance in the following applications: robust compression and numerical quantization in digital communication systems, robust wireless channel estimation in digital communication systems, and robust magnetic resonance imaging reconstruction from undersampled measurements. The choice to investigate magnetic resonance imaging (MRI) in the same dissertation as digital communication systems may seem arbitrary, but it is not. For the past two decades, clinical high-field MRI scanners operate using the principles of parallel imaging by acquiring undersampled electromagnetic field measurements using a set of sensitivity coils placed around the target anatomy (brain, knee, abdomen, etc.). This has led to the formulation of MRI image reconstruction as an inverse problems, and to seminal research papers that use compressed sensing to solve it. Incidentally, this is also akin to a single-input multiple-input (SIMO) digital communication system, leading to a large overlap of signal processing and optimization algorithms being reused between the two fields. Representing information in digital communication systems using as few bits as possible is a fundamental requirement for power-efficient devices that either have limited memory or communication bandwidth. Storing or forwarding such compressed information is a requirement that arises in different aspects of digital communication: soft bit storage is important in re-transmission request protocols, where the receiver wants to hold on to as much information as possible from failed transmissions, while forwarding soft bits to a third-party using a finite communication budget is an essential component in fronthaul and relay communications. In the first part of this dissertation, I introduce my work on deep learning for soft bit quantization in high-dimensional communication systems. The approach builds on a fundamental observation concerning the sufficient feature representation of the vector of maximum-likelihood (ML) soft bits derived from a scalar channel use, and extends this to multi-carrier and arbitrary multiple-input multiple-output (MIMO) channels. A closely related problem is that of soft bit estimation, which has two components: (i) developing algorithms that can efficiently approximate the ML solution and (ii) accurately estimating channel state information in high dimensional systems. I present data-driven methods to address both these problems. For the first, I connect soft bit compression from the previous paragraph with soft bit estimation, and show that a feature learning approach can lead to efficient, high-fidelity estimation that is on par with supervised approaches. For the second problem, I propose a broader solution that uses a score-based generative model to learn the distribution of wireless channels from synthetic data. This model can be then used for downstream tasks, such as sampling and estimation, where we show robust performance under severe environment changes in synthetic scenarios, with reductions of more than an order of magnitude in estimation error compared to competing deep learning approaches. To address the problem of robust, undersampled MRI reconstruction, the final chapter of this dissertation introduces two deep learning methods. The Deep J-Sense approach combines iterative optimization with deep learning to learn reconstructions that are robust to measurement patterns. The key insight used to achieve this is inspired by previous work that treats the coil sensitivity profiles as dynamic optimization variables. In a different line of research, I have contributed to the use of score-based generative models for learning the distribution of MRI images, and using this for robust downstream reconstruction under anatomy and measurement pattern shifts, as well as introducing efficient, single-shot adaptation methods under these shifts. Altogether, these methods have tried to touch on different aspects and tasks that are present in high-dimensional systems (whether their role is for communication or imaging), and propose solutions that increase efficiency - whether related to storage budget, resource overhead, energy consumption, or human discomfort.