Browsing by Subject "Convolutional neural networks"
Now showing 1 - 6 of 6
- Results Per Page
- Sort Options
Item Exploring protein biochemistry with deep learning(2023-12) Kulikova, Anastasiya Vitalievna; Wilke, C. (Claus); Davies, Bryan W.; Klivans, Adam R.; Russell, RickDeep learning has become widely used in biological sciences. More specifically, the development of protein deep learning models has leveraged the evergrowing collection of biological data to learn the patterns that govern protein biochemistry. Here, we focus on the assessment of different protein deep learning models to better understand each of their capabilities, benefits and drawbacks. Our work aims to provide insights for future protein engineering efforts and for the discovery of protein homologs. In Chapter 2 we assessed a structure-based protein ML model in its ability to make biochemically meaningful predictions and tested weather or not the model can predict specific allowed amino acids in a protein. We compared the performance of models trained on different input sizes and correlated model predictions with natural variation in order to better understand how these models learn protein structure and biochemistry. In Chapter 3, we compared the predictions of two structure models and two language models to determine if different protein representations affect what information each model type learns and their performance. Finally, in Chapter 4, we apply a sequence-based protein model to searching for antibacterial microcin peptides in bacterial genomes.Item Machine learning applications for porous media(2021-12-09) Estrada Santos, Javier Andres; Prodanovic, Masa; Pyrcz, Michael; DiCarlo, David; Lake, Larry; Lubbers, NicholasUnderstanding how fluids flow through permeable structures in the subsurface is paramount in the design and execution of projects for hydrocarbon extraction, CO₂ sequestration, and contaminant tracing in aquifers. The most important processes that impact fluid behavior of a field happen at the pore-scale. Therefore, these will be studied in detail throughout this dissertation. In particular, the main focus of this work is to estimate the permeability of 3D volumes. The permeability is arguably the most important property of a subsurface formation since it describes the ease for fluids to travel through a porous domain of interest. This, in turn, will control how fast a CO₂ plume will migrate, how much oil can be recovered from a reservoir, or when will a contaminant reach a certain region, among other relevant applications. This quantity can be computed via direct flow simulation, which provides accurate results, nevertheless, it is computationally very expensive. In particular, the simulation convergence time scales poorly as the domains become less porous or more heterogeneous. On the other hand, semi-analytical models that rely on averaged structural properties (i.e., porosity and tortuosity) have been proposed, but these properties only partly describe the domain, resulting in limited generalizability of these models. Alternatively, machine learning models have shown to be effective tools for finding complex relationships in structured data. These models have had great success at otherwise difficult tasks like natural language processing, video frame prediction, and semantic segmentation of natural scenes in real time. Nevertheless, the pore-scale world has remained vastly unexplored due to the the great effort needed to create labeled datasets, the complexity of obtaining and processing data, and the size of the 3D data arrays that concern subsurface applications. Having said that, the primary goal of this work is to show how to label datasets selectively, how to train models in cases where only a limited data is available, how to overcome the memory bottleneck of hardware, and how to bridge information from multiple scales. The goal of the models proposed in here is not to replace simulations or laboratory experiments, but to complement them. Throughout this dissertation, it is shown how a trained model can carry-out accurate predictions in a variety of domains. These models can provide good approximations, and can be specially useful when exploring a new reservoir, when tuning a segmentation, as initial conditions for a physics simulations, or when high performance computing resources are not available. It is our hope that the work presented here can be employed to build better and more accurate machine learning models that can advance the field of real-time data-driven alternatives for digital rock applicationsItem Machine learning-based uncertainty models for reservoir property prediction and forecasting(2023-04-19) Maldonado Cruz, Eduardo; Pyrcz, Michael; Lake, Larry W; Foster, John T; Sepehrnoori, Kamy; Isebor, Obiajulu JIn subsurface data analytics and machine learning, advances enable new methods and workflows for spatio-temporal, geoscience, and engineering property estimation and forecasting. These advances allow new and detailed models that contribute to field development planning cycles, such as reservoir modeling, volumetric assessment, pre-drill uncertainty, and production allocation. Uncertainty is caused by incomplete information and a lack of knowledge about continuous or discrete features. The presence of data uncertainty resulting from measurement errors, recording, data processing, sampling bias, and sample heterogeneity further exacerbates the need for reliable and interpretable uncertainty models, which are essential for effective subsurface prediction and forecasting. Therefore, developing robust, accurate, and precise models that provide the best possible estimates and account for the associated uncertainties is critical to improve decision-making. This research aims to develop innovative workflows to validate uncertainty models, predict properties with uncertainty, and create interpretable forecasting models. This research presents the following subjects: (1) The evaluation of machine learning-based uncertainty models, (2) the use of a deep convolutional encoder-decoder network to generate ensemble predictions and evaluate the uncertainty based on development parameters and geological information, (3) the use of virtual ensembles in gradient-boosted decision trees for well-log imputation, and (4) model interpretability in well performance forecasting with temporal fusion transformers. The developed workflows offer a reliable and understandable approach to uncertainty models critical for subsurface resource modeling and forecasting.Item Modeling of fluid imbibition and chemical tracer transport in porous media for oil recovery applications(2023-08-11) Velasco Lozano, Moises; Balhoff, Matthew; Pope, Gary; Delshad, Mojdeh; Pyrcz, Michael; Javadpour, FarzamModeling of fluid and solute transport in porous media is fundamental to describing driving mechanisms of recovery methods before their field application, however, conventional simulations and experiments demand time and expertise. Therefore, this research work presents novel real-time solutions for spontaneous imbibition (SI) and chemical tracer transport in porous media for two-phase flow. Although imbibition tests are critical to evaluating the displacement of oil by water and chemical solutions, the existing models fail to properly estimate the entire imbibition process. Therefore, a new semi-analytical solution for SI, valid during the infinite-acting and boundary-dominated regimes, was derived. The solution was validated with experimental data for different flow geometries under diverse flow conditions and capillary pressure functions, obtaining differences of less than 5%. Additionally, a numerical model is presented to examine SI in cores with a discrete fracture by including a new transfer function in the fracture equation to account for the fluid exchange at the matrix-fracture boundary. As a result, the flow model is reduced to a one-dimensional equation that is numerically solved using finite differences, leading to the accurate and rapid modeling of fluid displacement, obtaining results comparable to two-dimensional simulations. In addition, first-ever solutions are presented for the modeling of chemical tracer transport in two-phase flow in capillary- and advective-dominated systems at core scale, accounting for hydrodynamic dispersion, partitioning, and adsorption. These novel solutions are derived using Laplace transform and a series of transformation variables that simplify the highly nonlinear advection-dispersion equation, resulting in real-time analysis with simple mathematical expressions that do not require complex numerical calculations or inversion methods. Finally, a convolutional neural network is developed to estimate residual oil saturation based on the generation of partitioning tracer responses as a function of ideal tracer profiles, where the results obtained demonstrate that this machine learning method serves as a complementary tool to significantly reduce the number of reservoir simulations. Thus, the models described in this work are innovative approaches that facilitate the analysis of fluid and tracer dynamics at core and field scales for oil recovery and subsurface applications.Item Perceiving pixels and bits : perceptual optimization of image and video encoding pipelines(2022-05-09) Chen, Li-heng, 1989-; Bovik, Alan C. (Alan Conrad), 1958-; Ghosh, Joydeep; Tamir, Jon; Geisler, Wilson S.; Bampis, Christos G.The use of ℓ [subscript p] norms has largely dominated the measurement of distortion in video encoding or loss in neural networks due to their simplicity and analytical properties. However, when used to assess the loss of visual information, these simple norms are not very consistent with human perception. Given the continuously growing demand for online videos, improving the performance of video compression in perceptual ways has become an important, yet challenging problem, as humans are the ultimate receiver of visual signals. The main contribution of this thesis is to provide new directions for optimization of components in video workflows, in which the topics of hybrid video codecs, resizer, and learned image compression models are covered. The first part of this thesis studies the chroma distortions in conventional video compression standards. It is empirically known that the chroma components are less sensitive to human perception, yet has not been studied as much in the application in video compression. To this end, we carried out a subjective experiment to understand the interplay between luma and chroma distortions. We also found that there is room for reducing bitrate consumption in modern video codecs by creatively increasing the compression factor on chroma channels. On the other hand, video downsampling is also a crucial module in adaptive streaming scenarios. This thesis introduces a new data-driven downsampling model realized using deep neural networks. Since the layers of convolutional neural networks can only be used to alter the resolutions of their inputs by integer scale factors, we seek new ways to achieve fractional scaling, which is crucial in many video processing applications. The second part of this thesis explores the perceptual aspect of optimizing learning-based lossy image compression models. Although numerous powerful perceptual models have been proposed to predict the perceived quality of a distorted picture, most other image quality indexes have never been adopted as deep network loss functions, because they are generally non-differentiable. To address this problem, we propose a new "proximal" approach, called the ProxIQA training, to optimize image analysis networks against quantitative perceptual models. We also describe a search-free resizing framework that can further improve the rate-distortion tradeoff of recent learned image compression models. Our approach is simple: compose a pair of differentiable downsampling/upsampling layers that sandwich a neural compression model. To determine resize factors for different inputs, we utilize another neural network jointly trained with the compression model, with the end goal of minimizing the rate-distortion objective. Among these, quantitative simulations and subjective quality studies show that the proposed methods yield significant improvements in coding efficiency. The thesis concludes with some remarks on future directions and open problems.Item Using convolutional neural networks to improve branch prediction(2022-08-12) Zangeneh Kamali, Siavash; Patt, Yale N.; Erez, Mattan; Gerstlauer, Andreas; Lin, Calvin; Yeh, Tse-YuThe state-of-the-art branch predictor, TAGE, remains inefficient at identifying correlated branches deep in a noisy global branch history. This dissertation argues this inefficiency is a fundamental limitation of runtime branch prediction and not a coincidental artifact due to the design of TAGE. To further improve branch prediction, we need to relax the constraint of runtime only training and adopt more sophisticated prediction mechanisms. To this end, I propose using convolutional neural networks (CNNs) that are trained at compile-time to accurately predict branches that TAGE cannot. Given enough profiling coverage, CNNs learn input-independent branch correlations that can accurately predict branches when running a program with unseen inputs. I describe two practical approaches for using CNNs. First, I build on the work of Tarsa et al. and introduce BranchNet, a CNN with a storage-efficient on-chip inference engine tailored to the needs of branch prediction. At runtime, BranchNet predicts a few hard-to-predict branches, while TAGE-SC-L predicts the remaining branches. This hybrid approach reduces the MPKI of SPEC2017 Integer benchmarks by 9.6% (and up to 17.7%) compared to a 64KB TAGE-SC-L without increasing the prediction latency. Alternatively, instead of using BranchNet as a black-box predictor, I use it to explicitly identify correlated branches and filter the global branch history of TAGE to include only the outcomes of correlated branches. Filtering the branch history leads to less allocation pressure and faster warmup time in TAGE, resulting in improved prediction accuracy and better storage-efficiency. Filtering TAGE histories achieves a notable fraction of BranchNet's accuracy improvements (average 3.7% MPKI reduction, up to 9.4%) with a simpler predictor design.