Browsing by Subject "Variational inference"
Now showing 1 - 2 of 2
- Results Per Page
- Sort Options
Item Bayesian modeling with tractable inference and applications to online peer-to-peer lending(2020-08-13) Zhang, Quan; Zhou, Mingyuan (Assistant professor); Gao, Qiang; Lin, Mingfeng; Sager, Thomas W.; Shively, Thomas S.While modern machine learning and deep learning seem to dominate the areas where scalability and modeling flexibility are required, Bayesian methods shine out when people are seeking interpretation and high-quality uncertainty estimation. Appreciating the beauty of Bayesian statistics, I have been dedicated to tractable Bayesian inference and interpretable modeling, and especially interested in Markov chain Monte Carlo (MCMC) on which Bayesian inference has highly depended until variational inferences was invented to provide an alternative solution. Therefore, I develop novel algorithms in MCMC and variation inference and interpretable models to explain complex mechanisms. Proposed in the thesis are novel Bayesian inference algorithms and modeling framework to solve various fundamental problems in statistics, including general-purpose statistical inference with uncertainty quantification, multinomial classification and survival analysis. Tremendous help was given by my Ph.D. advisor, the committee members and other collaborators and a lot of inspirations were ignited amid cooperation. In the first chapter of the thesis, we introduce MCMC-interactive variational inference that utilizes the complementary advantages of MCMC and variational inference. This inference algorithm not only accurately and efficiently estimates the posteriors, but also facilitates designs of stochastic gradient MCMC and transition kernels for Gibbs sampling. In the second chapter, we propose a permuted and augmented stick-breaking construction as sequential decision making to extend any binary classifier using a cross-entropy loss to a Bayesian multinomial one, so that favorable properties of the binary classifier can be preserved. We develop a data augmentation scheme and an efficient Metropolis-Hastings algorithm to transform the sequential decision making problem into several conditionally independent ones so that parallel computing can be used. In the third chapter, we propose Weibull delegate racing to explicitly model surviving under competing events and to interpret non-monotonic covariate effects by an intuitive two-phase racing mechanism. For inference, we develop a Gibbs-sampling-based MCMC algorithm with data augmentations along with a maximum a posteriori estimation algorithm for big data analysis. As an application, we analyze time to loan payoff and default on Prosper.com, demonstrating not only distinguished model performance, but also the value of standard and soft information on this peer-to-peer lending platform.Item Infinite-word topic models for digital media(2014-05) Waters, Austin Severn; Miikkulainen, RistoDigital media collections hold an unprecedented source of knowledge and data about the world. Yet, even at current scales, the data exceeds by many orders of magnitude the amount a single user could browse through in an entire lifetime. Making use of such data requires computational tools that can index, search over, and organize media documents in ways that are meaningful to human users, based on the meaning of their content. This dissertation develops an automated approach to analyzing digital media content based on topic models. Its primary contribution, the Infinite-Word Topic Model (IWTM), helps extend topic modeling to digital media domains by removing model assumptions that do not make sense for them -- in particular, the assumption that documents are composed of discrete, mutually-exclusive words from a fixed-size vocabulary. While conventional topic models like Latent Dirichlet Allocation (LDA) require that media documents be converted into bags of words, IWTM incorporates clustering into its probabilistic model and treats the vocabulary size as a random quantity to be inferred based on the data. Among its other benefits, IWTM achieves better performance than LDA while automating the selection of the vocabulary size. This dissertation contributes fast, scalable variational inference methods for IWTM that allow the model to be applied to large datasets. Furthermore, it introduces a new method, Incremental Variational Inference (IVI), for training IWTM and other Bayesian non-parametric models efficiently on growing datasets. IVI allows such models to grow in complexity as the dataset grows, as their priors state that they should. Finally, building on IVI, an active learning method for topic models is developed that intelligently samples new data, resulting in models that train faster, achieve higher performance, and use smaller amounts of labeled data.