Browsing by Subject "Unsupervised learning"

Now showing 1 - 11 of 11

Advances and application of positive matrix factorization for source attribution of air pollution in megacities
(2021-02-22) Bhandari, Sahil; Hildebrandt Ruiz, Lea; Apte, Joshua S.; Sharma, Mukul M; Allen, David T
Air pollution is considered the greatest current environmental health threat to humanity, with an estimated mortality burden of 7 million per year. More than half the world’s population is exposed to increasing air pollution. Reduction of air pollution is essential to global health and can be expected to generate long-term societal benefits. Receptor models are efficient mathematical tools for identification of sources of air pollution. A popular receptor modeling technique is Positive Matrix Factorization (PMF). However, PMF is limited by the assumption of constant source profiles throughout the modeling period—while the contribution of each source is modeled to change over time, its profile (e.g., mass spectrum, when PMF is applied to mass spectrometer data) stays constant. PMF is frequently applied to data on air pollution from fine particulate matter (PM), particularly in megacities. Megacities are centers of economic activity, harbor very large populations, and have high PM levels, especially in the developing world, posing acute challenges to public health. One such city is Delhi, India. Delhi is the second most populated city in the world and routinely experiences some of the highest particulate matter concentrations of any megacity on the planet. However, the current understanding of the sources and dynamics of PM pollution in Delhi is limited. Measurements at the Delhi Aerosol Supersite (DAS) provide long-term chemical characterization of ambient submicron aerosol in Delhi, with near-continuous online measurements of aerosol composition. In this dissertation, I apply PMF on data collected in the DAS study to characterize sources and atmospheric dynamics of submicron aerosols in Delhi. In study 1 (chapter 2), I report on source apportionment based on unsupervised (unconstrained) positive matrix factorization (PMF), conducted on 15 months of highly time-resolved speciated submicron non-refractory PM₁ (NR-PM₁) between January 2017 and March 2018. This dataset was collected in the DAS study. I report on seasonal variability across four seasons of 2017 and interannual variability using data from the two winters and springs of 2017 and 2018. I also show that a modified tracer-based organic component analysis provides an opportunity for a real-time source apportionment approach for organics in Delhi. Phase equilibrium modeling of aerosols using the extended aerosol inorganics model (E-AIM) predicts equilibrium gas-phase concentrations and allows evaluation of the importance of the ventilation coefficient (VC) and temperature in controlling primary and secondary organic aerosol. I also find that primary aerosol dominates severe air pollution episodes, and secondary aerosol dominates seasonal averages. An edited version of this chapter has been published in Atmospheric Chemistry and Physics. In study 2 (chapter 3), we develop the approach of conducting supervised (constrained) PMF on long-term datasets separated into 4 hour periods with limited variability in emissions and meteorology and statistically demonstrate its viability. I apply this time-of-day PMF approach on two seasons of highly time-resolved NR-PM₁ organics. This approach improves upon the seasonal source apportionment previously employed in Delhi by capturing the diurnal variability in source mass spectral profiles and retaining low computational intensity. Use of the EPA PMF tool allows application of constraints and quantifies random errors and rotational ambiguity in PMF solutions. Results in this study demonstrate that time-of-day PMF approach gives a greater number of more appropriate PMF factors compared to the traditional seasonal PMF approach. The time-of-day PMF approach fits data better, improving fits at specific time points, and at key m/zs. Portions of this chapter will be submitted to Atmospheric Measurement Techniques. Previous receptor modeling studies have identified vehicular emissions and fossil fuel combustion as prevalent factors contributing to fine PM pollution in Delhi. However, cooking and biomass burning have not been consistently identified in ambient studies. Bottom-up (source-oriented) studies have recognized the high exposure to residential energy emissions from cooking and heating and associated biomass burning emissions. In study 3 (chapter 4), I address these limitations of receptor modeling studies by applying PMF on two seasons of highly time-resolved NR-PM₁ organics. I utilize the time-of-day PMF approach (chapter 3) to separate primary organics into component primary factors. Hydrocarbon-like organic aerosol, or HOA, the fuel combustion and traffic primary organic aerosol surrogate, occurs in every season, and shows strong diurnal patterns. Biomass burning organic aerosol, or BBOA, separates only in winter, and exhibits time series peaks associated with space heating and solid-fuel combustion. Cooking organic aerosol, or COA, separates only in monsoon and reports stable diurnal patterns, suggesting the presence of cooking sources all-day. Equilibrium modeling of organic aerosols using volatility basis sets (VBS) suggests that differences in ventilation coefficient and temperature can explain the differences in factor separation between winter and monsoon. Overall, I show that traffic, and cooking and biomass burning contribute almost equally to the primary organic aerosol burden in Delhi, in broad agreement with several bottom-up studies. Portions of this chapter will be submitted to Atmospheric Chemistry and Physics
Computational fashion understanding
(2021-11-15) Hsiao, Wei-Lin Kimberly; Grauman, Kristen Lorraine, 1979-; Parikh, Devi; Durrett, Greg; Huang, Qixing
Fashion is very much a social and cultural statement. It documents the tastes and values of an era, and can even reflect the political orientation and economic development of a nation. While a fashion style can be defined by combinations of basic elements, fashion itself can take on indefinite forms. It is shaped by group acceptance, social forces important in an era, people's desire to relate to specific lifestyles, and it evolves over time. Fashion being so tightly wrapped up in our daily lives, our fashion appearance becomes one of the most direct ways to express ourselves. Understanding fashion computationally would allow better personalization, help the industry analyze its enterprise, and bring us closer towards building socially intelligent agents. Unfortunately, current computer vision systems for fashion are lacking in multiple ways. They fail to consider the synergy between multiple garments that compose an outfit, the garments' relation to the human wearer, and the underlying social factors that influence our fashion choices. The goal of this thesis is to devise computational models that understand fashion in the full context of outfits, humans and society. Our solution is to develop models that learn to utilize "in-the-wild" fashion images with their associated meta-data (e.g., geotags, timestamps, user tags), and capture human's aesthetics and rationales for fashion in an unsupervised manner. These freely available Internet images come at large-scale and are natural examples of what people choose to wear, allowing models with easy adaptation to new definitions of fashion as it evolves. The thesis begins by introducing our method that accounts for the synergy between multiple garments in an outfit. Specifically, we devise a topic-model-based method that automatically analyzes the interplay between garments, and discovers underlying combinations that form fashion styles. As outfits are eventually worn by humans, the thesis moves on to introduce new fashion recommendation systems that are aware of an individual's body shape and give suggestions on dressing in a more fashionable way. A 3D mesh of a human body is recovered and mapped into a visual embedding to identify flattering garments for the given body shape. In order to communicate with users how to dress fashionably, our model learns to score fashionability of outfits by seeing tens of thousands of natural examples from Internet images, and renders the optimized look of an outfit via an image generation neural network. Finally, the thesis zooms out to consider fashion in the history of human culture, where we conduct our study on millions of news articles and fashion images spanning from the beginning of the twentieth century to nowadays. We first extend our style topic model to adapt to the fluid and ever-evolving nature of fashion styles, and then devise a multi-modal statistical model to identify the links from world events to fashion evolution. All together, I hope my research lays down foundations for human-aware, society-aware, and evolving fashion understanding.
Database of planar and three-dimensional periodic orbits and families near the moon
(2021-08-20) Franz, Carter Joseph; Russell, Ryan Paul, 1976-; Jones, Brandon
The renewed interest in Lunar exploration prompts a need to better understand the dynamics of spacecraft in the vicinity of the moon. Here, a detailed survey is conducted via a broad grid search to find, characterize, and archive periodic orbits. The resulting database contains over 13 million planar and three-dimensional solutions in the circular restricted three body problem. The work is a direct follow-on to previous periodic orbit grid searches, with a focus on the Earth-Moon mass ratio, the addition of x-z symmetric orbits, and family clustering. Each periodic orbit successfully identified is evaluated for stability properties, perilune distance, number and center of revolutions, and other defining properties. DBSCAN, an unsupervised learning clustering algorithm, is used to isolate family curves in phase space and group orbits into family or sub-family clusters. The clustered orbits are then sorted to form smooth, ordered curves in phase space, allowing for arbitrary data resolution and improved labeling of the discrete grid solutions. A custom cluster confidence measure is introduced and applied. Over 80% of 3D data and over 62% of planar data are clustered with high confidence, with better results at lower revolutions. Approximately 4.25% of 3D solutions and less than 0.5% of planar solutions are classified as outliers. The resulting database is an extension of several recent lunar periodic orbit studies, and can be considered a modern update to Broucke's seminal database of planar cislunar periodic orbits. This new public database is a tool for future mission design and has potential use in a variety of catalogue maintenance space situational awareness applications
Embodied learning for visual recognition
(2017-08-10) Jayaraman, Dinesh; Grauman, Kristen Lorraine, 1979-; Efros, Alexei; Ghosh, Joydeep; Niekum, Scott; Thomaz, Andrea
The field of visual recognition in recent years has come to rely on large expensively curated and manually labeled "bags of disembodied images". In the wake of this, my focus has been on understanding and exploiting alternate "free" sources of supervision available to visual learning agents that are situated within real environments. For example, even simply moving from orderless image collections to continuous visual observations offers opportunities to understand the dynamics and other physical properties of the visual world. Further, embodied agents may have the abilities to move around their environment and/or effect changes within it, in which case these abilities offer new means to acquire useful supervision. In this dissertation, I present my work along this and related directions.
Energy storage-aware prediction/control for mobile systems with unstructured loads
(2013-08) LeSage, Jonathan Robert, 1985-; Longoria, Raul G.
Mobile systems, such as ground robots and electric vehicles, inherently operate in stochastic environments where load demands are largely unknown. Onboard energy storage, most commonly an electrochemical battery system, can significantly constrain operation. As such, mission planning and control of mobile systems can benefit from a priori knowledge about battery dynamics and constraints, especially the rate-capacity and recovery effects. To help overcome overly conservative predictions common with most existing battery remaining run-time algorithms, a prediction scheme was proposed. For characterization of a priori unknown power loads, an unsupervised Gaussian mixture routine identifies/clusters the measured power loads, and a jump-Markov chain characterizes the load transients. With the jump-Markov load forecasts, a model-based particle filter scheme predicts battery remaining run-time. Monte Carlo simulation studies demonstrate the marked improvement of the proposed technique. It was found that the increase in computational complexity from using a particle filter was justified for power load transient jumps greater than 13.4% of total system power. A multivariable reliability method was developed to assess the feasibility of a planned mission. The probability of mission completion is computed as the reliability integral of mission time exceeding the battery run-time. Because these random variables are inherently dependent, a bivariate characterization was necessary and a method is presented for online estimation of the process correlation via Bayesian updating. Finally, to abate transient shutdown of mobile systems, a model predictive control scheme is proposed that enforces battery terminal voltage constraints under stochastic loading conditions. A Monte Carlo simulation study of a small ground vehicle indicated significant improvement in both time and distance traveled as a result. For evaluation of the proposed methodologies, a laboratory terrain environment was designed and constructed for repeated mobile system discharge studies. The test environment consists of three distinct terrains. For each discharge study, a small unmanned ground vehicle traversed the stochastic terrain environment until battery exhaustion. Results from field tests with a Packbot ground vehicle in generic desert terrain were also used. Evaluation of the proposed prediction algorithms using the experimental studies, via relative accuracy and [alpha]-[lambda] prognostic metrics, indicated significant gains over existing methods.
The Fern algorithm for intelligent discretization
(2012-08) Hall, John Wendell; Djurdjanovic, Dragan; Fernandez, Benito R.
This thesis proposes and tests a recursive, adpative, and computationally inexpensive method for partitioning real-number spaces. When tested for proof-of-concept on both one- and two- dimensional classification and control problems, the Fern algorithm was found to work well in one dimension, moderately well for two-dimensional classification, and not at all for two-dimensional control. Testing ferns as pure discretizers - which would involve a secondary discrete learner - has been left to future work.
Leveraging pervasive data to study and support mother-infant dyads in the wild
(2023-04-20) Yao, Xuewen; De Barbaro, Kaya; Thomaz, Edison; Yang, Diyi; Julien, Christine; Li, Jessy; Barber, Suzanne
The ubiquity of mobile devices and wearable sensors coupled with fast-evolving machine learning algorithms has transformed people's daily life, specifically in the healthcare domain. These advancements can be leveraged to not only detect and infer people's behavior patterns with great precision but also provide "just-in-time" support in a seamless, non-intrusive, and cost-effective fashion. The first year of a child's life is a particularly challenging period for the mother, and also a vital period for child developments. I hypothesize that pervasive data, such as motion, audio, text data, collected online or in people’s daily life, can be leveraged to provide support to postpartum women and their families in need in the wild. In my dissertation, I developed models that can detect two clinically-relevant parent and infant behaviors in naturalistic home interactions, namely, infant crying and parent holding. Traditional methods by developmental scientists rely heavily on behavioral observations and self-reported data while computer scientists build models using data collected in controlled environments, such as lab and hospital. These methods limit researchers’ understanding of the natural variation in mother-infant interactions across families, and its specific impacts on child development. In my work, I leveraged data collected in longitudinal home environments and built detection models that provide objective, unobtrusive, and continuous measurements of parent holding and infant crying with accuracy 0.870 and 0.613 respectively. Additionally, I evaluated both models based on assessment scenarios specific to developmental science such as event-based accuracy and contingency analysis. Another piece of my work focuses on using natural language processing to understand the experience of postpartum women experiencing or at risk of postpartum mood and anxiety disorders and to provide them with empathy in the form of a conversational agent, chatbot. Specifically, I collaborated with Postpartum Support International (PSI) and obtained text transcripts between trained volunteers and support seekers. After analyzing 7014 conversations using a combination of human annotations, dictionary models and unsupervised techniques, I find stark differences between the experiences of "distressed" and "healthy" mothers in psychological states, concerns, and goals. Additionally, incorporating the insights from the descriptive analysis as well as empathy and open questions, I designed, built, and evaluated three chatbots that accept open-ended user input to provide postpartum women with support.
Machine learning algorithms in political research
(2022-07-27) Freedman, Guy, Ph. D.; Theriault, Sean M., 1972-; Jones, Bryan D; Craig, Alison W; Wilkerson, John D
In recent years, political science has witnessed an explosion of data. Political scientists have begun turning to machine learning methods to provide reliable and scalable measurements of such large datasets. Building on the emerging literature on the use of machine learning in political science, I contribute four major lessons to the students and scholars who wish to make the most of these methods. These lessons include the advantage of treating machine learning as a process, combining text as data with standard data practices, the strength of pooling together supervised and unsupervised learning and the importance of understanding a model’s strengths and limits. Through two rigorous empirical chapters, I trace the process of machine learning in two case studies, with actual outcomes for two widely-used datasets in the discipline. The first centers on a model for identifying agency-creation in historical data of congressional hearings. In the second case study, I tackle a multi-classification problem of predicting one of 20 major policy topics (and over 220 minor topics) in congressional bills. I conclude with a look to the future of machine learning in the discipline as we shift from a first wave of the literature that served as an introduction to machine learning, to a second wave of utilizing machine learning in actual research on political data and the challenges that these data present.
Semantic interpretation with distributional analysis
(2012-05) Glass, Michael Robert; Barker, Ken, 1959-; Porter, Bruce, 1956-; Mooney, Ray; Erk, Katrin; Dhillon, Inderjit
Unstructured text contains a wealth of knowledge, however, it is in a form unsuitable for reasoning. Semantic interpretation is the task of processing natural language text to create or extend a coherent, formal knowledgebase able to reason and support question answering. This task involves entity, event and relation extraction, co-reference resolution, and inference. Many domains, from intelligence data to bioinformatics, would benefit by semantic interpretation. But traditional approaches to the subtasks typically require a large annotated corpus specific to a single domain and ontology. This dissertation describes an approach to rapidly train a semantic interpreter using a set of seed annotations and a large, unlabeled corpus. Our approach adapts methods from paraphrase acquisition and automatic thesaurus construction to extend seed syntactic to semantic mappings using an automatically gathered, domain specific, parallel corpus. During interpretation, the system uses joint probabilistic inference to select the most probable interpretation consistent with the background knowledge. We evaluate both the quality of the extended mappings as well as the performance of the semantic interpreter.
Unsupervised learning for large-scale data
(2019-09-20) Wu, Shanshan, Ph. D.; Sanghavi, Sujay Rajendra, 1979-; Dimakis, Alexandros G.; Caramanis, Constantine; Klivans, Adam R; Ward, Rachel A
Unsupervised learning involves inferring the inherent structures or patterns from unlabeled data. Since there is no label information, the fundamental challenge of unsupervised learning is that the objective function is not explicitly defined. The ubiquity of large-scale datasets adds another layer of complexity to the overall learning problem. When the data size or dimension is large, even algorithms with quadratic runtime may be prohibitive. This thesis presents four large-scale unsupervised learning problems. We start with two density estimation problems: given samples from a one-layer ReLU generative model or a discrete pairwise graphical model, the goal is to recover the parameters of the generative model. We then move to representation learning of high-dimensional sparse data coming from one-hot encoded categorical features. We assume that there are additional but a-priori unknown structures in their support. The goal is to learn a lossless low-dimensional embedding for the given data. Our last problem is to compute low-rank approximations of a matrix product given the individual matrices. We are interested in the setting where the matrices are too large and can only be stored in the disk. For every problem presented in this thesis, we (i) design novel and efficient algorithms to capture the inherent structure from data in an unsupervised manner; (ii) establish theoretical guarantees and compare the empirical performance with the state-of-the-art methods; and (iii) provide source code to support our experimental findings
Visual object category discovery in images and videos
(2012-05) Lee, Yong Jae, 1984-; Grauman, Kristen Lorraine, 1979-; Ghosh, Joydeep; Efros, Alexei; Bovik, Al; Geisler, Wilson; Aggarwal, J. K.
The current trend in visual recognition research is to place a strict division between the supervised and unsupervised learning paradigms, which is problematic for two main reasons. On the one hand, supervised methods require training data for each and every category that the system learns; training data may not always be available and is expensive to obtain. On the other hand, unsupervised methods must determine the optimal visual cues and distance metrics that distinguish one category from another to group images into semantically meaningful categories; however, for unlabeled data, these are unknown a priori. I propose a visual category discovery framework that transcends the two paradigms and learns accurate models with few labeled exemplars. The main insight is to automatically focus on the prevalent objects in images and videos, and learn models from them for category grouping, segmentation, and summarization. To implement this idea, I first present a context-aware category discovery framework that discovers novel categories by leveraging context from previously learned categories. I devise a novel object-graph descriptor to model the interaction between a set of known categories and the unknown to-be-discovered categories, and group regions that have similar appearance and similar object-graphs. I then present a collective segmentation framework that simultaneously discovers the segmentations and groupings of objects by leveraging the shared patterns in the unlabeled image collection. It discovers an ensemble of representative instances for each unknown category, and builds top-down models from them to refine the segmentation of the remaining instances. Finally, building on these techniques, I show how to produce compact visual summaries for first-person egocentric videos that focus on the important people and objects. The system leverages novel egocentric and high-level saliency features to predict important regions in the video, and produces a concise visual summary that is driven by those regions. I compare against existing state-of-the-art methods for category discovery and segmentation on several challenging benchmark datasets. I demonstrate that we can discover visual concepts more accurately by focusing on the prevalent objects in images and videos, and show clear advantages of departing from the status quo division between the supervised and unsupervised learning paradigms. The main impact of my thesis is that it lays the groundwork for building large-scale visual discovery systems that can automatically discover visual concepts with minimal human supervision.