Browsing by Subject "Transfer learning"
Now showing 1 - 9 of 9
- Results Per Page
- Sort Options
Item A prototype-oriented framework for deep transfer learning applications(2023-04-05) Tanwisuth, Korawat; Zhou, Mingyuan (Assistant professor); Mueller, Peter; Ho, Nhat; Qian, XiaoningDeep learning models achieve state-of-the-art performance in many applications but often require large-scale data. Deep transfer learning studies the ability of deep learning models to transfer knowledge from source tasks to related target tasks, enabling data-efficient learning. This dissertation develops novel methodologies that tackle three different transfer learning applications for deep learning models: unsupervised domain adaptation, unsupervised fine-tuning, and source-private clustering. The key idea behind the proposed methods relies on minimizing the distributional discrepancy between the prototypes and target data with the transport framework. For each scenario, we design our algorithms to suit different data and model requirements. In unsupervised domain adaptation, we leverage the source domain data to construct class prototypes and minimize the transport cost between the prototypes and target data. In unsupervised fine-tuning, we apply our framework to prompt-based zero-shot learning to adapt large pre-trained models directly on the target data, bypassing the source data requirement. In source-private clustering, we incorporate a knowledge distillation framework with our prototype-oriented clustering to address the problem of data and model privacy. All three approaches show consistent performance gains over the baselines.Item Adaptive trading agent strategies using market experience(2011-05) Pardoe, David Merrill; Stone, Peter, 1971-; Miikkulainen, Risto; Mooney, Raymond; Saar-Tsechansky, Maytal; Wellman, MichaelAlong with the growth of electronic commerce has come an interest in developing autonomous trading agents. Often, such agents must interact directly with other market participants, and so the behavior of these participants must be taken into account when designing agent strategies. One common approach is to build a model of the market, but this approach requires the use of historical market data, which may not always be available. This dissertation addresses such a case: that of an agent entering a new market in which it has no previous experience. While the agent could adapt by learning about the behavior of other market participants, it would need to do so in an online fashion. The agent would not necessarily have to learn from scratch, however. If the agent had previous experience in similar markets, it could use this experience to tailor its learning approach to its particular situation. This dissertation explores methods that a trading agent could use to take advantage of previous market experience when adapting to a new market. Two distinct learning settings are considered. In the first, an agent acting as an auctioneer must adapt the parameters of an auction mechanism in response to bidder behavior, and a reinforcement learning approach is used. The second setting concerns agents that must adapt to the behavior of competitors in two scenarios from the Trading Agent Competition: supply chain management and ad auctions. Here, the agents use supervised learning to model the market. In both settings, methods of adaptation can be divided into four general categories: i) identifying the most similar previously encountered market, ii) learning from the current market only, iii) learning from the current market but using previous experience to tune the learning algorithm, and iv) learning from both the current and previous markets. The first contribution of this dissertation is the introduction and experimental validation of a number of novel algorithms for market adaptation fitting these categories. The second contribution is an exploration of the degree to which the quantity and nature of market experience impact the relative performance of methods from these categories.Item Combining classifier and cluster ensembles for semi-supervised and transfer learning(2012-05) Acharya, Ayan; Ghosh, Joydeep; Mooney, Raymond J.Unsupervised models can provide supplementary soft constraints to help classify new, "target" data since similar instances in the target set are more likely to share the same class label. Such models can also help detect possible differences between training and target distributions, which is useful in applications where concept drift may take place, as in transfer learning settings. This contribution describes two general frameworks that take as input class membership estimates from existing classifiers learnt on previously encountered "source" data, as well as a set of cluster labels from a cluster ensemble operating solely on the target data to be classified, and yield a consensus labeling of the target data. One of the proposed frameworks admits a wide range of loss functions and classification/clustering methods and exploits properties of Bregman divergences in conjunction with Legendre duality to yield a principled and scalable approach. The other approach is built on probabilistic mixture models and provides additional flexibility of distributed computation that is useful when the target data cannot be gathered in a single place for privacy or security concerns. A variety of experiments show that the proposed frameworks can yield results substantially superior to those provided by popular transductive learning techniques or by naively applying classifiers learnt on the original task to the target data.Item Curriculum learning in reinforcement learning(2021-05-06) Narvekar, Sanmit Santosh; Stone, Peter, 1971-; Niekum, Scott; Mooney, Raymond; Brunskill, EmmaIn recent years, reinforcement learning (RL) has been increasingly successful at solving complex tasks. Despite these successes, one of the fundamental challenges is that many RL methods require large amounts of experience, and thus can be slow to train in practice. Transfer learning is a recent area of research that has been shown to speed up learning on a complex task by transferring knowledge from one or more easier source tasks. Most existing transfer learning methods treat this transfer of knowledge as a one-step process, where knowledge from all the sources are directly transferred to the target. However, for complex tasks, it may be more beneficial (and even necessary) to gradually acquire skills over multiple tasks in sequence, where each subsequent task requires and builds upon knowledge gained in a previous task. This idea is pervasive throughout human learning, where people learn complex skills gradually by training via a curriculum. The goal of this thesis is to explore whether autonomous reinforcement learning agents can also benefit by training via a curriculum, and whether such curricula can be designed fully autonomously. In order to answer these questions, this thesis first formalizes the concept of a curriculum, and the methodology of curriculum learning in reinforcement learning. Curriculum learning consists of 3 main elements: 1) task generation, which creates a suitable set of source tasks; 2) sequencing, which focuses on how to order these tasks into a curriculum; and 3) transfer learning, which considers how to transfer knowledge between tasks in the curriculum. This thesis introduces several methods to both create suitable source tasks and automatically sequence them into a curriculum. We show that these methods produce curricula that are tailored to the individual sensing and action capabilities of different agents, and show how the curricula learned can be adapted for new, but related target tasks. Together, these methods form the components of an autonomous curriculum design agent, that can suggest a training curriculum customized to both the unique abilities of each agent and the task in question. We expect this research on the curriculum learning approach will increase the applicability and scalability of RL methods by providing a faster way of training reinforcement learning agents, compared to learning tabula rasa.Item Deep domain adaptation for label-efficient and generalizable bearing fault diagnosis(2022-05-06) Liu, Chenkuan; Ward, Rachel, 1983-; Bajaj, ChandrajitThis study presents the application of deep domain adaptation techniques in bearing fault diagnosis. Deep domain adaptation is a type of data-centric, label-efficient transfer learning approach based on deep neural network construction. Bearing fault diagnosis is a field which aims to perform analysis on the vibration signals generated from bearing apparatus. Different types of model structure and experiments are implemented on selected datasets, with associated ablation studies to analyze the effectiveness of domain adaptation compared to vanilla approaches. Our experiments show great performance and potential of the combination between bearing fault diagnosis and advanced machine learning techniques. In addition, detailed analysis also suggests the influence of the ways the bearing data are collected over the resulting performance, which leads to future directions for more accurate and specific approaches for bearing fault diagnosis tasks.Item Knowledge transfer using latent variable models(2015-08) Acharya, Ayan; Ghosh, Joydeep; Mooney, Raymond J. (Raymond Joseph); Shakkottai, Sanjay; Sanghavi, Sujay; Rajan, SujuIn several applications, scarcity of labeled data is a challenging problem that hinders the predictive capabilities of machine learning algorithms. Additionally, the distribution of the data changes over time, rendering models trained with older data less capable of discovering useful structure from the newly available data. Transfer learning is a convenient framework to overcome such problems where the learning of a model specific to a domain can benefit the learning of other models in other domains through either simultaneous training of domains or sequential transfer of knowledge from one domain to the others. This thesis explores the opportunities of knowledge transfer in the context of a few applications pertaining to object recognition from images, text analysis, network modeling and recommender systems, using probabilistic latent variable models as building blocks. Both simultaneous and sequential knowledge transfer are achieved through the latent variables, either by sharing these across multiple related domains (for simultaneous learning) or by adapting their distributions to fit data from a new domain (for sequential learning).Item Learning with Markov logic networks : transfer learning, structure learning, and an application to Web query disambiguation(2009-08) Mihalkova, Lilyana Simeonova; Mooney, Raymond J. (Raymond Joseph)Traditionally, machine learning algorithms assume that training data is provided as a set of independent instances, each of which can be described as a feature vector. In contrast, many domains of interest are inherently multi-relational, consisting of entities connected by a rich set of relations. For example, the participants in a social network are linked by friendships, collaborations, and shared interests. Likewise, the users of a search engine are related by searches for similar items and clicks to shared sites. The ability to model and reason about such relations is essential not only because better predictive accuracy is achieved by exploiting this additional information, but also because frequently the goal is to predict whether a set of entities are related in a particular way. This thesis falls within the area of Statistical Relational Learning (SRL), which combines ideas from two traditions within artificial intelligence, first-order logic and probabilistic graphical models to address the challenge of learning from multi-relational data. We build on one particular SRL model, Markov logic networks (MLNs), which consist of a set of weighted first-order-logic formulae and provide a principled way of defining a probability distribution over possible worlds. We develop algorithms for learning of MLN structure both from scratch and by transferring a previously learned model, as well as an application of MLNs to the problem of Web query disambiguation. The ideas we present are unified by two main themes: the need to deal with limited training data and the use of bottom-up learning techniques. Structure learning, the task of automatically acquiring a set of dependencies among the relations in the domain, is a central problem in SRL. We introduce BUSL, an algorithm for learning MLN structure from scratch that proceeds in a more bottom-up fashion, breaking away from the tradition of top-down learning typical in SRL. Our approach first constructs a novel data structure called a Markov network template that is used to restrict the search space for clauses. Our experiments in three relational domains demonstrate that BUSL dramatically reduces the search space for clauses and attains a significantly higher accuracy than a structure learner that follows a top-down approach. Accurate and efficient structure learning can also be achieved by transferring a model obtained in a source domain related to the current target domain of interest. We view transfer as a revision task and present an algorithm that diagnoses a source MLN to determine which of its parts transfer directly to the target domain and which need to be updated. This analysis focuses the search for revisions on the incorrect portions of the source structure, thus speeding up learning. Transfer learning is particularly important when target-domain data is limited, such as when data on only a few individuals is available from domains with hundreds of entities connected by a variety of relations. We also address this challenging case and develop a general transfer learning approach that makes effective use of such limited target data in several social network domains. Finally, we develop an application of MLNs to the problem of Web query disambiguation in a more privacy-aware setting where the only information available about a user is that captured in a short search session of 5-6 previous queries on average. This setting contrasts with previous work that typically assumes the availability of long user-specific search histories. To compensate for the scarcity of user-specific information, our approach exploits the relations between users, search terms, and URLs. We demonstrate the effectiveness of our approach in the presence of noise and show that it outperforms several natural baselines on a large data set collected from the MSN search engine.Item Object-model transfer in the general video game domain(2019-09-17) Braylan, Alexander Eric; Miikkulainen, RistoReinforcement learning agents often benefit from learning models that predict their environment. However, learned models may not generalize well to novel situations. This thesis investigates the potential for a transfer learning approach to address the challenge in the video game domain. The approach helps agents learn models of new games by transferring knowledge from previously learned games. Transfer is facilitated by decomposing games into the objects they contain. The assumption is that it is easier to relate features between objects from different games than features between whole environments of different games. Experiments show that predictions made by this method are more accurate than predictions made without transferred knowledge, and this improvement is demonstrated to result in increased efficiency in a task where an agent explores a maze-like game. The conclusion is that model learning can be enhanced by transferring object models from previously learned environments.Item Phoneme segmentation using self-supervised speech models(2023-08) Strgar, Luke Vincent; Harwath, DavidWe apply transfer learning to the task of phoneme segmentation and demonstrate the utility of representations learned in self-supervised pre-training for the task. Our model extends transformer-style encoders with strategically placed convolutions that manipulate features learned in pre-training. Using the TIMIT and Buckeye corpora we train and test the model in the supervised and unsupervised settings. The latter case is accomplished by furnishing a noisy label-set with the predictions of a separate model, it having been trained in an unsupervised fashion. Results indicate our model eclipses previous state-of-the-art performance in both settings and on both datasets. Finally, following observations during published code review and attempts to reproduce past segmentation results, we find a need to disambiguate the definition and implementation of widely-used evaluation metrics. We resolve this ambiguity by delineating two distinct evaluation schemes and describing their nuances. We provide a publicly available implementation of our work on Github.