Browsing by Subject "Discrete-time systems"
Now showing 1 - 3 of 3
- Results Per Page
- Sort Options
Item Discrete-time partially observed Markov decision processes: ergodic, adaptive, and safety control(2002) Hsu, Shun-pin; Arapostathis, Ari, 1954-In this dissertation we study stochastic control problems for systems modelled by discrete-time partially observed Markov decision processes. The issues we consider include ergodic control, adaptive control, and safety control. For ergodic control we propose a new condition that weakens the elegant interior accessibility assumption suggested recently. Using the standard procedure to transform the partially observed control problem to its completely observed equivalent, and then applying the vanishing discount method, we obtain Bellman’s ergodic optimality equation, which characterizes the optimal policy. We also provide an example to compare our assumption with those of previous work. When there are more than one decision maker in the system, we formulate our problem as a stochastic non-cooperative game where each decision maker seeks to minimize his or her own long-run average cost. A special class of systems with two decision makers and mixed observation structure is considered, and the existence of a Nash equilibrium for the policies is proved. In the study of adaptive control we extend settings of the ergodic control to the ones where the transition matrix is parameterized by a unknown vector. Motivated by notions of weak ergodicity, we propose a condition on the structure of the transition matrix that results in the ergodic behavior of the underlying controlled process. Under additional hypotheses, we show that the proposed adaptive policy is self-optimizing in appropriate sense. A new concept designated safety control is introduced in our work where the notion of safety is specified in terms of membership in a set called safe set. We study the choices of an appropriate policy (called safe policy) and an initial state probability distribution such that a safety request, which asks the state probability distribution of the system to lie in a given convex set at each time step, is met. Since the choice of a safe policy is not unique in general, we apply techniques of constrained Markov decision processes to find an optimal policy in appropriate sense among the candidates. We also develop an algorithm to find the largest set of initial state probability distributions corresponding to a given safe policy to meet the safety request. The algorithm is proved to terminate in finite steps under reasonable assumptions. Finally we investigate the safety control under partial observations. A machine replacement problem is studied in detail and numerical simulations are presented.Item A time-centered split for implicit discretization of unsteady advection problems(2008-05) Fu, Shipeng, 1975-; Hodges, Ben R.Environmental flows (e.g. river and atmospheric flows) governed by the shallow water equations (SWE) are usually dominated by the advective mechanism over multiple time-scales. The combination of time dependency and nonlinear advection creates difficulties in the numerical solution of the SWE. A fully-implicit scheme is desirable because a relatively large time step may be used in a simulation. However, nonlinearity in a fully implicit method results in a system of nonlinear equations to be solved at each time step. To address this difficulty, a new method for implicit solution of unsteady nonlinear advection equations is developed in this research. This Time-Centered Split (TCS) method uses a nested application of the midpoint rule to computationally decouple advection terms in a temporally second-order accurate time-marching discretization. The method requires solution of only two sets of linear equations without an outer iteration, and is theoretically applicable to quadratically-nonlinear coupled equations for any number of variables. To explore its characteristics, the TCS algorithm is first applied to onedimensional problems and compared to the conventional nonlinear solution methods. The temporal accuracy and practical stability of the method is confirmed using these 1D examples. It is shown that TCS can computationally linearize unsteady nonlinear advection problems without either 1) outer iteration or 2) calculation of the Jacobian. A family of the TCS method is created in one general form by introducing weighting factors to different terms. We prove both analytically and by examples that the value of the weighting factors does not affect the order of accuracy of the scheme. In addition, the TCS method can not only computationally linearize but also decouple an equation system of coupled variables using special combinations of weighting factors. Hence, the TCS method provides flexibilities and efficiency in applications.Item A time-centered split for implicit discretization of unsteady advection problems(Center for Research in Water Resources, University of Texas at Austin, 2008-05) Fu, Shipeng; Hodges, Ben R.