Fault diagnosis in dynamic systems using reinforcement learning




Pany, Rajeev Vadakicharla

Journal Title

Journal ISSN

Volume Title



This thesis is an effort to extend the concepts of reinforcement learning to Fault Diagnosis and Detection. The focus will be in trying to detect incipient faults in dynamic systems which are incompletely specified, or which have unmodeled and uncertain dynamics, or in the presence of noise. The method of Temporal Differences has been used to train a recurrent multi-layer perceptron architecture, known as an adaptive critic to predict the incipient faults occurring in dynamic systems. The effort is in trying to detect and classify faults based on their dynamics rather than just the states, and hence be in a position to make early predictions on the final fault condition. This paradigm has been demonstrated by applying the scheme to a second order spring-mass-damper system and then extending it to a fourth order continuous stirred tank reactor. Additionally, we have also presented an analysis scheme based on Lie Algebra and the concepts of observability of nonlinear systems, to ensure the detectability and distinguishability of faults. The fault detection scheme presented within can be used in tandem with any analytical redundancy technique such as the Luenberger observer, Kalman filter, Parity space or the parameter estimation technique