Analysis of classical and advanced control techniques tuned with reinforcement learning

Abstract

Control theory is used to stabilize systems and obtain specific responses for each type of process. Classic controllers, such as the PID used in this research, are spread globally in industries because they have well studied topologies in the literature and are easily applied in microcontrollers or programmable lógic devices; advanced ones,such as GMV, GPC and LQR, also used in this work, have some resistance in common applications in base industries, but are widely used in energy, aerospace and robotic systems, since the complexity and structure of these methods generate robustness and reach satisfactory performances for processes that are difficult to control. In this work, these methods are studied and evaluated with a tuning approach that uses re inforcement learning. The tuning methods are used in two forms and are applied to the controllers, these are the Repeat and Improve method and the Differential Games method. The first works using offline iterations, where the process agent is the chosen control technique, which selects performance and robustness indexes as an environment (metric of how the process is evolving), being able to organize an adjustment policy for the controller, which is based on rewarding the weighting factor until reaching the process stopping criterion (desired response). The second method uses reinforcement strategies that reward the controller as the response changes, so the LQR learns the ideal control policies, adapting to changes in the environment, which allows for better performance by recalculating the traditional gains found. With the Ricatti equation for tuning the regulator; in this method, differential games are used as a framework to model and analyze dynamic systems with multiple agents. To validate what is presented, the Tachogenerator Motor and the Ar Drone have been chosen. The Tachogenerator Motor is modeled with least squares estimation in an ARX-SISO topology, in order to evaluate the first tuning method. The Ar Drone is modeled with a state space approach to evaluate the second tuning method.