Online Reinforcement Learning for Flight Control

Online Reinforcement Learning for Flight Control: An Adaptive Critic Design without prior model knowledge

Kroezen, Dave (TU Delft Aerospace Engineering; TU Delft Control & Simulation)

van Kampen, E. (mentor)
de Croon, G.C.H.E. (graduation committee)
Mitici, M.A. (graduation committee)
Pan, W. (graduation committee)

Delft University of Technology

Aerospace Engineering | Control & Simulation

2019-04-10

Online Reinforcement Learning is a possible solution for adaptive nonlinear flight control. In this research an Adaptive Critic Design (ACD) based on Dual Heuristic Dynamic Programming (DHP) is developed and implemented on a simulated Cessna Citation 550 aircraft. Using an online identified system model approximation, the method is independent of prior model knowledge. The agent consists of two Artificial Neural Networks (ANNs) which form the Adaptive Critic Design and is supplemented with a Recursive Least Squares (RLS) online model estimation. The implemented agent is demonstrated to learn a near optimal control policy for different operating points, which is capable of tracking pitch and roll rate while actively minimizing the sideslip angle in a faster than real-time simulation. Providing limited model knowledge is shown to increase the learning, performance and robustness of the controller.

Reinforcement Learning (RL)
Adaptive Control
Online Learning
Adaptive Critic Designs
Flight Control Systems
Adaptive Flight Control
Machine Learning

http://resolver.tudelft.nl/uuid:38547b1d-0535-4b30-a348-67ac40c7ddcc

Student theses

master thesis

© 2019 Dave Kroezen