Human demonstrations for fast and safe exploration in reinforcement learning

Human demonstrations for fast and safe exploration in reinforcement learning

Schonebaum, G.K. (TU Delft Education AE)
Junell, J. (TU Delft Control & Simulation)
van Kampen, E. (TU Delft Control & Simulation)

2017

Reinforcement learning is a promising framework for controlling complex vehicles with a high level of autonomy, since it does not need a dynamic model of the vehicle, and it is able to adapt to changing conditions. When learning from scratch, the performance of a reinforcement learning controller may initially be poor and -for real life applications- unsafe. In this paper the effects of using human demonstrations on the performance of reinforcement learning is investigated, using a combination of offline and online least squares policy iteration. It is found that using the human as an efficient explorer improves learning time and performance for a benchmark reinforcement learning problem. The benefit of the human demonstration is larger for problems where the human can make use of its understanding of the problem to efficiently explore the state space. Applied to a simplified quadrotor slung load drop off problem, the use of human demonstrations reduces the number of crashes during learning. As such, this paper contributes to safer and faster learning for model-free, adaptive control problems.

http://resolver.tudelft.nl/uuid:e860e94c-ea45-4793-a2f3-e84fde614bd1

https://doi.org/10.2514/6.2017-1069

American Institute of Aeronautics and Astronautics Inc. (AIAA)

2018-01-31

9781624104497

AIAA Information Systems-AIAA Infotech at Aerospace, 2017

AIAA Information Systems-Infotech At Aerospace Conference, 2017, 2017-01-09 → 2017-01-13, Grapevine, United States

Institutional Repository

conference paper

© 2017 G.K. Schonebaum, J. Junell, E. van Kampen