Reinforcement learning of potential fields to achieve limit-cycle walking

Reinforcement learning of potential fields to achieve limit-cycle walking

Feirstein (student), D.S. (Student TU Delft)
Koryakovskiy, I. (TU Delft Biomechatronics & Human-Machine Control)
Kober, J. (TU Delft OLD Intelligent Control & Robotics)
Vallery, H. (TU Delft OLD Biorobotics)

Nijmeijer, Henk (editor)

2016

Reinforcement learning is a powerful tool to derive controllers for systems where no models are available. Particularly policy search algorithms are suitable for complex systems, to keep learning time manageable and account for continuous state and action spaces. However, these algorithms demand more insight into the system to choose a suitable controller parameterization. This paper investigates a type of policy parameterization for impedance control that allows energy input to be implicitly bounded: Potential fields. In this work, a methodology for generating a potential field-constrained impedance controller via approximation of example trajectories, and subsequently improving the control policy using Reinforcement Learning, is presented. The potential field-const rained approximation is used as a policy parameterization for policy search reinforcement learning and is compared to its unconstrained counterpart. Simulations on a simple biped walking model show the learned controllers are able to surpass the potential field of gravity by generating a stable limit-cycle gait on flat ground for both parameterizations. The potential field-constrained controller provides safety with a known energy bound while performing equally well as the unconstrained policy.

Energy Control
Limit cycles
Machine learning
Robot control
Walking

http://resolver.tudelft.nl/uuid:b4693303-4c1a-40f6-b20b-f19ba845701f

https://doi.org/10.1016/j.ifacol.2016.07.994

Elsevier

Proceedings of the 6th IFAC Workshop on Periodic Control Systems (PSYCO 2016)

PSYCO 2016: 6th IFAC Workshop on Periodic Control System, 2016-06-29 → 2016-07-01, Eindhoven, Netherlands

IFAC-PapersOnLine, 2405-8963, 49 (14)

Institutional Repository

conference paper

© 2016 D.S. Feirstein (student), I. Koryakovskiy, J. Kober, H. Vallery