Print Email Facebook Twitter Residential demand response of thermostatically controlled loads using batch Reinforcement Learning Title Residential demand response of thermostatically controlled loads using batch Reinforcement Learning Author Ruelens, F (Katholieke Universiteit Leuven) Claessens, BJ (Flemish Institute for Technological Research) Vandael, S (Katholieke Universiteit Leuven) De Schutter, B.H.K. (TU Delft Team Bart De Schutter) Babuska, R. (TU Delft Learning & Autonomous Control) Belmans, R (Katholieke Universiteit Leuven) Date 2017 Abstract Driven by recent advances in batch Reinforcement Learning (RL), this paper contributes to the application of batch RL to demand response. In contrast to conventional model-based approaches, batch RL techniques do not require a system identification step, making them more suitable for a large-scale implementation. This paper extends fitted Q-iteration, a standard batch RL technique, to the situation when a forecast of the exogenous data is provided. In general, batch RL techniques do not rely on expert knowledge about the system dynamics or the solution. However, if some expert knowledge is provided, it can be incorporated by using the proposed policy adjustment method. Finally, we tackle the challenge of finding an open-loop schedule required to participate in the day-ahead market. We propose a model-free Monte Carlo method that uses a metric based on the state-action value function or Q-function and we illustrate this method by finding the day-ahead schedule of a heat-pump thermostat. Our experiments show that batch RL techniques provide a valuable alternative to model-based controllers and that they can be used to construct both closed-loop and open-loop policies. Subject Load managementWater heatingResistance heatingAtmospheric modelingLoad modelingFeature extractionLearning (artificial intelligence) To reference this document use: http://resolver.tudelft.nl/uuid:2a6f9e25-d538-45d5-a8dd-2c2f9b453bdb DOI https://doi.org/10.1109/TSG.2016.2517211 ISSN 1949-3053 Source IEEE Transactions on Smart Grid, 8 (5), 2149-2159 Bibliographical note Accepted Author Manuscript Part of collection Institutional Repository Document type journal article Rights © 2017 F Ruelens, BJ Claessens, S Vandael, B.H.K. De Schutter, R. Babuska, R Belmans Files PDF 07401112_1_3.pdf 1.38 MB Close viewer /islandora/object/uuid:2a6f9e25-d538-45d5-a8dd-2c2f9b453bdb/datastream/OBJ/view