Discrete state-space active inference and unknown controlled state transitions

Discrete state-space active inference and unknown controlled state transitions

van Roessel, L. (TU Delft Mechanical, Maritime and Materials Engineering; TU Delft Cognitive Robotics; TU Delft Biomechanical Engineering)

Wisse, M. (mentor)
Anil Meera, A. (graduation committee)
Kober, J. (graduation committee)

Delft University of Technology

Mechanical Engineering

2020-08-06

Active inference is a process theory arising from neuroscience which casts perception, action, planning and learning under one optimisation criterion: minimisation of free energy. Current literature on the implementation of discrete state-space active inference focuses on scalability, the comparison to reinforcement learning and its performance to learn the state-observation mapping of the environment. In these implementations, active inference is supplied with accurate predefined information (knowledge) about the transition dynamics (controlled state transitions) of the environment. However, it is likely that in practical settings, knowledge gaps about controlled state transitions are involved. This thesis analyses the influence of knowledge gaps about controlled state transition on the active inference process. First, we provide a comprehensive overview of active inference and related principles. Secondly, we expand current active inference formulations with the power to actively evaluate the consequences of actions in their ability to resolve transition knowledge gaps. Thirdly, we analyse the behaviour of this expanded version of active inference once concerned with uncertain transition knowledge in a pure explorative setting (no task set). We found that active inference can deal with minor transition uncertainty, however, fails to operate when no initial transition knowledge is supplied due to a failing parameter updating mechanism. After proposing an alternative parameter updating mechanism, we found that active inference can deal with total uncertainty. Finally, we analysed the performance of active inference (with alternative parameter updating mechanism) to reach goals, starting without knowledge about transitions. We found that the resulting behaviour is a trade-off between exploration and exploitation and that the active inference agent gains transition knowledge while reaching the goals. In short, by investigating the influence of knowledge gaps in controlled state transitions, we aim to bring discrete state-space active inference one step closer to practical applications.

Active Inference
Free Energy Principle
Generative Model
Grid world
Epistemic value
Exploration
Exploitation
Transition Uncertainty
Active Learning
Transition Novelty

http://resolver.tudelft.nl/uuid:6a7f1da8-88bd-44a4-9ef2-e9e7903e832f

Student theses

master thesis

© 2020 L. van Roessel