Multi-leader Adaptive Cruise Control Systems considering Sensor Measurement Uncertainties based on Deep Reinforcement Learning

Multi-leader Adaptive Cruise Control Systems considering Sensor Measurement Uncertainties based on Deep Reinforcement Learning

Ni, Ying-Chuan (TU Delft Civil Engineering and Geosciences)

Knoop, V.L. (mentor)
van Arem, B. (graduation committee)
Kooij, J.F.P. (graduation committee)

Delft University of Technology

Civil Engineering | Transport and Planning

2022-08-25

Adaptive Cruise Control (ACC) relieves human drivers’ tasks by taking over the control of the throttle and braking of the vehicles automatically. However, it has been demonstrated in many empirical studies that current production ACC systems fail to guarantee string stability. It is believed that if vehicles can take the longitudinal dynamics further downstream into account and react to the propagating disturbance earlier, the string stability in the platoon may be improved. Instead of relying on inter-vehicle communication technologies, the ego-vehicle should be able to detect the second leading vehicle by leveraging the power of on-board sensors. Still, the second leader measurements can be highly erroneous. Therefore, it is important to consider the entailed measurement uncertainties when designing and evaluating such ACC systems. This study proposes several ACC systems which possess the property of multi-anticipation and uncertainty handling.

The possible sensor technology which can collect the second leader measurements is first investigated. Based on the considered setup, the measurement uncertainties are modelled to reflect the real-world conditions. The ACC system architecture and control system design method are then proposed. Deep reinforcement learning is applied for the controller design in light of its great potential in describing the complex non-linear control task and handling the uncertainties. Kalman filters and recurrent policies with a Long-Short-Term-Memory network are applied to cope with uncertain measurements. The first method estimates the state information before feeding it back to the controller agent, while the latter incorporates the state estimator into the controller to actively consider the uncertainties while making decisions.

A numerical simulation approach is adopted to theoretically assess the performance of the proposed ACC systems. A traffic disturbance event and multiple levels of measurement noise are considered in the experiment. To analyze the performance in terms of string stability and ride comfort and understand the car-following behavior mechanism resulted from the proposed systems, a quantitative analysis framework is developed.

The evaluation results demonstrate the applied learning-based approach succeeds to train ACC control policies which can ensure string stability. It is also found that the multi-anticipation ability significantly improves the string stability and ride comfort performance. In the scenarios with measurement noise, systems using the tuned Kalman filters exhibit the ideal level of string stability performance. However, ride comfort cannot be guaranteed in scenarios with large measurement noise. On the other hand, systems using recurrent policies can better ensure ride comfort performance while maintaining string stability at certain levels. Based on the results, the performance limits of the proposed ACC systems in the handling of measurement uncertainties are explored. In addition, with the different policy training setups, the trade-off between these two performance aspects is shown.

The findings of this study are anticipated to trigger the development of advanced multi-leader ACC system by automakers, sensor manufacturers, and traffic engineers. Future work can be directed to an enhanced controller design. Robustness of the systems with respect to other sources of measurement uncertainties, more types of traffic disturbance, and platoon heterogeneity is worth further design consideration and analysis.

adaptive cruise control
multi-anticipation
deep reinforcement learning
measurement uncertainties
Kalman filter
string stability
ride comfort

http://resolver.tudelft.nl/uuid:08dea442-af73-4b35-b4de-e257287bfc6a

Student theses

master thesis

© 2022 Ying-Chuan Ni