Evaluating Dynamic Environment Difficulty for Obstacle Avoidance Benchmarking

Evaluating Dynamic Environment Difficulty for Obstacle Avoidance Benchmarking

SHI, MOJI (TU Delft Mechanical, Maritime and Materials Engineering; TU Delft Cognitive Robotics)

Alonso Mora, J. (mentor)
Chen, G. (mentor)
Wisse, M. (graduation committee)
Hamaza, S. (graduation committee)

Delft University of Technology

Mechanical Engineering | Vehicle Engineering | Cognitive Robotics

2023-08-30

Dynamic obstacle avoidance remains a crucial research area for autonomous systems, such as Micro Aerial Vehicles (MAVs) and service robots.
Efforts to develop dynamic collision avoidance techniques in unknown environments have proliferated in recent years. While these methods exhibit impressive and reliable performance in simpler environments, their efficacy in more challenging settings remains an area ripe for enhancement. The difficulty of these environments arises from a multitude of factors, and currently, no standardized approach exists to quantify this complexity. Additionally, to fairly compare different dynamic collision avoidance strategies, it's essential to assess them in environments with a similar degree of difficulty. Therefore, devising a metric capable of accurately gauging the intricacy of dynamic environments becomes imperative.


Building on this context, this master's thesis endeavors to fill this critical gap through three contributions: 1) The establishment and validation of map difficulty metrics that represent the difficulty of dynamic environments, 2) The introduction of a robust benchmarking pipeline to critically validate the representativeness of the proposed metrics and evaluate various collision avoidance strategies, and 3) The provision of a framework for comparative analysis of different planning strategies, utilizing the introduced map difficulty metric.

The proposed survivability metric effectively captures environmental complexity. Its validity is evidenced by a notable correlation with the success rates of typical collision avoidance methods, with over 1.7 million collision avoidance trials on over six hundred maps, securing a Spearman's Rank correlation coefficient (SRCC) of over 0.9. This metric serves as an indispensable tool for facilitating fair comparisons in this dynamic research domain. More importantly, it offers valuable insights for the future refinement and improvement of dynamic collision avoidance strategies, making a contribution to the continuous advancement of autonomous systems.

Dynamic Collision Avoidance
Benchmark Study
Measurement

http://resolver.tudelft.nl/uuid:ca95c8cb-8df3-4d43-9d17-c7b7f54eb1ea

Student theses

master thesis

© 2023 MOJI SHI