Automatic feature discovery

Automatic feature discovery: A comparative study between filter and wrapper feature selection techniques

Mânăstireanu, Andrei (TU Delft Electrical Engineering, Mathematics and Computer Science)

Ionescu, A. (mentor)
Katsifodimos, A (mentor)
Isufi, E. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2023-06-28

The curse of dimensionality is a common challenge in machine learning, and feature selection techniques are commonly employed to address this issue by selecting a subset of relevant features. However, there is no consistently superior approach for choosing the most significant subset of features. We conducted a comprehensive analysis comparing filter and wrapper techniques to guide future work in selecting the most appropriate method based on specific circumstances. We quantified the performance of these techniques using a diverse collection of datasets. We utilised simple decision trees, linear machine learning algorithms, and support vector machines to assess the performance with varying percentages of features selected by the filter and wrapper techniques. The findings demonstrate that filter methods (Chi-Squared and ANOVA) perform better than wrapper methods (Forward Selection and Backward Elimination) regarding the classification accuracy, regression root mean squared error, and runtime.

Feature selection
Filter feature selection
Wrapper feature selection
Chi-Squared
ANOVA
Forward selection
Backward elimination

http://resolver.tudelft.nl/uuid:a4ac9242-e8c7-4e92-a281-cc5e53cebe28

Student theses

bachelor thesis

© 2023 Andrei Mânăstireanu