Print Email Facebook Twitter A Systematic Comparison of Search-Based Approaches for LDA Hyperparameter Tuning Title A Systematic Comparison of Search-Based Approaches for LDA Hyperparameter Tuning Author Panichella, A. (TU Delft Software Engineering) Date 2021 Abstract Context: Latent Dirichlet Allocation (LDA) has been successfully used in the literature to extract topics from software documents and support developers in various software engineering tasks. While LDA has been mostly used with default settings, previous studies showed that default hyperparameter values generate sub-optimal topics from software documents. Objective: Recent studies applied meta-heuristic search (mostly evolutionary algorithms) to configure LDA in an unsupervised and automated fashion. However, previous work advocated for different meta-heuristics and surrogate metrics to optimize. The objective of this paper is to shed light on the influence of these two factors when tuning LDA for SE tasks. Method: We empirically evaluated and compared seven state-of-the-art meta-heuristics and three alternative surrogate metrics (i.e., fitness functions) to solve the problem of identifying duplicate bug reports with LDA. The benchmark consists of ten real-world and open-source projects from the Bench4BL} dataset. Results: Our results indicate that (1) meta-heuristics are mostly comparable to one another (except for random search and CMA-ES), and (2) the choice of the surrogate metric impacts the quality of the generated topics and the tuning overhead. Furthermore, calibrating LDA helps identify twice as many duplicates than untuned LDA when inspecting the top five past similar reports. Conclusion: No meta-heuristic and/or fitness function outperforms all the others, as advocated in prior studies. However, we can make recommendations for some combinations of meta-heuristics and fitness functions over others for practical use. Future work should focus on improving the surrogate metrics used to calibrate/tune LDA in an unsupervised fashion Subject Duplicate bug reportHyperparameter optimizationLatent dirichlet allocationMetaheuristic searchSearch-based software engineeringTopic modeling To reference this document use: http://resolver.tudelft.nl/uuid:b5eece97-0427-4415-aa21-0900183ae116 DOI https://doi.org/10.1016/j.infsof.2020.106411 ISSN 0950-5849 Source Information and Software Technology, 130 Part of collection Institutional Repository Document type journal article Rights © 2021 A. Panichella Files PDF 1_s2.0_S0950584920300069_main.pdf 1.22 MB Close viewer /islandora/object/uuid:b5eece97-0427-4415-aa21-0900183ae116/datastream/OBJ/view