Exploiting visual-based intent classification for diverse social image retrieval

Exploiting visual-based intent classification for diverse social image retrieval

Wang, Bo (External organisation)
Larson, M.A. (TU Delft Multimedia Computing; Radboud Universiteit Nijmegen)

Gravier, Guillaume (editor)
Bischke, Benjamin (editor)
Demarty, Claire-Hélène (editor)
Zaharieva, Maia (editor)
Riegler, Michael (editor)
Dellandrea, Emmanuel (editor)
Bogdanov, Dmitry (editor)
Sutcliffe, Richard (editor)
Jones, Gareth J.F. (editor)
Larson, Martha (editor)

2017

In the 2017 MediaEval Retrieving Diverse Social Images task, we (TUD-MMC team) propose a novel method, namely an intent-based approach, for social image search result diversification. The underlying assumption is that the visual appearance of social images is impacted by the underlying photographic act, i.e., why the images were taken. Better understanding the rationale behind the photographic act could potentially benefit social image search result diversification. To investigate this idea, we employ a manual content analysis approach to create a taxonomy of intent classes. Our experiments show that a CNN-based neural network classifier is able to capture the visual difference between the classes in the intent taxonomy. We cluster images of the Flickr baseline based on predicted intent class and generate a re-ranked list by alternating images from different clusters. Our results reveal that, compared to conventional diversification strategies, intent-based search result diversification is able to bring a considerable improvement in terms of cluster recall with several extra benefits.

http://resolver.tudelft.nl/uuid:c15a255b-65b6-4328-8df7-9f579c48b14d

Working Notes Proceedings of the MediaEval 2017 Workshop

MediaEval 2017, 2017-09-13 → 2017-09-15, Dublin, Ireland

CEUR Workshop Proceedings, 1613-0073, 1984

Institutional Repository

conference paper

© 2017 Bo Wang, M.A. Larson