Print Email Facebook Twitter Generalization by Visual Attention Title Generalization by Visual Attention Author Collé, Baptiste (TU Delft Electrical Engineering, Mathematics and Computer Science) Contributor Bohmer, Wendelin (mentor) Poulsen, C.B. (graduation committee) Degree granting institution Delft University of Technology Programme Computer Science and Engineering Project CSE3000 Research Project Date 2022-06-24 Abstract Most deep learning models fail to generalize in production. Indeed, sometimes data used during training does not completely reflect the deployed environment. The test data is then considered out-of-distribution compared to the training data. In this paper, we focus on out-of-distribution performance for image classification. In fact, transformers, which are a novel neural network architecture compared to the more traditionally used convolutional neural networks (CNN), have been shown to work well for image classification. This is why, in this paper, we firstly explore the different capabilities of both models on out-of-distribution. This is then followed by an in-depth investigation of individual architectural components of the transformer and their impact on the generalization capability of the model. Subject CNNout-of-distributionTransformerMHAattention To reference this document use: http://resolver.tudelft.nl/uuid:44653bb4-bc7a-42c4-9621-7d12fb759c4b Part of collection Student theses Document type bachelor thesis Rights © 2022 Baptiste Collé Files PDF baptiste_colle_bachelor_thesis.pdf 2.43 MB Close viewer /islandora/object/uuid:44653bb4-bc7a-42c4-9621-7d12fb759c4b/datastream/OBJ/view