Deepfake Image detection using Anchored Pairwise Learning Approach

Deepfake Image detection using Anchored Pairwise Learning Approach

Sudharsan, S. (TU Delft Electrical Engineering, Mathematics and Computer Science)

Tax, D.M.J. (mentor)
Hung, H.S. (graduation committee)
Gadiraju, U.K. (graduation committee)

Delft University of Technology

Computer Science | Data Science and Technology

2020-08-24

Deep learning has enabled technologies that have been perceived complex or impossible a few years ago. Deep learning models can be used to solve several complex problem statements thereby making it a prominent field of research. With the advancements of Deep learning models, their application in domains have diversified. One prominent use-case is the generation of synthetic images or videos targeting unethical media manipulation. This raises concerns on implications caused in the society where there is loss of trust in digital media. The synthetic media generated using Deep learning models are called Deepfakes (portmanteau of "Deep learning" and "fake"). Deepfake generation leverages powerful techniques using Deep learning and Artificial Intelligence (AI) to manipulate or generate visual content with a high potential prospect of deception to human perception. The application of deepfakes is primarily in face images or videos. The use-case of synthetically generated media is a powerful tool to communicate information with false motives to public. However, synthetic media generation of faces is not new with Deep learning. Prior to deepfakes, synthetic face images and videos were generated using photo-editing softwares, primarily in the computer graphics domain. The inclusion of Deep learning methods has only facilitated higher realism of synthetic media with lesser time computation required.

With synthetic media being made easier to generate using Deep learning models, appropriate detection methods should also be deployed. This thesis focuses on detecting deepfakes using Face Recognition Systems. Deepfake detection methods have been proposed and implemented since 2018. However, the fundamental challenge in deepfake detection is the capability of the model to learn discriminatory features between real and deepfake media (owing to high similarity and realism). Furthermore, the availability of real images or videos of target individuals is lesser in comparison with the number of deepfakes available thereby posing a challenge for the detection models to further learn the real vs. deepfake features.

In this thesis, a modification to traditional Siamese Network using Pairwise Learning approach is made called Anchored Pairwise Learning, to learn the similarity between a pair of input faces to address the challenges in deepfake detection with regard to availability of real faces per individuals class. Anchored Pairwise Learning combines two proposed methods called Anchor Siamese Network (ASN) and Anchor Face Detector Network (AFDN) to perform a binary classification of images belonging to real or deepfake class by anchoring one of the test input pairs. The reported best AUC performance of the method on a test set of real and deepfake images is 91.9% and 81% on an independent hold-out set. Additionally, a comparative study between the proposed method and a baseline traditional learning face recognition method is made and their results based on model performances are discussed. Furthermore, the proposed method is evaluated with other state-of-the-art deepfake detection methods from a generalization performance on the reserved independent hold-out set. The results conclude that the Pairwise learning techniques outperform traditional learning techniques in scenarios where the number of real images are lesser compared to the deepfake images.

Deep Learning
Face Recognition
Media Forensics
Deepfake

http://resolver.tudelft.nl/uuid:aa4c6431-6880-406c-8429-af5b04bc3b05

Student theses

master thesis

© 2020 S. Sudharsan