Introduction
With the evolution of AI technology, Deepfake technology has emerged, which makes it easier to forge the content of film or pictures and used to make pornographic, political propaganda or mocking films.We build a website which can generate deepfake videos and detect deepfake videos in real time. The implementation is based on deep learning and computer vision.
Flow
Dataset
We selected 5 datasets provided by the Faceforensics paper, including the original video and the video generated by 4 different fake video methods. The four methods are "Deepfakes" , "FaceSwap" , "Face2Face" and "NeuralTextures"; each dataset has a total of 1000 videos, and then divided into 720, 140, and 140 videos according to the training set, validation set, and test set.
Model
(1) Lightweight Model: The original paper used XceptionNet as training model, it reached high detection accuracy, but we choose MobileNet as our model, because the architecture of MobileNet is much more smaller, also the performance is not much worse than the XceptionNet.
(2) Cross Dataset: In the beginning, we separately trained the original videos and the fake videos of each generation method with a ratio of true to false 1:1. The training results are good, but since the manipulation method of the fake videos to be detected is unknown, so we also trained a Cross Dataset model, using the dataset consists of the original video and four types of fake videos for training. After testing, although the accuracy has dropped slightly, the model still can detect fake videos from unknown sources efficiently.
Performance
In Table 1, we can see that our cross-dataset model have different performance on different datasets. Except Pristine data, the detection accuracy using mobilenet is higher than the accuracy using XceptionNet. The reason that the detection accuracy of pristine data is relatively low is because we have unbalanced class of training data
In Table 2 and 3, we can see that after using MobileNet, the detection accuracy is a bit lower than using XceptionNet, but the inference time has improved a lot, it can save a bunch of times when detecting fake videos.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Team
張廖祐祺
李禹承
清大資工四年級
清大資工四年級