Skip to main content Skip to journal navigation menu Skip to site footer
contact@ukscip.com
Article

Evaluating Deep Learning Architectures for Effective Detection of Manipulated Media across Multiple Datasets

Achraf Ibnouzaher ORCID ,  Noureddine Moumkine ORCID
    Received: 1 February 2026; Revised: 4 March 2026; Accepted: 25 March 2026; Published: 15 July 2026

    Abstract

    The advancement of artificial intelligence has enabled the creation of highly realistic synthetic media raising serious concerns about misinformation, identity manipulation, and digital security. As these technologies continue to evolve and become more accessible the potential misuse of manipulated media increases significantly making it easier to spread deceptive content across digital platforms. This growing threat highlights the urgent need for reliable and efficient deepfake detection techniques capable of identifying manipulated content and limiting its potential societal and technological impact. Deep Learning (DL) has emerged as a promising solution for automatically identifying manipulated multimedia content. In this study, we harnessed Transfer Learning (TL) techniques to fine-tune and evaluate the performance of seven different DL models including EfficientNetB5, XceptionNet, EfficientNet3D, Vision Transformer (ViT), CrossViT, ViViT, and CoAtNet. These models were trained and evaluated on three widely used deepfake detection datasets: FaceForensics++, the DeepFake Detection Challenge (DFDC), and CelebDF. Our experimental results demonstrate that hybrid approaches combining Convolutional Neural Networks (CNNs) with Vision Transformer-based architectures outperform individual models in detecting manipulated media. Furthermore, the findings indicate that models trained on the FaceForensics++ and DFDC datasets exhibit stronger generalization capabilities when applied to previously unseen deepfake samples.

    Keywords

    References

    1. 1. Nguyen, T.T.; Nguyen, Q.V.H.; Nguyen, D.T.; et al. Deep learning for deepfakes creation and detection: A survey. Comput. Vis. Image Underst. 2022, 223, 103525. DOI: https://doi.org/10.1016/j.cviu.2022.103525
    2. 2. Flattery, T.; Miller, C.B. Deepfakes and dishonesty. Philos. Technol. 2024, 37, 120. DOI: https://doi.org/10.1007/s13347-024-00812-1
    3. 3. Salman, S.; Shamsi, J.A.; Qureshi, R. Deep fake generation and detection: Issues, challenges, and solutions. IT Prof. 2023, 25, 52–59. DOI: https://doi.org/10.1109/MITP.2022.3230353
    4. 4. Agarwal, S.; Farid, H.; Gu, Y.; et al. Protecting world leaders against deep fakes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Long Beach, CA, USA, 16–20 June 2019.
    5. 5. Klepper, D. Creating realistic deepfakes is getting easier than ever. Fighting back may take even more AI. Available online: https://www.ap.org/news-highlights/spotlights/2025/creating-realistic-deepfakes-is-getting-easier-than-ever-fighting-back-may-take-even-more-ai/ (accessed on 12 December 2025).
    6. 6. Heidari, A.; Jafari Navimipour, N.; Dag, H.; et al. Deepfake detection using deep learning methods: A systematic and comprehensive review. WIREs Data Min. Knowl. Discov. 2024, 14, e1520. DOI: https://doi.org/10.1002/widm.1520
    7. 7. Malik, A.; Kuribayashi, M.; Abdullahi, S.M.; et al. DeepFake Detection for Human Face Images and Videos: A Survey. IEEE Access 2022, 10, 18757–18775. DOI: https://doi.org/10.1109/ACCESS.2022.3151186
    8. 8. Gupta, G.; Raja, K.; Gupta, M.; et al. A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods. Electronics 2024, 13, 95. DOI: https://doi.org/10.3390/electronics13010095
    9. 9. Rajput, T.; Arora, B. A Systematic Review of Deepfake Detection Using Learning Techniques and Vision Transformer. In Lecture Notes in Networks and Systems; Springer: Singapore, 2024. DOI: https://doi.org/10.1007/978-981-97-2550-2_17
    10. 10. Lu, T.; Bao, Y.; Li, L. Deepfake Video Detection Based on Improved CapsNet and Temporal–Spatial Features. Comput. Mater. Contin. 2023, 75, 715–740. DOI: https://doi.org/10.32604/cmc.2023.034963
    11. 11. Yang, W.; Zhou, X.; Chen, Z.; et al. AVoiD-DF: Audio-Visual Joint Learning for Detecting Deepfake. IEEE Trans. Inf. Forensics Secur. 2023, 18, 2015–2029. DOI: https://doi.org/10.1109/TIFS.2023.3262148
    12. 12. Elhassan, A.; Al-Fawa’reh, M.; Jafar, M.T.; et al. DFT-MF: Enhanced deepfake detection using mouth movement and transfer learning. SoftwareX 2022, 19, 101115. DOI: https://doi.org/10.1016/j.softx.2022.101115
    13. 13. Ibnouzaher, A.; Moumkine, N. Enhanced Deepfake Detection Using a Multi-model Approach. In Digital Technologies and Applications; Springer: Cham, Switzerland, 2024; pp. 317–325. DOI: https://doi.org/10.1007/978-3-031-68675-7_31
    14. 14. Ibnouzaher, A.; Moumkine, N. Vgg-ViT: A Framework for Deepfakes Images Detection. In Networked Systems; Springer: Cham, Switzerland, 2026; pp. 241–252. DOI: https://doi.org/10.1007/978-3-032-00347-8_16
    15. 15. Stanciu, D.C.; Ionescu, B. Autoencoder-based Data Augmentation for Deepfake Detection. In Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation, Thessaloniki, Greece, 12–15 June 2023. DOI: https://doi.org/10.1145/3592572.3592840
    16. 16. Rössler, A.; Cozzolino, D.; Verdoliva, L.; et al. FaceForensics++: Learning to detect manipulated facial images. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea, 27 October–2 November 2019. DOI: https://doi.org/10.1109/ICCV.2019.00009
    17. 17. Chollet, F. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017. DOI: https://doi.org/10.1109/CVPR.2017.195
    18. 18. Echizen, I.; Afchar, D.; Nozick, V.; et al. MesoNet: A compact facial video forgery detection network. In Proceedings of the 2018 IEEE International Workshop on Information Forensics and Security (WIFS), Hong Kong, China, 11–13 December 2018. DOI: https://doi.org/10.1109/WIFS.2018.8630761
    19. 19. Cheng, J.; Sabir, E.; Jaiswal, A.; et al. Recurrent Convolutional Strategies for Face Manipulation Detection in Videos. arXiv preprint 2019, arXiv:1905.00582.
    20. 20. Wu, J.; Hu, W.; Wen, Y.; et al. Skin lesion classification using densely connected convolutional networks with attention residual learning. Sensors 2020, 20, 7080. DOI: https://doi.org/10.3390/s20247080
    21. 21. He, K.; Zhang, X.; Ren, S.; et al. Deep residual learning for image recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016. DOI: https://doi.org/10.1109/CVPR.2016.90
    22. 22. Ciftci, U.A.; Demir, I.; Yin, L. FakeCatcher: Detection of synthetic portrait videos using biological signals. IEEE Trans. Pattern Anal. Mach. Intell. 2023. DOI: https://doi.org/10.1109/TPAMI.2020.3009287
    23. 23. Li, Y.; Yang, X.; Sun, P.; et al. Celeb-DF: A large-scale challenging dataset for deepfake forensics. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020. DOI: https://doi.org/10.1109/CVPR42600.2020.00327
    24. 24. Zhu, X.; Wang, H.; Fei, H.; et al. Face forgery detection by 3D decomposition. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 20–25 June 2020. DOI: https://doi.org/10.1109/CVPR46437.2021.00295
    25. 25. Dolhansky, B.; Bitton, J.; Pflaum, B.; et al. The DeepFake Detection Challenge (DFDC) dataset. arXiv preprint 2020, arXiv:2006.07397.
    26. 26. Gully, A.; Dufour, N. Contributing data to deepfake detection research. Available online: https://research.google/blog/contributing-data-to-deepfake-detection-research/ (accessed on 12 December 2025).
    27. 27. Wang, J.; Wu, Z.; Ouyang, W.; et al. M2TR: Multi-modal multi-scale transformers for deepfake detection. In Proceedings of the 2022 International Conference on Multimedia Retrieval, Newark, NJ, USA, 27–30 June 2022. DOI: https://doi.org/10.1145/3512527.3531415
    28. 28. Coccomini, D.A.; Messina, N.; Gennaro, C.; et al. Combining EfficientNet and Vision Transformers for video deepfake detection. In Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2022. DOI: https://doi.org/10.1007/978-3-031-06433-3_19
    29. 29. Zhao, C.; Wang, C.; Hu, G.; et al. ISTVT: Interpretable spatial-temporal video transformer for deepfake detection. IEEE Trans. Inf. Forensics Secur. 2023, 18, 1335–1348. DOI: https://doi.org/10.1109/TIFS.2023.3239223
    30. 30. Ramadhani, K.N.; Munir, R.; Utama, N.P. Improving video vision transformer for deepfake video detection using facial landmark, depthwise separable convolution and self attention. IEEE Access 2024, 12, 8932–8939. DOI: https://doi.org/10.1109/ACCESS.2024.3352890
    31. 31. Face detection guide for Python. Available online: https://ai.google.dev/edge/mediapipe/solutions/vision/face_detector/python (accessed on 4 January 2023).
    32. 32. MarekKowalski/FaceSwap. Available online: https://github.com/MarekKowalski/FaceSwap/ (accessed on 16 February 2025).
    33. 33. Thies, J.; Zollhöfer, M.; Stamminger, M.; et al. Demo of Face2Face: Real-time face capture and reenactment of RGB videos. In Proceedings of the ACM SIGGRAPH 2016 Emerging Technologies, Anaheim, CA, USA, 24–28 July 2016. DOI: https://doi.org/10.1145/2929464.2929475
    34. 34. Thies, J.; Zollhöfer, M.; Nießner, M. Deferred neural rendering: Image synthesis using neural textures. ACM Trans. Graph. 2019, 38, 66. DOI: https://doi.org/10.1145/3306346.3323035
    35. 35. Deepfakes/faceswap. Available online: https://github.com/deepfakes/faceswap (accessed on 16 February 2025).
    36. 36. Melnik, A.; Miasayedzenkau, M.; Makaravets, D.; et al. Face generation and editing with StyleGAN: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 2024, 46, 3557–3576. DOI: https://doi.org/10.1109/TPAMI.2024.3350004
    37. 37. Nirkin, Y.; Keller, Y.; Hassner, T. FSGAN: Subject-agnostic face swapping and reenactment. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea, 27 October–2 November 2019. DOI: https://doi.org/10.1109/ICCV.2019.00728
    38. 38. Doukas, M.C.; Ververas, E.; Sharmanska, V.; et al. Free-HeadGAN: Neural talking head synthesis with explicit gaze control. IEEE Trans. Pattern Anal. Mach. Intell. 2023, 45, 9743–9756. DOI: https://doi.org/10.1109/TPAMI.2023.3253243
    39. 39. Das, M.K.; Kumar, M.; Kapil, I.K.; et al. Deepfake creation using GANs and autoencoder and deepfake detection. In Proceedings of the 2023 2nd International Conference on Vision Towards Emerging Trends in Communication and Networking Technologies (ViTECoN), Vellore, India, 5–6 May 2023. DOI: https://doi.org/10.1109/ViTECoN58111.2023.10157962
    40. 40. Reinhard, E.; Ashikhmin, M.; Gooch, B.; et al. Color transfer between images. IEEE Comput. Graph. Appl. 2001, 21, 36–43. DOI: https://doi.org/10.1109/38.946629
    41. 41. Ma, L.; Jia, X.; Sun, Q.; et al. Pose Guided Person Image Generation. arXiv preprint 2017, arXiv:1705.09368.
    42. 42. Szegedy, C.; Liu, W.; Jia, Y.; et al. Going deeper with convolutions. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015. DOI: https://doi.org/10.1109/CVPR.2015.7298594
    43. 43. Fei, H.; Zhu, X.; Zhang, B.; et al. Face forgery detection by 3D decomposition and composition search. IEEE Trans. Pattern Anal. Mach. Intell. 2023, 45, 8342–8357. DOI: https://doi.org/10.1109/TPAMI.2022.3233586
    44. 44. Tan, M.; Le, Q.V. EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, Long Beach, CA, USA, 9–15 June 2019.
    45. 45. Noor, A.; Benjdira, B.; Ammar, A.; et al. DriftNet: Aggressive driving behavior classification using 3D EfficientNet architecture. arXiv preprint 2020, arXiv:2004.11970.
    46. 46. Zheng, B.; Gao, A.; Huang, X.; et al. A modified 3D EfficientNet for the classification of Alzheimer’s disease using structural magnetic resonance images. IET Image Process. 2023, 17, 77–87. DOI: https://doi.org/10.1049/ipr2.12618
    47. 47. Vaswani, A.; Shazeer, N.; Parmar, N.; et al. Attention is all you need. In Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017.
    48. 48. Khan, S.; Naseer, M.; Hayat, M.; et al. Transformers in vision: A survey. ACM Comput. Surv. 2022, 54, 200. DOI: https://doi.org/10.1145/3505244
    49. 49. Chen, C.F.; Fan, Q.; Panda, R. CrossViT: Cross-attention multi-scale vision transformer for image classification. In Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 10–17 October 2021. DOI: https://doi.org/10.1109/ICCV48922.2021.00041
    50. 50. Wang, Y.; Deng, Y.; Zheng, Y.; et al. Vision transformers for image classification: A comparative survey. Technologies 2025, 13, 32. DOI: https://doi.org/10.3390/technologies13010032
    51. 51. Arnab, A.; Dehghani, M.; Heigold, M.; et al. ViViT: A video vision transformer. In Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 10–17 October 2021. DOI: https://doi.org/10.1109/ICCV48922.2021.00676
    52. 52. Dai, Z.; Liu, H.; Le, Q.V.; et al. CoAtNet: Marrying convolution and attention for all data sizes. In Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021), Online, 6–14 December 2021.
    53. 53. Hanley, J.A.; McNeil, B.J. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982, 143. DOI: https://doi.org/10.1148/radiology.143.1.7063747
    54. 54. Trisha, M.; Bhattacharya, U.; Chandra, R.; et al. Emotions don’t lie: An audio-visual deepfake detection method using affective cues. In Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA, 12–16 October 2020. DOI: https://doi.org/10.1145/3394171.3413570
    55. 55. Saikia, P.; Dholaria, D.; Yadav, P.; et al. A hybrid CNN-LSTM model for video deepfake detection by leveraging optical flow features. In Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN), Padua, Italy, 18–23 July 2022. DOI: https://doi.org/10.1109/IJCNN55064.2022.9892905
    56. 56. Nguyen, H.H.; Yamagishi, J.; Echizen, I. Capsule-forensics: Using capsule networks to detect forged images and videos. In Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 12–17 May 2019. DOI: https://doi.org/10.1109/ICASSP.2019.8682602
    57. 57. Hu, J.; Liao, X.; Wang, W.; et al. Detecting compressed deepfake videos in social networks using frame-temporality two-stream convolutional network. IEEE Trans. Circuits Syst. Video Technol. 2022, 32, 1089–1102. DOI: https://doi.org/10.1109/TCSVT.2021.3074259
    ×