AI-based Forensic Analysis of Lip-Sync Consistency in Digital Video Content

Abstract

Deepfake technology has advanced rapidly, enabling the creation of highly realistic synthetic videos that pose significant threats to information integrity, security, and digital forensics. Among these, lip-sync forgeries, in which mouth movements are altered to match unrelated audio, are particularly difficult to detect because of their natural-looking synchronisation. This study proposes an AI-based forensic system that analyses lip-sync consistency by comparing audio and visual features to identify manipulated videos. The system integrates a pre-trained SyncNet model that extracts Mel-Frequency Cepstral Coefficients (MFCCs) as audio features and Convolutional Neural Network (CNN)-based visual features for lip movements. We trained a Random Forest classifier using the sync confidence score and audio visual offset to classify videos as real or fake. The system was evaluated on a FakeAVCeleb dataset, achieving 99% accuracy in distinguishing real from fake videos. Fake videos generally showed lower audio visual sync confidence compared to real videos. We also developed a web-based application for real-time forensic analysis. This study contributes to digital forensics by providing a practical and high accuracy tool for detecting lip-sync deepfakes. This tool helps forensic investigators, journalists, and cybersecurity agencies verify video authenticity, strengthen media integrity, and combat misinformation.