A Hybrid Machine Learning Model for Streaming Frequency-Based Anomaly Detection in Banking Transactions

Abstract

The increasing sophistication of fraudulent activities in digital financial systems necessitates real-time anomaly detection models that can adapt to evolving transactional behaviours. While prior research has explored machine learning approaches for fraud detection, most rely on static datasets and overlook temporal dependencies inherent in financial transaction streams. This study addresses this critical gap by proposing a hybrid anomaly detection model that integrates dynamic streaming-frequency analysis via a 7-day sliding window, an unsupervised Long Short-Term Memory (LSTM) -autoencoder for anomaly scoring, and a supervised Artificial Neural Network (ANN) for classification. The sliding window mechanism enables the model to capture short-term temporal fluctuations and behavioural patterns, aligning with the streaming nature of financial data. The LSTM-autoencoder is trained exclusively on normal transaction sequences to learn temporal dependencies and compute reconstruction errors, which serve as deep anomaly features. These features are then fed into the ANN to classify transactions as normal or anomalous. Experimental results on the IBM Anti-Money Laundering (AML) dataset demonstrate the effectiveness of the proposed framework, achieving a classification accuracy of 99.92%, precision of 94.12%, recall of 88.43%, and F1 score of 91.19%. This layered architecture not only enhances early detection of anomalous behaviour but also provides a scalable, adaptive solution for real-time fraud detection in streaming financial environments.