Habeeb, Riyaz Ahamed Ariyaluran and Nasaruddin, Fariza and Gani, Abdullah and Amanullah, Mohamed Ahzam and Hashem, Ibrahim Abaker Targio and Ahmed, Ejaz and Imran, Muhammad (2022) Clustering-based real-time anomaly detection-A breakthrough in big data technologies. Transactions on Emerging Telecommunications Technologies, 33 (8, SI). ISSN 2161-3915, DOI https://doi.org/10.1002/ett.3647.
Full text not available from this repository.Abstract
Off late, the ever increasing usage of a connected Internet-of-Things devices has consequently augmented the volume of real-time network data with high velocity. At the same time, threats on networks become inevitable; hence, identifying anomalies in real time network data has become crucial. To date, most of the existing anomaly detection approaches focus mainly on machine learning techniques for batch processing. Meanwhile, detection approaches which focus on the real-time analytics somehow deficient in its detection accuracy while consuming higher memory and longer execution time. As such, this paper proposes a novel framework which focuses on real-time anomaly detection based on big data technologies. In addition, this paper has also developed streaming sliding window local outlier factor coreset clustering algorithms (SSWLOFCC), which was then implemented into the framework. The proposed framework that comprises BroIDS, Flume, Kafka, Spark streaming, SparkMLlib, Matplot and HBase was evaluated to substantiate its efficacy, particularly in terms of accuracy, memory consumption, and execution time. The evaluation is done by performing critical comparative analysis using existing approaches, such as K-means, hierarchical density-based spatial clustering of applications with noise (HDBSCAN), isolation forest, spectral clustering and agglomerative clustering. Moreover, Adjusted Rand Index and memory profiler package were used for the evaluation of the proposed framework against the existing approaches. The outcome of the evaluation has substantially proven the efficacy of the proposed framework with a much higher accuracy rate of 96.51% when compared to other algorithms. Besides, the proposed framework also outperformed the existing algorithms in terms of lesser memory consumption and execution time. Ultimately the proposed solution enable analysts to precisely track and detect anomalies in real time.
| Item Type: | Article |
|---|---|
| Funders: | King Saud University [RG-1435-051] |
| Uncontrolled Keywords: | Detection system; Framework; Internet |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
| Divisions: | Faculty of Computer Science & Information Technology |
| Depositing User: | Ms. Juhaida Abd Rahim |
| Date Deposited: | 21 Sep 2023 02:53 |
| Last Modified: | 21 Sep 2023 02:53 |
| URI: | http://eprints.um.edu.my/id/eprint/41377 |
Actions (login required)
 |
View Item |
