Saadoon, Muntadher and Hamid, Siti Hafizah Ab and Sofian, Hazrina and Altarturi, Hamza and Nasuha, Nur and Azizul, Zati Hakim and Sani, Asmiza Abdul and Asemi, Adeleh (2021) Experimental analysis in Hadoop MapReduce: A closer look at fault detection and recovery techniques. Sensors, 21 (11). ISSN 1424-8220, DOI https://doi.org/10.3390/s21113799.
Full text not available from this repository.Abstract
Hadoop MapReduce reactively detects and recovers faults after they occur based on the static heartbeat detection and the re-execution from scratch techniques. However, these techniques lead to excessive response time penalties and inefficient resource consumption during detection and recovery. Existing fault-tolerance solutions intend to mitigate the limitations without considering critical conditions such as fail-slow faults, the impact of faults at various infrastructure levels and the relationship between the detection and recovery stages. This paper analyses the response time under two main conditions: fail-stop and fail-slow, when they manifest with node, service, and the task at runtime. In addition, we focus on the relationship between the time for detecting and recovering faults. The experimental analysis is conducted on a real Hadoop cluster comprising MapReduce, YARN and HDFS frameworks. Our analysis shows that the recovery of a single fault leads to an average of 67.6% response time penalty. Even though the detection and recovery times are well-turned, data locality and resource availability must also be considered to obtain the optimum tolerance time and the lowest penalties.
| Item Type: | Article |
|---|---|
| Funders: | Malaysia Ministry of Education [Grant No: GPF097C-2020] |
| Uncontrolled Keywords: | Hadoop MapReduce; Experiment analysis; Fault-tolerance; Fault detection; Fault recovery |
| Subjects: | Q Science > QD Chemistry T Technology > TA Engineering (General). Civil engineering (General) |
| Divisions: | Faculty of Computer Science & Information Technology |
| Depositing User: | Ms Zaharah Ramly |
| Date Deposited: | 12 Jul 2022 04:45 |
| Last Modified: | 12 Jul 2022 04:45 |
| URI: | http://eprints.um.edu.my/id/eprint/33921 |
Actions (login required)
 |
View Item |
