Leong, Chin Poh and Liew, Chee Sun and Chan, Chee Seng and Rehman, Muhammad Habib Ur (2021) Optimizing workflow task clustering using reinforcement learning. IEEE Access, 9. pp. 110614-110626. ISSN 2169-3536, DOI https://doi.org/10.1109/ACCESS.2021.3101454.
Full text not available from this repository.Abstract
Scientific workflows are composed of many fine-grained computational tasks. Generally, a large number of small tasks will slow down the workflow performance due to the scheduling overhead incurs during the execution time. Task clustering is an optimization technique that aggregates multiple small tasks into a large task to reduce the scheduling overhead, and thus it will reduce the overall workflow makespan, i.e. the total execution time taken by the resources to complete the execution of all of the tasks. However, finding an optimal clustering number is a big challenge as it usually requires manual intervention of experienced researchers to define the clustering parameter. In this paper, we proposed the use of reinforcement learning to tackle this problem by automating the discovery of the optimal clustering number for the submitted workflow. First, we model the workflow environment that allows the reinforcement learning agent to interact by determining the cluster number for every round of workflow execution. Then, based on the provenance records after the execution, the workflow environment will analyze the performance data and then determine either a reward or a punishment as the feedback to the reinforcement learning agent. The evaluation experiments are performed using real-world scientific workflow (e.g. Montage in this research), to demonstrate our model capability to identify the optimal cluster number and thus lay the groundwork for the adoption of reinforcement learning in workflow task clustering.
| Item Type: | Article |
|---|---|
| Funders: | Ministry of Education, Malaysia (LRGS-LR002-2020) |
| Uncontrolled Keywords: | Task analysis; Optimization; Reinforcement learning; Scheduling; Clustering algorithms; Processor scheduling; Monitoring; Clustering method; machine learning; workflow management software |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
| Divisions: | Faculty of Computer Science & Information Technology > Department of Computer System & Technology |
| Depositing User: | Ms. Juhaida Abd Rahim |
| Date Deposited: | 28 Mar 2022 03:25 |
| Last Modified: | 28 Mar 2022 03:25 |
| URI: | http://eprints.um.edu.my/id/eprint/26597 |
Actions (login required)
 |
View Item |
