Alkawaz, Ali Najem and Kanesan, Jeevan and Khairuddin, Anis Salwa Mohd and Badruddin, Irfan Anjum and Kamangar, Sarfaraz and Hussien, Mohamed and Baig, Maughal Ahmed Ali and Ahammad, N. Ameer (2023) Training multilayer neural network based on optimal control theory for limited computational resources. Mathematics, 11 (3). ISSN 2227-7390, DOI https://doi.org/10.3390/math11030778.
Full text not available from this repository.Abstract
Backpropagation (BP)-based gradient descent is the general approach to train a neural network with a multilayer perceptron. However, BP is inherently slow in learning, and it sometimes traps at local minima, mainly due to a constant learning rate. This pre-fixed learning rate regularly leads the BP network towards an unsuccessful stochastic steepest descent. Therefore, to overcome the limitation of BP, this work addresses an improved method of training the neural network based on optimal control (OC) theory. State equations in optimal control represent the BP neural network's weights and biases. Meanwhile, the learning rate is treated as the input control that adapts during the neural training process. The effectiveness of the proposed algorithm is evaluated on several logic gates models such as XOR, AND, and OR, as well as the full adder model. Simulation results demonstrate that the proposed algorithm outperforms the conventional method in terms of improved accuracy in output with a shorter time in training. The training via OC also reduces the local minima trap. The proposed algorithm is almost 40% faster than the steepest descent method, with a marginally improved accuracy of approximately 60%. Consequently, the proposed algorithm is suitable to be applied on devices with limited computation resources, since the proposed algorithm is less complex, thus lowering the circuit's power consumption.
| Item Type: | Article |
|---|---|
| Funders: | Faculty Research Grant (FRG) of Universiti Malaya [Grant No: GPF055A-2020], King Khalid University [Grant No: RGP.1/74/43] |
| Uncontrolled Keywords: | Multilayer neural network; Optimal control; Pontryagin minimum principle; Backpropagation; Logic gates |
| Subjects: | T Technology > T Technology (General) T Technology > TK Electrical engineering. Electronics Nuclear engineering |
| Divisions: | Faculty of Engineering > Department of Electrical Engineering |
| Depositing User: | Ms Zaharah Ramly |
| Date Deposited: | 12 Nov 2024 06:47 |
| Last Modified: | 12 Nov 2024 06:47 |
| URI: | http://eprints.um.edu.my/id/eprint/38665 |
Actions (login required)
 |
View Item |
