Teh, Xin Yu and Yeoh, Pauline Shan Qing and Wang, Tao and Wu, Xiang and Hasikin, Khairunnisa and Lai, Khin Wee (2024) Knee Osteoarthritis Diagnosis With Unimodal and Multi-Modal Neural Networks: Data From the Osteoarthritis Initiative. IEEE Access, 12. pp. 146698-146717. ISSN 2169-3536, DOI https://doi.org/10.1109/ACCESS.2024.3472654.
Full text not available from this repository.Abstract
Knee osteoarthritis (OA) is a prevalent musculoskeletal condition affecting millions worldwide, posing significant health and economic burdens. Characterized by the degeneration of joint cartilage, the progression of knee OA varies significantly among individuals, making its prediction a complex issue. Previous studies on automated knee OA diagnosis have primarily relied on unimodal data, often overlooking the valuable information present in multi-modal data. Multi-modal learning, which integrates information from various modalities, is increasingly recognized for its potential to enhance diagnostic performance in medical applications. However, such models incur a higher computational load due to the additional data required. This research investigates the feasibility of multi-modal neural networks in knee OA diagnosis by integrating structural demographic data with unstructured imaging data. Three deep learning unimodal models (InceptionV3, DIKO, and EfficientNetv2) were transformed into multi-modal architectures (MF_InceptionNet, MF_DIKO, and MF_Eff) to compare their diagnostic capabilities. The proposed multi-modal models share a common architecture, with unimodal models acting as image feature extraction backbones and separate embedding layers for demographic data. The image features and demographic embeddings are combined into a unified vector before classification. Extensive experiments were conducted to evaluate the performance of these models across different class categories and dataset sizes. MF_DIKO and InceptionV3 emerged as the best multi-modal and unimodal neural networks, respectively, with overall accuracies of 0.67 and 0.75 for 3-class severity classification. Contrary to existing literature, our findings reveal that unimodal neural networks using only imaging features outperform multi-modal networks, suggesting unimodal models might suffice in certain applications.
| Item Type: | Article |
|---|---|
| Funders: | Xuzhou Science and Technology Project (KC21182), Unveiling & Leading Project of Xuzhou Medical University (XZHMU) (JBGS202204), Ministry of Education, Malaysia, Universiti Malaya (FRGS/1/2023/SKK05/UM/02/2) |
| Uncontrolled Keywords: | Knee osteoarthritis; deep learning; X-ray; multi-modal fusion; multi-modal fusion; classification; classification; X-ray; multi-modal fusion; classification |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science R Medicine > R Medicine (General) T Technology > TK Electrical engineering. Electronics Nuclear engineering |
| Divisions: | Faculty of Engineering > Biomedical Engineering Department |
| Depositing User: | Ms. Juhaida Abd Rahim |
| Date Deposited: | 09 Dec 2024 03:37 |
| Last Modified: | 09 Dec 2024 03:37 |
| URI: | http://eprints.um.edu.my/id/eprint/47130 |
Actions (login required)
 |
View Item |
