Biswas, Kunal and Shivakumara, Palaiahnakote and Pal, Umapada and Liu, Cheng-Lin and Lu, Yue (2023) VQAPT: A New visual question answering model for personality traits in social media images. Pattern Recognition Letters, 175. pp. 66-73. ISSN 0167-8655, DOI https://doi.org/10.1016/j.patrec.2023.10.016.
Full text not available from this repository.Abstract
Visual Question Answering (VQA) for personality trait images on social media is challenging because of multiple emotions and actions with complex backgrounds in social media images. This work aims at developing a new VQA model for different personality traits (VQAPT) identification in a single image. This work considers the Big Five Factors (BFF) for personality traits namely, Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism. VQA is proposed based on the observation that multiple personality traits can be seen in a single image. We propose a model integrating text recognition and person/face recognition to derive the unique relationship between the text and the person's action in the image. Furthermore, a dynamic text-object graph for personality traits identification is constructed according to the query. For understanding a query, we explore the Contrastive Language-Image Pre-trained (CLIP) transformer encoder in this work. Since it is the first work of its kind, we have created a new dataset under this work for evaluation and the dataset is available publicly as mentioned in Section 4. The effectiveness of the proposed method is also evaluated on two benchmark datasets, namely TextVQA for VQA and PTI for personality traits identification.
| Item Type: | Article |
|---|---|
| Funders: | Ministry of Education, Malaysia (FRGS/1/2020/ICT02/UM/02/4), University Grants Commission, India |
| Uncontrolled Keywords: | Personality trait images; Multimodal concept; Text recognition; Social media images; Natural language processing; Visual question answering |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
| Divisions: | Faculty of Computer Science & Information Technology |
| Depositing User: | Ms. Juhaida Abd Rahim |
| Date Deposited: | 09 Sep 2025 01:51 |
| Last Modified: | 09 Sep 2025 01:51 |
| URI: | http://eprints.um.edu.my/id/eprint/50606 |
Actions (login required)
 |
View Item |
