Liong, Gen-Bing and See, John and Chan, Chee Seng (2023) Spot-then-recognize: A micro-expression analysis network for seamless evaluation of long videos. Signal Processing-Image Communication, 110. ISSN 0923-5965, DOI https://doi.org/10.1016/j.image.2022.116875.
Full text not available from this repository.Abstract
Facial Micro-Expressions (MEs) reveal a person's hidden emotions in high stake situations within a fraction of a second and at a low intensity. The broad range of potential real-world applications that can be applied has drawn considerable attention from researchers in recent years. However, both spotting and recognition tasks are often treated separately. In this paper, we present Micro-Expression Analysis Network (MEAN), a shallow multi-stream multi-output network architecture comprising of task-specific (spotting and recognition) networks that is designed to effectively learn a meaningful representation from both ME class labels and location-wise pseudo-labels. Notably, this is the first known work that addresses ME analysis on long videos using a deep learning approach, whereby ME spotting and recognition are performed sequentially in a two-step procedure: first spotting the ME intervals using the spotting network, and proceeding to predict their emotion classes using the recognition network. We report extensive benchmark results on the ME analysis task on both short video datasets (CASME II, SMIC-E-HS, SMIC-E-VIS, and SMIC-E-NIR), and long video datasets (CAS(ME)2 and SAMMLV); the latter in particular demonstrates the capability of the proposed approach under unconstrained settings. Besides the standard measures, we promote the usage of fairer metrics in evaluating the performance of a complete ME analysis system. We also provide visual explanations of where the network is ``looking''and showcasing the effectiveness of inductive transfer applied during network training. An analysis is performed on the in-the-wild dataset (MEVIEW) to open up further research into real-world scenarios.
| Item Type: | Article |
|---|---|
| Funders: | None |
| Uncontrolled Keywords: | Facial micro-expression; Spotting; Recognition; Optical flow features; Shallow neural networks |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
| Divisions: | Faculty of Computer Science & Information Technology > Department of Artificial Intelligence |
| Depositing User: | Ms Zaharah Ramly |
| Date Deposited: | 28 Nov 2023 08:17 |
| Last Modified: | 28 Nov 2023 08:17 |
| URI: | http://eprints.um.edu.my/id/eprint/39363 |
Actions (login required)
 |
View Item |
