Choong, Ren-Jun and Yap, Wun-She and Hum, Yan Chai and Lai, Khin Wee and Ling, Lloyd and Vodacek, Anthony and Tee, Yee Kai (2024) Impact of visual enhancement and color conversion algorithms on remote sound recovery from silent videos. Journal of the Society for Information Display, 32 (3). pp. 112-125. ISSN 1938-3657, DOI https://doi.org/10.1002/jsid.1275.
Full text not available from this repository.Abstract
The visual microphone is a technique for remote sound recovery that extracts sound information from tiny pixel-scale vibrations in a video. Despite having demonstrated success in sound recovery, the impact of various visual enhancement and color conversion algorithms applied on the video before the sound recovery process has not been explored. Thus, it is important to investigate these effects have on the recovered sound quality, as the vibrations are so small the effects play an important role. This work experimented with different color to grayscale conversions and visual enhancement algorithms on 576 videos, and found that the recovered sound quality is indeed greatly affected by the choice of algorithms. The best conversion algorithms were found to be the average of the red, green and blue color channels and the perceptual lightness in the CIELAB color space, improving the recovered sound quality by up to 23.22%. Furthermore, visual enhancement techniques such as gamma correction have been found to corrupt vibration information, leading to a 22.47% drop in recovered sound quality in one of the tested videos. Therefore, it is advisable to avoid or minimize the use of visual enhancement techniques for remote sound recovery to prevent the elimination of useful subtle vibrations. Different color to grayscale conversion and visual enhancement algorithms were applied to high-speed videos before performing sound recovery using the visual microphone. It was found that color to grayscale conversion algorithms prioritizing the green color channel led to a better sound recovery, while visual enhancements degraded the quality of the recovered sound. image
| Item Type: | Article |
|---|---|
| Funders: | Ministry of Higher Education Malaysia, Fundamental Research Grant Scheme (FRGS/1/2019/ICT04/UTAR/02/1); (8073/Y01), Fundamental Research Grant Scheme (IPSR/RMC/UTARRF/2016-C2/T04), UTAR Research Fund (8060/000), Fulbright-MCMC Specialist Grant (P5000 GPU), Nvidia Corporation |
| Uncontrolled Keywords: | color to grayscale conversions; remote sound acquisition; sound recovery; visual enhancement; visual microphone |
| Subjects: | T Technology > TA Engineering (General). Civil engineering (General) T Technology > TK Electrical engineering. Electronics Nuclear engineering |
| Divisions: | Faculty of Engineering |
| Depositing User: | Ms. Juhaida Abd Rahim |
| Date Deposited: | 22 Oct 2024 03:39 |
| Last Modified: | 22 Oct 2024 03:39 |
| URI: | http://eprints.um.edu.my/id/eprint/45448 |
Actions (login required)
 |
View Item |
