Emotion Recognition Technologies in Virtual Reality Film Experiences: A Review
DOI:
https://doi.org/10.65150/EP-jsshrs/V2E5/2026-10Keywords:
Virtual Reality Film, Emotion Recognition, Physiological Signal Analysis, Affective Computing, Immersive StorytellingAbstract
Virtual reality (VR) film has emerged as a representative form of immersive media, providing audiences with highly engaging and emotionally rich viewing experiences. Compared with traditional screen-based films, VR films place viewers within a three-dimensional narrative environment and offer a stronger sense of presence, embodiment, and interactivity. As emotional responses play a critical role in audience engagement, narrative comprehension, and overall user experience, emotion recognition has become an increasingly important research direction in VR film studies.
This paper reviews recent advances in emotion recognition technologies applied to virtual reality film experiences. It summarizes commonly used physiological and behavioral signals, including electroencephalography (EEG), electrodermal activity (EDA), heart rate variability (HRV), eye tracking, and facial expression analysis. In addition, representative machine learning and deep learning methods, such as support vector machines, convolutional neural networks, long short-term memory networks, and Transformer-based models, are discussed. The paper further examines key application scenarios, including audience experience evaluation, adaptive storytelling, therapeutic VR, and educational environments. However, existing studies still face challenges regarding multimodal data synchronization, individual variability, and the lack of standardized evaluation frameworks for VR film experiences.
The review indicates that multimodal physiological signal analysis provides an effective and objective approach for understanding emotional responses in immersive narrative experiences. Despite challenges related to individual differences, data synchronization, motion artifacts, and privacy concerns, continuous advances in sensor technology and artificial intelligence are expanding the potential of emotion-aware VR filmmaking. This study provides a theoretical reference for future research and practical guidance for creators seeking to develop more responsive and emotionally engaging virtual reality films.
References
1) Picard, R. W. (1997). Affective Computing. MIT Press.
2) Slater, M., & Sanchez-Vives, M. V. (2016). Enhancing our lives with immersive virtual reality. Frontiers in Robotics and AI, 3, 74. https://doi.org/10.3389/frobt.2016.00074
3) Carpio, R., Baumann, O., & Birt, J. (2023). Evaluating the viewer experience of interactive virtual reality movies. Virtual Reality, 27(4), 3181-3190. https://doi.org/10.1007/s10055-023-00824-y
4) Koelstra, S., et al. (2012). DEAP: A database for emotion analysis using physiological signals. IEEE Transactions on Affective Computing, 3(1), 18-31. https://doi.org/10.1109/T-AFFC.2011.15
5) Soleymani, M., et al. (2012). A multimodal database for affect recognition and implicit tagging. IEEE Transactions on Affective Computing, 3(1), 42-55. https://doi.org/10.1109/T-AFFC.2011.25
6) Katsigiannis, S., & Ramzan, N. (2018). DREAMER: A database for emotion recognition through EEG and ECG signals. IEEE Journal of Biomedical and Health Informatics, 22(1), 98-107. https://doi.org/10.1109/JBHI.2017.2688239
7) Correa, J. A. M., et al. (2018). AMIGOS: A dataset for affect, personality and mood research. IEEE Transactions on Affective Computing, 12(2), 479-493. https://doi.org/10.1109/T-AFFC.2018.2884461
8) Calvo, R. A., & D'Mello, S. (2010). Affect detection: An interdisciplinary review. IEEE Transactions on Affective Computing, 1(1), 18-37. https://doi.org/10.1109/T-AFFC.2010.4
9) Jenke, R., Peer, A., & Buss, M. (2014). Feature extraction and selection for emotion recognition from EEG. IEEE Transactions on Affective Computing, 5(3), 327-339. https://doi.org/10.1109/T-AFFC.2014.2336234
10) Shu, L., et al. (2018). A review of emotion recognition using physiological signals. Sensors, 18(7), 2074.
https://doi.org/10.3390/s18072074
11) Hurst, B. S. (2020). Cinematic virtual reality: Inside the story. In Handbook of Research on the Global Impacts and Roles of Immersive Media. https://doi.org/10.4018/978-1-5225-9815-6.ch007
12) Freeman, D., et al. (2017). Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychological Medicine, 47(14), 2393-2400. https://doi.org/10.1017/S003329171700040X
13) D'Mello, S., & Kory, J. (2015). A review and meta-analysis of multimodal affect detection systems. ACM Computing Surveys, 47(3), 43.
https://doi.org/10.1145/2689744
14) Li, M., et al. (2022). Multimodal emotion recognition using deep learning: A review. Information Fusion, 82, 1-15.
https://doi.org/10.1016/j.inffus.2021.12.008
15) Du, P. (2025). Investigating emotional responses in virtual reality films using physiological signal analysis. PhD Design Colloquium 2025 Proceedings.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Pengcheng Du , Rina Binti Abd Shukor , Zhijing Chen (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
