PERTANIKA JOURNAL OF SCIENCE AND TECHNOLOGY

Emotions play a significant role in both verbal and nonverbal communication. Facial emotion recognition has applications in various sectors where we can get real-time feedback about student activeness by detecting their expression. In this paper, we aim to provide an improved deep-learning technique to detect emotions by using publicly available datasets to perform this detection. To get more data for the well-being of the Machine Learning Model, we have used data augmentation using the TensorFlow framework. Visual Geometry Group-16 (VGG16) is a convolutional neural network of 16 layers deep. There has been an alteration to the default VGG16 structure to get better classification results. Various optimization algorithms and loss functions increase the model’s accuracy. We have used many evaluation parameters from the technical side, like precision, accuracy, recall, Area Under the Receiver Operating Characteristic Curve (AUC), and F1 Score. The proposed model has an accuracy of 89% while having a precision of 81 percent for classification. We have achieved an F1 Score of 0.42 and an area under the ROC curve (AUC) of 0.734. Overall, it would be beneficial for analyzing and categorizing positive and negative emotions, which would aid in detecting signs of stress, anxiety, and burnout, as well as taking preventive actions to enhance well-being.

• Alamsyah, T. M. S. N., Abidin, T. F., Ferdhiana, R., Dirhamsyah, M., & Chaidir, M. (2022, December 8-9). Analysis of face data augmentation in various poses for face recognition model. [Paper presentation]. Seventh International Conference on Informatics and Computing (ICIC), Bali, Indonesia. https://doi.org/10.1109/ICIC56845.2022.10006997

• Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930

• Darwin, C. (1872). The expression of the emotions in man and animals. John Murray. https://doi.org/10.1037/10001-000

• Devi, B., & Preetha, M. M. S. J. (2023). A descriptive survey on face emotion recognition techniques. International Journal of Image and Graphics, 23(1), Article 2350008. https://doi.org/10.1142/S0219467823500080

• Ekman, P. (2009). Darwin’s contributions to our understanding of emotional expressions. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), 3449–3451. https://doi.org/10.1098/rstb.2009.0189

• Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2), 124–129. https://doi.org/10.1037/h0030377

• Garcia-Garcia, J. M., Penichet, V. M. R., Lozano, M. D., & Fernando, A. (2022). Using emotion recognition technologies to teach children with autism spectrum disorder how to identify and express emotions. Universal Access in the Information Society, 21(4), 809–825. https://doi.org/10.1007/s10209-021-00818-y

• Jaymon, N., Nagdeote, S., Yadav, A., & Rodrigues, R. (2021, February 19-20). Real-time emotion detection using deep learning. [Paper presentation]. International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT), Bhilai, India. https://doi.org/10.1109/ICAECT49130.2021.9392584

• Mandal, M. K. (2004). Emotion. Affiliated East-West Press.

• Ortony, A. (2022). Are all “basic emotions” emotions? A problem for the (Basic) emotions construct. Perspectives on Psychological Science, 17(1), 41–61. https://doi.org/10.1177/1745691620985415

• Pabst, A., Bollen, Z., Masson, N., Billaux, P., Timary, P. D., & Maurage, P. (2023). An eye-tracking study of biased attentional processing of emotional faces in severe alcohol use disorder. Journal of Affective Disorders, 323, 778–787. https://doi.org/10.1016/j.jad.2022.12.027

• Salve, P. (2022, October 3). India’s suicide rate has increased. But is it because of better reporting or rising distress? Croll.in. https://scroll.in/article/1034045/indias-suicide-rate-has-increased-but-is-it-because-of-better-reporting-or-rising-distress

• Saini, A., Khaparde, A. R., Kumari, S., Shamsher, S., Joteeswaran, J., & Kadry, S. (2023). An investigation of machine learning techniques in speech emotion recognition. Indonesian Journal of Electrical Engineering and Computer Science, 29(2), Article 875. https://doi.org/10.11591/ijeecs.v29.i2.pp875-882

• Shank, D. B. (2014). Technology and emotions. In J. E. Stets & J. H. Turner (Eds.), Handbook of the Sociology of Emotions: Volume II (pp. 511–528). Springer. https://doi.org/10.1007/978-94-017-9130-4_24

• Siam, A. I., Soliman, N. F., Algarni, A. D., El-Samie, F. E. A., & Sedik, A. (2022). Deploying machine learning techniques for human emotion detection. Computational Intelligence and Neuroscience, 2022, Article 8032673. https://doi.org/10.1155/2022/8032673

• Song, S., Zhao, S., Gao, Z., Lu, M., Zhang, M., Gao, S., & Zheng, Y. (2022). Influence of affective verbal context on emotional facial expression perception of social anxiety. International Journal of Psychophysiology, 181, 141–149. https://doi.org/10.1016/j.ijpsycho.2022.09.002

• Song, Z. (2021). Facial expression emotion recognition model integrating philosophy and machine learning theory. Frontiers in Psychology, 12, Article 759485. https://doi.org/10.3389/fpsyg.2021.759485

• Subhashini, R., & Niveditha, P. R. (2015). Analyzing and detecting employee’s emotions for amelioration of organizations. Procedia Computer Science, 48, 530–536. https://doi.org/10.1016/j.procs.2015.04.131