Video annotation tool to detect mental health problems

18 June 2022

The prevalence of stress, anxiety, and depression among the younger generation, especially university students, has been increasing over the past few decades. Malaysian Health Ministry statistics reveal a worsening state of mental health problems among Malaysian students, from one in ten individuals in 2011 to one in five in 2016. According to the National Health and Morbidity Survey (NHMS) 2017 conducted by the Institute for Public Health, National Institutes of Health, Ministry of Health Malaysia, it was found that Malaysian youngsters between the ages of 13 to 17 suffer from various types of mental health problems.

The findings suggest that one in five students suffers from depression, two in five have anxiety issues, and one in ten suffers from stress. The Education Ministry had registered 87,574 special education needs (SEN) students as of June 30, 2019. According to the National Health and Morbidity Survey (NHMS) 2019, 424,000 children have mental health problems.

The Mental Health Advisory Council in Malaysia acknowledges a need for more psychiatrists and psychologists for hospitals and clinics in Malaysia. Currently, the ratio of clinical psychologists in the Ministry of Health (MOH) to the population is relatively low. According to the MOH, Malaysia currently has 459 psychiatrists. Ideally, it would need to add 261 such specialists in four more years amid a sharper focus on people's mental health and wellbeing. Based on the Health Ministry's latest publicly available data, the country would need an additional 493 psychiatrists by 2030 to cater to the country's projected population then.

"With the increased number of individuals to be treated, psychotherapists have limited time to spend on each individual. Hence, we believe a video annotation tool will benefit psychotherapists and psychiatrists in clinically assessing their patients' conditions. Furthermore, since it will tag the most triggering situations, the video annotation tool can be utilised in schools and universities as an early detecting mechanism of stress, anxiety, and depression," stated Dr Dharmaratne Anuja Thimali from the School of Information Technology, Monash University Malaysia.

According to Dr Anuja, the proposed video annotation tool will be the first of its kind to use micro expressions, facial action units, and speech patterns to achieve behaviour analysis using Artificial Intelligence-based methods. "Our main focus is a consolidated algorithm for recognising emotions through the analysis of micro-expressions, facial action units, and speech patterns. We will explore these three aspects and produce a video annotation mechanism based on a consolidated algorithm that could identify specific behavioural patterns found among depressed individuals using machine learning."

Dr Anuja shared that the research project is motivated by the high prevalence of psychological disorders (stress, anxiety, and depression) in the young population, especially university students.

"Micro-expressions are brief, involuntary facial expressions that are hard to spot and recognise using conventional methods. Thus, we are utilising the latest technique called visual transformers with self-attention modules to increase the accuracy of the spotting and recognition of micro-expressions. Our model is further enhanced by incorporating the facial action units associated with the selected psychological disorders. We will also use Natural Language Processing (NLP) transformer methods to spot and recognise speech patterns related to stress, anxiety, and depression," Dr Anuja further elaborated.

Due to the pandemic, the plan to collect high-quality videos of individuals' facial expressions had to be put on hold, and instead, experiments were carried out using other available annotated datasets.

"We have already analysed the accuracy of micro-expression analysis using a convolutional neural network (Resnet 50) as well as a vision transformer approach (VIT 16). We are currently working on combining the vision transformer with a bottleneck attention module," Dr Anuja said.

The research group consisting of academics from the School of Information Technology and the Jeffrey Cheah School of Medicine and Health Sciences anticipates that the research tool can be assessed using some real data in 2023.