Multimodal analysis of depression in unconstrained environments
Publication Date
December 12, 2023
Creator
Abstract
Mental health disorders, such as depression and anxiety, are a significant global problem affecting millions of people, leading to disability, increased mortality from suicide, and reduced quality of life. Traditional diagnostic and evaluation methods rely on subjective approaches and are limited by resource availability, driving the need for more accessible and efficient methods using technology. Digital mental health, a rapidly growing field, merges digital technologies into mental health care, utilizing the Internet and mobile phone software to deliver mental health services. The use of mobile health technologies, such as Ecological Momentary Assessments and digital phenotyping, can improve depression diagnostics by generating objectively measurable markers in natural environments. Technological progress in computer vision, natural language processing, and affective computing has also led to the emergence of automated behavior analysis methods, improving depression assessment and understanding.
This thesis addresses the problem of mood assessment and analysis for detecting depression from multimodal data in unconstrained, natural environments. This thesis presents a novel, multi-modal dataset collected from a purpose-
built smartphone app for depression recognition in real-world, unconstrained environments and proposes a state-of-the-art, automated depression recognition system leveraging advancements in multimodal analysis. The research outcomes have the potential to be applied in automated patient monitoring or therapy administering platforms. The thesis contributes by: 1) collecting a novel, longitudinal, and multi-modal, Mood-Seasons dataset in real-world settings, 2) benchmarking state-of-the-art video analysis techniques on newly collected and publicly available datasets, 3) building a multimodal spatio-temporal transformer model for automated depression severity prediction, 4) presenting a new framework for face generation that learns to synthesize novel face images that adhere to a given pose and appearance from exemplar image in a semantically meaningful way and 5) applying the face manipulation method for anonymizing the Mood-Seasons dataset for privacy preservation.
In conclusion, this thesis addresses the limitations of current depression diagnostics and assessments by integrating smartphone-driven digital phenotyping technologies to advance and personalize depression care. By collecting a novel dataset, proposing state-of-the-art methods for depression recognition, and addressing privacy concerns, this work has the potential to significantly improve mental health care delivery and accessibility.
Item Type
ethesis
Thesis Type
PhD
Subjects (LC)
Associated Schools / Departments
School of Computer Science (UK)
eprints ID
76647
UoN Repository URI
Except where otherwise noted, this item's license is described as
File(s)![Thumbnail Image]()
Name
phd_thesis_keerthy _corrected.pdf
Type
Full-text
Description
Examined. School of computer science PhD thesis submitted after corrections
Size
13.73 MB
Format
Adobe PDF
Checksum (MD5)
0c6f9bc6509e0fd2518391388aaa8608