NLP
A multi-modal, multi-diagnostic approach to language-based inference of mental disorders
Together with Lasse Hansen and Riccardo Fusaroli from Aarhus University, I am working on developing text-, audio-, and combined text-and-audio models for language-based inference of psychiatric disorders in a multimodal and multiclass settings.
We have engineered a number of baseline models (using XGBoost on text and audio features) as well as transformers-based architectures, and trained them to predict clinical diagnoses for a cohort of individuals diagnosed with ASD, schizophrenia or major depressive disorders and matched controls. In our forthcoming manuscript, we show that performance in multiclass settings decreases significantly compared to binary (diagnosis vs. control) prediction problems, highlighting the need for more research (and larger datasets!) aimed at improving the specificity and the real-world clinical utility of language- and voice-based diagnostic approaches.
We also show that ensemble approaches (text + audio) can improve specificity in multiclass settings, efficiently leveraging information from multiple modalities at a low computational cost.
NLP
psychiatry
clinical diagnostics
machine learning
language
Text transformer for context-aware encoding
This project focuses on training transformer encoders whose representations incorporate information about higher-order context, i.e., characteristics of the author and/or the pragmatic context. We feed models a target sequence and a number of ‘context’ sequences (i.e., text from the same author, or from the same subreddit) as a single example, and train models on a variant of MLM where the MLM head is fed the combination of token-level representations of the input sequence and an aggregate representation of context sequences.
We experiment with three DistilBERT-inspired architectures: a bi-encoder (where context and target are fed to two separate encoders), a ‘batch’ encoder (single encoder with added context aggregation and target-context combination layers) and a hierarchical encoder (applying attention across [CLS] tokens in between standard transformer layers to integrate information across contexts and target sequence). The benefits of this training protocol are evaluated both by comparing their MLM performance with no-context MLM training and to random-context training, as well as on extrinsic tasks.
This project is still in progress.
NLP transformers DistilBERT TensorFlow huggingface ML