Together with Lasse Hansen and Riccardo Fusaroli from Aarhus University, I am working on developing text-, audio-, and combined text-and-audio models for language-based inference of psychiatric disorders in a multimodal and multiclass settings.
We have engineered a number of baseline models (using XGBoost on text and audio features) as well as transformers-based architectures, and trained them to predict clinical diagnoses for a cohort of individuals diagnosed with ASD, schizophrenia or major depressive disorders and matched controls. In our forthcoming manuscript, we show that performance in multiclass settings decreases significantly compared to binary (diagnosis vs. control) prediction problems, highlighting the need for more research (and larger datasets!) aimed at improving the specificity and the real-world clinical utility of language- and voice-based diagnostic approaches.
We also show that ensemble approaches (text + audio) can improve specificity in multiclass settings, efficiently leveraging information from multiple modalities at a low computational cost.
Spatial demonstratives are words like ‘this’ and ‘that’ used to direct manipulate people’s attentional focus. They are extremely frequent, yet far from simple. Understanding what they refer to requires not only knowing language, but also the context in which they are pronounced.
As part of my PhD, I ran a naturalistic fMRI study combining synthesized dialogical narratives, fast multiband acquisition, and finite impulse response modeling to understand how the brain makes sense of them.
I found that spatial words engaged dorsal regions of the brain implicated not only in language, but in various aspects of visuospatial cognition, supporting distributed views of language processing.
This study has been published in NeuroImage, and it is available here https://www.sciencedirect.com/science/article/pii/S1053811919307190
Humans perceive space as functional to action. Several studies have shown that humans organize space into a peripersonal (i.e., within reach) and an extrapersonal (i.e., outside reach) region.
Interestingly, a lot of the actions we perform in our daily life are performed together with others. Do we adapt the way in which we parse space as near vs. far oto the position of other people when action goals are shared?
We tested this hypothesis over two interactive experiments using language as a proxy for spatial representations. We found that, in the context of joint action, linguistic coding of locations as proximal vs. distal is based on the position of the partner rather than oneself’s.
These studies (part of my PhD) are published in Nature Scientific Reports. The article is available here: https://www.nature.com/articles/s41598-019-51134-8
Spatial demonstratives (words like ‘this’ and ‘that’) are thought to map onto a distinction between near and far space. Yet, when people are asked to pair a noun with a demonstrative without any spatial context, choices tend to be non-random. Over a number of large-scale online experiments, I investigated which semantic features of a referent determine which demonstrative people tend to use to refer to it.
Using PCA and multilevel linear modeling, we found that demonstrative choice is systematically influenced by a range of factors including manipulability, valence, and potential for motion. Importantly, the resulting experimental paradigm (the ‘demonstrative choice task’) has been used across a number of languages displaying consistent results, and it is currently being used in follow-up studies to investigate whether linguistic behavior in the demonstrative choice task can be used as predictor of personality and clinical traits.
Studies from my PhD using this paradigm have been published in PlosOne, Frontiers in Psychology, and Language and Cognition, and two more are currently in progress.