Together with Lasse Hansen and Riccardo Fusaroli from Aarhus University, I am working on developing text-, audio-, and combined text-and-audio models for language-based inference of psychiatric disorders in a multimodal and multiclass settings.
We have engineered a number of baseline models (using XGBoost on text and audio features) as well as transformers-based architectures, and trained them to predict clinical diagnoses for a cohort of individuals diagnosed with ASD, schizophrenia or major depressive disorders and matched controls. In our forthcoming manuscript, we show that performance in multiclass settings decreases significantly compared to binary (diagnosis vs. control) prediction problems, highlighting the need for more research (and larger datasets!) aimed at improving the specificity and the real-world clinical utility of language- and voice-based diagnostic approaches.
We also show that ensemble approaches (text + audio) can improve specificity in multiclass settings, efficiently leveraging information from multiple modalities at a low computational cost.
At Psychoinformatics Lab, I have contributing to the development of Neuroscout, an end-to-end platform for the analysis of naturalistic fMRI data. You can read more about Neuroscout in our eLife paper: https://elifesciences.org/articles/79277.
I am focusing on expanding Neuroscout’s annotation set by implementing feature extraction pipelines that use pretrained deep learning models (e.g., from HuggingFace’s transformers and TensorflowHub) in pliers.
I contributed to validating the platform and showing its potential to increase the generalizability of neuroimaging findings through a series of large-scale meta-analyses presented in the paper, and available as a Jupyter book here.
Quantitative research in psychology and neighboring field emphasizes explanation and in-sample effect sizes over demonstrating models’ ability to predict on unseen data (generalization).
In a methods paper that interleaves theoretical arguments with empirical demonstrations (code available in this repo), we show how psychology would benefit from adopting benchmarking as a consensus paradigm for model evaluation.
We discuss how psychology can learn from both the strengths and the known weaknesses (e.g., biases, overfitting) of benchmarking in ML, discuss first steps for introducing these new practices in the field, and outline their potential to increase the practical utility of the outputs of psychological research.
This article has been published in Advances in Methods and Practices in Psychological Sciences, and it available at: https://journals.sagepub.com/doi/full/10.1177/25152459211026864