Marco Baroni

Marco Baroni

Universitat Pompeu Fabra

Social & Behavioural Sciences

Marco Baroni received a PhD in Linguistics from the University of California, Los Angeles, in the year 2000. After several experiences in research and industry, he joined the Center for Mind/Brain Sciences of the University of Trento, where he became associate professor in 2013. In 2016, Marco joined the Facebook Artificial Intelligence Research team. In 2019, he became ICREA research professor, affiliated with the Linguistics Department of Pompeu Fabra University in Barcelona. Marco's work in the areas of multimodal and compositional distributed semantics has received widespread recognition, including a Google Research Award, an ERC Starting Grant, and the ICAI-JAIR best paper prize and the ACL test of time award. Marco's current research focuses on the analysis of communication in deep-learning-trained artificial neural networks. In 2021, he was awarded an ERC Advanced Grant to work on this topic.

Research interests

While deep-learning-based artificial neural networks have revolutionized science, engineering and our daily life, we still know surprisingly little about how they work. Indeed, they can sometimes behave in completely unexpected ways, exposing weaknesses that can be used for harmful purposes. My current interest lies in opening the black box of modern neural networks, specifically in the domain of language, where I study so-called "large language models" such as ChatGPT. I am focusing on two main approaches to this challenge. On the one hand, I am studying how large language models react to inputs that are outside their training distribution. On the other, I use tools from probability, linear algebra and information theory to measure the complexity of the internal representations of large language models.

Selected publications

- Lake B & Baroni M 2023, 'Human-like systematic generalization through a meta-learning neural network', Nature, 623, 115-121.

- Dessì R, Bevilacqua M, Gualdoni E, Rakotonirina N, Franzon F & Baroni M 2023. Cross-Domain Image Captioning with Discriminative Finetuning. Proceedings of CVPR 2023 (IEEE/CVF Conference on Computer Vision and Pattern Recognition), 6935 - 6944.

- Rakotonirina NC, Dessi R, Petroni F, Riedel S & Baroni M 2023, 'Can discrete information extraction prompts generalize across language models?', Proceedings of the 11th International Conference on Learning Representations (ICLR), 2023 Mai 1-5; Kigali, Rwanda.

- Mahaut M, Franzon F, Dessi R & Baroni M 2023, 'Referential communication in heterogeneous communities of pre-trained visual deep networks', Proceedings of AAMAS (22nd International Conference on Autonomous Agents and Multiagent Systems): 2619-2621.
- Cheng E, Kervadec C & Baroni M 2023. 'Bridging information-theoretic and geometric compression in language models'. Proceedings of EMNLP 2023 (Conference on Empirical Methods in Natural Language Processing), East Stroudsburg PA: ACL.

Selected research activities

  • Member of the ERC SH4 Consolidator panel
  • Remote panel evaluator for the ERC Synergy Grant panel
  • Area chair for ICLR
  • In two international PhD thesis committees and a US tenure track evaluation panel