The Brains and Machines Lab

Mission

Our brain continuously processes complex visual information, constructing a comprehensive understanding of the world around us from fleeting 2D retinal impressions. At the Brains and Machines Lab at BGU, we endeavor to understand human visual cognition by employing neural network modeling and model-driven experiments. Our research utilizes a range of techniques, including psychophysics, eye-tracking, and functional magnetic resonance imaging (fMRI), to probe the mechanisms underlying high-order human vision. By leveraging deep learning, we develop computational hypotheses that could explain the observed behavioral and neural data. We then design and conduct innovative model-driven experiments to empirically test these hypotheses, closing the loop between theory and experiment.

Team

Tal Golan

Principal investigator

golan.neuro@bgu.ac.il

Gal Shubeli

Master's student
Co-advised with Dan Vilenchik

shubeli@post.bgu.ac.il

Itamar Avitan

Undergraduate researcher

avitanit@post.bgu.ac.il

Itay Inbar

Undergraduate researcher

itayin@post.bgu.ac.il

Yaar Litvin Harari

Undergraduate researcher

yaarl@post.bgu.ac.il

Selected Publications/Preprints

Tal Golan, Prashant C. Raju, Nikolaus Kriegeskorte. "Controversial stimuli: Pitting neural networks against each other as models of human cognition." Proceedings of the National Academy of Sciences, vol. 117, no. 47, pp. 29330–29337, 2020. DOI: 10.1073/pnas.1912334117.

This is a seminal work for our research program. It introduces the method of controversial stimuli, which are stimuli designed to elicit distinct predictions in alternative models. Using this method, we discovered that generative neural network classifiers are more human-like than discriminative neural network classifiers.

Tal Golan*, Matthew Siegelman*, Nikolaus Kriegeskorte, Christopher Baldassano. "Testing the limits of natural language models for predicting human language judgements." Nature Machine Intelligence, 1–13, 2023. DOI: 10.1038/s42256-023-00718-1.

In this work, we translated the controversial stimuli method to the domain of language models, creating pairs of sentences where one sentence was highly likely according to one language model and highly unlikely according to another model, and vice versa. This manipulation allowed us to effectively compare language models in terms of their validity as cognitive models of human relative acceptability judgments.

Kamila M. Jozwik*, Jonathan O'Keeffe*, Katherine R. Storrs*, Wenxuan Guo, Tal Golan, Nikolaus Kriegeskorte. "Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models." Proceedings of the National Academy of Sciences, vol. 119, no. 27, 2022. DOI: 10.1073/pnas.2115047119.

Similarity and dissimilarity relations among stimuli define a geometry in a representational space. In this work, led by several graduates of Niko Kriegeskorte's old Cambridge lab, the representational geometry of human face similarity judgments is compared to that of multiple computational models, including the Basel Face Model and convolutional neural networks trained on various tasks.

Tal Golan*, Wenxuan Guo*, Heiko H. Schütt, Nikolaus Kriegeskorte. "Distinguishing representational geometries with controversial stimuli: Bayesian experimental design and its application to face dissimilarity judgments." SVRHM 2022 Workshop @ NeurIPS, 2022. openreview.net/forum?id=a3YPu2-Mf2h.

In the previous study, we encountered a challenge: multiple representational models predicted the human similarity ratings equally well. In this work, we introduced a method for generating stimuli that elicit distinct representational geometries in different models. Using the synthetic controversial face stimuli in a human psychophysical experiment, we found that training on "inverse rendering" (estimating structure from pixels) results in a neural network that is more human-like than when training the same architecture on alternative tasks.

Tal Golan, JohnMark Taylor, Heiko Herbert Schütt, Benjamin Peters, Rowan Paolo Sommers, Katja Seeliger, Adrien Doerig, Paul Linton, Talia Konkle, Marcel van Gerven, Konrad Kording, Blake Richards, Tim Christian Kietzmann, Grace W. Lindsay, Nikolaus Kriegeskorte. "Deep neural networks are not a single hypothesis but a language for expressing computational hypotheses." PsyArXiv, 2023. DOI: 10.31234/osf.io/tr7gx. In press at Behavioral and Brain Sciences (BBS).

This is a concise statement from various cognitive computational neuroscience labs, explaining how challenges posed by cognitive psychology findings should drive progress in neural network-based cognitive modeling rather than a return to traditional box-and-arrow models.

Amirhossein Farzmahdi, Wilbert Zarco, Winrich Freiwald, Nikolaus Kriegeskorte, Tal Golan. "Emergence of brain-like mirror-symmetric viewpoint tuning in convolutional neural networks." BioRxiv, 2023. DOI: 10.1101/2023.01.05.522909. Preprint.

The AL face-patch in macaque monkeys is sensitive to faces' head orientations, but it does not differentiate between mirrored orientations (e.g., left and right profiles). In this study, we show that convolutional neural networks (CNNs) trained to discriminate between real-world objects (which are mostly bilaterally symmetric) provide a parsimonious explanation for this property.

Guy Gaziv*, Roman Beliy*, Niv Granot*, Assaf Hoogi, Francesca Strappini, Tal Golan, Michal Irani. "Self-supervised Natural Image Reconstruction and Large-scale Semantic Classification from Brain Activity." NeuroImage, vol. 254, pp. 119121, 2022. DOI: 10.1016/j.neuroimage.2022.119121.

In this cool collaboration with Michal Irani's lab, self-supervised neural networks are trained to reconstruct seen images from fMRI signals. The self-supervision "trick" allows the networks to learn also from images for which no fMRI data is available.

Nikolaus Kriegeskorte, Tal Golan. "Neural network models and deep learning." Current Biology, vol. 29, no. 7, pp. R231–R236, 2019. DOI: 10.1016/j.cub.2019.02.034.

This is an accessible tutorial on neural network models and deep learning.

Openings

• Fully funded positions for Master's and Doctoral students are available! We are looking for highly motivated individuals with a passion for the intersection between deep learning and human cognition. If that sounds like you, please send your CV, current grade sheet, and a one-paragraph description of your scientific interests to golan.neuro@bgu.ac.il.

Contact