cover michael arbib

Michael Arbib on mirror neurons and schema theory

  • cover play_arrow

    PLAY EPISODE


cover michael arbib
Season 2010
Season 2010
Description arrow_drop_down

Description

How did a brain system for grasping objects become the foundation for human language? Michael Arbib traces the evolutionary path from mirror neurons to speech, arguing that schema theory provides the missing link between neural circuits and cognitive architecture. Subscribe for more from the Convergent Science Network podcast series. Michael Arbib has spent decades developing schema theory , a framework for decomposing complex behaviors into interacting functional units that can be mapped onto neural circuits. In this interview, he explains how this approach bridges the gap between high-level cognitive descriptions and low-level neural implementations, using two case studies: the visual control of hand movements and the evolution of language. The story begins with the premotor cortex, where Arbib’s collaborator Giacomo Rizzolatti discovered mirror neurons , cells active both when a monkey performs a hand action and when it observes the same action performed by another. Brain imaging revealed that the human homologue of this mirror region overlaps with Broca’s area, traditionally considered a speech center. This anatomical coincidence opened a research program connecting manual action to linguistic communication. Arbib outlines eleven evolutionary steps from our common ancestor with monkeys to the language-ready human brain, each representing what he calls a “small miracle” , a plausible transition requiring only modest genetic changes. The key transitions include: extending action recognition to imitation of novel actions, developing pantomime from practical object manipulation, conventionalizing gestures through social interaction, and finally recruiting the vocal apparatus for proto-speech. Arbib emphasizes that language likely did not evolve as a single package but was gradually discovered by human cultures exploiting brain capacities that evolved for other purposes. Sign language demonstrates that the linguistic capacity is not inherently vocal , it is a general-purpose system for structured communication. Schema theory serves as the computational backbone of this framework. Arbib positions schemas as intermediate-level descriptions, analogous to high-level programming languages, that capture the functional decomposition of behavior without committing to specific neural implementations. But unlike purely abstract computational theories, schemas are meant to be iteratively refined against neurophysiological data, creating a loop between cognitive-level hypotheses and circuit-level constraints. Arbib insists on causal completeness: a model must account for the full chain from sensory input through internal processing to behavioral output, not just correlate with isolated neural recordings.

Tagged as:

About the author call_made

CSN Podcasts

Both the triumphs of humanity and its most evil deeds have resulted from collaboration. In a time where humanity is required to aspire to the former and minimize the latter, the question arises of how collaboration arises and why it fails. Surprisingly, this phenomenon, so central to who we are, is not well understood. Hence, a collaborative effort is required to understand collaboration in its full biological, psychological, sociological, cultural, and economic complexity and to translate this understanding into operational impact. This series of podcasts is one step toward achieving these complementary goals. The Collaboration Podcast presents interviews with people who are central orchestrators of collaboration in various domains including business, government, science, art, health, sustainability, and the military. The discussions were conducted by Prof. Dr. Paul F.M.J. Verschure and members of the Program Advisory Committee of the Ernst Strungmann Forum on Collaboration (https://www.esforum.de/forums/ESF32_Collaboration.html) during 2021 and had the goal to sketch a map of opportunities, challenges, and obstacles in human collaboration. The forum took place in May 2022, and now we would like to share this series of interviews with a broader audience. The full report of the Forum will be published in 2023 by MIT Press. The podcast was produced by the Convergent Science Network (https://www.convergentsciencenetwork.org/). Context: The stability of social systems depends critically on realizing sustainable methods of “collaboration,” yet how and by which means collaboration is achieved is not clearly understood; neither are the conditions or processes that lead to its breakdown or failure. Collaboration can be understood as cooperation between agents toward mutually constructed goals. Part of the reason for our lack of understanding is that the phenomenon of collaboration is, by nature, a highly multidisciplinary problem, and effective research into its complexities has been difficult to achieve across the broad range of scientific and technical disciplines involved. The need for a fundamental understanding of collaboration, however, has become increasingly important. Not only does humankind demand answers as it attempts to address critical challenges at multiple scales (e.g., climate change, migration, enhanced automation, social and economic inequality), but ever-increasing technological and economic means of interconnecting people and societies are disrupting long-established, familiar patterns of how we interact. Radical technological changes that are ongoing have the potential to reshape collaboration in ways that are currently hard to predict or influence (e.g., by altering configurations in interaction, information creation, and modes of communication). On one hand, such changes could disrupt hitherto stable forms of collaboration by affecting critical communication channels and traditional roles, as can be observed in the rapidly changing patterns in governance, commerce, and social interaction. Conversely, technology could lead to the emergence of novel, successful forms of collaboration that deviate from traditional “hierarchical” architectures. Evidence of this can be seen in areas as diverse as highly automated manufacturing plants, the open science movement, collaborative software repositories, user-centered services, and the sharing of economy-based modes of organization. Without a fundamental understanding of the mechanisms, processes, and boundary conditions of collaboration, it is not possible to evaluate or predict which of these possible scenarios are sustainable or even plausible. The Forum “How Collaboration Arises and Why it Fails” (May 8–13, 2022, Location: Frankfurt am Main, Germany) Chairs: Andreas Roepstorff and Paul Verschure Program Advisory Committee: Jenna Bednar, Julia R. Lupp, Bhavani R. Rao , Andreas Roepstorff, Ferdinand von Siemens, and Paul Verschure

More posts

Timestamp

  • fast_forward00:00:00 - All right. So this is Tony Prescott and Paul Verschure talking with Michael
  • fast_forward00:00:05 - Arbib after his presentation at the Barcelona Brain, Cognition and Technology Summer School.
  • fast_forward00:00:15 - And we want to revisit some of the main themes that Michael has been talking
  • fast_forward00:00:20 - about to us. So, Michael, would you like to give a short summary in a few words?
  • fast_forward00:00:25 - So what few of the key messages were in your two lectures?
  • fast_forward00:00:32 - Well, for me, the grounding interest has been how vision is related to action.
  • fast_forward00:00:39 - And for that, I've been looking at two different approaches and trying to integrate them.
  • fast_forward00:00:46 - One is what I call the schema-based approach, is to try and take an overall
  • fast_forward00:00:49 - behavior and think about what processes must interact in parallel and distributed
  • fast_forward00:00:54 - way with each other to explain that behavior.
  • fast_forward00:00:58 - And then that's balanced by how could those schemas, as those units are called,
  • fast_forward00:01:04 - play out over particular neural networks of the brain.
  • fast_forward00:01:08 - And of course, sometimes the original schema model dies because it's not consistent
  • fast_forward00:01:13 - with the available neurophysiology, but there's a loop then of explanation.
  • fast_forward00:01:17 - And the other part is that at any time, we, I claim, cannot model every detail of the brain.
  • fast_forward00:01:25 - So we're always making selections as to which brain regions we will implicate in our models.
  • fast_forward00:01:31 - At what level of detail will we look at those particular brain regions?
  • fast_forward00:01:36 - And then as time goes by, we learn which details have to be added, which can be ignored.
  • fast_forward00:01:41 - So I looked first at the control of rapid eye movements, saccadic eye movements,
  • fast_forward00:01:47 - and stressed that we have below the sort of standard brain, the cortical structures,
  • fast_forward00:01:55 - there is the brain stem, the superior colliculus, which can take visual input
  • fast_forward00:02:00 - and control these movements.
  • fast_forward00:02:01 - But once we get into interesting things like, don't look now,
  • fast_forward00:02:05 - but you can look later, or you just heard two noises, look towards the first,
  • fast_forward00:02:10 - then towards the second, where you have to bring in memory and sequencing of
  • fast_forward00:02:14 - actions, then you have to bring in cortical structures.
  • fast_forward00:02:18 - And then you get this balance between the back of the brain,
  • fast_forward00:02:21 - the parietal system that seems to be saying, what do I need to pay attention
  • fast_forward00:02:24 - to that's relevant to my action?
  • fast_forward00:02:27 - And the front of the brain that's saying, well, what actions should I do?
  • fast_forward00:02:30 - And then we bring in another part of the brain called the basal ganglia that
  • fast_forward00:02:34 - do the scheduling or even scheduling of these actions. So that was the framework there.
  • fast_forward00:02:40 - And then I moved on to another system where, again, we have this interaction
  • fast_forward00:02:45 - of prefrontal and parietal systems, namely the visual control of hand movements.
  • fast_forward00:02:52 - Then I reported that my colleague Giacomo Ruzzolati and his group at Palmer
  • fast_forward00:02:58 - had made a discovery that within the premotor area involved in hand movements,
  • fast_forward00:03:03 - there was a subset called mirror neurons,
  • fast_forward00:03:05 - which had this amazing property that they were active not only during the animal's
  • fast_forward00:03:12 - execution of particular hand movements, but also when he recognized other hand movements.
  • fast_forward00:03:19 - And then that suddenly got interesting when we turned to human brain imaging
  • fast_forward00:03:27 - and said, well, we can't monitor individual mirror neurons in the human the way we can in the monkey,
  • fast_forward00:03:33 - but at least we can look for a brain region that lights up in a way that indicates
  • fast_forward00:03:38 - it might contain the mirror system.
  • fast_forward00:03:40 - This plus anatomical data converged to say that the area of mirror neurons seemed
  • fast_forward00:03:50 - to exist in the human brain in what had been thought of as a speech area.
  • fast_forward00:03:55 - What speech got to do with recognition of hand movements? Well,
  • fast_forward00:03:59 - we know there is sign language.
  • fast_forward00:04:01 - Language can exist in the manual domain as well.
  • fast_forward00:04:05 - This gets us into what is called the gestural origins of Jim's theory of language
  • fast_forward00:04:09 - that maybe, although for most of us speech is predominant, we all use co-speech
  • fast_forward00:04:14 - gestures to embellish our speech.
  • fast_forward00:04:18 - And so, I outlined a fairly elaborate,
  • fast_forward00:04:24 - I would say, network of models rather than a single model for what might have
  • fast_forward00:04:29 - been the 11 evolutionary changes from our common ancestor with the monkey 20 million years ago,
  • fast_forward00:04:33 - our common ancestor with the chimpanzee of 7 million years ago,
  • fast_forward00:04:38 - to build a system where the mirror neurons were still a core system,
  • fast_forward00:04:45 - but we'd also gone beyond the mirror.
  • fast_forward00:04:46 - How do we get from just recognizing actions to imitating novel actions?
  • fast_forward00:04:51 - How can our use of actions to practical effect on objects.
  • fast_forward00:04:56 - Provide the basis for pantomime and beginning to use hand movements for communication.
  • fast_forward00:05:01 - What social interactions yield a system of conventionalized gestures rather than ad hoc pantomime?
  • fast_forward00:05:07 - How does speech come into the picture so that we can move from purely gestural
  • fast_forward00:05:12 - proto-sign to a proto-language that is in the spoken domain?
  • fast_forward00:05:18 - And so there we got fairly elaborately into the back and forth between what
  • fast_forward00:05:24 - happens in in terms of biological evolution,
  • fast_forward00:05:27 - opening up new possibilities for brain activity and the cultural historical
  • fast_forward00:05:33 - development of the human species which finds new ways of exploiting the brain
  • fast_forward00:05:38 - that were not exploited before.
  • fast_forward00:05:41 - One of the concluding suggestions was to emphasize the notion that it's probably
  • fast_forward00:05:46 - not the case that our brain evolved to give us language in the sense of a big
  • fast_forward00:05:53 - lexicon, lots of grammatical rules and so on,
  • fast_forward00:05:55 - but it rather evolved to allow us over many tens of millennia to discover more
  • fast_forward00:06:00 - and more aspects of what now constitute what we take for granted as part of human language.
  • fast_forward00:06:07 - That's a great summary, Michael. It's a good thing I was listening.
  • fast_forward00:06:13 - But what was interesting is, what's now the role of schema theory in this second part?
  • fast_forward00:06:22 - Should we consider these as two separate proposals, or did the schema theory,
  • fast_forward00:06:27 - the idea of schemas that you have been pushing for quite some time,
  • fast_forward00:06:30 - give you leverage to look at this mirroring system and to think about language?
  • fast_forward00:06:36 - So what's the relationship between these two?
  • fast_forward00:06:40 - Okay, well, two parts. When we were looking at the control of hand movements,
  • fast_forward00:06:43 - then in fact, the idea that we made a preliminary analysis, what do you have
  • fast_forward00:06:48 - to notice about an object to be able to interact with it?
  • fast_forward00:06:52 - So there are perceptual schemas for just the shape of the object.
  • fast_forward00:06:56 - Because the details of that are going to be very important to the shaping of
  • fast_forward00:07:00 - the hand, recognizing the location of the object, very important to how we move
  • fast_forward00:07:04 - the arm to get the hand into place.
  • fast_forward00:07:07 - So we had, therefore, the shape of the model was in terms of what are the perceptual
  • fast_forward00:07:14 - schemas to know how to interact with the object, but also what are the perceptual
  • fast_forward00:07:19 - schemas to know about the object.
  • fast_forward00:07:21 - We recognize something as a coffee mug, then we can call on knowledge about
  • fast_forward00:07:26 - the use of the handle to lift it.
  • fast_forward00:07:28 - Whereas if we had a nonsense object, we wouldn't be able to call on those more
  • fast_forward00:07:32 - meaningful schemas, and there are some correlates of where in the brain the processes might occur.
  • fast_forward00:07:37 - So our big analysis of visual control of grasping is essentially a schema level,
  • fast_forward00:07:44 - and then we said we can now go in, because we have recording data from the monkey,
  • fast_forward00:07:50 - where we can begin to say, where are those schemas computed?
  • fast_forward00:07:53 - How do we have to modify our understanding of those schemas to see how different
  • fast_forward00:07:57 - brain regions must interact to support it?
  • fast_forward00:08:00 - So in the case of visual control of grasping and then bringing in the mirror
  • fast_forward00:08:04 - neurons, well, that's where we had a very solid neurophysiological data to then
  • fast_forward00:08:09 - reflect back into the schema-level theory.
  • fast_forward00:08:12 - That's part one. But part two is when we turn to language, we essentially have
  • fast_forward00:08:17 - zero in the way of cellular data.
  • fast_forward00:08:20 - We are at the level of analogies from the monkey brain to the human brain and
  • fast_forward00:08:27 - homologies from the monkey brain to the human brain.
  • fast_forward00:08:30 - But our other data are just people have a brain lesion, the system doesn't work
  • fast_forward00:08:35 - so well, or we do some brain imaging and part of the brain lights up.
  • fast_forward00:08:39 - But the trouble with part of the brain lights up is that it's just giving you
  • fast_forward00:08:42 - information that one part of the brain is perhaps more active in task A than
  • fast_forward00:08:48 - in task B, but it doesn't rule it out as being vital for task B.
  • fast_forward00:08:52 - So we now develop, as it were, our informed data analysis is at the level of
  • fast_forward00:09:01 - schemas. How are the different processes?
  • fast_forward00:09:03 - How do I know what a word is? That's a schema-level description.
  • fast_forward00:09:06 - It's not yet a neuron-level description. But what we're hoping to do in future
  • fast_forward00:09:10 - is to say that because of what we've learned from the monkey brain,
  • fast_forward00:09:14 - we can make informed hypotheses about the circuitry in the human brain for which
  • fast_forward00:09:21 - we don't have detailed neurophysiological recordings to come up with better
  • fast_forward00:09:25 - and better neural models.
  • fast_forward00:09:26 - So in the end, we can render a consistent understanding at the schema level
  • fast_forward00:09:31 - and the neural level that embeds our understanding of hand movements for practical
  • fast_forward00:09:37 - ends where we can share a lot with other creatures with this refined use of
  • fast_forward00:09:42 - language, which is particularly human.
  • fast_forward00:09:44 - But would you equate a schema level with a computational level or an algorithmic
  • fast_forward00:09:50 - level, or how should I relate these levels of description, these constructs? Okay.
  • fast_forward00:09:56 - So if we look at computers, there is a machine language, which is the basic
  • fast_forward00:10:02 - language of zeros and ones.
  • fast_forward00:10:04 - And then above that, there will be something like an assembly language,
  • fast_forward00:10:08 - which allows you to say, well, how can I think of these patterns of zeros and
  • fast_forward00:10:11 - ones as recognizing letters or symbols or patches of pixels on a graphic screen?
  • fast_forward00:10:17 - But the language that people who program in is a level up from that,
  • fast_forward00:10:23 - something like Java or C++, which is using relatively high-level constructs,
  • fast_forward00:10:28 - and they don't know actually how that plays out over the hardware.
  • fast_forward00:10:32 - And then for most of us, we're at an even higher level where we just have an
  • fast_forward00:10:37 - app which somebody else has programmed at that level.
  • fast_forward00:10:40 - So schemas are probably describing computation at that level from the high-level
  • fast_forward00:10:48 - programming language up to the app hierarchical levels there,
  • fast_forward00:10:52 - and then the neurons are the computations.
  • fast_forward00:10:55 - They correspond to the machine code in the computer.
  • fast_forward00:10:59 - So it's an intermediate to high-level description of how computations occur in the brain. Right.
  • fast_forward00:11:07 - But, Tony, so you said that you don't want to model the brain in all this detail.
  • fast_forward00:11:14 - Oh, I do, but I know I won't. Okay. So what are your criteria for deciding which
  • fast_forward00:11:19 - phenomena in the brain are important for informing the models?
  • fast_forward00:11:23 - How do you apply those criteria in the process of modeling?
  • fast_forward00:11:26 - Is it that you look for some key aspects of data that you try and switch at
  • fast_forward00:11:34 - your model and then extend what you're looking at to try and bring in more phenomena?
  • fast_forward00:11:39 - Right. So we now know a lot of details about the synaptic structure of the brain.
  • fast_forward00:11:44 - We know a lot of details about how different molecules provide the ability of
  • fast_forward00:11:49 - a synapse to take signals from one neuron to another and apply learning rules and so on.
  • fast_forward00:11:55 - So one could make a model which basically loses itself in just the details of one synapse.
  • fast_forward00:12:01 - Or one could be a little simpler about the synapse and blow a whole supercomputer
  • fast_forward00:12:06 - on just a few interacting neurons.
  • fast_forward00:12:09 - And for some people, that's the career path. For me, it really is starting from this cognitive level,
  • fast_forward00:12:15 - visual control of hand movements, visual perception control
  • fast_forward00:12:18 - of action language and so
  • fast_forward00:12:22 - there i'm taking a sort of
  • fast_forward00:12:24 - survey approach where i say what is known at the neurophysiological level of
  • fast_forward00:12:29 - correlates what is known at the psychological level how can i make a preliminary
  • fast_forward00:12:34 - model perhaps just using pure schemas to make sense of the psychological data
  • fast_forward00:12:40 - now how can i constrain that to meet the neurophysiological Now,
  • fast_forward00:12:44 - how can I refine those schemas so that they not only yield the behavior and
  • fast_forward00:12:48 - how the behavior is damaged by lesions, but also can give me explanations for
  • fast_forward00:12:54 - how individual cells are farmed?
  • fast_forward00:12:56 - And then the literature just keeps pouring in.
  • fast_forward00:13:01 - Filtering in a perhaps not very intelligent way of saying oh
  • fast_forward00:13:04 - here's a new paper that looks really important i have to be
  • fast_forward00:13:06 - able to either show my model can explain it
  • fast_forward00:13:09 - or expand my model to be able to address those data
  • fast_forward00:13:12 - here's something else to somebody to another
  • fast_forward00:13:16 - person that might appear very important but i'm finite
  • fast_forward00:13:19 - so i'll hope they'll address it and i'll have
  • fast_forward00:13:21 - to leave that out so it's a it's opportunistic once
  • fast_forward00:13:25 - the first set of big models is in
  • fast_forward00:13:28 - place i think so you're interested fit in these sort of cognitive behavioral
  • fast_forward00:13:31 - phenomena and your goal is a decomposition of that
  • fast_forward00:13:35 - task that you observe the personal monkey doing into computation
  • fast_forward00:13:39 - elements that you call schema and you're not necessarily at that first stage
  • fast_forward00:13:44 - particularly concerned about mapping the schema onto the brain is that no i
  • fast_forward00:13:49 - would say that no i i'm very much engaged in mapping it onto the brain but what
  • fast_forward00:13:53 - i'm suggesting is if you just look at the neurons and try to make sense of them.
  • fast_forward00:13:58 - You may not succeed so by starting with a hypothesis about what the schemas
  • fast_forward00:14:03 - are you're not saying how is this complex thing being able to speak english,
  • fast_forward00:14:08 - mapped to neurons you're saying here is
  • fast_forward00:14:11 - how do we recognize a particular auditory profile
  • fast_forward00:14:14 - as a word for example then that's a tractable problem
  • fast_forward00:14:17 - and recognizing the words of a vocabulary would be would be schemas within a
  • fast_forward00:14:23 - language understanding system for example so the notion is that you go top-down
  • fast_forward00:14:28 - from the psychology and the behavior to negotiate what seemed to be the necessary,
  • fast_forward00:14:33 - intermediate-level processes.
  • fast_forward00:14:35 - Then you use whatever data are available to say, but I'm not happy as a neurocomputational
  • fast_forward00:14:44 - type to just say I've got schemas as abstract computational processes.
  • fast_forward00:14:48 - That might be enough if I'm building a robot to say, okay, that's a good architecture for the robot.
  • fast_forward00:14:53 - But if I really want to understand the human brain, as indeed I do,
  • fast_forward00:14:56 - then I don't rest with the schema analysis if there are data available which
  • fast_forward00:15:02 - will let me be more explicit about how plausible neural networks in the brain
  • fast_forward00:15:08 - will actually implement those schemas and that means the original schema level
  • fast_forward00:15:11 - model may get restructured to accommodate more neural level data.
  • fast_forward00:15:15 - In a comment on another talk you mentioned that you weren't entirely happy with
  • fast_forward00:15:19 - the split proposed by David Mark which is algorithms and the implementations.
  • fast_forward00:15:24 - Your suggestion was that if you want to understand the brain algorithms,
  • fast_forward00:15:28 - then you might want to come through the implementation.
  • fast_forward00:15:31 - So that's what we're talking about. It's looking at what we can see about the
  • fast_forward00:15:35 - implementation in the decomposition of the brain and then say your schema-level
  • fast_forward00:15:41 - system sounds like the algorithm, but it is informed by... Right.
  • fast_forward00:15:46 - So I think the problem was that a lot of David Maher's writing and a lot of
  • fast_forward00:15:50 - the way people quote his statement is the idea I can specify the problem,
  • fast_forward00:15:54 - then I can come up with the algorithm, and then I can implement it.
  • fast_forward00:15:58 - The point is that if we want to understand the brain, you've really got to look
  • fast_forward00:16:02 - at a dynamic loop where, yes, you already understand the behavior and have the top level fixed,
  • fast_forward00:16:09 - but the algorithm is going to depend so crucially on whether you're using neural
  • fast_forward00:16:13 - nets or serial computers that that's an ongoing loop.
  • fast_forward00:16:17 - I will have a schema model as an initial algorithm.
  • fast_forward00:16:21 - Then I will see how well I can implement it. The feedback from that may change
  • fast_forward00:16:24 - my schema level model so I have a loop of understanding.
  • fast_forward00:16:27 - But in that approach, you also were using this concept of causal completeness
  • fast_forward00:16:33 - to guide your choices with respect to the constraints you want to consider.
  • fast_forward00:16:38 - So, but how complete can you actually be in reality and in defining these kinds of models?
  • fast_forward00:16:46 - So, is causal completeness a hope or a reality of building? Oh, no, it's a reality.
  • fast_forward00:16:51 - I mean, the point of a computational model is that it's causally complete in
  • fast_forward00:16:55 - the sense that when you provide the appropriate input and you build on the appropriate
  • fast_forward00:17:00 - memories, you get the observed behavior. behavior.
  • fast_forward00:17:05 - But if I'm causally complete with respect to, let's say, an analysis of saccadic
  • fast_forward00:17:12 - eye movements, that same model is not going to be causally complete with respect
  • fast_forward00:17:18 - to arm movements, let alone language.
  • fast_forward00:17:20 - So it's not going to be causally complete for every possible behavior.
  • fast_forward00:17:24 - It's causally complete with respect to that behavior. Again,
  • fast_forward00:17:27 - the level of description that you start with will determine if I'm looking at
  • fast_forward00:17:32 - subtle learning effects,
  • fast_forward00:17:33 - then I may have to go back and iterate the model to include data about synaptic
  • fast_forward00:17:39 - plasticity to understand the timing of learning so that the causal completeness
  • fast_forward00:17:46 - is not saying I have covered everything in the universe.
  • fast_forward00:17:49 - The causal completeness is saying that where the experimentalist could just
  • fast_forward00:17:53 - go in and say say, I'm monitoring activity in a part of the brain,
  • fast_forward00:17:56 - and it correlates with some particular behavior.
  • fast_forward00:17:59 - He doesn't have to say how the sensory stimuli got to the point where they could cause that.
  • fast_forward00:18:04 - He doesn't have to specify how that activity could get to the muscles to yield the overt behavior.
  • fast_forward00:18:10 - He's just saying, here's a fascinating correlate. I have to say.
  • fast_forward00:18:14 - The model is causally complete in the sense that if I stimulate my model with
  • fast_forward00:18:18 - a representation of the sensory stimuli,
  • fast_forward00:18:21 - then my representation of those cells will fire in that way,
  • fast_forward00:18:24 - and I have a network which will show how that emerges in the behavior.
  • fast_forward00:18:30 - So it's causally complete with respect to the level of description.
  • fast_forward00:18:35 - Right, and also then given the assumptions I have made about the primitive elements
  • fast_forward00:18:40 - that are playing the key role in my model, that you say, okay,
  • fast_forward00:18:45 - below that level, I don't need to go.
  • fast_forward00:18:47 - And again, the point I made yesterday was that in some learning models,
  • fast_forward00:18:53 - we have the idea that we have an input signal and a training signal,
  • fast_forward00:18:57 - which tells you to remember that input or to change your response to that input.
  • fast_forward00:19:02 - And a lot of those models are at the event level.
  • fast_forward00:19:05 - So you say, here is the input at this event, here is the training signal at
  • fast_forward00:19:09 - this event, what's the output?
  • fast_forward00:19:10 - And they were saying in some cases in the real brain, though.
  • fast_forward00:19:13 - The output might be planning a movement, and the actual work by the muscles
  • fast_forward00:19:22 - follows later, and then the observable effect of that result.
  • fast_forward00:19:25 - So the training signal might well be 200 milliseconds later than the brain activity.
  • fast_forward00:19:30 - And then that forces me to say,
  • fast_forward00:19:32 - what is going on in the brain that bridges across that fifth of a second?
  • fast_forward00:19:37 - So there I was forced to look at details of synaptic function that were not
  • fast_forward00:19:42 - engaged in the initial model, which was just event by event,
  • fast_forward00:19:46 - rather than looking at the actual time course of action.
  • fast_forward00:19:54 - Coming on to your later work stop eating or lie like a shell so just can you
  • fast_forward00:20:00 - just clarify why you moved into this area of the evolution of language from
  • fast_forward00:20:05 - trying to understand how the brain implements some of these very important,
  • fast_forward00:20:11 - actions and cognitions and perceptions that you've now it's actually a different
  • fast_forward00:20:16 - it's not that I moved into it But firstly,
  • fast_forward00:20:18 - as a schoolboy, I was very much intrigued by the history of the English language.
  • fast_forward00:20:24 - So that has always been an interest. Then in my mid-career at the University
  • fast_forward00:20:31 - of Massachusetts in the 80s, I helped found the Cognitive Science Program,
  • fast_forward00:20:37 - where I worked with the linguists.
  • fast_forward00:20:39 - So therefore, the forging of connections between my work on visual control of
  • fast_forward00:20:45 - action and their work on language became a very important topic,
  • fast_forward00:20:49 - and we had about four PhD theses on that.
  • fast_forward00:20:51 - Then I moved to the University of Southern California in 1986.
  • fast_forward00:20:56 - I got busy with other things. And then when Ritz-Zolartes' group discovered
  • fast_forward00:21:01 - the mirror neurons in the early 90s, I was already working with that group on
  • fast_forward00:21:06 - visual control of hand movements.
  • fast_forward00:21:08 - And my group at USC did brain imaging, which established that the activity that
  • fast_forward00:21:15 - looked like mirror neurons in the human brain was a broker's area or a speech
  • fast_forward00:21:19 - area, which we now understand more as a language area.
  • fast_forward00:21:23 - That provided the path after a 10-year break back into the study of language
  • fast_forward00:21:28 - because now there was a really strong connection that exploited what I had said before.
  • fast_forward00:21:35 - In fact, from 1979 to 1985, I was publishing in the area of models of language.
  • fast_forward00:21:43 - The work you're doing now is always anticipating the models.
  • fast_forward00:21:48 - The modeling is is pulling behind in some sense, or has it caught up with where
  • fast_forward00:21:52 - you are theoretically in terms of your ideas about language evolution?
  • fast_forward00:21:57 - Or is that the way it's always been, that you've always had the theories first,
  • fast_forward00:22:01 - and then the model followed on?
  • fast_forward00:22:04 - No, I think, again, it's a loop that at times you raise questions and form hypotheses,
  • fast_forward00:22:11 - and then look for data to test the resulting model.
  • fast_forward00:22:16 - At other times, you're confronted with a body of data and you're trying to make sense of it.
  • fast_forward00:22:22 - In the case of language, I think the state of neurolinguistics is very fragmentary
  • fast_forward00:22:29 - from a computational point of view.
  • fast_forward00:22:30 - So here we are establishing a range of models that we hope will begin to fill in the landscape.
  • fast_forward00:22:39 - But meanwhile, the overarching model I have is a conceptual model,
  • fast_forward00:22:43 - not an implemented model.
  • fast_forward00:22:45 - But it is already engaged in many conversations, talking to primatologists.
  • fast_forward00:22:50 - What do we know about communication in monkeys and apes?
  • fast_forward00:22:54 - How does that make one set of hypotheses more plausible than another at the conceptual level?
  • fast_forward00:23:00 - Looking at people working with sign languages,
  • fast_forward00:23:06 - How do we change our view of what language is because we realize it's not speech,
  • fast_forward00:23:11 - it's a more general capability?
  • fast_forward00:23:14 - And then just getting into debates about what is the nature of language.
  • fast_forward00:23:18 - We have Noam Chomsky on the one hand looking at syntax as an abstract structure.
  • fast_forward00:23:24 - There are other people who are looking at language as a flexible means of communication.
  • fast_forward00:23:27 - So I find myself moving over there to saying, how do I capture certain aspects
  • fast_forward00:23:33 - of that more action-oriented approach to language that I can now bring back
  • fast_forward00:23:38 - to the brain in a way that I think is consistent with my other work on the brain?
  • fast_forward00:23:42 - So there's a great deal of work going on to create this framework in which some
  • fast_forward00:23:48 - models already exist, but in which we're also defining spaces for future modeling.
  • fast_forward00:23:53 - But if I look at this from the outside,
  • fast_forward00:23:57 - that does give the impression of a discontinuity from a very much action-oriented
  • fast_forward00:24:01 - view with the schemas to this view of the mirror system and language that actually
  • fast_forward00:24:07 - seems to start with the primitive element of gestures or any communicative actions.
  • fast_forward00:24:13 - So it seems to me that there's a discontinuity between actions as in relation
  • fast_forward00:24:19 - to objects in the world and now communicative gestures as the starting point
  • fast_forward00:24:24 - of developing language. So how should I relate to these two?
  • fast_forward00:24:27 - Well, my joke is that my work on language evolution is to replace one big miracle
  • fast_forward00:24:33 - by a series of small miracles.
  • fast_forward00:24:34 - And so what I'm trying to say is that getting all the way from a monkey-like
  • fast_forward00:24:39 - brain, that's a simplification, but let's say from whatever our common ancestor
  • fast_forward00:24:43 - had 20 million years ago to today in one leak is too much.
  • fast_forward00:24:47 - But if I can break it into, okay, getting from recognizing other actions.
  • fast_forward00:24:52 - As one similar to those they already have, to understanding other actions as
  • fast_forward00:24:57 - a means to imitating them, to being able to understand complex actions in terms
  • fast_forward00:25:02 - of the structure of goals and movements,
  • fast_forward00:25:04 - then these are reasonable miracles in which to address specific modeling, as we are now doing.
  • fast_forward00:25:11 - And then again, the transition from the use of these actions for praxis to others
  • fast_forward00:25:20 - imitating them to get praxis, to being able to build pantomime is another small miracle.
  • fast_forward00:25:26 - And then once I get to pantomime, then the social ritualization of those into
  • fast_forward00:25:31 - symbolic gestures is, again, a meaningful step.
  • fast_forward00:25:35 - And then once I've got that use of arbitrary gestures for communication,
  • fast_forward00:25:40 - bringing the vocal apparatus back into play is a reasonable thing.
  • fast_forward00:25:44 - So in some sense, what I'm doing is, as I say, I'm breaking it into miracles
  • fast_forward00:25:47 - that are small enough that, yes, they're discontinuities. Evolution is a discontinuity.
  • fast_forward00:25:53 - We have forelimbs. Birds have wings. We have a common ancestor.
  • fast_forward00:25:58 - So there's always going to be divergence points.
  • fast_forward00:26:01 - But the issue is, how can we define that in a way that it becomes plausible
  • fast_forward00:26:05 - that a relatively small suite of genetic changes could support that divergence?
  • fast_forward00:26:11 - So, at the moment, I'm trying to consolidate data from many different disciplines
  • fast_forward00:26:18 - to come up with what I think is a plausible set of bifurcations in our evolutionary history,
  • fast_forward00:26:24 - and then to build before and after models.
  • fast_forward00:26:26 - And say, this is what the brain was like before, this is what the brain was like after.
  • fast_forward00:26:31 - And then hopefully, in the end.
  • fast_forward00:26:34 - Evolving work in genetics and molecular biology will catch up and say,
  • fast_forward00:26:39 - we can begin to understand the genetic correlates of the changes in neural architecture,
  • fast_forward00:26:45 - brain architecture, that you've posited to provide that set of stepping stones
  • fast_forward00:26:50 - from the common ancestor of 20 million years ago to the language-ready brain of the human.
  • fast_forward00:26:55 - So to get to a conclusion of our short interview, I have two generic questions.
  • fast_forward00:27:04 - So you are in this field now for a really long time.
  • fast_forward00:27:07 - You trained with some of the heroes of the cybernetic age, Wiener and McCulloch.
  • fast_forward00:27:14 - So in your long experience in this field, what's the law of RBEAP that we,
  • fast_forward00:27:20 - sort of younger, representatives of the younger generation should take on board
  • fast_forward00:27:25 - to actually help us understand the brain and cognition? Yeah.
  • fast_forward00:27:31 - Well, I think schema theory has been a very minor theme in the field,
  • fast_forward00:27:39 - and I have a certain pride of ownership, and I think it should be a bigger theme.
  • fast_forward00:27:44 - The idea that there is a functional decomposition to be placed in conversation
  • fast_forward00:27:48 - with a neural decomposition.
  • fast_forward00:27:50 - Often we'll get something like, well, I want to look at vision,
  • fast_forward00:27:53 - and oh, I want to look at stereo, and then I jump immediately to the neural networks.
  • fast_forward00:27:59 - And the idea of thinking about stereo in terms of how it contributes to an overall
  • fast_forward00:28:06 - set of interacting schemas for vision in the service of planning behavior gets lost.
  • fast_forward00:28:13 - So I think my big lesson is I want people to think more about how to develop
  • fast_forward00:28:20 - schema theory as a high-level, if you will, brain programming language,
  • fast_forward00:28:24 - which can then be either played out on circuitry for the design of robots or
  • fast_forward00:28:30 - put in conversation with lesion data.
  • fast_forward00:28:33 - Imaging data, neurophysiological data to try and move into very complex systems
  • fast_forward00:28:38 - where approaching it from the level of,
  • fast_forward00:28:42 - here's a lot of synapses or here's a lot of neurons is doomed to
  • fast_forward00:28:44 - failure and i think as as we go into
  • fast_forward00:28:47 - this world of really large integrated
  • fast_forward00:28:52 - systems with complex suites of behavior this
  • fast_forward00:28:55 - will become more and more necessary than it has
  • fast_forward00:28:58 - been in the past okay and then the concluding question
  • fast_forward00:29:01 - for me would be um if we're going to meet up again five years from now you know
  • fast_forward00:29:06 - in science it's all about prediction so what's what's the prediction i can hold
  • fast_forward00:29:10 - you to five years from now whether we can find out whether it was true or false
  • fast_forward00:29:15 - what is one prediction you want to stick your neck out for today.
  • fast_forward00:29:22 - I don't really have, in the sense of a limited prediction.
  • fast_forward00:29:26 - It's rather what I've laid out, especially in my second talk here,
  • fast_forward00:29:30 - has been a framework for the study of how the brain supports language that is integrated with,
  • fast_forward00:29:39 - despite evolutionary divergences, from the way in which sensory data are processed
  • fast_forward00:29:46 - to perceive our relation with the world and move us on.
  • fast_forward00:29:50 - It also has emphasized the way that neuroscience has to go from a focus on the
  • fast_forward00:29:59 - isolated individual responding to sensory data with a course of action to more
  • fast_forward00:30:04 - and more thought about social interactions, a trend that's already started.
  • fast_forward00:30:08 - So my prediction is a very wishy-washy prediction.
  • fast_forward00:30:11 - It is that five years from now, the framework that I have presented to this
  • fast_forward00:30:19 - point will still be seen as correct in its overall structure,
  • fast_forward00:30:23 - but there will be a lot of very specific models,
  • fast_forward00:30:27 - embedded within that structure, which will change some of the details,
  • fast_forward00:30:31 - but not change the overall conceptual framework. Very good.
  • fast_forward00:30:35 - Michael Arbique, thank you very much for joining us, and we hope to see you
  • fast_forward00:30:38 - back very soon. I look forward to it.

Be the first to leave a comment

Leave a comment

Your email address will not be published. Required fields are marked *

Convergent PozitivLinia Logo

Exploring the convergence of neuroscience, robotics, and AI through conversations with leading researchers since 2010.

A project of the Convergent Science Network Foundation.

© CSN Podcasts. Developed by IMCreativeWEBC

0%

Login to enjoy full advantages

Please login or subscribe to continue.

Go Premium!

Enjoy the full advantage of the premium access.

Stop following

Unfollow Cancel

Cancel subscription

Are you sure you want to cancel your subscription? You will lose your Premium access and stored playlists.

Go back Confirm cancellation