Listeners:

Top listeners:

00:00 00:00

Go to album

Paul Verschure on consciousness and distributed adaptive control CSN Podcasts
Edvard Moser on grid cells and entorhinal cortex CSN Podcasts
Giacomo Rizzolatti on mirror neurons and action understanding CSN Podcasts
Robert Axelrod on game theory and prisoner's dilemma CSN Podcasts
Adrian Owen on disorders of consciousness and vegetative state CSN Podcasts
Jonathan Whitlock on markerless motion capture and posterior parietal cortex CSN Podcasts
Luis Puelles on neuroanatomy and prosomeric model CSN Podcasts
Zoltan Molnar on subplate neurons and cortical development CSN Podcasts

Gary Marcus on canonical microcircuit and variable binding

Episode 5 15.03.2014

PLAY EPISODE

Season 2014

0 Followers 11 Episodes

Description

What if the search for a single canonical cortical microcircuit is leading neuroscience in the wrong direction? Cognitive scientist Gary Marcus argues that the brain’s apparent uniformity masks functionally critical variations , and that understanding higher cognition requires computational primitives we have barely begun to identify. Subscribe for more from the Convergent Science Network podcast series. Gary Marcus joins Paul Verschure and Tony Prescott at the BCBT summer school to challenge the dominant idea that a single repeated circuit underlies all cortical computation. Drawing on evolutionary biology’s principle of duplication and divergence, Marcus argues that cortical areas may share a common template but differ in ways that are functionally decisive , much like a hand and a foot share most of their genes yet serve very different purposes. He contends that the field’s attraction to parsimony, while productive in physics, is misleading in biology where complexity is the rule. The discussion identifies what Marcus sees as the most critical gap in computational neuroscience: variable binding. While hierarchical feature detection is reasonably well understood and modeled, the ability to represent variables, instantiate them with particular values, maintain structured representations, and distinguish types from tokens remains unexplained at the neural level. Marcus argues these operations are non-negotiable for higher cognition, particularly language, and that no current neural network architecture adequately captures them. He also revisits his earlier claim about tree structures, now arguing that humans lack true location-addressable memory, which limits our ability to represent unbounded hierarchical structures. Key topics include why the cortex appears uniform under a magnifying glass but differs in functionally important ways, how duplication and divergence applies to cortical circuit evolution, what variable binding is and why it matters for language and reasoning, the limitations of simple recurrent networks for capturing syntax, why labeled-line architectures cannot scale to handle novel representations, and how a phylogenetic approach to cortical circuit types could advance our understanding of human cognition. Part of the Convergent Science Network podcast series from the BCBT Summer School.

Tagged as:
canonical microcircuit Circuit Cortical Duplication Divergence Functionally Higher Cognition variable binding

About the author

CSN Podcasts

Both the triumphs of humanity and its most evil deeds have resulted from collaboration. In a time where humanity is required to aspire to the former and minimize the latter, the question arises of how collaboration arises and why it fails. Surprisingly, this phenomenon, so central to who we are, is not well understood. Hence, a collaborative effort is required to understand collaboration in its full biological, psychological, sociological, cultural, and economic complexity and to translate this understanding into operational impact. This series of podcasts is one step toward achieving these complementary goals. The Collaboration Podcast presents interviews with people who are central orchestrators of collaboration in various domains including business, government, science, art, health, sustainability, and the military. The discussions were conducted by Prof. Dr. Paul F.M.J. Verschure and members of the Program Advisory Committee of the Ernst Strungmann Forum on Collaboration (https://www.esforum.de/forums/ESF32_Collaboration.html) during 2021 and had the goal to sketch a map of opportunities, challenges, and obstacles in human collaboration. The forum took place in May 2022, and now we would like to share this series of interviews with a broader audience. The full report of the Forum will be published in 2023 by MIT Press. The podcast was produced by the Convergent Science Network (https://www.convergentsciencenetwork.org/). Context: The stability of social systems depends critically on realizing sustainable methods of “collaboration,” yet how and by which means collaboration is achieved is not clearly understood; neither are the conditions or processes that lead to its breakdown or failure. Collaboration can be understood as cooperation between agents toward mutually constructed goals. Part of the reason for our lack of understanding is that the phenomenon of collaboration is, by nature, a highly multidisciplinary problem, and effective research into its complexities has been difficult to achieve across the broad range of scientific and technical disciplines involved. The need for a fundamental understanding of collaboration, however, has become increasingly important. Not only does humankind demand answers as it attempts to address critical challenges at multiple scales (e.g., climate change, migration, enhanced automation, social and economic inequality), but ever-increasing technological and economic means of interconnecting people and societies are disrupting long-established, familiar patterns of how we interact. Radical technological changes that are ongoing have the potential to reshape collaboration in ways that are currently hard to predict or influence (e.g., by altering configurations in interaction, information creation, and modes of communication). On one hand, such changes could disrupt hitherto stable forms of collaboration by affecting critical communication channels and traditional roles, as can be observed in the rapidly changing patterns in governance, commerce, and social interaction. Conversely, technology could lead to the emergence of novel, successful forms of collaboration that deviate from traditional “hierarchical” architectures. Evidence of this can be seen in areas as diverse as highly automated manufacturing plants, the open science movement, collaborative software repositories, user-centered services, and the sharing of economy-based modes of organization. Without a fundamental understanding of the mechanisms, processes, and boundary conditions of collaboration, it is not possible to evaluate or predict which of these possible scenarios are sustainable or even plausible. The Forum “How Collaboration Arises and Why it Fails” (May 8–13, 2022, Location: Frankfurt am Main, Germany) Chairs: Andreas Roepstorff and Paul Verschure Program Advisory Committee: Jenna Bednar, Julia R. Lupp, Bhavani R. Rao , Andreas Roepstorff, Ferdinand von Siemens, and Paul Verschure

Timestamp

00:00:03 - This is the Convergent Science Network podcast. Leading researchers in the domain
00:00:10 - of neuroscience, brain theory and technology are interviewed by Paul Verschoor and Tony Prescott.
00:00:19 - This is Paul Verschoor with Tony Prescott and the Cognitive Science Network podcast.
00:00:26 - Today, we're here with Gary Marcus, who's one of the speakers at our BCPT summer school.
00:00:32 - Gary, you really tried to come up with an alternative view on how we can think
00:00:40 - about, if you want, cortical computation, contrasting your proposal to this
00:00:45 - notion of a canonical microcircuit.
00:00:47 - Right so so what what
00:00:51 - do you find problematic with this notion
00:00:54 - of a canonical microcircuit now do we define a canonical microcircuit
00:00:57 - it looks like a search for the one true ring that will rule them all and searches
00:01:03 - for one true ring to rule them all usually fail and i have some specific reasons
00:01:07 - why i'm skeptical about this one like i said the broadest level i don't think
00:01:11 - we're going to find one true ring i don't think we're going to find a silver
00:01:14 - bullet i think that the brain is really complicated,
00:01:16 - and I think you can see that complication at any level at which you try to analyze
00:01:20 - the brain, whether you're talking about connectivity between different areas
00:01:23 - or whether you're looking at the number of neuron types or all of the different
00:01:27 - proteins that are trafficked in synapses. There's enormous complexity.
00:01:31 - I think that in physics, people seek a kind of parsimony. They're trying to
00:01:35 - find a grand unified theory in a few tiny principles.
00:01:39 - I don't think that that's plausible for biology. There was a quote I used to
00:01:43 - like from Francis Crick.
00:01:44 - I may not get the words exactly right, but it's something like,
00:01:46 - parsimony is a valuable tool in physics.
00:01:52 - In biology, it's a dangerous implement.
00:01:54 - And then I met him once, and I told him that this was my favorite quote.
00:01:57 - And he said, yeah, in physics, we have laws. In biology, there are gadgets.
00:02:04 - And hoping that you're going to find the entire answer in a single gadget is,
00:02:08 - I think, unrealistic. But now, in just celebrating complexity,
00:02:12 - you might not gain understanding either.
00:02:14 - No, I'm not saying that, for example, the right way forward is to just have
00:02:18 - a big massive computer simulation where we don't know any of the operating principles
00:02:22 - and we just sort of worship the complexity.
00:02:24 - I think what we want are intermediate levels of explanation,
00:02:28 - intermediate levels of components that map on to other levels.
00:02:34 - In understanding a computer, you want to understand a series of levels starting
00:02:38 - from the transistor and moving up to AND gates and OR gates,
00:02:40 - microprocessors, operating system, software, etc.
00:02:45 - And I think that we probably want to find something similar in understanding
00:02:49 - probably any biological system.
00:02:50 - There are going to be low-level components that contribute to higher-level components.
00:02:54 - But then what are we trying to explain?
00:02:57 - Is it like, in the MAR sense, some overall functionality that we can push into
00:03:03 - a computational level of description, which implementation we have to identify?
00:03:08 - Or is it another set of functionalities that will decompose along a different
00:03:12 - set of levels of description?
00:03:15 - Well, I think that Marsense is a really good starting place where you want to
00:03:18 - understand the relation between computation and specific algorithms and how
00:03:23 - those specific algorithms are realized.
00:03:25 - And I think you might ask for even more than that, but I think that that's a
00:03:30 - really good starting point.
00:03:31 - I think sometimes in neuroscience, people lose sight of the mapping between these things.
00:03:36 - And I think that that's really what the field should be about. out so
00:03:39 - if you're talking about like mapping the whole brain that's a great thing
00:03:42 - to do but mapping the whole brain isn't by itself a question about
00:03:45 - how that relates to computation and i think that we
00:03:47 - need to keep our eye on that ball what we're trying to do is to find
00:03:50 - let's say motifs that are represented neurally that are repeated over and over
00:03:54 - again and how those connect in turn to the computational right but so let's
00:03:59 - set up at the context of the discussion if we then want to let's say declare
00:04:04 - some higher level functions that we want to explain in the anti-implementational terms,
00:04:08 - what are the higher level functions that we need to explain?
00:04:11 - Well, we don't fully know that. I mean, I think we're, I use this metaphor in
00:04:16 - the talk about we're working from two ends of the tunnel and we're trying to meet in the middle.
00:04:19 - The truth is we don't have a well-defined starting point on the higher level cognition side.
00:04:25 - We have guesses. the place where
00:04:28 - we best have answers are like we know some very specific
00:04:31 - things about individual channels and things like that
00:04:34 - on the neural side and then on the other side we
00:04:36 - have some guesses about what's going on cognition we don't know for
00:04:39 - sure I can give you my own set of guesses so for example
00:04:43 - I wrote a book called the algebraic mind and I laid out basically what I would
00:04:47 - take to be the tenets of symbol manipulation and said these are these are non-negotiable
00:04:51 - these are things as far as I can tell really have to be part of any theory of
00:04:54 - mine and that included the ability to represent variables that you can instantiate
00:04:59 - with particular values at particular times,
00:05:01 - to have operations over those variables,
00:05:04 - to have structured representation so A, B is not the same as B, A,
00:05:09 - to represent a type token distinction, and ultimately to represent tree structures.
00:05:14 - Now, I wrote that book almost 15 years ago, I recant one of those claims to
00:05:19 - some degree, which is I don't think that the human mind actually has the ability
00:05:22 - to represent arbitrary tree structures.
00:05:24 - It'd be really handy if we had it and computers make use of that all the time.
00:05:28 - So the directory structure for your files, for example, is a tree structure.
00:05:32 - I don't think that we can do that in unbounded ways. And there's some psycholinguistic
00:05:35 - phenomena that suggest to me
00:05:37 - that we can't like sort of in the mind's eye apprehend the full sentence.
00:05:41 - It's analogous to these change blind experiments where, you know,
00:05:44 - you see a parking lot full of soldiers and there's a jet plane and in between
00:05:49 - frames with a mask jet engine comes and goes and you don't even notice because
00:05:52 - you're watching the soldiers and the plane and so forth.
00:05:54 - So you have the illusion of representing the image as a whole,
00:05:58 - but we don't really. We represent pieces of it and we can reconstruct some of
00:06:01 - it, hoping that the world is stable.
00:06:02 - I think that in representing sentences, we don't really have the full sentence in our head.
00:06:07 - And so you're subject to kind of linguistic illusions that are like optical illusions.
00:06:11 - Like I can say the sentence, more people have been to Russia than I have.
00:06:15 - And ostensibly it's a reasonable sentence, but you sit there,
00:06:18 - you realize it's what's called ellipsis. Something's missing.
00:06:20 - More people have been to Russia than I have.
00:06:22 - Been to Russia, that doesn't actually make sense. But you don't immediately
00:06:25 - notice the problematic nature of that sentence because you don't really have
00:06:30 - the full sentence in your head.
00:06:32 - Take another example. If I say it was the boxer that the sailor loved or something
00:06:37 - like that, you can get confused pretty quickly about what the relations are
00:06:40 - between the particular actors in the scene.
00:06:43 - If you really had a full tree structure in your head, you probably wouldn't get confused.
00:06:47 - Instead, what I argued in my book Kluge is that we don't have location addressable
00:06:52 - memory in our brains the way that computers do. So in a computer,
00:06:55 - you have essentially a set of
00:06:56 - safe deposit boxes that are numbered and you store things in those boxes.
00:07:00 - And once you put something in box 113, you can expect to get it back out again.
00:07:04 - And if you have that, then you can build tree structures in a very straightforward
00:07:08 - way where you just map the boxes onto the trees.
00:07:11 - But if you don't have location addressable memory, it's very difficult to actually
00:07:15 - represent a tree structure. So I think that one was actually an incorrect claim
00:07:19 - that I made in the Algebraic Mind.
00:07:21 - I stand by the others, and I would say there are other things too.
00:07:24 - I come from a language perspective, but if you were doing vision,
00:07:27 - you could have your own wish list.
00:07:29 - But those would start my wish list. And I would say that type tokens is one,
00:07:34 - the distinction between like this bottle of water and bottles of water in general
00:07:38 - is something that you need at some level in vision and that we know surprisingly
00:07:42 - little about how the brain does that.
00:07:44 - But again, I think it's non-negotiable, at least if you have human cognition,
00:07:47 - that's part of what you traffic in is the difference between a kind and a particular instance of a kind.
00:07:52 - Right so the example that you
00:07:54 - gave of our difficulty in processing these sentences with
00:07:57 - embedded clauses uh is one of the ones that uh
00:08:00 - jeffrey ellman i think used to motivate his simple recurrent network which you
00:08:05 - were quite critical of in your talk and he said look i have a network which
00:08:09 - can generate some things which look a bit like grammar and it has difficulties
00:08:14 - with particular types of constructions that are like the constructions that
00:08:18 - human minds have difficulty with.
00:08:21 - Now, you felt that the simple recurrent network wasn't powerful enough and that
00:08:24 - there needs to be something more.
00:08:26 - But in your talk, you weren't specific on how we go from the more computer-like
00:08:31 - FPGA-type mechanisms onto something which is neural circuitry.
00:08:37 - So have you got some examples in mind when you think of how that maps onto circuit designs?
00:08:43 - Well, I think we don't know enough to understand, say, a parser as a whole.
00:08:47 - So So what Ellman was trying to do in that model was to capture a lot about
00:08:50 - both syntax and semantics in one model.
00:08:53 - And I think too much, in fact. So he had a model where you could predict the
00:08:58 - next word in a sentence based on prior context. Yeah.
00:09:02 - Syntax and semantics weren't even explicitly represented. All you had were individual
00:09:06 - words that followed one another.
00:09:08 - And it's true that some of what it did had some superficial similarity to what
00:09:13 - people did, but I think it got it right for the wrong reasons.
00:09:16 - And I think it's possible to break down systems in many ways and think that
00:09:19 - you understood the system.
00:09:21 - But if you look at whether the system as a whole works, well,
00:09:24 - his system as a whole didn't work. There are lots of things about language it didn't really capture.
00:09:28 - And so I think people made more of it than they probably should.
00:09:32 - That's one part of your question. The other part of the question is,
00:09:35 - you know, where do we go from here?
00:09:37 - How do we handle these things? Part of what I'd say is we're not in a position
00:09:41 - to understand a complete parser now or a complete seeding segment or anything like that.
00:09:45 - What I'm arguing is that we need intermediate units before we can even hope
00:09:50 - to take on that question.
00:09:51 - So I think parsers, for example, have to do some type token kind of stuff.
00:09:56 - They certainly have to do a lot of variable binding.
00:09:58 - They're constantly doing variable binding. Even if the variable binding system
00:10:01 - itself is not perfect, they're doing a lot of it.
00:10:04 - They're binding syntax and semantics together and particular elements of the
00:10:09 - syntax trying to figure out the roles of things.
00:10:11 - And I don't think we're really going to be able to unravel that circuitry until
00:10:15 - we can at least say recognize the circuitry that underlies a binding and say,
00:10:20 - okay, it's invoked here in this way.
00:10:21 - Otherwise, it's just too unconstrained. I mean, you could think from a computer
00:10:25 - science perspective, having Lisp enables you to do variable binding and to represent tree structures.
00:10:31 - But once you have Lisp, then you can build lots of different parsers,
00:10:34 - starting from this corner or that corner, this kind of search or that kind of search.
00:10:38 - But you couldn't even understand the differences between those if you didn't
00:10:42 - first understand the basic elements, the atoms of the language.
00:10:45 - But before we get to that, Tony, because I think we made a bit of a jump now
00:10:49 - into the modeling exercise that I think we have a lot to discuss there,
00:10:53 - Because, Gary, you took a very specific line of attack, if you want,
00:10:58 - to substantiate this point that we also need to give more room to,
00:11:03 - let's say, a functional level of consideration that could guide,
00:11:06 - let's say, this more neuroscience-oriented exercise, to which I'm very sympathetic.
00:11:11 - Then your approach was to say, well, you know, we ended up with this intuition,
00:11:15 - if you want, in neuroscience of a canonical circuit, in this case, a neocortex.
00:11:20 - But you can also say the same claim would hold for other main structures in the brain.
00:11:25 - And what you wanted to say is, look, this is a misleading construct.
00:11:31 - It's a misleading line of thought in trying to understand the brain.
00:11:36 - So first, how would you define a canonical microcircuit? So what do you consider
00:11:42 - to be a canonical microcircuit?
00:11:44 - Why do you think it had such an impact in the field that you see it now as a
00:11:48 - dominant, let's say, paradigm, if you want?
00:11:51 - And then, of course, question three, what then is wrong with it?
00:11:56 - Well, I'm not going to try to define it. It's not my term, and I don't think
00:11:59 - it's a satisfactory approach.
00:12:01 - But I think the notion is supposed to be that there's one kind of circuit that's
00:12:05 - repeated throughout the cortex, that you'll see many instantiations of it,
00:12:09 - and that experience will tune different instantiations in different ways,
00:12:13 - but that they're all at some level identical.
00:12:15 - I think this comes from a superficial reading of anatomy and also maybe from
00:12:20 - a kind of particular scientific aesthetic, let's say.
00:12:25 - So the superficial reading of anatomy is the cortex is relatively uniform throughout its extent.
00:12:31 - Of course, if you say that in a room full of anatomists, they'll all get exercise
00:12:35 - and say, well, that's not really true.
00:12:36 - I mean, I look at this area and that area and they're very different to me.
00:12:40 - But if you just looked under a magnifying glass and you weren't an expert,
00:12:44 - you might reasonably say Broca's area and occipital cortex, they don't really look that different.
00:12:49 - They don't look as different as you might have a priori expected,
00:12:51 - given how different vision and language turn out to be.
00:12:55 - And so I think it starts from that. Then I think there's an aesthetic.
00:12:59 - I think people are looking for kind of simple principles to explain the brain.
00:13:04 - I don't see any reason to think that we're actually going to get a few simple
00:13:07 - principles to explain the brain, that that's going to be sufficient. But I think,
00:13:10 - A lot of people are attracted to those kinds of theories. And this is not something
00:13:14 - unique to neuroscience.
00:13:15 - So people are looking for that in physics. There are actually arguments in physics
00:13:18 - about whether it's plausible.
00:13:20 - In linguistics, Chomsky lately, of all people, has been looking for a kind of
00:13:23 - explanation for linguistics in a few principles.
00:13:26 - I don't think he's gotten a lot of mileage out of it. He's gotten a lot of followers,
00:13:29 - but I don't think that he's made a lot of convincing progress doing that.
00:13:32 - But I see it over and over again in lots of different fields.
00:13:36 - In economics, people start with this model of rational man. They think this
00:13:39 - one principle is going to explain things that it doesn't really.
00:13:42 - Yeah, but wait, there's an issue now, right? I don't think you're really denying the anatomy as such.
00:13:47 - I mean, in that sense, if you do inject in the thalamus, you will see that most
00:13:51 - projections end up in layer four of a six-layered cortex in our case, right?
00:13:57 - Well, I won't deny that there's a common background, let's say, of similarity.
00:14:01 - But a The common background of similarity is not the same thing as identity.
00:14:05 - Exactly right. So I think it hinges on that issue of identity,
00:14:08 - right? It does. So you can think like the hand and the foot,
00:14:11 - they're clearly genetically related.
00:14:12 - I used to know the number, like 95% of the genes, maybe this is in another organism,
00:14:17 - but there's a very heavy overlap between the hand and the foot in terms of their genes.
00:14:21 - But there's also some fine grain tuning that makes them do different things.
00:14:25 - And you might expect the same thing for cortical circuits. In some ways,
00:14:29 - my talk is a meditation on that thought.
00:14:31 - I mean, I could have arranged it differently, but if you assume that duplication
00:14:36 - and divergence is kind of the dominant paradigm of evolution,
00:14:38 - you've got some set of genes, and there are copies, and that copy allows you
00:14:42 - to build something new, whether it's a new photoreceptor with a new wavelength
00:14:46 - distribution, or it's a new vertebrae, whatever it is, there's lots of duplication and divergence.
00:14:52 - And if you saw that in the cortex, then what you might see is from a distance,
00:14:56 - all this stuff looks the same.
00:14:58 - But if you zeroed in, you might see the kind of tweakings that make a hand different
00:15:03 - from a foot. So what you're saying to conclude this bit then is to say,
00:15:06 - look, we might have a common template.
00:15:09 - But in its ultimate expression, there's a lot of variability.
00:15:13 - And if we want to understand function, we must actually focus on that variability
00:15:16 - as opposed to these common templates.
00:15:19 - Well, or the conjoined function of the two.
00:15:22 - I don't want to ignore the template. I mean, a good example here,
00:15:25 - though, might be like if I gave you a breadboard with transistors and lights
00:15:28 - and so forth, you could build an AND gate or an OR gate.
00:15:32 - Or if I had enough of them, you could build all kinds of gates and you could
00:15:34 - say, well, it's just one breadboard.
00:15:36 - But the logical or functional consequences of a breadboard where you have slightly
00:15:42 - different wires are immense.
00:15:43 - I don't know if you probably are old enough to have played with these old RadioShack
00:15:47 - kits with the springs and the wires and stuff like that. So in some sense,
00:15:51 - you had a template, they would call them like the 150 in one kit.
00:15:54 - You could build a radio or an
00:15:56 - alarm or whatever by just instantiating that template in different ways.
00:16:01 - And all the world's difference in the structural ways led to the functional
00:16:06 - difference. Right, exactly.
00:16:08 - One of the themes of this week has been, because of the complexity that we see
00:16:12 - in an area like neocortex,
00:16:13 - let's look across species and see how other animals' cortex is organized,
00:16:19 - and to see if there are common principles and perhaps extrapolate from the variety
00:16:25 - that we see to what might have been some ancestral form of cortex,
00:16:29 - which I think we all will expect will be simpler and will have less variety.
00:16:33 - And, you know, John Cass was talking about maybe five areas as opposed to,
00:16:38 - you know, dozens of areas in primates.
00:16:41 - So one strategy to address the problem you're describing would be to say,
00:16:45 - let's not try and build a canonical microcircuit based on mouse brain,
00:16:49 - but let's think about what the original first mammal microcircuit might have been.
00:16:54 - And perhaps there were only a small number of different microcircuits in cortex.
00:16:58 - And once we've solved that problem, then we could use it to address problems
00:17:02 - that mammals commonly have.
00:17:04 - Then we can think about how you modify that circuit to get towards human cognition.
00:17:10 - Is that a viable strategy?
00:17:11 - I think so. And you could look at, for example, central pattern generators.
00:17:15 - And a couple of people have made suggestions, like Sten Grillner we were mentioning a minute ago.
00:17:19 - You could look at central pattern generators and ask, just there you could say,
00:17:23 - Has there been duplication and divergence in that?
00:17:25 - Are there different variations on central pattern generators that you could
00:17:28 - use to do different kinds of computations?
00:17:30 - I don't know that anybody's pursued that in any great detail,
00:17:33 - but that's kind of a version of the strategy that you're talking about.
00:17:36 - And you could do that with coupled oscillators.
00:17:39 - There are lots of domains in which you could say, here is an ancestral circuit.
00:17:44 - What's happened to that ancestral circuit?
00:17:46 - I mean, in some sense, people are trying to map out vision that way.
00:17:49 - So we know that there's PAC-6 at the top of this cascade.
00:17:52 - And in some creatures, it goes on to guide the construction of a compound eye.
00:17:56 - And in some, it constructs a mammalian eye. And there are these beautiful experiments
00:17:59 - that Walter Goering did where you take PAC-6 from a mouse, you express it in
00:18:03 - a fly, and you get an eye in the fly.
00:18:06 - You put in the fly's antenna, but it's not a mouse eye, it's a fly's eye.
00:18:09 - And so that says there's a very ancient code here that specifies something like
00:18:13 - build your light sensitive organ here.
00:18:15 - And then, you know, the cascades diverge and people, because they're more accessible
00:18:19 - tools, they are starting to really work out, you know, when did this gene change?
00:18:23 - When did these downstream genes change? And I would like to see us do the same in human.
00:18:27 - And the big problem, which Asif mentioned in his talk, is that we're mostly
00:18:32 - limited to kind of comparative methods with people.
00:18:36 - And it turns out that the nearest neighbor species, which is the one you'd think
00:18:39 - you want to study, doesn't have language.
00:18:42 - And language is obviously pivotal in our experience. So it's a bit unfortunate
00:18:46 - from our perspective as scientists that we can't look at chimps and have sort
00:18:51 - of 80% of language there and ask what happened with their 80% of language.
00:18:55 - At some level, it's hard, but at some level, I think that's exactly right,
00:18:59 - that it should be one of the prongs of the research strategy is to try to figure
00:19:02 - out basically a phylogeny of computational or cortical circuit types.
00:19:06 - I think that that's essential. potential.
00:19:07 - In fact, maybe the best we'll be able to do with language is to figure out these
00:19:11 - are the circuits that we share with mammals, or maybe these are the ones we
00:19:14 - share with vertebrates, these are the ones we share with mammals,
00:19:16 - these are the ones with primates, and there are these one or two that are different.
00:19:19 - And how do those one or two that we don't see attested, and we may not even
00:19:23 - be able to see them until we really get the EM under our belts in the right
00:19:27 - way, but eventually we might find one or two that are new.
00:19:30 - And it's really not going to be that language is those one or two,
00:19:33 - but how they work together with with the rest of that phylogeny, right?
00:19:35 - Language is gonna put together a whole bunch, in my view, of different cortical
00:19:40 - circuit types that are all maybe going back to one or two common ancestors,
00:19:44 - but are really different versions of those, and that's what we need.
00:19:46 - So now if you try to figure out the properties of this archetypical mammalian
00:19:51 - brain, which actually is the target of our own work.
00:19:55 - Then in some sense, you also showed in your talk that there are certain,
00:20:00 - let's say, theoretical approaches, like the recurrent networks of Elman you
00:20:04 - mentioned and so on, that are not going to get us there, right?
00:20:07 - So you were saying that you were making the point there that in the theoretical
00:20:10 - approaches, at least the ones you presented today, something fundamental is missing.
00:20:15 - So what's that? What's missing there? I mean, for me, the biggest thing that's
00:20:19 - missing is an understanding of variable binding.
00:20:21 - So I think we as a collective field of neuroscientists and computational neuroscientists
00:20:27 - and psychologists have a pretty good grip on hierarchical feature detection.
00:20:31 - We don't know everything that there is to know about it, but we have very good
00:20:34 - reason to think that it happens.
00:20:35 - We have some notion about some of the tricks you might need,
00:20:38 - like divisive normalization and so forth, to make it all work out. We have a grip on it.
00:20:44 - We have some idea of where it might live, how to build computer models of it.
00:20:47 - We don't have anything like that for variable binding. We have a few kind of
00:20:51 - stray voices speaking in the wilderness about it.
00:20:54 - But I think we need it. I think variable binding is as important to higher-level
00:20:57 - cognition as hierarchical feature perception is division.
00:21:00 - So you would be saying these hierarchical feature detection approaches might
00:21:05 - be nice if you talk about perception. perception, but if we start to talk about
00:21:09 - higher level cognition or feeding into language, it's not going to follow the
00:21:12 - same principles. Is that right?
00:21:14 - I don't want to overstate that. I think that hierarchical feature perception
00:21:17 - plays a role in, say, speech perception.
00:21:20 - So it's not that language doesn't use this stuff. I'm basically talking about
00:21:24 - like the Hubel and Wiesel ideas here.
00:21:26 - It's not that I don't think that that has any role, but I think there's something else too.
00:21:29 - So it'd be like, I'm not sure what analogy, it'd be like, I'm trying to do math
00:21:33 - and I've already got addition down and addition is great. I'm never going to
00:21:37 - get rid of addition, but I need some multiplication and maybe some square roots too.
00:21:40 - And we don't really know so much about how to do the square roots.
00:21:43 - We have a hint about the multiplication.
00:21:44 - We're really good at the addition, but we need a bit of multiplication.
00:21:48 - Bit of a better toolkit if we're going to grasp higher level cognition,
00:21:52 - not because it's not going to use these other tools.
00:21:53 - I mean, probably every good idea
00:21:56 - in developmental biology toolkits gets exploited in language in some way.
00:22:00 - And one of the findings, I'm not a big fan of fMRI, but one of the big findings
00:22:05 - from fMRI, I think, is that language is really distributed way across the brain.
00:22:10 - It's not just Broca's area, something like you read in a textbook.
00:22:13 - I think language is exploiting all
00:22:15 - kinds of things from theory of mind to hierarchical perception in general.
00:22:22 - But the part of it that we least understand is how we concatenate all the symbols
00:22:27 - together and manipulate them and move them around.
00:22:29 - And that's pretty essential. It's actually a dirty secret to these hierarchical
00:22:33 - models of when you talk about perception.
00:22:35 - That is that on the one hand, they scale really badly. Like if you talk about
00:22:42 - perception, you want to have different kinds of invariances,
00:22:45 - position, orientation, scale.
00:22:48 - But that means duplication of wires at a massive scale.
00:22:52 - So that means on anatomical rounds, it's very questionable whether even perceptual
00:22:57 - structures in the cortex can follow such a wiring strategy.
00:23:01 - And there's a second, I think, massive problem if you talk about variable binding,
00:23:05 - which is that ultimately it's a labeled line system, right?
00:23:09 - Right. So I think if you talk about higher level cognition and you think about
00:23:12 - working memory and you think about variable binding, how to achieve that with
00:23:16 - labeled lines, I think it's going to be really a long shot. I think we have to rethink.
00:23:21 - I don't know exactly what you mean by labeled lines, but I would say that for
00:23:26 - me, that's like a moment where you pause and you say, maybe one of my assumptions are wrong.
00:23:30 - I don't know precisely what you mean by the labeled lines, but I would say in
00:23:33 - general that it would be very hard to get from the set of ideas that are floating
00:23:39 - around computational neuroscience to variable binding. So there are two options.
00:23:43 - One is to say variable binding doesn't exist. And a lot of people actually have
00:23:46 - tried to make that argument.
00:23:48 - The other is to say we're missing something and this is a big clue.
00:23:51 - And that's the line that I'm taking. Right.
00:23:53 - The labeled line you can think of, take the Hubel-Mises model,
00:23:56 - simple cells to complex cells.
00:23:58 - Every single synapse that defines now
00:24:01 - the complex cell has to be uniquely labeled with
00:24:04 - a certain feature otherwise your complex cell cannot do this encoding right
00:24:07 - so but that means that synapse cannot be used for anything else anymore now
00:24:11 - you're stuck with it right so I do think wires are cheap and so I'm not completely
00:24:16 - convinced by your first part of your argument but I think the labeled line stuff
00:24:19 - actually relates to the issue that I'm worried about so labeled lines are fine
00:24:23 - if you're limited to encountering things that you've seen before so you can.
00:24:28 - You can grow a grandmother node literally for your grandmother through a bunch
00:24:33 - of experience and use that to recognize your grandmother.
00:24:36 - At least, you know, there's some data that suggests you might be able to do that.
00:24:39 - But what about the things that are unfamiliar to us, like the sentences that
00:24:42 - we haven't heard before?
00:24:44 - There, it starts to, just on exponential grounds, it becomes implausible,
00:24:48 - like that you've got a node for my last sentence, right?
00:24:50 - You have to construct something on the fly. You don't have a pre-labeled node
00:24:54 - for my last sentence or probably any of the sentences we said during the course of the podcast.
00:24:58 - You might have labeled nodes for some idioms like kick the bucket.
00:25:01 - You might really have a node in your brain that recognizes because it's an idiom.
00:25:06 - It's not compositional.
00:25:07 - You can't figure out the kick the bucket means death from the words kick and
00:25:10 - bucket. And so you may have some of those, but at the same time,
00:25:14 - language is generative.
00:25:15 - And there are some things that you can understand. In fact, a lot of them for
00:25:18 - which a solution that relies on a pre-labeled node is not going to cut it.
00:25:23 - There's got to be a way of constructing a novel representation on the fly.
00:25:27 - And I think we lack understanding of how that works. Exactly.
00:25:31 - The talk very much focused on the cortical microcircuit, but to get the functionality
00:25:37 - that you're looking for, then we might look outside cortex,
00:25:40 - and you might want to combine microcircuit types, assuming that the concept
00:25:45 - has some validity in basal ganglia,
00:25:47 - cerebellum, who knows amygdala, and together between these different structures,
00:25:51 - which everyone agrees have very different internal architecture,
00:25:55 - we might approach the kind of complexity that you're looking for.
00:25:58 - So yes, there's variability within the cortex, but it's variability on a theme,
00:26:02 - and you get the extra power that you need for something like language by having
00:26:05 - a systems theory of how the brain does something like linguistic cognition.
00:26:10 - Well, first of all, I totally agree we want a systems theory.
00:26:12 - And I think you're right to point out that in the talk I didn't give enough...
00:26:18 - Attention to these other systems. So I think you're exactly right.
00:26:22 - The cortex doesn't work on its own, right?
00:26:24 - I mean, if you get rid of the subcortical things, for example,
00:26:26 - the cortex is not going to be able to do its usual work.
00:26:29 - It's clear that the way that the cortex works is in interaction with the rest of the brain.
00:26:34 - And I think you're also right that some of what I'm calling differences in circuit
00:26:39 - types may have to do with basically how those other resources are accessed.
00:26:42 - So maybe Maybe two things that I want to call different circuits,
00:26:46 - what they really chiefly vary in is in how much they're calling these other
00:26:50 - systems and what they're doing with those other systems.
00:26:52 - So I'm very sympathetic with that. And I think I just did a poor job of discussing it.
00:26:56 - There's a written but not published version where we did a little bit better
00:26:59 - job of at least pointing to that possibility.
00:27:02 - So I'm completely sympathetic to it. But I think probably if there's any difference
00:27:07 - between me and you, it's that I think some of it's going to be at the level
00:27:11 - of like the difference between an AND and an OR gate that's actually local.
00:27:15 - I totally agree that it's not going to be all local. It's going to be really
00:27:18 - important stuff that isn't local that has to do with long range projections and so forth.
00:27:23 - And I agree that I didn't emphasize that enough in the lecture.
00:27:27 - But I also think that there are probably going to be some important differences
00:27:30 - within the immediate circuits as well. So one of the questions is how those
00:27:35 - differences come about.
00:27:36 - And the other theme this week has been not just comparing species,
00:27:41 - but also thinking about development within a species.
00:27:44 - And that was an aspect of your talk, because I think that you would agree that
00:27:48 - at some point in development, there is something relatively homogenous,
00:27:52 - and then we get heterogeneity increasing across cortex.
00:27:56 - And at some point, there's no going back. This sort of equipotentiality of cortex ceases to happen.
00:28:03 - And I think one slide that I left out, by the way, is just stuff from the Allen
00:28:08 - Institute showing that gene expression differs from front to back and that the
00:28:13 - gene expression is more similar the closer you are to cortex.
00:28:16 - And that's one clue. And there are many other clues we need to put together.
00:28:19 - But that's one clue that says there is some differentiation here that's important.
00:28:23 - And it's important to realize you don't need a lot of genes to be different
00:28:26 - in order to build something really different.
00:28:28 - So you can get sickle cell anemia from one nucleotide change,
00:28:31 - you could make the difference between an AND gate and an OR gate with,
00:28:34 - you know, probably a single gene or less.
00:28:36 - I'm not sure we literally have those things, but I think that relatively small
00:28:40 - numbers of genes can differ against a background of many shared genes and lead
00:28:45 - to significant differences.
00:28:47 - Yeah, I think, I mean, one of the, we had previous Terence Deacon as a speaker
00:28:51 - for BCBT, and he gave a talk which would have been very appropriate this week
00:28:55 - on this idea of relaxed selection,
00:28:57 - that there could be sort of junk DNA there which is floating around and it could
00:29:01 - be recruited for something like language.
00:29:05 - In order to create new functionality very rapidly and it might even be recruited
00:29:09 - without necessarily having a mutation It could be recruited by some epigenetic
00:29:15 - mechanism as we also heard this week.
00:29:16 - So What was you could maybe clarify for it?
00:29:21 - So a bit more is how much you think that these developmental processes that
00:29:25 - are creating these qualitatively different circuits?
00:29:29 - How much of that is?
00:29:31 - Not requiring any external input and how much?
00:29:35 - Would that be experience-dependent, given what we know about the importance
00:29:38 - of culture and motheries and all these things for something like language?
00:29:42 - I think it's hard to put a number on it.
00:29:43 - I think that the right way to think about it is that the molecular constraints
00:29:47 - and the activity dependence are both critically important and they play together.
00:29:51 - So you can't really say, I mean, like people give these heritability numbers
00:29:55 - or something. They say IQ is, you know, 70% heritable.
00:29:58 - All that really means is we can correlate this much of the variation with the
00:30:02 - genes. The molecular processes themselves don't really work that way.
00:30:06 - They're not separable in a way that you can kind of partial out variance in a meaningful way.
00:30:12 - I think the molecular processes are really important and must be understood,
00:30:15 - and the activity processes are really important and must be understood.
00:30:18 - I mentioned some alternative splicing mechanisms that might actually integrate
00:30:22 - these things, and I think we should be looking for that too.
00:30:25 - I, in my talks, tend to emphasize the molecular at the expense of the activity
00:30:29 - almost as a political thing.
00:30:31 - So I think that most of the field pays attention to the activity-driven part
00:30:36 - of the equation and doesn't pay as much attention to the molecular side of things.
00:30:40 - So in computational neuroscience, except in a couple of small areas like topographic maps...
00:30:45 - Where people actually know what the molecules are, people kind of ignore the
00:30:49 - molecular contributions, and they focus on the activity.
00:30:53 - And I don't doubt that the activity is really, really important. I mean, that's obvious.
00:30:57 - But I do doubt that we can get a complete model without having some grasp on
00:31:01 - the developmental molecular mechanisms and how they might shape,
00:31:04 - say, two patches of cortical tissue to be subtly but importantly different,
00:31:08 - such that they respond different ultimately to that
00:31:11 - activity i think part of the reason people are enthusiastic
00:31:15 - about activity dependent uh generation of
00:31:19 - structure is that you can perhaps more likely automate
00:31:22 - it so if you get the right powerful learning algorithm and this of course was
00:31:25 - so seductive and exciting about connectionist models and and i think more recently
00:31:30 - deep learning is the idea that if the learning algorithm is right the system
00:31:35 - will just build itself given the activity and and the The alternative,
00:31:39 - which you're suggesting,
00:31:40 - does imply for the people who want to model computationally what's going on,
00:31:44 - that we're going to have to understand and build more of the infrastructure
00:31:48 - before those kinds of activity-dependent learning systems can take off.
00:31:53 - I think you're exactly right at multiple levels. So one is, I think you're right about the appeal.
00:31:57 - I mean, it would be great if we could have one algorithm that ruled them all.
00:32:00 - That would make everybody's life easier. It'd be much easier to build much better AI, for example.
00:32:07 - But at the same time, just because it's easier doesn't mean that it's right.
00:32:11 - I think that if you look at what machine learning, for example,
00:32:14 - has been able to do well and what it hasn't been able to do well,
00:32:16 - there are domains where bottom-up learning just doesn't seem to do the trick.
00:32:21 - So we are still struggling with common sense reasoning. We are still struggling
00:32:24 - with natural language understanding.
00:32:26 - We've got machines that can read license plates very well. It's kind of a variation
00:32:31 - on the Hubel and Wiesel kind of stuff,
00:32:33 - but we don't have machines that can understand discourse. Of course.
00:32:36 - Something else I'm involved in now is trying to have a successor to the Turing test.
00:32:40 - And one of the proposals that I made is that we have a comprehension test where
00:32:45 - you ask a machine to watch a video and then answer questions like,
00:32:48 - why did Walter White take out a hit on Jesse or something like that.
00:32:51 - Stuff that would be easy for any 14-year-old.
00:32:53 - But so far, computers don't know how to do that, to watch some general kind
00:32:58 - of scene and understand what's going on. They could label things.
00:33:00 - They could say that's a person and that's a person, but they couldn't necessarily
00:33:03 - tell you much about the scene.
00:33:05 - And that's because I think people haven't been investing the time to get the
00:33:10 - basis structure from which you can do the learning.
00:33:12 - Another way I think about this, and I see you want to jump in,
00:33:17 - but the way I think about it is that a lot of work in developmental robotics
00:33:20 - basically starts with a blank slate.
00:33:22 - And I don't think that work has gotten us that far.
00:33:25 - The intuition that you'd like a robot that's embodied, I think, is a very good one.
00:33:29 - But I think that people shy away from what is fairly hard work,
00:33:33 - I think, to build the right basis set so that you can then go out and learn.
00:33:37 - But now, we shouldn't sell computational neuroscience. That's short,
00:33:41 - right? Because yes, even with the interest in activity-dependent processes,
00:33:45 - these are often studied on the basis of some defined circuit.
00:33:50 - And implicitly, that means that's the contribution of these Evo-Devo processes
00:33:55 - that give you this template, if you want, on which an activity-dependent process
00:33:59 - is sculpted, if you want, the ultimate functional circuit.
00:34:03 - Well, yes, but…,
00:34:06 - like the fundamental intuition i think or
00:34:09 - result from evo devo is about conservation and
00:34:12 - duplication and divergence that you have families of
00:34:15 - mechanisms that that are variations on themes like if i had to say there's one
00:34:19 - thing that evo devo has really shown us it's like you know hox genes for example
00:34:23 - getting used over and over again in lots of different kinds of contexts and
00:34:26 - i don't see any reflex of that intuition in computational neuroscience really
00:34:31 - anywhere except maybe be the topographic map.
00:34:34 - But now, so, okay, so we started with this notion of canonical microcircuits
00:34:39 - in European not really delivering on understanding higher level cognition,
00:34:44 - in particular issues around variable binding that you need in language, okay?
00:34:49 - But then what's now your counterproposal, right? What should we be looking at?
00:34:55 - I think we should be trying to find a set of circuits rather than one that probably
00:35:01 - have a family resemblance structure to one another.
00:35:03 - So in a computer, that set would start with things like ANDs and ORs and NANDs and XCORs.
00:35:09 - There's a family resemblance between those, but with very different computational repercussions.
00:35:13 - And ultimately, I think we're looking for something similar,
00:35:17 - maybe at that grain level, maybe a bit higher.
00:35:19 - I made some specific proposals, like we want mechanisms for understanding sequencing, for example.
00:35:24 - And that's a simplification, right? The sequencing itself is probably going
00:35:28 - to require that we understand things about working memory and copying things
00:35:32 - from buffers and so forth.
00:35:34 - So the top-down suggestions that I gave in my talk are really collapsing a number of levels.
00:35:39 - And I think really we need to iteratively do this process of finding intermediate computational.
00:35:46 - What can you do with a set of neurons or what does biology do with a set of neurons?
00:35:50 - What do you do with sets of sets of neurons? What do you do with sets of sets of sets and so forth?
00:35:54 - And so I think of a parser, for example, as being made up of very structured
00:35:58 - combinations of all of these kinds of processes.
00:36:01 - Yeah, but Gary, and sometimes you make a dual proposal, right?
00:36:05 - Because the one that you're saying, well, to make progress, we need some functional guidance.
00:36:09 - And I can give you a list of that. That's what you now sort of was elaborating.
00:36:13 - On the other hand, you're telling, you also told us, look, we should rethink
00:36:17 - cortex in terms of a more, let's say, configurable and variable substrate of a set of computations.
00:36:25 - I think these are two complementary proposals that you're making, or am I wrong here?
00:36:29 - Well, I see a connection between them, but I mean, it's an open empirical question.
00:36:33 - But the notion is that we're going to find motifs structurally in the cortex.
00:36:39 - Projects. The motifs may be variations on a theme, but there will be motifs.
00:36:44 - And I'm talking at the neuron level. I know there's interesting work at the
00:36:47 - sort of voxel level, but I think at the neuron level, we're going to find motifs,
00:36:52 - and those motifs are going to map onto functions.
00:36:54 - And we want to say, you know, there are 10 or 20 or 40 different motifs that
00:36:59 - come in these different flavors. These are the kinds of computations that they do.
00:37:02 - And once we have that level, then we can say, well, When you put those together,
00:37:06 - what kind of computations do you get out of there?
00:37:08 - But now, in some sense, you're advocating, I think, a position that actually
00:37:14 - is also coming out of the community that's pushing canonical microcircuits.
00:37:18 - Like for instance, Rodney Douglas, who has been together with Kevin Martin doing
00:37:22 - a lot of the anatomy on this, the notion of cortical canonical microcircuits
00:37:27 - is now suggesting that cortical circuits can be seen as finite state machines.
00:37:32 - So that means you have computational configurability given a standard hardware template.
00:37:38 - And that sounds very compatible to what you're having in mind.
00:37:41 - Or is there a difference?
00:37:42 - I mean, I think it's in the, in some sense, I think they're in the same family
00:37:46 - of hypotheses and in some sense not.
00:37:48 - So one hypothesis is you look around the brain and there are these configurable
00:37:52 - finite state automata in different places or something like that.
00:37:56 - Another is that the grain level of these circuits is smaller and they're not
00:38:00 - all finite state machines. They're doing other kinds of things.
00:38:03 - So for example, finite state machines don't have memory and memory is actually
00:38:06 - one of the components that I think is critical. And so if all you had was a
00:38:10 - bunch of finite state machines, it would be complicated.
00:38:12 - You might be able to pull the system out of it. I mean, you can think about
00:38:15 - Turing machines and so forth as a possible argument.
00:38:18 - Of course, this is the direction they would like to go then.
00:38:22 - My intuition is that's not the right way to go. I can't say for sure that it's wrong.
00:38:26 - And some level what I'm saying is that the empirical research direction needs
00:38:30 - to go towards itemizing these things or enumerating these things.
00:38:34 - So it is possible, but I think it is unlikely, for reasons I've been explaining,
00:38:40 - that you really will have this one circuit. You have many copies.
00:38:43 - This circuit might be adaptable to do different things. I think the part of
00:38:46 - their view that I'm most sympathetic to is this idea that you could essentially
00:38:51 - reconfigure that circuit to do different kinds of computations.
00:38:53 - Um another possibility is there more variations on themes the way i'm describing
00:38:59 - where maybe particular genes tell you to build this kind of gate versus that
00:39:02 - kind of gate or things like that they're certainly in the same school let me
00:39:07 - just say one other thing that's a different um.
00:39:09 - Thinking about that is different from saying well let's just build a
00:39:13 - map of the entire brain right and run the simulation and
00:39:16 - see what happens but if you do talk about these these primitive
00:39:19 - computational functions how big
00:39:22 - is that set in your opinion as a
00:39:25 - prediction right i don't know i i would say the
00:39:27 - lower bound is like the 10 or 20
00:39:30 - that i put up on the the screen and the upper bound is is you know sort of unknowable
00:39:35 - we we have to do the empirical work my my guess from looking at other parts
00:39:42 - of biology is that there are you know hundreds or maybe thousands but not
00:39:48 - hundreds of thousands, that the ones that are there, most of them get used a lot.
00:39:53 - You know, there might be a kind of Zipf's Law thing where, you know,
00:39:56 - a few of them get used only very rarely, and maybe those are actually critical for language.
00:40:00 - But a lot of motifs get used over and over again. I mean, that's sort of how
00:40:03 - biology tends to work, and that's maybe the best guidance we've got.
00:40:08 - Another way to think about it, I guess, would be coming from psychology.
00:40:10 - You could say, well, working memory is something you need in every computation,
00:40:14 - or practically every computation.
00:40:16 - Sequencing is something you need in a lot of computations normalization is something
00:40:20 - you need over and over again tree structures you might only need in planning
00:40:24 - and language and so you know you could try to get a handle on it that way mm-hmm.
00:40:30 - I understand where you're coming from, but I think that where we go from there,
00:40:35 - it becomes problematic in a way.
00:40:37 - Partly what you're saying is we don't know enough about the richness of the
00:40:41 - cortex to really understand how it operates.
00:40:43 - And that is feeding into a kind of frenzy of let's get more data on the brain.
00:40:50 - But I think at the other time you're saying, look, hang on, we don't understand
00:40:54 - enough of the data that we already have to do this effectively.
00:40:57 - You know so so i i think we're in
00:41:00 - a position now in our field
00:41:03 - where uh if you like the people
00:41:06 - that want to get more data on the brain are in a bit of an ascendancy
00:41:09 - and a lot of the money that's coming into the field is moving towards data gathering
00:41:14 - uh and there's a risk that the people that in the past who've been thinking
00:41:19 - about building functional models of this um that that approach is not being
00:41:25 - supported to the extent it was.
00:41:26 - Perhaps because we haven't succeeded, because we've been working at different
00:41:29 - levels of description and fighting battles between symbolism and connectionism,
00:41:33 - for instance, which really weren't helping the overall cause.
00:41:38 - So how do you see where we go from here? Because my concern, possibly yours, is that,
00:41:45 - alongside all this data collection, we need to build theories.
00:41:48 - We need to build theories at multiple levels of description.
00:41:51 - And I think at some point, we need a principle of parsimony,
00:41:54 - which says, look, we can't possibly include all the data.
00:41:57 - We have to leave some things out and see how far we can get with a subset of data.
00:42:03 - I mean, I'm basically very sympathetic to what you just said.
00:42:06 - The parts that I'm not 100% sympathetic is, I don't think we have all the data that we need now.
00:42:12 - I mean, I do think that we need to do more data collection. We will never have
00:42:14 - all the data. We probably never have all the data. I think there's some specific
00:42:17 - places where I would like to see more data.
00:42:20 - In particular, I would like to see more comparisons between different cortical areas.
00:42:25 - So I would like to see not a whole brain map, which I don't think we'd know
00:42:29 - what to do with, but like focus comparisons between some prefrontal areas and
00:42:34 - some motor areas and some occipital areas. What do they have in common? What do they have?
00:42:39 - What's different about them? And I think you need to map that at multiple levels.
00:42:43 - So I think it has to be at the neuron level.
00:42:46 - I think you have to have activity. I think you probably need to know a lot about
00:42:50 - protein expression, you know, all the way down to the synapse level.
00:42:54 - But I think the key there is that you want to look at different bits of cortex,
00:42:57 - a small number, and really try to understand what is uniform about them?
00:43:01 - What is different about them? How do the computations vary?
00:43:04 - And I think that should be the starting point where it's very focused on trying
00:43:07 - to ultimately give accounts of what computation is done there.
00:43:11 - So using like optogenetic techniques to probe these kinds of circuits and say,
00:43:15 - you know, if I alter the input, what happens?
00:43:18 - Does the same thing happen in occipital cortex as it happens in prefrontal cortex?
00:43:22 - There's lots of problems. This is not trivial to do, but in outline,
00:43:26 - that's what I would like to see done on the empirical side. And then I'm completely
00:43:29 - sympathetic on the theoretical side.
00:43:32 - I think that theory just doesn't have enough prestige in neuroscience,
00:43:35 - doesn't have enough money behind it.
00:43:37 - I don't think there are enough institutions in place to support theorists.
00:43:40 - I think theorists sort of are generally routed towards modeling very...
00:43:45 - Kind of narrow, straightjacketed pieces of empirical data. They're not given
00:43:49 - enough room to think broadly and not given enough prestige.
00:43:53 - And I think we need to build institutions to strengthen the theory side.
00:43:58 - I don't think that there's nearly enough for that relative to the tool building itself.
00:44:02 - But maybe the thing that's missing there as well for theory is that we always
00:44:07 - think about computation as opposed to behavior, because what really matters is behavior.
00:44:13 - I agree with that in the long run, but maybe not entirely in the short run.
00:44:16 - So my concern is that I don't think we can go from wiring diagrams to behavior
00:44:22 - without some intermediate theories of computation.
00:44:25 - So I really do think that the behavior is crucial. I worry that too about these
00:44:32 - brain initiatives and behaviors not paid attention to enough.
00:44:37 - But I think that we need to understand the primitives before we're going to
00:44:40 - have hope of understanding the behavior.
00:44:42 - So you couldn't understand how Microsoft Word works unless you had some theory
00:44:47 - of computation underlying it. You want to know about registers and subroutines
00:44:51 - and object-oriented programming languages and things like that.
00:44:56 - You need some intermediate things before you can understand some complex cognitive artifact.
00:45:01 - I take Microsoft Word to be a kind of cognitive artifact in the sense that it
00:45:05 - responds to different commands in different ways and so forth.
00:45:08 - It's sort of at the right grain level.
00:45:12 - And we just don't have that intermediate connectivity. I'm not sure if I agree with that.
00:45:15 - I mean, in terms of behaviors such as seen in classical conditioning or in operant
00:45:21 - conditioning, forging,
00:45:23 - right there, I think, I'm not saying it's a close case, but links to behavior
00:45:28 - are also established on theoretical grounds, and they're pretty coherent stories.
00:45:34 - Well, there's two things to say there. One is, I didn't have in mind retracting
00:45:38 - your gill when you're doing classical conditioning. I think we have most of
00:45:43 - the tools that we already need, but I don't think it's the kind of behavior that I had in mind.
00:45:47 - I was thinking about behavior like understanding a sentence,
00:45:49 - or foraging might be a good example, where you need a rich set of internal representations.
00:45:57 - For those, I think we need this firm computational grounding.
00:46:01 - The other thing I would say is I don't see theory in computation as at all exclusive.
00:46:04 - I see it as the theory that we're trying to develop is a theory that links the
00:46:09 - neurophysiology and so forth, neuroanatomy, with the computation.
00:46:14 - So the theory that I want to see us develop is really one that goes from the
00:46:19 - neural instantiation to the computation, ultimately to the behavior.
00:46:22 - It's just that I think we can't immediately go from the neural instantiation
00:46:27 - to the behavior in any meaningful way, because it's just too complicated without
00:46:31 - this intervening layer of explanation.
00:46:33 - And do you think we should be paying more attention to embodiment and society
00:46:37 - and trying to understand these systems, or do you think the focus on the brain is appropriate?
00:46:43 - I think those things are important. I'm not sure we're to the point yet where
00:46:47 - for the kinds of questions that I'm asking about, they're going to make a difference.
00:46:51 - I mean, ultimately, in the grand scheme of society, we want to understand how
00:46:55 - embodied people participate in society and so forth.
00:46:58 - But understanding social structure
00:47:00 - is not something where I think neuroscience has that much to say yet.
00:47:04 - So there's a field of neuroeconomics that tries to derive economic principles
00:47:08 - from neural wiring and so forth. I don't think we're really in a position to do that well yet.
00:47:13 - And language, the way that parent-child interactions scaffold children's language acquisition.
00:47:20 - I mean, how critical is that to our understanding of how humans gain language?
00:47:26 - For me, it's something that would come later. And I guess that's partly having
00:47:30 - to do with my background in language acquisition.
00:47:33 - I would say that all kids acquire language, even in a very broad range of social
00:47:36 - circumstances, ranging from ones where parents are kind of, the term I've heard
00:47:41 - is helicopter parents, where they're hovering around their kid,
00:47:44 - every utterance, they're, I'm a helicopter,
00:47:47 - I will disclose.
00:47:50 - And then there are parents that don't really interact with their kids,
00:47:53 - and the kids learn from their siblings, or mostly by observation and so forth.
00:47:57 - And the system is relatively robust to a very wide range of inputs.
00:48:01 - There's an interesting question. It's not fully robust.
00:48:04 - So kids that have parents that talk more have bigger vocabularies that might
00:48:08 - be partly genetic, which most people worry, and it's probably partly experiential.
00:48:12 - And I think those are interesting questions.
00:48:13 - But I don't think that we know enough about the basics of how the universal
00:48:17 - part of the system is put together to really be able to make sense yet of those
00:48:22 - kinds of things at a mechanistic level. So I still want to know how we represent
00:48:25 - one sentence in the brain.
00:48:27 - And once you can tell me that, then I'll move on to like, you know,
00:48:30 - why you learned this one a little bit faster than the other.
00:48:32 - But now, so to come back to the issue of behavior versus computation,
00:48:37 - from a methodological perspective, the three sources of information we have
00:48:42 - understanding mind and brain is anatomy, physiology, and behavior.
00:48:46 - And actually, what I believe is the correct methodology is to have a conversion
00:48:53 - validation of these sources of information on our model so we can identify what
00:48:56 - the computation is they perform, okay?
00:48:58 - As opposed to first identifying computation and then going to behavior.
00:49:03 - Well, I'm not sure that's a substantive difference. So,
00:49:08 - I agree we can't access the computation directly, and we're trying to converge
00:49:12 - on what the computation is.
00:49:14 - But I'm not sure what the alternative is that you think that I'm endorsing.
00:49:17 - And what I'm saying is that the computation, the characterizing the computation
00:49:22 - is absolutely central to putting the system together.
00:49:25 - And I can't tell you how many neuroscience conferences I've been to lately where
00:49:28 - the word computation scarcely is even mentioned. And I think in some sense,
00:49:32 - that's what I'm railing against here is the idea that, you know,
00:49:35 - once we have the circuit, the
00:49:36 - computation comes for free and that we don't have hard work to do there.
00:49:41 - You go to neuroscience talks and people explain their channels and never invoke
00:49:45 - the word computation and you don't know what computation they're even thinking about.
00:49:48 - I think that's problematic. But that means you would use the word computation
00:49:51 - on a broad sense, like what are the transformations or operations that these circuits perform?
00:49:57 - Form it is not necessarily like in a
00:50:00 - true machine sense of computation well i'm interested in both i think
00:50:03 - that the right full account of
00:50:06 - things has to involve both the fine
00:50:09 - level i mean like if you if you're talking about a computer again you in order
00:50:13 - to understand how word works you need to understand both the fine grain of like
00:50:18 - transistors and how they make gates and you need to understand something like
00:50:21 - about the api of an operating system if you really want to understand how it works.
00:50:26 - And not everybody does, but to the extent that we're trying to reverse engineer
00:50:30 - the mind, it's sort of comparable to someone who would try to reverse engineer
00:50:33 - Microsoft Word and build their own.
00:50:36 - Well, in order to build your own copy of Microsoft Word, you'd at least need
00:50:39 - to know what an API is and what a programming language is.
00:50:42 - And the people who built the programming languages would have to know what assembly
00:50:45 - code is. Maybe you could survive without it because their different levels become insulated.
00:50:49 - So maybe when we study the brain, not everybody who is connecting to behavior
00:50:53 - needs to understand every intermediate level, but somebody's got to be able
00:50:56 - to make the mappings between each of these levels that we're talking about.
00:51:00 - So when we talk about a computer, somebody can map between the transistors and the microprocessors.
00:51:05 - And even if I can't personally, I know there's an ordered mapping that explains
00:51:09 - it. I know roughly how it works, and we need that level of it.
00:51:12 - But I think with your Microsoft Word analogy, we don't want to start by reverse
00:51:16 - engineering a system of that complexity.
00:51:18 - If you're an alien, and you wanted to know how that program worked,
00:51:22 - you'd probably want to get a hold of a much simpler text editor and try and figure that out first.
00:51:26 - And we can do the same, obviously, in neuroscience, so the comparative approach.
00:51:31 - And I think the other thing I want to push on here a bit.
00:51:34 - Development because you know children don't start
00:51:37 - talking in multi-word sentences uh until
00:51:41 - they're several years old and before that there are various earlier grammars
00:51:45 - which are my son's 20 months he does a fair number of multiple yeah but come
00:51:50 - on he's your your son gary please he's my primary source so yeah so there's
00:51:56 - this phase when they're they're doing a lot of two-word utterances.
00:51:59 - And, you know, and surely there's some kind of substrate for that,
00:52:05 - which will be interesting to understand as a precursor for the substrate for adult language.
00:52:10 - And to get to adult language, perhaps we should understand the substrate for
00:52:14 - that and build that, and then think about the mechanisms that we'll construct from there.
00:52:18 - I totally agree that we want smaller systems. I mean, when I say a parser is too big, I mean it.
00:52:22 - I think that on the behavioral side, we
00:52:25 - want to understand things like how can you repeat a word you know very
00:52:29 - small things that doing a whole language system is
00:52:32 - just outside the scope of what we can do now it's like trying to do microsoft word
00:52:35 - when we don't know what a text editor is we don't know what it means to draw
00:52:38 - you know characters on the display like we're really at a a primitive level
00:52:43 - of understanding these things and we i totally agree we want to find simpler
00:52:47 - pieces so how do you imitate a word how How do you recognize the difference
00:52:52 - between blue car and car blue?
00:52:55 - You know, these kind of very basic things we need to work out before we understand.
00:52:59 - How do you comprehend language in the context of a discourse and you know all
00:53:04 - the things that are going on around you and how do you integrate that?
00:53:06 - We don't have the basic tools out of which such systems are built yet.
00:53:12 - But in the case of language, for instance, and this question of how we come
00:53:16 - to be able to use variables when we think, but perhaps that's something we develop
00:53:21 - as we practice language,
00:53:22 - you become able to use more and more abstract tokens and to realize that tokens
00:53:28 - are interchangeable and so on.
00:53:29 - Some of my own work gives a piece of an argument that the variable binding itself might be innate.
00:53:34 - I did some work showing that seven-month-olds could do a kind of variable binding.
00:53:39 - It was actually on the right side of one of my slides.
00:53:42 - So I showed that kids could learn ABA structures or ABB structures and then
00:53:47 - generalize them to new words.
00:53:48 - So they don't seem to be just using transitional probabilities.
00:53:50 - And then, as in what is typical in developmental psychology,
00:53:54 - someone said, well, I can do that even younger.
00:53:56 - And so now we know that the paradigm that I invented, minus a control that I'd
00:54:00 - like to see run, can be done in newborns.
00:54:03 - So even newborns, and I know that's not a perfect argument for nativism,
00:54:07 - but it's at least evidential that newborns apparently can do a computation that
00:54:12 - I believe requires variable binding.
00:54:15 - So on the particulars of variable binding, I actually think that that's part
00:54:18 - of our innate armamentarium.
00:54:21 - As to language as a whole, I think there's a lot of learning.
00:54:23 - So you might plausibly think that kids are born with the ability to represent
00:54:28 - arbitrary relationships, which you need for words.
00:54:31 - They might be born with the ability to concatenate symbols, even if they don't
00:54:34 - know what those symbols are.
00:54:36 - They have to learn lots of things that are language particular.
00:54:39 - They may have to learn that you map syntax to semantics, or maybe that part
00:54:43 - is known, but a lot of the detail about how you do that might have to be learned.
00:54:47 - So, I mean, the most extreme nativist theories are like some of the ones that
00:54:51 - Chomsky was pushing in the 1980s and that I was trained on in graduate school,
00:54:56 - where there are a lot of very specific principles, like of what is a legal tree
00:54:59 - structure and what is an illegal tree structure.
00:55:01 - So if these two items are not in this geometric relation to one another,
00:55:05 - the sentence is ruled Stuff like that might not really be innate,
00:55:07 - even though Chomsky argued that it is.
00:55:09 - The ability to represent something like a tree structure, I'm guessing is,
00:55:13 - it's probably not exactly a tree structure for reasons that I mentioned before.
00:55:17 - But my guess is actually that either chimps don't have that structure at all,
00:55:22 - or they don't know how to use it for new things.
00:55:23 - Maybe they can use it for motor planning, but they don't have the ability to
00:55:27 - say, hey, this is a useful mental representation that I can do,
00:55:30 - a representational format that I can do other useful work with. Yeah.
00:55:34 - So getting to the finish line, there's two issues I would like to clarify with
00:55:40 - respect to your proposal, right?
00:55:42 - So after criticizing the canonical microcircuit as being too restricted in thinking
00:55:47 - about the kinds of cognitive function we want to get.
00:55:50 - I was rather surprised that you were proposing field programmable gate arrays
00:55:54 - as, let's say, an example of the configurable kind of computation you want.
00:56:00 - Because FPGAs, which are widely used for, let's say, real-time processing because
00:56:05 - of their parallel operation, are actually...
00:56:10 - A very paradigmatic example, if you want, of a canonical microcircuit repeated many times in silicon.
00:56:18 - But a configurable one, crucially. Right. So, I mean, I chose that deliberately.
00:56:23 - So there is this at least superficial similarity, and yet it needs to be resolved
00:56:27 - with the functional diversity.
00:56:29 - And I think that's what the FPGA gives you is superficially and initially,
00:56:33 - in fact, not just superficially, it is literally identical across its extent.
00:56:36 - Then the configuration comes from instructions that say, you know,
00:56:42 - I want this to behave in this way.
00:56:43 - And the real difference ultimately comes down to, I think that that configuration
00:56:47 - can be partly done molecularly just as in any other part of the body.
00:56:50 - And that indeed illustrates your point to say, look, there might be an initial
00:56:54 - infrastructure as in the FPGA, but that then gets configured and that gives
00:56:58 - you variability across the microcircuits. That's right. This is roughly the idea.
00:57:02 - Yes. So the second thing is, to me, your proposal sounds very reminiscent of
00:57:06 - Jerry Edelman's idea of neurodharmonism, where you would say,
00:57:08 - look, developmental factors give rise to what he then called a primary repertoire,
00:57:14 - highly redundant, with lots of possible mappings.
00:57:18 - Upon that, selection takes place due to the engagement with the real world.
00:57:23 - So now you have your secondary repertoire.
00:57:25 - Secondary repertoire can perform complex functions through what he calls reentry,
00:57:30 - including recursion. and then you would rely on what he calls value-based learning
00:57:34 - to accept rules of engagement with the world.
00:57:37 - So would it be fair to say that you're now a neo-Edelmanian?
00:57:43 - I'm more sympathetic to that view than I might have been before.
00:57:47 - I'm not sure that I would put as much weight on selection.
00:57:52 - I do think there's a basic stock of elements. I don't remember his exact way
00:57:57 - of thinking about that. We may share that.
00:58:01 - I think that the basic elements can be fairly sophisticated computation.
00:58:06 - So let me rephrase that. I think some of the basic elements.
00:58:14 - I need another word here. So I've been talking about building blocks all along.
00:58:17 - Some of the basic assemblies of building blocks can do fairly sophisticated
00:58:21 - things, probably without learning.
00:58:23 - So my paradigm example of this would be imprinting, where an organism sees a
00:58:28 - stimulus that falls in a certain class, makes basically a one trial decision
00:58:32 - about that. I don't think that the Edelman notion gives you a good handle on that.
00:58:37 - I don't think it's necessarily incompatible. I think ultimately it's a broad
00:58:40 - umbrella and you could work it out in different kinds of ways.
00:58:43 - But for me, I want to know why there are circuits like that.
00:58:46 - I think there are going to be some circuits at that level in language where
00:58:50 - you're really looking for specific stimuli and doing specific things with those stimuli.
00:58:55 - And, you know, in that work, we know that, you know, Conrad Lorenz is not,
00:58:59 - in fact, as good an example.
00:59:01 - You won't imprint on Lorenz if you have a proper duct to imprint on.
00:59:05 - So, you know, there's some specificity there to how those work.
00:59:08 - And that's got to be part of the picture. And I see how to shoehorn that into
00:59:13 - his theory, but I don't see it as sort of following from it. Right. Okay.
00:59:17 - Unfortunately, we cannot ask Jerry anymore because he died in May this year.
00:59:23 - So you're now in this business of understanding the brain, also certainly from
00:59:28 - the perspective of language.
00:59:30 - Also, I think you made a really good case for linking functional considerations
00:59:36 - with structural considerations, right?
00:59:38 - And not to decouple the two and say, well, let's just worry about structure, then it will all happen.
00:59:43 - So given all that experience and also given your objectives perspectives
00:59:46 - in terms of resetting neuroscience what's the
00:59:50 - what's what's gary's law we should follow in
00:59:53 - studying the brain and mind pay attention
00:59:56 - to the bridges don't i mean you just said it i mean i'm not sure i have one
01:00:00 - law but i think what you said is right that that you can't study these things
01:00:04 - in isolation maybe that's the phrase if it's you know you can't study these
01:00:08 - things in isolation you can't study the structure or the function and expect
01:00:11 - to really understand the cognitive neurosciences. You have to think about the bridges.
01:00:15 - And then, so four years from now, we're going to come visit you in New York
01:00:19 - or wherever you're going to be four years from now.
01:00:21 - And we're going to challenge you on a prediction you're going to make today.
01:00:26 - So what's the one specific prediction you're willing to make that you will find
01:00:31 - confirmed four years from now when we visit you?
01:00:35 - Four years from now? Yeah, four years. It might be three. It depends how fast
01:00:39 - you're going to be. Okay.
01:00:43 - I mean, these things are so subject to, the kinds of things that I'm talking
01:00:47 - about are research programs that aren't done by one person, they're done by societies.
01:00:52 - And so there's a lot that's dependent on how society allocates its resources
01:00:56 - for how far along we get in these problems.
01:00:59 - Problems maybe the first thing that i think will be confirmed is
01:01:03 - that there'll be important differences in recurrent
01:01:06 - motifs at the neuron level so we'll we'll
01:01:09 - find sets of neural motifs we won't initially know what
01:01:12 - they do computationally but we'll say hey that's a number another one of these
01:01:16 - number 17s now that we can drill down to the multicellular i mean to the to
01:01:22 - the circuits containing multiple neurons we keep seeing this kind of uh connectivity
01:01:27 - and this kind of connectivity over and over again.
01:01:30 - We don't know what it means yet, but I think three or four years from now,
01:01:33 - there's a good chance with all the money that's being poured into EM,
01:01:36 - for example, with good analytic techniques, people will be able to pick out
01:01:40 - those motifs and say, hey, these are interesting.
01:01:44 - And with luck, we'll have this not just in, say, visual cortex,
01:01:47 - and be able to say the distribution of these motifs is different in prefrontal
01:01:52 - cortex than visual cortex.
01:01:54 - That's got to be telling us something. We won't know four years from now know
01:01:57 - what it's telling us, but I hope in four years we'll at least be able to say
01:02:01 - that much. We'll be able to say, I see the stock of motifs is different in these
01:02:04 - two areas. And that's something that we can try to leverage now.
01:02:07 - Exactly. Okay. Gary Marcus, thank you very much for this conversation.
01:02:10 - Thank you very much. That was great.
01:02:14 - The CSN Podcast was produced by the Convergent Science Network of Biometrics
01:02:19 - and Biohybrid Systems, a project funded by the European Sevens Research Framework Program.
01:02:27 - For more interviews, recorded lectures, or upcoming conferences in the field
01:02:33 - of biometrics and biohybrid systems, go to csnnetwork.com.
01:02:39 - Music.

CSN Podcasts

Gary Marcus on canonical microcircuit and variable binding

Season 2014

Description

CSN Podcasts

Episodes

Zoltan Molnar on neocortex evolution and homology

Yaki Setty on synthetic biology and agent-based modeling

Stefano Ferraina on transitive inference and prefrontal cortex

Murray Shanahan on metastability and chimera states

Jon Kaas on motor cortex and posterior parietal cortex

Timestamp

Be the first to leave a comment

Leave a commentCancel

Legal

Follow us on

Gary Marcus on canonical microcircuit and variable binding

Description

Episodes

Timestamp

Be the first to leave a comment

Leave a commentCancel

Tags

Legal

Follow us on

Login to enjoy full advantages

Please login or subscribe to continue.

Go Premium!

Enjoy the full advantage of the premium access.

Stop following

Cancel subscription