cover fetz eb low res

Eberhard Fetz on brain-computer interface and neurochip

  • cover play_arrow

    PLAY EPISODE


cover fetz eb low res
Season 2013
Season 2013
Description arrow_drop_down

Description

Can a monkey learn to control a single neuron in its motor cortex independently of the muscles it normally drives? Eberhard Fetz traces five decades of work on volitional neural control from biofeedback to brain-computer interfaces.

Subscribe for more from the Convergent Science Network podcast series.

Eberhard Fetz recounts the intellectual journey from his pioneering 1969 experiments on operant conditioning of single motor cortex neurons to the development of the Neurochip, an autonomous neural interface that creates artificial connections between brain sites, spinal cord, and muscles. His early work demonstrated that monkeys could learn to volitionally increase or decrease the firing rate of individual neurons to earn rewards, with the key insight that the animal was not simply conditioning a neuron in isolation but learning to control the pattern of activation flowing through a fixed circuit in novel ways.

A central theme is the remarkable flexibility of neural control. Fetz showed that neurons with consistent relationships to specific muscles could be operantly dissociated: a cell that always co-activated with the biceps could be driven to fire without any muscle activity, and vice versa. He interprets this not as rewiring of anatomical connections but as the brain exploiting its existing circuitry in variable patterns, analogous to trains taking different routes over fixed railroad tracks. This distinction between structure and activation patterns has profound implications for understanding how the brain achieves flexible behavior without constantly modifying its physical connectivity.

The Neurochip technology, developed with Andy Jackson and Jaideep Mavoori, enabled a breakthrough: autonomous, battery-powered devices mounted on the monkey’s skull could record neural activity and deliver spike-triggered stimulation during days of free behavior. This allowed the creation of artificial recurrent connections between cortical sites, between cortex and spinal cord, and between cortex and muscles. In one paradigm, monkeys with temporarily paralyzed wrist muscles learned to drive a cursor into targets by activating motor cortex cells that directly stimulated the denervated muscles through the Neurochip.

The episode explores the relationship between volitional control and mental imagery, the challenges of reverse recruitment order when electrically stimulating muscles, and the potential for bidirectional brain-computer interfaces to restore function after spinal cord injury. Fetz draws a provocative parallel between the self’s relationship to the brain and the brain’s relationship to external devices, suggesting that the same mechanisms of flexible volitional control apply in both domains.

Tagged as:

About the author call_made

CSN Podcasts

Both the triumphs of humanity and its most evil deeds have resulted from collaboration. In a time where humanity is required to aspire to the former and minimize the latter, the question arises of how collaboration arises and why it fails. Surprisingly, this phenomenon, so central to who we are, is not well understood. Hence, a collaborative effort is required to understand collaboration in its full biological, psychological, sociological, cultural, and economic complexity and to translate this understanding into operational impact. This series of podcasts is one step toward achieving these complementary goals. The Collaboration Podcast presents interviews with people who are central orchestrators of collaboration in various domains including business, government, science, art, health, sustainability, and the military. The discussions were conducted by Prof. Dr. Paul F.M.J. Verschure and members of the Program Advisory Committee of the Ernst Strungmann Forum on Collaboration (https://www.esforum.de/forums/ESF32_Collaboration.html) during 2021 and had the goal to sketch a map of opportunities, challenges, and obstacles in human collaboration. The forum took place in May 2022, and now we would like to share this series of interviews with a broader audience. The full report of the Forum will be published in 2023 by MIT Press. The podcast was produced by the Convergent Science Network (https://www.convergentsciencenetwork.org/). Context: The stability of social systems depends critically on realizing sustainable methods of “collaboration,” yet how and by which means collaboration is achieved is not clearly understood; neither are the conditions or processes that lead to its breakdown or failure. Collaboration can be understood as cooperation between agents toward mutually constructed goals. Part of the reason for our lack of understanding is that the phenomenon of collaboration is, by nature, a highly multidisciplinary problem, and effective research into its complexities has been difficult to achieve across the broad range of scientific and technical disciplines involved. The need for a fundamental understanding of collaboration, however, has become increasingly important. Not only does humankind demand answers as it attempts to address critical challenges at multiple scales (e.g., climate change, migration, enhanced automation, social and economic inequality), but ever-increasing technological and economic means of interconnecting people and societies are disrupting long-established, familiar patterns of how we interact. Radical technological changes that are ongoing have the potential to reshape collaboration in ways that are currently hard to predict or influence (e.g., by altering configurations in interaction, information creation, and modes of communication). On one hand, such changes could disrupt hitherto stable forms of collaboration by affecting critical communication channels and traditional roles, as can be observed in the rapidly changing patterns in governance, commerce, and social interaction. Conversely, technology could lead to the emergence of novel, successful forms of collaboration that deviate from traditional “hierarchical” architectures. Evidence of this can be seen in areas as diverse as highly automated manufacturing plants, the open science movement, collaborative software repositories, user-centered services, and the sharing of economy-based modes of organization. Without a fundamental understanding of the mechanisms, processes, and boundary conditions of collaboration, it is not possible to evaluate or predict which of these possible scenarios are sustainable or even plausible. The Forum “How Collaboration Arises and Why it Fails” (May 8–13, 2022, Location: Frankfurt am Main, Germany) Chairs: Andreas Roepstorff and Paul Verschure Program Advisory Committee: Jenna Bednar, Julia R. Lupp, Bhavani R. Rao , Andreas Roepstorff, Ferdinand von Siemens, and Paul Verschure

More posts

Timestamp

  • fast_forward00:00:03 - This is the Convergent Science Network podcast. Leading researchers in the domain
  • fast_forward00:00:10 - of neuroscience, brain theory and technology are interviewed by Paul Verschure and Tony Prescott.
  • fast_forward00:00:26 - Is paul for sure with a conversion science network and
  • fast_forward00:00:29 - here i'm speaking with abehart fetz and
  • fast_forward00:00:32 - ab spoke this morning in our
  • fast_forward00:00:35 - in our summer school about what he called the self in the brain so but what
  • fast_forward00:00:41 - what did you really have in mind when you described the self in the brain So
  • fast_forward00:00:46 - there's a common experience of everybody knows that there's a self in their
  • fast_forward00:00:52 - brain that is interacting with the world.
  • fast_forward00:00:54 - And I brought it up as a model for understanding how the brain interacts with
  • fast_forward00:01:00 - external devices, how the brain would interact with brain-computer interfaces,
  • fast_forward00:01:05 - and also how the self would deal with recurrent brain-computer interfaces,
  • fast_forward00:01:13 - that is, bidirectional brain-computer interfaces.
  • fast_forward00:01:15 - Faces right because this is if you
  • fast_forward00:01:18 - want this is also a metaphor right of how right how
  • fast_forward00:01:21 - volitional control could be exerted over even
  • fast_forward00:01:24 - single neurons in some sense yes because usually we we think about okay we are
  • fast_forward00:01:29 - controlling the body we control the world given our goals and our wishes our
  • fast_forward00:01:33 - intentions but what you have have specialized in uh with also really very exciting
  • fast_forward00:01:39 - results is in some sense how this mapping, if you want,
  • fast_forward00:01:43 - between a brain and the volitional control of the brain itself and external
  • fast_forward00:01:48 - devices can be almost arbitrarily mapped, right?
  • fast_forward00:01:50 - That's right. Your first experiments were in the 1960s in this domain.
  • fast_forward00:01:55 - That's correct. So what was the key observation that really pushed this forward?
  • fast_forward00:02:00 - So these experiments were done as a postdoctoral fellow.
  • fast_forward00:02:05 - I came from MIT just having a degree in physics, which was totally useless for what I was going to do.
  • fast_forward00:02:11 - But I needed to know about neurophysiology and behavior, and I thought it would
  • fast_forward00:02:16 - be fun to put these two together and do experiments on training monkeys to volitionally
  • fast_forward00:02:23 - control the neural activity in their brain.
  • fast_forward00:02:26 - And it was a great deal of fun, and I wish I'd stuck with it, but...
  • fast_forward00:02:30 - Because it turns out, in retrospect, that what I had looked at as biofeedback
  • fast_forward00:02:36 - for these operant conditioning studies, which is the deflection of a meter arm
  • fast_forward00:02:39 - controlled by brain activity,
  • fast_forward00:02:42 - is sort of the paradigm that's now being used in brain-machine interfaces.
  • fast_forward00:02:48 - And I wish I had had the connection between meter arm and prosthetic arm.
  • fast_forward00:02:52 - My career would have been quite different.
  • fast_forward00:02:54 - Right. It is a demonstration of your narrow vision that this meter arm is just
  • fast_forward00:03:00 - feedback, not a prosthetic device.
  • fast_forward00:03:02 - Right, because in those experiments, so I guess you were already exposed to
  • fast_forward00:03:07 - notions like operant conditioning and so on at the time.
  • fast_forward00:03:10 - Not so much. I mean, this work exposed me to that.
  • fast_forward00:03:13 - I was working in a lab of Mitch Glickstein, and he had some very bright postdoctoral
  • fast_forward00:03:21 - fellows who knew all about operant conditioning.
  • fast_forward00:03:23 - And I also worked with a practitioner of the art, Dom Finocchio,
  • fast_forward00:03:30 - who was really an expert on this, and taught me all the lingo and techniques of it.
  • fast_forward00:03:36 - So basically, it was something learned as we went along.
  • fast_forward00:03:40 - Right. But learned in the context of operantly conditioning neural activity instead of behavior.
  • fast_forward00:03:45 - Okay, but I think this very first experiment that you published in 69,
  • fast_forward00:03:51 - in science, in some sense set the tone for what followed.
  • fast_forward00:03:57 - Because in some, what you tried to do there is say, okay, the monkey was watching
  • fast_forward00:04:01 - a dial, but now it had to train.
  • fast_forward00:04:04 - The monkey was rewarded for moving the dial in a certain direction,
  • fast_forward00:04:08 - but the response that was being conditioned here was really neural activity.
  • fast_forward00:04:13 - Exactly. So he was actually, the way I saw it, he was being rewarded for controlling
  • fast_forward00:04:17 - the neural activity, and the dial was a help, a conditioned reinforcer, if anything,
  • fast_forward00:04:25 - and helped the monkey to zero in on what it was that was going to get rewarded.
  • fast_forward00:04:30 - And the monkey quickly learned, I'm anthropomorphizing, but his behavior was
  • fast_forward00:04:38 - as if he had learned that the rightward deflection of this meter arm was going
  • fast_forward00:04:42 - to be associated with applesauce reward,
  • fast_forward00:04:44 - and he would very quickly learn to generate whatever.
  • fast_forward00:04:49 - Neural activity or behavior would drive that meter arm to the right and get rewarded.
  • fast_forward00:04:54 - But now these neurons, prior to the experiment, were these neurons in any way
  • fast_forward00:04:59 - involved with similar kinds of movements in the world?
  • fast_forward00:05:04 - Definitely. These were motor cortex neurons. So very large numbers of them were,
  • fast_forward00:05:09 - a large fraction of the ones that we worked with were actually neurons that
  • fast_forward00:05:14 - were involved in generating movements.
  • fast_forward00:05:17 - And so, in fact, one of the original rationales for these studies was to determine
  • fast_forward00:05:22 - what the movements were that were associated with particular cells. Right.
  • fast_forward00:05:29 - So this gave you, I guess, the initial idea that you actually could change these
  • fast_forward00:05:34 - response properties of neurons and map them differently to the world.
  • fast_forward00:05:39 - So what was really the next step there?
  • fast_forward00:05:42 - And when did this notion of volitional really come in in the development of this whole approach?
  • fast_forward00:05:48 - Well, this concept of volitional control of neurons is simply just a way of
  • fast_forward00:05:54 - saying that the monkey was generating these responses as if he were making a movement.
  • fast_forward00:06:02 - And so what happened ironically is that as we were doing this work,
  • fast_forward00:06:09 - the question came up as to whether or not the cells were really causally involved
  • fast_forward00:06:15 - in generating the movements and the muscle activity.
  • fast_forward00:06:19 - And so I got distracted by a causal question, which unfortunately got me off
  • fast_forward00:06:29 - the track of operant conditioning.
  • fast_forward00:06:30 - And that technique was to demonstrate causality by doing things like spike-triggered
  • fast_forward00:06:37 - averaging of muscle activity, which is a way to determine that the cell actually
  • fast_forward00:06:41 - had an output effect on muscles.
  • fast_forward00:06:43 - And that was such a devilishly difficult procedure that that gobbled up my time
  • fast_forward00:06:50 - for the next decade or two.
  • fast_forward00:06:52 - And looking back, what I should have done is stuck with the more fun things,
  • fast_forward00:06:57 - which was to see how far we could exploit this operant conditioning paradigm.
  • fast_forward00:07:04 - Right. But then for me to understand the logic of that, why did you move to
  • fast_forward00:07:08 - a paradigm where you really started to map neural response to muscle activity?
  • fast_forward00:07:13 - What was the concept there? Well, the concept was to really,
  • fast_forward00:07:18 - in this time, the activity of neurons was being recorded in monkeys generating
  • fast_forward00:07:27 - behavior and correlating the changes in the neural activity to the changes in behavior.
  • fast_forward00:07:33 - But everybody was frustrated about the lack of real evidence that the cells
  • fast_forward00:07:40 - were causally related to the behavior. So this came out of...
  • fast_forward00:07:45 - Questions like, how would we show that the cell actually has a causal effect on the muscle activity?
  • fast_forward00:07:53 - And this, I think, was an issue that appealed to my physics background to get
  • fast_forward00:08:01 - to the bottom of mechanisms and pursue answers to questions like that.
  • fast_forward00:08:07 - And so that's what I did for the next couple of decades.
  • fast_forward00:08:10 - We did this correlation method of confirming that cells had a real output effect on the muscles.
  • fast_forward00:08:21 - And I think that actually did solve a lot of issues as to what the properties
  • fast_forward00:08:28 - of these output cells was and how their properties differed from other motor
  • fast_forward00:08:33 - cortex cells that didn't have this output on muscles.
  • fast_forward00:08:35 - So I think it was – I'm being a little facetious saying it was a waste of time,
  • fast_forward00:08:40 - but I think it was definitely a useful enterprise agenda to do this.
  • fast_forward00:08:46 - But I do believe that even though it might not be at the core of your current
  • fast_forward00:08:50 - interest, I mean, this mapping to muscle activity,
  • fast_forward00:08:52 - there are some interesting aspects to the learning dynamics you already observed
  • fast_forward00:08:57 - then that I think are also relevant for our current discussion about brain-computer interfaces.
  • fast_forward00:09:02 - For instance, that the learning, also what you talked about this morning,
  • fast_forward00:09:06 - already these early studies, I felt there was something very strange going on.
  • fast_forward00:09:10 - So you train this neuron to, let's say, control a certain muscle based on reward
  • fast_forward00:09:15 - or an aversive stimulus, depending on the conditions.
  • fast_forward00:09:19 - But it's not necessarily these mappings are static.
  • fast_forward00:09:22 - It seems that they're very transient in nature, right? So you induce a sort
  • fast_forward00:09:26 - of responsibility because there's reward, but as soon as, let's say,
  • fast_forward00:09:30 - the animal is moved into its own cage and it's outside of the experimental context,
  • fast_forward00:09:35 - the mapping very quickly disappears.
  • fast_forward00:09:37 - Well, I see it more like there is this mapping or this relationship,
  • fast_forward00:09:42 - let's say, between a motor cortex cell and a set of muscles that it may be linked
  • fast_forward00:09:48 - to functionally in the sense of co-varying with those muscles.
  • fast_forward00:09:52 - When we reward the cell, we're basically looking at not just the activity of
  • fast_forward00:10:01 - the cell, but also at the correlated muscle activity.
  • fast_forward00:10:04 - And so that's a natural relationship that exists in the cell.
  • fast_forward00:10:08 - Cage as well as in the experimental booth.
  • fast_forward00:10:12 - And we're not actually rewarding a relationship there.
  • fast_forward00:10:17 - We're basically rewarding one component of a circuit, which includes the cell and the muscles.
  • fast_forward00:10:23 - So part of the issue is how necessary is that relationship?
  • fast_forward00:10:29 - So one of the fun things we did was to operantly condition the dissociation
  • fast_forward00:10:34 - of very consistent relationships.
  • fast_forward00:10:38 - So there was a cell, for example, that always fired with the biceps.
  • fast_forward00:10:42 - But then when the monkey was rewarded for firing the cell without the muscle
  • fast_forward00:10:49 - activity, he very quickly learned to dissociate them.
  • fast_forward00:10:52 - So that was of interest and sort of flew in the face of the dogma that the motor
  • fast_forward00:10:58 - cortex cells have a relatively stable and reliable relationship to the muscles.
  • fast_forward00:11:04 - Right. So do you believe that there's a full dissociation there,
  • fast_forward00:11:08 - or are there some constraints acting upon that system?
  • fast_forward00:11:11 - Yeah, I think there's a full dissociation in the sense that one turns off while the other stays active.
  • fast_forward00:11:19 - But there are various ways that that can be achieved by just changing the balance
  • fast_forward00:11:26 - of inputs to the two to the cell and to the muscle so that they can be independently controlled,
  • fast_forward00:11:33 - but do you believe i could for instance retrain my my the homunculus of my motor
  • fast_forward00:11:38 - cortex to be inverted that's a big uh challenge if you're talking about the
  • fast_forward00:11:46 - whole homunculus we're just We're just talking about a single cell.
  • fast_forward00:11:48 - I know, I know, but I just tried to understand the boundaries, right?
  • fast_forward00:11:51 - Yeah, right. So, for example, relating it to the homunculus,
  • fast_forward00:11:54 - the question might be, can the cell in the hand area be related to movement of the foot?
  • fast_forward00:12:07 - And it's probably pretty straightforward to reward the animal and succeed in
  • fast_forward00:12:14 - getting him to co-activate the foot in the cell that was related to the hand.
  • fast_forward00:12:19 - It doesn't change any functional relationship in the sense of circuitry.
  • fast_forward00:12:27 - It just changes the pattern of activation, which is quite different.
  • fast_forward00:12:32 - I mean, that pattern of activation is sort of a transient thing that's modifiable
  • fast_forward00:12:36 - by these operant techniques.
  • fast_forward00:12:40 - But the underlying circuitry is a little bit more fixed and anatomical, as it were.
  • fast_forward00:12:49 - But the way these units, the cells and the muscles are activated can be quite flexible.
  • fast_forward00:12:58 - And that's actually essentially what this operant conditioning paradigm demonstrates.
  • fast_forward00:13:03 - Right. So then, in some of your saying, you could multiplex it.
  • fast_forward00:13:07 - You could say, look, there's sort of one layer, one frame of reference that
  • fast_forward00:13:10 - is maybe a bit more hardwired, but then on top of that, you could wire in almost
  • fast_forward00:13:14 - an arbitrary response set, and they can coexist. exist?
  • fast_forward00:13:18 - Well, the term wired for that second thing is a little too strong.
  • fast_forward00:13:22 - I think I see wired as more having to do with anatomical connections and activation
  • fast_forward00:13:28 - as being the pattern in which the activity is distributed through those hardwired circuits.
  • fast_forward00:13:37 - And so the activity patterns can vary quite a bit depending on how the brain
  • fast_forward00:13:43 - propagates or generates this activity.
  • fast_forward00:13:48 - And it can activate the elements in a fixed circuit in a variable way.
  • fast_forward00:13:53 - That's another way to put it. But still, also to do that. You're superimposing
  • fast_forward00:13:57 - that. I mean, that's superimposed on the structure.
  • fast_forward00:14:00 - That's correct. But still, to superimpose it, you do have to change some of
  • fast_forward00:14:03 - the wires to implement that activity pattern.
  • fast_forward00:14:07 - Well, I beg to differ. I don't know that you actually are changing wires so
  • fast_forward00:14:10 - much as working with the wired system in new ways.
  • fast_forward00:14:14 - You basically are propagating activity and ignoring some of the wires that exist
  • fast_forward00:14:20 - in this process and activating or propagating activities through some other wires that exist.
  • fast_forward00:14:29 - But the activity is sort of the waves on top of this relatively fixed structure.
  • fast_forward00:14:38 - But you would agree that you're changing synapses to accomplish that?
  • fast_forward00:14:42 - No. I'm sorry.
  • fast_forward00:14:45 - But look, how do you get selectivity then in that system?
  • fast_forward00:14:50 - That's a good question. You get selectivity by changing the pattern of excitation
  • fast_forward00:14:58 - and inhibition that exists in the circuitry and activating cells in different ways.
  • fast_forward00:15:04 - I mean, there's a very deep question you're asking is how the brain can use
  • fast_forward00:15:08 - a fixed circuit, which I believe it has, in variable ways.
  • fast_forward00:15:14 - Let's take one example. So one of these early experiments you described,
  • fast_forward00:15:19 - you're mapping two neurons to a muscle.
  • fast_forward00:15:23 - And then you showed that you could condition that neuron to go both up or down
  • fast_forward00:15:28 - its response, dependent on how you were conditioning it. That's right.
  • fast_forward00:15:34 - So now, given the reward contingency, you are modulating the response of this neuron.
  • fast_forward00:15:41 - It can go in any direction you want, up or down.
  • fast_forward00:15:45 - What's the substrate of that if it's not wires and it's not synapses?
  • fast_forward00:15:48 - It is obviously the wires that connect to those cells.
  • fast_forward00:15:54 - So we have these two cells that are neighboring cells that have very similar
  • fast_forward00:15:59 - normal relationships to a joint.
  • fast_forward00:16:03 - But when the monkey is rewarded for activating them independently,
  • fast_forward00:16:09 - he figures a way to independently control the synaptic input to those cells
  • fast_forward00:16:16 - to make one's activity go up and the other's activity go down.
  • fast_forward00:16:21 - So it's a flexible way of activating the circuitry, but it's not actually modifying
  • fast_forward00:16:28 - the physiological circuitry. the connections are there.
  • fast_forward00:16:33 - So how I read what you're saying is when you say, look, there's like a fixed wired system.
  • fast_forward00:16:39 - That's not really changing dramatically due to the conditioning.
  • fast_forward00:16:43 - Correct. But there's sort of a modulation of that circuit.
  • fast_forward00:16:46 - Exactly. Possibly expressed in, let's say, synaptic connectivity or whatever.
  • fast_forward00:16:51 - Okay. Yeah? That might lead to the specificity of these changes.
  • fast_forward00:16:55 - It's something along these lines.
  • fast_forward00:16:57 - Yes. Okay. So there... Well, so I'm trying to get you to say what the real substrate
  • fast_forward00:17:04 - is of these changes, right?
  • fast_forward00:17:09 - Yeah, well, I wish I could give you an answer as to exactly how that happens,
  • fast_forward00:17:13 - or even give you an analogy.
  • fast_forward00:17:15 - I guess one thing that comes to mind is you've got railroad tracks that go all
  • fast_forward00:17:22 - over the place. That's the structure.
  • fast_forward00:17:23 - And you can have trains riding in different ways over those tracks.
  • fast_forward00:17:29 - And one is structure and the other is activity.
  • fast_forward00:17:32 - Right. So here, in terms of activation of neurons, it's based on a circuit.
  • fast_forward00:17:40 - But the way that circuit is activated generates different patterns of activity.
  • fast_forward00:17:46 - And if your question is a demand for an explanation of how that happens.
  • fast_forward00:17:52 - I have to admit that I'm not totally sure other than wave my hands and say,
  • fast_forward00:17:58 - this is all top-down, as it were.
  • fast_forward00:18:02 - In other words, generated from within the brain in ways that are analogous to
  • fast_forward00:18:07 - the way you can move different muscles volitionally, independently.
  • fast_forward00:18:10 - It's pretty much the same thing because cell activity and muscles are similar.
  • fast_forward00:18:17 - So, okay, now that we have sort of an understanding of the substrate that's
  • fast_forward00:18:21 - implementing these changes, even though it's not completely clear. Right. Okay.
  • fast_forward00:18:25 - You did equate that with a substrate that would support mental imagery.
  • fast_forward00:18:30 - Yes. So what's the link there exactly? So the link is that the same cells that
  • fast_forward00:18:36 - generate a movement are,
  • fast_forward00:18:40 - in many cases, the same cells that are activated when you just imagine that movement.
  • fast_forward00:18:46 - And this principle of relationship between natural activity in relation to visual stimulus, for example,
  • fast_forward00:18:57 - and imagining that visual stimulus pertains to other areas.
  • fast_forward00:19:02 - So in general, imagery uses a lot of the same neural substrate.
  • fast_forward00:19:08 - But it doesn't actually activate that substrate enough to produce the movement.
  • fast_forward00:19:15 - You don't act out everything you think about.
  • fast_forward00:19:18 - So there's a switch that stops this imagery from being expressed.
  • fast_forward00:19:25 - Right. So here we see this flexibility of the brain to sort of remodel itself
  • fast_forward00:19:35 - and to change its mappings almost arbitrarily, but we looked only at output structures.
  • fast_forward00:19:41 - So do you see the same kind of superposition of states also acting out between
  • fast_forward00:19:48 - multiple modalities or motor systems and sensory systems?
  • fast_forward00:19:53 - So are the mappings also arbitrary in that respect?
  • fast_forward00:19:57 - If I understand you right, then the question is on the output side.
  • fast_forward00:20:01 - Actually, the motor cortex is sort of on the output side with regard to controlling
  • fast_forward00:20:07 - muscles because it's, at minimal,
  • fast_forward00:20:12 - it's one synapse away from the motor neurons, but even several synapses away.
  • fast_forward00:20:17 - It's functionally tied very closely to activation of motor neurons that contract
  • fast_forward00:20:23 - muscles. And there the flexibility is quite clear when you probe,
  • fast_forward00:20:30 - when you do experiments that probe that flexibility.
  • fast_forward00:20:32 - For example, this operant conditioning paradigm is a way of probing how flexible
  • fast_forward00:20:37 - the relationships actually are.
  • fast_forward00:20:40 - Does that have anything to do with your question? Yeah, sure.
  • fast_forward00:20:42 - So, but could I overlay, let's say, sensory responses over this motor core?
  • fast_forward00:20:49 - Could I properly condition these motor neurons to respond to sensory stimuli?
  • fast_forward00:20:56 - Well, first of all, the idea that you're conditioning the neurons is a little
  • fast_forward00:21:01 - bit misleading because you're conditioning the animal to activate the motor neuron in a certain way.
  • fast_forward00:21:07 - So you have to see it as being part of a distributed network of activity that
  • fast_forward00:21:16 - is generated by the subject,
  • fast_forward00:21:20 - the monkey or the human or whoever's in this experiment,
  • fast_forward00:21:23 - and not attribute this activity.
  • fast_forward00:21:28 - Um behavior specifically to the
  • fast_forward00:21:30 - cell in isolation okay so that's an important
  • fast_forward00:21:33 - distinction to understand how it's all being generated and
  • fast_forward00:21:38 - what it all means now there was something about the superposition of sensory
  • fast_forward00:21:42 - and motor that you were getting at in terms of arbitrary mapping of sensory
  • fast_forward00:21:47 - input to motor output exactly um because earlier yeah well go ahead because
  • fast_forward00:21:53 - Because earlier we just probed, let's say,
  • fast_forward00:21:54 - the boundaries of remapping within a motor system.
  • fast_forward00:21:58 - Yes. Right? And there was like, well, there might be some constraints on the
  • fast_forward00:22:02 - system, but within those, you can sort of freely map things around.
  • fast_forward00:22:05 - So then the obvious next question is to say like, okay, but what are then the
  • fast_forward00:22:10 - boundaries with respect to some sort of sensory motor mapping?
  • fast_forward00:22:13 - Okay. Okay, so that actually is implicitly addressed in these studies because
  • fast_forward00:22:21 - the motor cortex cells that were involved in these conditioning experiments
  • fast_forward00:22:28 - also respond to peripheral input.
  • fast_forward00:22:31 - So they have a sensory response.
  • fast_forward00:22:34 - For example, these two cells we were talking about that got dissociated had
  • fast_forward00:22:38 - a nice drive from knee extension, if I remember correctly. They were both driven
  • fast_forward00:22:43 - by knee extension. So that's a sensory input.
  • fast_forward00:22:46 - And then the operant conditioning got the monkey to individually activate the
  • fast_forward00:22:52 - cells without the knee moving at all.
  • fast_forward00:22:54 - So in that sense, that answers your question that, yes, the mapping between
  • fast_forward00:22:59 - the peripheral input to these cells and the motor output was dissociated by
  • fast_forward00:23:05 - this paradigm of conditioning.
  • fast_forward00:23:07 - And it was dissociated by virtue of the fact that we were requiring the monkey
  • fast_forward00:23:13 - to demonstrate that he has...
  • fast_forward00:23:16 - A central volitional input that activates a cell that's independent from the peripheral input.
  • fast_forward00:23:22 - So in that sense, I guess you'd have to say that any cell that has multiple
  • fast_forward00:23:28 - inputs that can be independently controlled are demonstrating this capacity
  • fast_forward00:23:36 - or can demonstrate the capacity for dissociation of the maps.
  • fast_forward00:23:41 - SL. Okay, but Tony? yeah
  • fast_forward00:23:44 - i i uh talking about extending it in your
  • fast_forward00:23:47 - talk you mentioned that this is a volitional control
  • fast_forward00:23:50 - yes um and but what that
  • fast_forward00:23:53 - made me think was well the animal or
  • fast_forward00:23:56 - if it was in a person has to actively think and concentrate on making this movement
  • fast_forward00:24:01 - and then they can make the movement and i'm wondering to what extent you think
  • fast_forward00:24:06 - uh that will be available then for automation so that But rather than having
  • fast_forward00:24:11 - to concentrate on moving your arm in the trajectory,
  • fast_forward00:24:14 - once you've learned that, will it be accessible as an automatic movement in
  • fast_forward00:24:19 - the same way that other kind of movements are ones that are made naturally?
  • fast_forward00:24:23 - Oh, yes, I think so. I think the natural movements are a good model of what
  • fast_forward00:24:28 - happens in this context as well.
  • fast_forward00:24:30 - Right. And I'm not even certain that volitional control requires an amount of conscious guidance.
  • fast_forward00:24:41 - I mean, I think it could be...
  • fast_forward00:24:47 - Analogous to reaching for something without thinking too much about it.
  • fast_forward00:24:50 - So, and if you take, for example, sort of a rhythmic pattern generation,
  • fast_forward00:24:55 - so for instance, walking.
  • fast_forward00:24:57 - So imagine you wanted to use your system to help somebody, a spinal patient,
  • fast_forward00:25:02 - who were unable to use their legs.
  • fast_forward00:25:04 - I mean, could you imagine that they would be able to, through their motor cortex,
  • fast_forward00:25:09 - drive the spinal pattern generators for walking and modulate them in a way that
  • fast_forward00:25:15 - people could walk relatively naturally, or would you imagine it would require
  • fast_forward00:25:19 - a lot of concentration through your motor cortex to control your legs in stepping patterns?
  • fast_forward00:25:24 - I mean, how do you see that developing? If you had a spinal cord injury,
  • fast_forward00:25:26 - you mean? Or if you… If you had a spinal cord injury, and imagining that we
  • fast_forward00:25:30 - could somehow wire the motor cortex directly into circuits that control spinal
  • fast_forward00:25:36 - pattern generators for leg locomotion.
  • fast_forward00:25:38 - Tony, I think we might get there.
  • fast_forward00:25:41 - We will get there because what I would like Ab to do first is to explain to
  • fast_forward00:25:46 - us more in detail how he has been wiring into muscles and spinal cords.
  • fast_forward00:25:51 - Then we can address that question a bit more.
  • fast_forward00:25:54 - Well, we can give him some time to come up with an answer. Yeah,
  • fast_forward00:25:57 - I'm going to need time to think of an answer to that one.
  • fast_forward00:26:00 - I want to make this transition out to the biofeedback, probably because this
  • fast_forward00:26:05 - has dominated certainly the last years of your work.
  • fast_forward00:26:10 - So why did you end up or how did you stumble into this whole biofeedback notion?
  • fast_forward00:26:15 - What was the transition there? Why did you go for that topic?
  • fast_forward00:26:19 - Well, like I was saying, I was a postdoc looking for a way to learn about recording
  • fast_forward00:26:24 - neural activity in awake monkeys and operant conditioning.
  • fast_forward00:26:28 - Motioning and ed everett said already um demonstrated
  • fast_forward00:26:34 - the paradigm of training a monkey to make a movement and recording
  • fast_forward00:26:37 - motor cortex cells in relation to movement so i thought i would turn that on
  • fast_forward00:26:41 - its head and see whether you can train the monkey to activate motor cortex cells
  • fast_forward00:26:46 - and see what sort of movements he made it was just a fun thing to do as a postdoc
  • fast_forward00:26:50 - right but i think the first studies came out in the early 60s right in this domain,
  • fast_forward00:26:56 - Yeah, you're talking about Everts. For instance, yeah. Right.
  • fast_forward00:27:00 - But now I think you sort of revolutionized a lot of that work also by using
  • fast_forward00:27:08 - this technology that you call Neurochip.
  • fast_forward00:27:11 - Right. This is a very recent development. Right, exactly. It's Neurochip. Right.
  • fast_forward00:27:16 - And the question is? Well, what's a Neurochip? Why was that such an important step in all this work?
  • fast_forward00:27:24 - Good question. I don't know. I have to think about whether I can give you a
  • fast_forward00:27:27 - rational sequence of where that came from. Uh.
  • fast_forward00:27:37 - I think it came out of the blue because the Neurochip was a device that had
  • fast_forward00:27:43 - been developed in Tom Daniels' lab.
  • fast_forward00:27:47 - And Jaideep Mavuri, a graduate student in electrical engineering,
  • fast_forward00:27:51 - was doing the programming and application of this to investigating the control
  • fast_forward00:27:59 - system of a moth, Manduka.
  • fast_forward00:28:01 - And I ran into Tom Daniels and he told me about this.
  • fast_forward00:28:04 - He's very excited. And he got me excited, and I said, well, good God,
  • fast_forward00:28:08 - if the moths can carry this thing, a monkey could do that even easier.
  • fast_forward00:28:13 - And we can add a lot of other stuff to it.
  • fast_forward00:28:16 - And so I started to think about this in the monkey.
  • fast_forward00:28:22 - And actually, this brilliant postdoctoral fellow, Andy Jackson and Jai Deep, got together.
  • fast_forward00:28:30 - This is a good example of the productive consequences of putting together two
  • fast_forward00:28:35 - people that have complementary talents, and they worked away at making this actually work.
  • fast_forward00:28:43 - So it's a serendipitous encounter with Tom Daniel, I guess.
  • fast_forward00:28:50 - But Neurochip essentially allows you to have an autonomous integrated package
  • fast_forward00:28:56 - sitting on the skull of the monkey to sort of train, condition,
  • fast_forward00:29:01 - remap, or exchange signals with that system.
  • fast_forward00:29:06 - So was that the key step there, that it would be completely wireless,
  • fast_forward00:29:11 - autonomous, locally programmed?
  • fast_forward00:29:13 - Yeah, that was all part of what the Neurochip could do. It could operate by
  • fast_forward00:29:18 - itself with battery power.
  • fast_forward00:29:21 - And then the other key development is development of wire electrodes.
  • fast_forward00:29:26 - Again, Andy Jackson gets credit for these wires that were embedded in the motor
  • fast_forward00:29:33 - cortex and could record neural activity even during free behavior,
  • fast_forward00:29:40 - the monkey jumping around and so forth.
  • fast_forward00:29:42 - This was not something that could have been anticipated, the stability of these recordings.
  • fast_forward00:29:50 - Because normally when you record neurons in the brain, it's a very fragile business
  • fast_forward00:29:56 - subject to artifacts, movement artifacts and stuff like that.
  • fast_forward00:30:00 - So these embedded wires actually are pretty solidly embedded and allow the recordings
  • fast_forward00:30:09 - to continue during days of free behavior.
  • fast_forward00:30:11 - So that was another pretty crucial
  • fast_forward00:30:14 - element to make this neurochip paradigm work over days of behavior.
  • fast_forward00:30:18 - And then Andy thought that it would be great, Andy Jackson thought it would
  • fast_forward00:30:24 - be great to test this Hebbian plasticity by doing spike-triggered stimulation.
  • fast_forward00:30:31 - And it worked. Right. So they're extracellular recordings, presumably.
  • fast_forward00:30:38 - Yes, they are. That they're making. And so presumably you're detecting several
  • fast_forward00:30:43 - neurons there, and you're sorting them using spike sorting?
  • fast_forward00:30:47 - No, this is actually typically the wires next to one neuron, so you don't have to...
  • fast_forward00:30:54 - Do the separation. But the neurochip is programmable so that it can detect the
  • fast_forward00:30:59 - waveform of a single cell even in the presence of other cells. That's right.
  • fast_forward00:31:03 - And how many wires can you put in at a time?
  • fast_forward00:31:07 - Well, I think the record now is Tim Lucas put in about 30 of these wires.
  • fast_forward00:31:12 - It's a real tour de force, but it actually worked. And it has one neuron per wire?
  • fast_forward00:31:18 - When you're lucky, yeah. That's right. Or if you're really lucky,
  • fast_forward00:31:22 - then you can get more than one.
  • fast_forward00:31:24 - But the neurochip that we've been working with up till now typically has up
  • fast_forward00:31:29 - to three channels of input.
  • fast_forward00:31:32 - But the next one that's coming down the pipeline is going to be really cool. Lots of channels.
  • fast_forward00:31:38 - Right. And then outputs. Our outputs, where do the outputs end up?
  • fast_forward00:31:42 - So the outputs are typically electrical stimuli like pulses that are triggered
  • fast_forward00:31:47 - from the action potentials of the cell.
  • fast_forward00:31:49 - And those are delivered in the motor cortex or or a spinal cord,
  • fast_forward00:31:55 - or eventually in muscles when we have a large enough or a potent enough stimulator
  • fast_forward00:32:04 - to activate the muscles.
  • fast_forward00:32:05 - So the first experiments, or one of the first, was creating what you called
  • fast_forward00:32:10 - artificial connections between cortex and spinal cord, if I'm correct, or that muscle.
  • fast_forward00:32:15 - Or even between cortex and cortex. So the very first experiments of Annie Jackson
  • fast_forward00:32:19 - and Jadeep Mavuri was connecting motor cortex sites.
  • fast_forward00:32:24 - So these were two separate sites in the precentral gyrus that could be functionally
  • fast_forward00:32:30 - connected by this artificial loop.
  • fast_forward00:32:32 - More recently, we've been working with artificial connections from motor cortex
  • fast_forward00:32:39 - cells to spinal cord, and that works very nicely too.
  • fast_forward00:32:44 - It's a little more distance, and the spinal cord is a little more challenging in the sense of stable.
  • fast_forward00:32:51 - Implantation of the electrodes, but it can be made to work.
  • fast_forward00:32:54 - But then one of the first experiments, I think, was one of these artificial
  • fast_forward00:32:58 - connections from motor cortex to muscles in the wrist, if I'm correct.
  • fast_forward00:33:03 - Yes. Through which the monkey, so the monkey had to control these muscles and
  • fast_forward00:33:08 - then using that response, it could control a cursor on a computer screen.
  • fast_forward00:33:13 - Well, this is the way it worked.
  • fast_forward00:33:14 - So this wasn't actually done with a neurochip, it was actually done with rack-mounted instrumentation.
  • fast_forward00:33:21 - But later on, the same thing was done with the neurochip.
  • fast_forward00:33:27 - But basically, the idea was to take the activity of a cell recorded in the motor
  • fast_forward00:33:33 - cortex and deliver stimuli into the muscle, or more accurately,
  • fast_forward00:33:39 - the nerves that go to the muscle.
  • fast_forward00:33:41 - And so the cell was directly controlling the stimulation of muscles,
  • fast_forward00:33:46 - which produced twitches that allowed the monkey to succeed in playing his video
  • fast_forward00:33:52 - game, which was to get forces into,
  • fast_forward00:33:54 - cursors that represent forces, into targets.
  • fast_forward00:33:58 - Okay, so it had to generate these forces with these muscles,
  • fast_forward00:34:02 - or how exactly? Yes, that's right. The muscles generated the forces.
  • fast_forward00:34:06 - And so this is work of Chet Moritz and the promoter that showed that if a monkey
  • fast_forward00:34:14 - is trained to get a cursor into targets with normal muscle activity,
  • fast_forward00:34:20 - and then you block the nerves to the muscles, paralyze them,
  • fast_forward00:34:24 - then you can bridge this lost connection with direct connection from the cell
  • fast_forward00:34:31 - to electrical stimulation of the muscles.
  • fast_forward00:34:33 - And the monkey will quickly learn to drive the cell to stimulate the muscle
  • fast_forward00:34:38 - to get the cursor into the target. Right. Okay?
  • fast_forward00:34:41 - You were saying in your talk that actually directly stimulating the muscle creates
  • fast_forward00:34:46 - this problem of recruiting the largest small cells in the wrong order.
  • fast_forward00:34:50 - That's right. It's not a natural way to activate the muscle.
  • fast_forward00:34:52 - So there's a natural recruitment order from what they're called small motor
  • fast_forward00:34:57 - neurons which have low thresholds and low forces to large motor neurons that
  • fast_forward00:35:03 - have more twitch tension but adapt very quickly.
  • fast_forward00:35:07 - And electrically stimulating this produces recruitment of the large before the
  • fast_forward00:35:14 - small, and more natural ways of activating the muscle recruits the small before the large.
  • fast_forward00:35:22 - And the small motor units have much longer, they can be activated much longer.
  • fast_forward00:35:28 - So it's much nicer to do it the natural way.
  • fast_forward00:35:33 - That having been said, it's possible to work around that and have the monkey
  • fast_forward00:35:40 - generate these movements through activating things electrically.
  • fast_forward00:35:43 - But now, in some sense, in this experiment, you have introduced a bias by using
  • fast_forward00:35:48 - muscles that in the life of the monkey are also used for moving objects.
  • fast_forward00:35:54 - Yes. Oh, definitely. Right. They're just temporarily disconnected through this
  • fast_forward00:35:57 - nerve block. Exactly right. Right.
  • fast_forward00:35:58 - So you're reactivating the subset of muscles that would have been used under
  • fast_forward00:36:04 - normal conditions as well to move an object in the world. Definitely. Absolutely.
  • fast_forward00:36:09 - But now, would you imagine that you could also have acquired this mapping,
  • fast_forward00:36:14 - could have twitched muscles just anywhere else in the body controlling that same cursor? Yes.
  • fast_forward00:36:23 - So if you had the cursor controlled by the contraction of any other muscle,
  • fast_forward00:36:28 - the experiment would be essentially the same.
  • fast_forward00:36:31 - So one of the factors here is that the cells don't necessarily have to be related
  • fast_forward00:36:38 - to the muscle that you're using.
  • fast_forward00:36:40 - The monkey just simply needs to learn how
  • fast_forward00:36:44 - to activate that cell which is similar to
  • fast_forward00:36:47 - the operant conditioning paradigm and then once
  • fast_forward00:36:50 - the monkey gets control of cell activity then
  • fast_forward00:36:54 - you link that cell activity to stimulation of any arbitrary muscle and if the
  • fast_forward00:36:59 - contraction of that muscle gets linked to the cursor then you're home free this
  • fast_forward00:37:06 - is going to work i think but it would be The acquisition would be equally fast
  • fast_forward00:37:11 - or it might take longer if it's, let's say,
  • fast_forward00:37:13 - your calf or something like that.
  • fast_forward00:37:16 - Yeah, I think it would be just as fast because I'm thinking that the speed of
  • fast_forward00:37:23 - acquisition is more a function of gaining control of the cell than it is of
  • fast_forward00:37:29 - the muscle, the nature of the muscle.
  • fast_forward00:37:30 - So once you've picked a muscle and you've connected it to the cursor,
  • fast_forward00:37:35 - it doesn't matter where it is physically so much, there might be some muscle
  • fast_forward00:37:41 - properties that would...
  • fast_forward00:37:43 - And if it's smooth muscle, would it also work?
  • fast_forward00:37:46 - Yeah, well, good question. I'm not sure. I'd have to make sure.
  • fast_forward00:37:54 - I think so. I would say so, but
  • fast_forward00:37:56 - I'm sort of guessing. It might be an interesting experiment to perform.
  • fast_forward00:37:59 - Definitely. Oh, yeah. I guess one of the issues there is how much the,
  • fast_forward00:38:05 - going back to this point of it being volitional, how much is the monkey aware
  • fast_forward00:38:11 - that it's using a muscle or a set of neurons that command a particular muscle to control it?
  • fast_forward00:38:17 - So I'm imagining, okay, so if this is my arm and I just have to imagine moving
  • fast_forward00:38:23 - my arm and it operates through your system to control those muscles,
  • fast_forward00:38:28 - that's much more natural than if I have to imagine moving my leg and as a result my arm moves.
  • fast_forward00:38:35 - So I can see that there might be some mapping conflicts there which wouldn't
  • fast_forward00:38:40 - exist if you can actually target the original set of neurons, motor neurons.
  • fast_forward00:38:47 - Or do you think I'm over-intellectualizing it? I'm thinking this through.
  • fast_forward00:38:51 - And I'm guessing that as far as the monkey is concerned, the idea is to get
  • fast_forward00:38:55 - that cursor into the target.
  • fast_forward00:38:58 - And once he gets the cursor into the target with cell activity,
  • fast_forward00:39:01 - the rest of the stuff is not something that he's cognitively concerned about.
  • fast_forward00:39:07 - I think he's just experiencing a sort of nonlinear relationship between the
  • fast_forward00:39:15 - cell activity and the cursor. and it's non-linear because of the recruitment
  • fast_forward00:39:20 - of the muscle electrically.
  • fast_forward00:39:22 - But which muscle it is, I don't think the monkey really… I guess in the case
  • fast_forward00:39:26 - of the cursor, there isn't anything that I could naturally do to move a cursor
  • fast_forward00:39:31 - without moving a hand or something.
  • fast_forward00:39:33 - But in the case where you're controlling your arm….
  • fast_forward00:39:37 - Then it would make sense, presumably, to find the arm area and see if you could
  • fast_forward00:39:41 - connect that to the arm muscles rather than use another area?
  • fast_forward00:39:46 - Well, you'd think so. I mean, that's the agenda of the decoding group.
  • fast_forward00:39:50 - And they approach it from that point of view is to find cells that are naturally
  • fast_forward00:39:57 - related to the limb that you want to control and work with that.
  • fast_forward00:40:03 - But one of the nice things about this study is that it demonstrated that's not necessary.
  • fast_forward00:40:09 - You can actually work with any motor cortex cell that the monkey can control
  • fast_forward00:40:16 - and link it to the muscle.
  • fast_forward00:40:18 - And pretty much any motor cortex cell can be volitionally controlled.
  • fast_forward00:40:24 - But now you also mentioned that the cell tuning itself does not predict the
  • fast_forward00:40:29 - ability to control the cursor.
  • fast_forward00:40:31 - Exactly. But in addition, you also said that you could triple the number of
  • fast_forward00:40:37 - neurons that can be recruited to now control the factor.
  • fast_forward00:40:41 - So what does that exactly mean? So what that means is in that experiment,
  • fast_forward00:40:44 - about two-thirds of the cells that were used in the study were not really showing
  • fast_forward00:40:52 - directional tuning in relation to wrist movements, torques around the wrist.
  • fast_forward00:40:57 - And all of those cells could be volitionally controlled and linked to a flexor
  • fast_forward00:41:05 - extensor musculature and the monkey would very quickly transition from control
  • fast_forward00:41:09 - of the cell to stimulating those muscles and generating the cursor movement.
  • fast_forward00:41:16 - And in that sense it liberates the paradigm from the necessity of finding cells related to the wrist,
  • fast_forward00:41:25 - and this two-thirds is a number that pertains to the sample in this particular sample.
  • fast_forward00:41:33 - But actually, the number is much larger than two-thirds.
  • fast_forward00:41:39 - I mean, when you start considering cells in other areas, leg area,
  • fast_forward00:41:45 - non-motor cortex, pre-motor cortex, who knows?
  • fast_forward00:41:48 - It's to be determined how many different cortical areas you can demonstrate
  • fast_forward00:41:53 - this volitional control.
  • fast_forward00:41:54 - Then you've increased the space of, let's say,
  • fast_forward00:42:02 - source cells or the number of cells that you could recruit into this sort of
  • fast_forward00:42:08 - a paradigm enormously as opposed to if you had to find cells that were related to the limb.
  • fast_forward00:42:18 - And this becomes a crucial issue in the case of stroke, where the area that
  • fast_forward00:42:23 - might control the hand muscle is lost, and you don't have any cells that are related to the hand.
  • fast_forward00:42:29 - But you'd have the possibility of going to cells in another area that would
  • fast_forward00:42:35 - normally have involved control of something else, but it's in an area that's still viable,
  • fast_forward00:42:43 - and those cells can be then used to control stimulation of hand muscles. Right.
  • fast_forward00:42:50 - So most of your experiments, as I understand it, then you're recording a train
  • fast_forward00:42:56 - of spikes from a motor neuron and motor cortex, and you're relaying that exact
  • fast_forward00:43:01 - train to the target with some latency.
  • fast_forward00:43:04 - Is that right? Well, first of all, I think the motor neurons,
  • fast_forward00:43:08 - strictly speaking, are the cells in the spinal cord that connect directly to the muscle.
  • fast_forward00:43:13 - Sorry, I mean the cortical. And so motor cortex cells are cells that may or
  • fast_forward00:43:19 - may not project to the spinal cord. Yeah, exactly.
  • fast_forward00:43:21 - But you can take their activity and use it to control the stimulation of the muscle.
  • fast_forward00:43:30 - I think that's… So they're cortical projection neurons. But what I found surprising
  • fast_forward00:43:35 - was that you get a hit with just about every neuron that you find,
  • fast_forward00:43:42 - and presumably in some of these areas of cortex there are interneurons that
  • fast_forward00:43:47 - could you be hitting, or do you know if you're always targeting projection neurons?
  • fast_forward00:43:52 - No, no, we don't know, and it doesn't matter, I don't think.
  • fast_forward00:43:56 - I'm pretty sure on the basis of how many cells in the motor cortex have been
  • fast_forward00:44:02 - successfully controlled, I would say that some of them are projection,
  • fast_forward00:44:08 - some of them are probably not projection.
  • fast_forward00:44:10 - Injection, there's a bias toward getting cells with very large action potentials
  • fast_forward00:44:16 - that are reliably isolated over long periods of time, so that would mean we're biasing toward.
  • fast_forward00:44:23 - Layer five pyramidal neurons. Pyramidal now meaning the morphology of the cell is called pyramidal.
  • fast_forward00:44:31 - But that doesn't mean they go into the pyramidal tract of the spinal cord.
  • fast_forward00:44:34 - It just means that that's the shape.
  • fast_forward00:44:36 - And so they generate these nice big action potentials that are pretty stable.
  • fast_forward00:44:41 - But where they project, we don't know.
  • fast_forward00:44:45 - And frankly, don't worry too much about because because the key is to use their
  • fast_forward00:44:51 - activity and use the ability of the animal to control that activity in a useful
  • fast_forward00:44:57 - way. But it does seem surprising.
  • fast_forward00:44:59 - I mean, if you think about cortical circuits, you don't think about all the
  • fast_forward00:45:02 - neurons being the same there.
  • fast_forward00:45:03 - And you think about interneurons having some modulatory role related to projection
  • fast_forward00:45:09 - neurons, and therefore possibly producing very different kinds of signals and
  • fast_forward00:45:13 - being used to very different kinds of inputs.
  • fast_forward00:45:15 - Outputs so whereas i can understand maybe a projection neuron
  • fast_forward00:45:19 - which is targeting a motor neuron you could
  • fast_forward00:45:22 - you could take the output of that and direct it down
  • fast_forward00:45:25 - to the the downstream muscle and that makes some
  • fast_forward00:45:28 - sense but if it's an interneuron you're having to really the brain is having
  • fast_forward00:45:32 - to really reprogram that interneuron to do something it's never done before
  • fast_forward00:45:35 - well i think you're over uh projecting function into cell types right first
  • fast_forward00:45:41 - of all uh your idea of how the The interneuron would modulate this activity,
  • fast_forward00:45:46 - also require some degree of flexibility of the way they're recruited.
  • fast_forward00:45:52 - So even interneurons could be as easily volitionally controllable, I would think.
  • fast_forward00:46:00 - There's another issue which has to do with the morphology of most of these interneurons
  • fast_forward00:46:04 - are sort of spherically,
  • fast_forward00:46:09 - as spherical dendrites and closed fields and so they're less likely to be actually
  • fast_forward00:46:14 - recorded by these tungsten electrodes But there's another aspect of course that
  • fast_forward00:46:20 - we might not want to interpret.
  • fast_forward00:46:23 - A circuit too literal in terms of a single cell. I mean, these neurons are embedded
  • fast_forward00:46:28 - in quite a dense volume of cells.
  • fast_forward00:46:30 - And maybe what you're training here is the response of this whole volume.
  • fast_forward00:46:33 - And activity within the volume will be highly correlated, whether that's layer
  • fast_forward00:46:38 - 5 pyramid or an interneuron or a layer 4 stellate cell.
  • fast_forward00:46:42 - Exactly. They will be tightly coupled in their responses. Is that how you think about it?
  • fast_forward00:46:46 - I do, yes. Yes, I think that we're talking about a large distributed group of
  • fast_forward00:46:52 - interconnected neurons of which we sample one or maybe two,
  • fast_forward00:46:58 - but they're all going to be co-activated more or less.
  • fast_forward00:47:03 - Do you have any physiological evidence for that or is there any?
  • fast_forward00:47:06 - Well, when you do record neighboring cells, originally when we recorded more
  • fast_forward00:47:12 - than one and conditioned one, often the neighboring cell was also modulated, but not always.
  • fast_forward00:47:18 - So, it depends on where the other cells are physically located.
  • fast_forward00:47:24 - Even though they're synaptically interconnected, they don't necessarily have to be neighbors.
  • fast_forward00:47:29 - And my view of it is that this is a fairly distributed population of cells.
  • fast_forward00:47:35 - Are you intending to measure this directly or you think that's not so much of a problem at this time?
  • fast_forward00:47:53 - Maybe even three in some cases. And the question is, what are the other cells doing?
  • fast_forward00:47:58 - So we'll have an answer once all that torrent of data gets analyzed.
  • fast_forward00:48:03 - Exactly. So now, but the next step here was that you sort of went straight.
  • fast_forward00:48:07 - So you bypassed out a muscle because then the next step became,
  • fast_forward00:48:11 - look, maybe we can just wire it straight into the spinal cord.
  • fast_forward00:48:14 - That became the next exercise. I mean, the last experiment we discussed was
  • fast_forward00:48:18 - really like moving the cursor with muscle twitches.
  • fast_forward00:48:21 - Right. But now the next step was spinal cord. Spinal cord. Why was that an important target?
  • fast_forward00:48:25 - So the spinal cord is important because it recruits the motor units more naturally, as we just discussed.
  • fast_forward00:48:33 - And it also, spinal stimuli tend to recruit synergistic groups of muscles together.
  • fast_forward00:48:43 - So that's what you ultimately want. If you try to achieve that by muscle stimulation,
  • fast_forward00:48:51 - you'd have to implant a lot of muscles and learn how to co-activate them.
  • fast_forward00:48:55 - Whereas in the spinal cord, you're at a place where you can recruit them sort of naturally as a group.
  • fast_forward00:49:03 - But that would suggest that you would know how the spinal cord is organized
  • fast_forward00:49:06 - and how activity in spinal motor neurons maps to coherent movement patterns.
  • fast_forward00:49:14 - Is that known? No, it's an empirical issue. So we have done the mapping of the spinal cord.
  • fast_forward00:49:22 - That is to say, drive an electrode systematically through the spinal cord and measure the output.
  • fast_forward00:49:29 - And what the take-home message from those experiments is that you can't really
  • fast_forward00:49:35 - predict what the output is.
  • fast_forward00:49:37 - It's an empirical issue. And the reason you can't predict is because what you're
  • fast_forward00:49:41 - stimulating is lots of fibers, and fibers are intertwined.
  • fast_forward00:49:46 - Tangled in unpredictable ways as a function of location in the spinal cord.
  • fast_forward00:49:51 - So ultimately, if you want to get a spinal site where you get contraction of
  • fast_forward00:49:59 - a particular muscle, you're going to have to hunt for it. And then when you
  • fast_forward00:50:03 - find it, you're going to have to hang on to it.
  • fast_forward00:50:05 - So it's not something that you can reliably predict beforehand.
  • fast_forward00:50:11 - So the mapping, another way to put it is the mapping of the cord isn't as clean
  • fast_forward00:50:16 - cut as the mapping of output effects in motor cortex for example. Right.
  • fast_forward00:50:23 - But then the neural response that you mapped onto spinal cord was response in
  • fast_forward00:50:29 - the gamma range. That's about 40 hertz type.
  • fast_forward00:50:31 - Oh, there's one study you're talking about. So there you were able to have the
  • fast_forward00:50:37 - monkey modulate 40 hertz responses in its motor cortex, I assume.
  • fast_forward00:50:42 - To then move a cursor along one dimension of movement.
  • fast_forward00:50:46 - Right yeah um but now do you
  • fast_forward00:50:49 - think you can move that much further could you go also again
  • fast_forward00:50:52 - from two single cells driving spinal cord absolutely
  • fast_forward00:50:55 - that's we've done that we have the data we just
  • fast_forward00:50:58 - need to analyze it okay so there are no limitations there in terms of the spinal
  • fast_forward00:51:02 - control you can get i think there probably are uh limitations in the number
  • fast_forward00:51:07 - of independent outputs you can expect to find and how practical it is to go
  • fast_forward00:51:13 - searching for the ones you need.
  • fast_forward00:51:16 - So there are going to be challenges to make this, for example,
  • fast_forward00:51:20 - a useful prosthetic agenda.
  • fast_forward00:51:22 - That's of course the obvious target, right, where you would say,
  • fast_forward00:51:25 - well, we can sort of, in case of spinal lesions, we can directly bypass the
  • fast_forward00:51:29 - lesion, go straight from M1 into spinal cord and move the legs or something
  • fast_forward00:51:34 - like this. Is that a feasible outlook?
  • fast_forward00:51:36 - Eventually, I think it will be there.
  • fast_forward00:51:41 - So, there's another thing that we're working on that might relate to that,
  • fast_forward00:51:45 - having to do with stimulating,
  • fast_forward00:51:48 - not intraspinally, which is an evasive technology, and the spinal cord likes
  • fast_forward00:51:53 - to eventually reject these electrodes that are in there.
  • fast_forward00:51:57 - So, the thing we're going to test next is whether we can get similar effects
  • fast_forward00:52:02 - by stimulating the surface of the spinal cord instead of doing it intraspinally.
  • fast_forward00:52:07 - So, the question is whether we'll be able to find enough differentiation of
  • fast_forward00:52:14 - the output effects that can be evoked from the surface of the spinal cord or
  • fast_forward00:52:20 - whether we actually need to poke these wires into the cord and search out.
  • fast_forward00:52:26 - But would the monkey be able to also correct now this mapping for itself?
  • fast_forward00:52:30 - Imagine we go from motor cortex into spinal cord.
  • fast_forward00:52:33 - We want to have, let's say, a certain walking gait. Imagine that's our target.
  • fast_forward00:52:37 - But initially we get some strange twitches because we're not placed correctly in the spinal cord.
  • fast_forward00:52:42 - Do you think there's a possibility that it's the motor commands coming in from
  • fast_forward00:52:48 - the motor cortex that induce that error and that can be remapped evolutionally?
  • fast_forward00:52:54 - Yes. Is that what you expect? To some extent, that's true, but the big factor
  • fast_forward00:53:01 - is whether your output effects from the spinal cord, which are sort of a given
  • fast_forward00:53:05 - basis functions, as it were,
  • fast_forward00:53:07 - include the movements that you want to generate.
  • fast_forward00:53:10 - If they don't include it, there's no way that cortical control is going to be
  • fast_forward00:53:16 - able to generate something that the stimulation doesn't produce.
  • fast_forward00:53:20 - But if you have it in your repertoire, then my prediction is that the brain,
  • fast_forward00:53:28 - given sufficient time to learn to optimize that stimulation,
  • fast_forward00:53:32 - can recruit those outputs to functionally useful ends.
  • fast_forward00:53:38 - So you have great confidence in the plastic capabilities of brains. I do.
  • fast_forward00:53:42 - Actually, yeah, that's right. It's an article of faith. We'll see.
  • fast_forward00:53:45 - I mean, there may be other complications.
  • fast_forward00:53:47 - I don't know. You know, things like you don't want to stimulate in the spinal
  • fast_forward00:53:52 - cord dorsal horn too much because that's pretty aversive and you need to be in the right place.
  • fast_forward00:53:58 - And so there's issues like that. Sure. But you have a backup plan,
  • fast_forward00:54:03 - I guess, which is to go directly to the target muscles.
  • fast_forward00:54:07 - Well, yes. If you can't go through the spinal cord. That's right.
  • fast_forward00:54:09 - But as we said, the target muscles directly have this recruitment problem.
  • fast_forward00:54:14 - Yeah, yeah. And the number of muscles that you can activate directly,
  • fast_forward00:54:19 - each one requires its own set of electrodes.
  • fast_forward00:54:23 - So that gets to be a lot of wires. I guess if you can understand more about
  • fast_forward00:54:28 - how the spinal cord is doing the recruitment, and then you could modulate your
  • fast_forward00:54:33 - cortical signal to be more like a spinal cord signal, is that?
  • fast_forward00:54:39 - Yeah, well... Then you're getting more towards the decoding, recoding guys, I guess.
  • fast_forward00:54:44 - I'm thinking more in terms of
  • fast_forward00:54:47 - the brain being able
  • fast_forward00:54:50 - to recruit the motor cortex cells that control the stimulation in a way that
  • fast_forward00:54:59 - makes that output from that stimulation be not practical for the movements that
  • fast_forward00:55:07 - the subject wants to attain.
  • fast_forward00:55:09 - And so, that whole thing involves a learning process that could take days and
  • fast_forward00:55:18 - could be supported by this sort of an implant.
  • fast_forward00:55:22 - That's part of this idea, the advantage of an implant is that it provides the
  • fast_forward00:55:27 - subject plenty of time to optimize this control.
  • fast_forward00:55:32 - So, that's an important… But now, in some sense, there must be a minimum command
  • fast_forward00:55:39 - set that has to go down from motor cortex into spinal cord to get coherent movement.
  • fast_forward00:55:45 - That's right. You need as many independent outputs as you want to independently
  • fast_forward00:55:50 - control the stimulus outputs.
  • fast_forward00:55:54 - So, in the healthy brain, what would that be?
  • fast_forward00:55:58 - So, how many, let's say, I want to induce a walking gait. so how many signals
  • fast_forward00:56:04 - do have to come out of my motor cortex to induce that or to control that.
  • fast_forward00:56:10 - Well, that's a good question.
  • fast_forward00:56:14 - So the answer to that question depends on how many components of the gate you want to control.
  • fast_forward00:56:21 - So you can use computers to help along a little bit to generate patterns of activity.
  • fast_forward00:56:27 - And then in the extreme, you might imagine you just have one cell that turns it on and off.
  • fast_forward00:56:33 - On the other hand, on the other extreme, you want outputs that control every
  • fast_forward00:56:37 - component of the gate. And that's a much more formidable challenge and probably
  • fast_forward00:56:43 - unlikely to be the way that we want to go.
  • fast_forward00:56:46 - So I think you're going to wind up with some combination of pre-programmed patterns
  • fast_forward00:56:52 - that are controlled by a smaller set of cells.
  • fast_forward00:56:57 - So getting back to your question, how many?
  • fast_forward00:56:59 - I'm guessing it would be useful to have at least a half a dozen cells.
  • fast_forward00:57:05 - Because it relates a little bit to this issue of bandwidth
  • fast_forward00:57:08 - and capacity right and now in some
  • fast_forward00:57:11 - sense we can get away with a very limited bandwidth
  • fast_forward00:57:14 - because we only control very few degrees of freedom we move cursors over one
  • fast_forward00:57:18 - or two dimensions and so on right so so do you see this as as a as a bottleneck
  • fast_forward00:57:24 - as a real limitation for technology or do you think it will just scale up to
  • fast_forward00:57:28 - any set of degrees of freedom. The independent control?
  • fast_forward00:57:33 - It's a good question. I don't know.
  • fast_forward00:57:36 - I don't think it's unlimited because there are ultimately relationships between these cells.
  • fast_forward00:57:42 - They're not totally independent, so they're probably going to be going to run
  • fast_forward00:57:48 - into the fact that it's not quite so easy to do them all individually. Right.
  • fast_forward00:57:54 - It is the cautious answer. My intuition is that, again, given enough time,
  • fast_forward00:58:00 - I bet you the brain is going to demonstrate that it can handle a goodly number
  • fast_forward00:58:06 - of degrees of freedom. But you have to give it enough time.
  • fast_forward00:58:09 - Right. So then another important aspect you touched upon was learning.
  • fast_forward00:58:15 - So you really also have configured your system that's in a very controlled condition.
  • fast_forward00:58:19 - You can look at different learning paradigms, different forms of Hebbian learning.
  • fast_forward00:58:25 - So what were the key things that you have learned?
  • fast_forward00:58:29 - Well, with regard to Hebbian learning, we've found that you can induce the synaptic
  • fast_forward00:58:38 - plasticity that is mediated by Hebbian mechanisms.
  • fast_forward00:58:42 - That is to say, you can strengthen the connections between neurons,
  • fast_forward00:58:48 - and in this motor cortex experiment where stimulation and recording were both done in motor cortex,
  • fast_forward00:58:57 - those changes lasted a surprisingly long time, 10 days in one case, and it never reverted.
  • fast_forward00:59:06 - But in other experiments, it hasn't lasted that long. So another lesson is you
  • fast_forward00:59:11 - can make those changes, but don't count on them staying around in the absence
  • fast_forward00:59:17 - of continued conditioning.
  • fast_forward00:59:19 - But now the first experiment in motor cortex, basically you use your neurochip
  • fast_forward00:59:24 - to impose certain correlation patterns between neurons with the idea that,
  • fast_forward00:59:32 - okay, what fires together wires together.
  • fast_forward00:59:34 - So if you force them to be synchronized in their response, you assume this experiment,
  • fast_forward00:59:39 - they also would wire together.
  • fast_forward00:59:41 - That's right. And then the idea would be the response properties would become more similar.
  • fast_forward00:59:46 - Well, yeah.
  • fast_forward00:59:49 - The connections are strengthened. We don't know really all that much about the response properties.
  • fast_forward00:59:54 - All we know is that the connections that were mediating the output effects by
  • fast_forward00:59:59 - stimulating those sites had changed.
  • fast_forward01:00:02 - And what that meant for the way these cells were active is an interesting question
  • fast_forward01:00:08 - for the future, but not one that we could… But wait, but your measure,
  • fast_forward01:00:13 - your performance measure was how the cell's response correlated with a movement
  • fast_forward01:00:19 - pattern, a movement of the monkey.
  • fast_forward01:00:21 - So, no, the measure in the motor cortex study was the movements that were generated
  • fast_forward01:00:28 - by electrically stimulating the sites in the cortex that were recorded from
  • fast_forward01:00:34 - and that were stimulated, and then there was a control site.
  • fast_forward01:00:37 - And so the measure was really a somewhat artificial measure of what you could
  • fast_forward01:00:44 - evoke with a train of stimuli from these sites.
  • fast_forward01:00:47 - But those outputs changed in a way that is most easily understood as strengthening
  • fast_forward01:00:55 - a connection between the recording site and the stimulation site.
  • fast_forward01:00:59 - The outputs were consistently in a direction that could be explained by strengthening those connections.
  • fast_forward01:01:05 - Now, the question of what that meant for the way cells in these two areas are
  • fast_forward01:01:12 - correlated, whether that was changing or not, is not something that we measured,
  • fast_forward01:01:17 - although it would be great to be able to do that.
  • fast_forward01:01:19 - But I thought your data were pointing to this because you stimulated the cell
  • fast_forward01:01:24 - or the site where you had been imposing a correlation with the recording site.
  • fast_forward01:01:29 - Then you looked at what movement it correlated later on with individual stimulation.
  • fast_forward01:01:34 - And you saw that it was sort of a similar movement as in the recording site
  • fast_forward01:01:39 - and dissimilar from the control site.
  • fast_forward01:01:42 - So you just have to remember that the movements we're talking about are movements
  • fast_forward01:01:45 - that are evoked by electrical stimulation, not movements that were generated by the monkey.
  • fast_forward01:01:49 - Right, sure, absolutely. So I think there's a lot to be.
  • fast_forward01:01:54 - Pursued here in terms of what the change in these synaptic connections means
  • fast_forward01:02:00 - for change in functional interactions between these sites.
  • fast_forward01:02:04 - So that would be a great follow-up experiment.
  • fast_forward01:02:09 - But then the other thing that you emphasized very strongly in these experiments
  • fast_forward01:02:13 - on plasticity was that it's not just any form of Hebbian learning that is driven
  • fast_forward01:02:18 - by correlation, but it really follows these ideas of spike-time dependent learning.
  • fast_forward01:02:22 - Yes. Where you depress or potentiate the synapse given the latency between pre-
  • fast_forward01:02:28 - and post-synaptic activity.
  • fast_forward01:02:30 - Exactly. Right. So that means at very short latencies, you would be depressing.
  • fast_forward01:02:35 - And at longer latencies, you would be potentiating.
  • fast_forward01:02:39 - And at some point, you don't do anything. So what was the exact observation there?
  • fast_forward01:02:44 - So the ultimate mechanism is that if you have a presynaptic input that's activated
  • fast_forward01:02:51 - relative to a postsynaptic cell activation,
  • fast_forward01:02:55 - the strengthening of the connection is dependent on the time between the presynaptic
  • fast_forward01:03:01 - and postsynaptic input.
  • fast_forward01:03:02 - So if that presynaptic input comes within 50 milliseconds of the postsynaptic
  • fast_forward01:03:08 - activation, You can strengthen the connection, and that's consistent with what
  • fast_forward01:03:12 - we saw in the motor cortex study and the corticospinal study.
  • fast_forward01:03:17 - It turns out that spike timing dependent plasticity function,
  • fast_forward01:03:21 - which I've just described in the sense of increasing the strength of connection,
  • fast_forward01:03:27 - actually goes in the other direction of predicting a decrease in the strength
  • fast_forward01:03:33 - of the connection if the postsynaptic cell is activated prior to the presynaptic input.
  • fast_forward01:03:39 - So that's the non-causal way of activating these cells. In other words...
  • fast_forward01:03:46 - The postsynaptic cell would probably be driven by the presynaptic input in the
  • fast_forward01:03:52 - normal causal way. That would strengthen connections.
  • fast_forward01:03:55 - But if you do it the other way around, then the bidirectional spike timing dependent
  • fast_forward01:04:02 - plasticity function, if I can use that mouthful of words, is actually predicting a decrease.
  • fast_forward01:04:09 - And this is the thing that we could demonstrate in the corticospinal system
  • fast_forward01:04:16 - could be achieved when the delay between the spike and the spinal stimulus was zero.
  • fast_forward01:04:23 - When the spinal stimulus was delivered as fast as possible after the cortical spike,
  • fast_forward01:04:29 - it actually activated the postsynaptic cells in the spinal cord prior to the
  • fast_forward01:04:34 - arrival of the corticospinal volley, and that resulted in a decrease in the connection.
  • fast_forward01:04:42 - But that's a part of this curve that is only possible to explore through this
  • fast_forward01:04:49 - means in situations where you have a long conduction time from the recording to the stimulated site.
  • fast_forward01:04:56 - But then there's something interesting about this, right? Because what you found,
  • fast_forward01:05:00 - this learning window you found, sort of shows the strongest potentiation about
  • fast_forward01:05:04 - 40 milliseconds and it showed a depression about 10 millisecond latency.
  • fast_forward01:05:10 - So this is telling you something about the causal structure that these circuits try to maintain, right?
  • fast_forward01:05:16 - Because they're saying, look, okay, if presynaptic activity is closer to 10
  • fast_forward01:05:22 - milliseconds to my post-synaptic activity, then I'm not going to learn this.
  • fast_forward01:05:25 - Then there's something wrong in the causal structure I'm dealing with.
  • fast_forward01:05:28 - So is there anything special about these latencies you find in that spinal circuit
  • fast_forward01:05:32 - with respect to the behavior that it controls?
  • fast_forward01:05:36 - The leap from this spike timing
  • fast_forward01:05:39 - dependent plasticity to behavior is an order
  • fast_forward01:05:43 - of magnitude difference in the number of cells that are involved in generating
  • fast_forward01:05:51 - the behavior in relation to the number of cells that are involved in demonstrating this plasticity.
  • fast_forward01:06:00 - So ultimately, one would expect a relationship to behavior, but it doesn't work.
  • fast_forward01:06:10 - Appear that our Hebbian plasticity studies are strong enough to mediate observable
  • fast_forward01:06:17 - changes in the behavior.
  • fast_forward01:06:19 - They're really small things, but they're nevertheless,
  • fast_forward01:06:24 - cellular mechanisms that underlie more broad changes in behavior, I think.
  • fast_forward01:06:30 - Right, but the first thing that's interesting, of course, is that you can induce
  • fast_forward01:06:36 - this kind of Hebbian type plasticity using spike-time dependent learning in
  • fast_forward01:06:40 - a spinal cord circuit that I find very surprising, right?
  • fast_forward01:06:44 - Because it means you can also reshape, remodel these spinal circuits dependent
  • fast_forward01:06:48 - on the kinds of pre-imposed signals that they pinch on it, which should also
  • fast_forward01:06:52 - give us hope in terms of rewiring spinal circuits, interestingly enough.
  • fast_forward01:06:56 - But then the other thing is, of course, that maybe these very short latencies
  • fast_forward01:06:59 - you want to maintain because you go from, let's say, your motor cortex.
  • fast_forward01:07:04 - Towards the periphery, projections from the spinal circuit into the skeletal
  • fast_forward01:07:09 - muscle system and its feedback will occur in a certain time window.
  • fast_forward01:07:13 - And these must be aligned correctly, right? Control signals from the cortex
  • fast_forward01:07:17 - should precede any type of activity you receive from the periphery.
  • fast_forward01:07:22 - So you can imagine that these latencies that seem now critical in this learning
  • fast_forward01:07:25 - window that you found map to the latencies you might find in this transduction
  • fast_forward01:07:29 - of control signals to the skeletal muscle system that you in the end are controlling.
  • fast_forward01:07:34 - That's a little bit what I was after.
  • fast_forward01:07:37 - Yeah, the changes that you see, are any of those long-lasting?
  • fast_forward01:07:42 - I mean, can we think of it as remodeling the circuit or is it something that
  • fast_forward01:07:46 - goes away? Well, that was one of the interesting observations in the study of
  • fast_forward01:07:50 - the corticospinal plasticity is how long does it last?
  • fast_forward01:07:53 - So in some cases, it lasted for at least two days after the end of the conditioning.
  • fast_forward01:07:59 - And possibly even longer. We just didn't measure it.
  • fast_forward01:08:02 - In other cases, in the majority probably, it dropped after the end of the conditioning
  • fast_forward01:08:09 - in the next couple of days. And so...
  • fast_forward01:08:14 - That raises the question of how one would make these changes more long-lasting.
  • fast_forward01:08:19 - It's possible that you could use other interventions.
  • fast_forward01:08:22 - Well, first of all, you could just do this conditioning much longer than we
  • fast_forward01:08:26 - did. We just did it for a couple days.
  • fast_forward01:08:29 - This is Yukio. We is actually Yukio Nishimura and Steve Promutter.
  • fast_forward01:08:33 - And they did it for a couple days. delays.
  • fast_forward01:08:38 - If you did it much longer, it's conceivable that this plasticity could lead
  • fast_forward01:08:45 - to structural changes that would be more permanent.
  • fast_forward01:08:49 - Number one, amount of conditioning. Number two, you could use neural modulators to change the strength.
  • fast_forward01:08:58 - Like BDNF, for example, could be something that could make these changes last longer.
  • fast_forward01:09:05 - And third, you could do things like polarization.
  • fast_forward01:09:09 - DC polarization could also make these things last longer.
  • fast_forward01:09:14 - But now, the next step in the process, the last experiment you were discussing,
  • fast_forward01:09:20 - where you really try to now modulate or change reward systems themselves in the brain, right?
  • fast_forward01:09:29 - Where you try to train towards nucleus accumbens, for instance.
  • fast_forward01:09:33 - So what was the experiment there? So the experiment there was to use the neurochip
  • fast_forward01:09:38 - to record activity somewhere.
  • fast_forward01:09:41 - Somewhere, in our case it started with muscles, and then deliver stimuli that
  • fast_forward01:09:48 - were driven by that activity in a rewarding site.
  • fast_forward01:09:52 - So this is an intracranial site where electrical stimulation is behaviorally
  • fast_forward01:09:58 - rewarding in the sense that the monkey will make movements to generate stimulation.
  • fast_forward01:10:05 - And so in this paradigm with the neurochip,
  • fast_forward01:10:11 - the idea was to close that loop from muscle activity to intracranial stimulation
  • fast_forward01:10:18 - of the reinforcement site, nucleus accumbens.
  • fast_forward01:10:21 - And the monkey very quickly learned to do what could track that muscle and amazingly
  • fast_forward01:10:26 - did it for quite a long period of alternating on and off periods. Would it not rest?
  • fast_forward01:10:34 - If you would leave it, would it do it forever? Well,
  • fast_forward01:10:37 - we took it out to 20 hours, and the monkey was still doing pretty well at the
  • fast_forward01:10:42 - end of 20 hours, although I think the data shows that control dropped somewhere
  • fast_forward01:10:48 - in between, probably at night.
  • fast_forward01:10:50 - But did you also observe that the monkey did not eat or drink?
  • fast_forward01:10:53 - It was just self-stimulating? No, I wouldn't say that.
  • fast_forward01:10:57 - I think it's not as potent as in the rat.
  • fast_forward01:11:02 - I think those are fairly dramatic demonstrations in the rat that that's all
  • fast_forward01:11:07 - they'll do until they die. But monkeys are...
  • fast_forward01:11:12 - It's not that compelling a stimulation.
  • fast_forward01:11:16 - He can drop it. I wish it were more compelling because we're now trying to obviously
  • fast_forward01:11:21 - do this with neural activity, neural patterns.
  • fast_forward01:11:23 - And it can work, but the results have been not quite as robust yet.
  • fast_forward01:11:29 - But now you mentioned that you actually started the Lulis experiments because
  • fast_forward01:11:33 - you wanted to get a handle on temporal coding, which I found a rather surprising step.
  • fast_forward01:11:38 - Oh, yeah. Very interesting. Right. So that's a speculation that if this could
  • fast_forward01:11:43 - all work, this is way out here now.
  • fast_forward01:11:46 - So if this could all work, this is a paradigm that could be used to test the
  • fast_forward01:11:51 - idea of temporal coding.
  • fast_forward01:11:53 - So what we mean by that is, first of all, temporal coding is the assumption
  • fast_forward01:11:59 - that information is coded in a temporal sequence of spikes,
  • fast_forward01:12:04 - as opposed to rate coding, which says that information is coded in the average firing rate of cells.
  • fast_forward01:12:11 - So, we know that rate coding works in the sensory and the motor system,
  • fast_forward01:12:19 - and probably in a lot of the association areas.
  • fast_forward01:12:24 - The question of whether temporal coding is actually being exploited by the brain
  • fast_forward01:12:29 - is an interesting question, because if it were true,
  • fast_forward01:12:32 - it would increase enormously the bandwidth, as it were, for neural computation.
  • fast_forward01:12:40 - It would mean that you could use this additional dimension of time of spikes,
  • fast_forward01:12:47 - in the ultimate case, for information coding.
  • fast_forward01:12:51 - So that's a great idea. It's very enticing. It's, as I said,
  • fast_forward01:12:56 - in dire need of experimental support. port.
  • fast_forward01:12:59 - And the idea, the fantasy here is that if we implemented this neurochip to reward
  • fast_forward01:13:06 - patterns of activity, temporal patterns, we could test whether those temporal
  • fast_forward01:13:11 - patterns were really part of a volitionally controllable,
  • fast_forward01:13:14 - repertoire of behavior.
  • fast_forward01:13:15 - Right, exactly. That would be very exciting. It would be great.
  • fast_forward01:13:19 - I should live that long, yeah.
  • fast_forward01:13:21 - So then, so where do you see the future of this going?
  • fast_forward01:13:26 - I mean, this is all really it's amazing work it really shows also the incredible
  • fast_forward01:13:30 - plasticity of the brain and of course it raises all sorts of questions about
  • fast_forward01:13:35 - application and neuroprosthetics and all sorts of amazing things and also something
  • fast_forward01:13:40 - we are confronted in the field but rather,
  • fast_forward01:13:43 - you know amazing demonstrations what might be possible.
  • fast_forward01:13:48 - So where do you see this really go in this field? Well, I think the exciting
  • fast_forward01:13:52 - prospect, looking in the long run, decades ahead, or do you want to talk just
  • fast_forward01:13:58 - about the next couple of years?
  • fast_forward01:13:59 - Both. Well, next couple of years will be modest increments.
  • fast_forward01:14:04 - Well, we'll have many more channels in the next neurochip, so it'll be more than modest, I think.
  • fast_forward01:14:09 - But in the long run, the exciting thing for me is to imagine that we would get
  • fast_forward01:14:17 - to the point where the brain could directly access the computational power of an implant, let's say.
  • fast_forward01:14:26 - And instead of interacting with our iPhone computers through normal sensory
  • fast_forward01:14:35 - channels, we would have that computer chip available for direct interaction.
  • fast_forward01:14:40 - So this is total science fiction today because the major hurdle that I can see
  • fast_forward01:14:47 - is the problem of tapping the right parts of the brain to connect to the computer
  • fast_forward01:14:53 - directly in both directions.
  • fast_forward01:14:56 - Probably more seriously in the direction of the stimulation being delivered
  • fast_forward01:15:03 - back into the brain in a way that the brain can decode and use.
  • fast_forward01:15:09 - Because electrical stimulation is a very crude way of activating a mess of cells
  • fast_forward01:15:14 - with different functions.
  • fast_forward01:15:17 - And so what we need is a much more specific way of delivering the feedback from
  • fast_forward01:15:26 - the computer into the brain.
  • fast_forward01:15:28 - Right now, the best way to do it is through normal sensory channels,
  • fast_forward01:15:32 - through tactile input or maybe stimulating the...
  • fast_forward01:15:39 - But what you envision there is also that you can create completely new capabilities to a brain.
  • fast_forward01:15:44 - Exactly. Yeah, yeah. Yeah, right. So the big fun thing to think about is what
  • fast_forward01:15:49 - would you do to complement the computational power of the brain with the computational
  • fast_forward01:15:58 - power of the computer chip?
  • fast_forward01:16:01 - And, well, of course, the science fiction movies and writers have anticipated all this.
  • fast_forward01:16:07 - And so we have to look to what they can imagine could happen.
  • fast_forward01:16:12 - And I think there are some serious problems to direct transfer and bidirectional ways of information.
  • fast_forward01:16:26 - I think right now, the most efficient way to do it is through normal sensory motor channels.
  • fast_forward01:16:32 - But in the future, if we want to fantasize about where things could go,
  • fast_forward01:16:37 - it would be interesting to see how you could uh.
  • fast_forward01:16:42 - Implant a chip that the brain could access the computational power of directly.
  • fast_forward01:16:48 - So right now, even the Google Glass is a device that makes this interaction,
  • fast_forward01:16:56 - makes the computer small enough and the interaction easy enough that it could
  • fast_forward01:17:05 - operate, but it's still operating through normal channels.
  • fast_forward01:17:08 - Right. So the question is, and it's a serious question as to whether this is
  • fast_forward01:17:12 - actually going to be possible.
  • fast_forward01:17:14 - The question is whether you can bypass the normal channels and get useful functional
  • fast_forward01:17:20 - interactions by directly stimulating and directly recording from the brain.
  • fast_forward01:17:27 - I'm guessing that's a pretty much open question. And if I were to bet real money
  • fast_forward01:17:32 - on it, I would say it's 50-50, maybe even.
  • fast_forward01:17:36 - And maybe my imagination isn't robust enough to imagine this actually working in a practical way.
  • fast_forward01:17:46 - So that's a kind of cognitive prosthesis, a sort of chip that helps you think better.
  • fast_forward01:17:51 - And in terms of sort of a physical prosthetic, like a hand or an arm.
  • fast_forward01:17:56 - How's the work you're going to do going to impact in that perhaps more near term?
  • fast_forward01:18:01 - Well, there are other people that are doing more practical work on the more
  • fast_forward01:18:08 - standard brain-machine interface, that is to say, an output to a robotic arm.
  • fast_forward01:18:15 - But they're also now working toward a bidirectional device where it's not just
  • fast_forward01:18:20 - output, but also there's some feedback from the, let's say, it's a prosthetic arm.
  • fast_forward01:18:27 - And so you want to know what the joint positions are and what the forces,
  • fast_forward01:18:35 - contact forces are, and you want to feed that information back somehow.
  • fast_forward01:18:40 - So that does create a recurrent loop through this process.
  • fast_forward01:18:44 - External machine this this artificial arm
  • fast_forward01:18:48 - and that's where i
  • fast_forward01:18:51 - think we'll be seeing progress because a lot
  • fast_forward01:18:53 - of people are hammering away at it and right yeah so now so so we started out
  • fast_forward01:18:59 - with your issue a metaphor of the self in the brain yes and with that you meant
  • fast_forward01:19:04 - that that there are systems in the brain that can sort of control other systems
  • fast_forward01:19:08 - in the brain right right so where Where is that self-system?
  • fast_forward01:19:12 - Wonderful question. I wish I knew. It's somewhere behind my eyes.
  • fast_forward01:19:17 - No, I'm just kidding. It's somewhere in there. It's distributed, but it's very robust.
  • fast_forward01:19:23 - And I think it's a key element of the brain that needs further investigation.
  • fast_forward01:19:32 - I think a lot of… But the key thing is you really see some distinction here.
  • fast_forward01:19:35 - There's really a controller, some central system that is, again,
  • fast_forward01:19:40 - controlling then these parts of the brain you are measuring from. Yes. Distinct systems.
  • fast_forward01:19:45 - That's right. And it's merged anatomically, really distinct. Oh, not necessarily.
  • fast_forward01:19:50 - I'm guessing that this controller includes this peripheral component of output,
  • fast_forward01:19:59 - because it actually has to modify its control of that output if you have situations like I described,
  • fast_forward01:20:09 - where you change the relationship between the output that you get and the output
  • fast_forward01:20:15 - that you want to generate.
  • fast_forward01:20:18 - Then the volitional controller part of this brain is adapting to the new contingencies.
  • fast_forward01:20:29 - But that adaptation obviously requires the peripheral elements as well as the central ones.
  • fast_forward01:20:35 - But, you know, in a sense, functionally, you can imagine that there's a separate
  • fast_forward01:20:40 - question of why you want to get from A to B and why you want to adapt your ability to get from A to B.
  • fast_forward01:20:48 - I'm thinking now of this Moussa Valdi and Shadmir study where A and B were points
  • fast_forward01:20:56 - on a plane and getting from A to B required overcoming some strange force field.
  • fast_forward01:21:04 - You still have the volitional controller part intending to get from A to B.
  • fast_forward01:21:10 - And the mechanisms that adapt to this artificial force field that you have to
  • fast_forward01:21:16 - overcome to get there as smoothly as possible,
  • fast_forward01:21:21 - that's all conventionally considered part of the motor system.
  • fast_forward01:21:27 - But I'm guessing that...
  • fast_forward01:21:30 - That it could also be usefully considered as part of the volitional controller.
  • fast_forward01:21:34 - So it's a semantic quibble. But is that self in the brain beyond the conditioning?
  • fast_forward01:21:45 - Is there a core system that cannot in itself be conditioned following these
  • fast_forward01:21:49 - methods you have been describing?
  • fast_forward01:21:51 - That's a good question. That is immune to this manipulation?
  • fast_forward01:21:54 - I don't know. What do you think? I don't think so. Right. it's really part of...
  • fast_forward01:22:01 - I think it's more integrated. I don't think there's a clear distinction between
  • fast_forward01:22:04 - the self and then the rest of the brain. I think it's much more integrated.
  • fast_forward01:22:09 - So then to finish up, I have two questions.
  • fast_forward01:22:13 - So look, we were talking earlier, right? So your first experiments in the 60s
  • fast_forward01:22:18 - as a physicist, so you're in this field for really a long time.
  • fast_forward01:22:22 - You have accomplished an amazing amount of work, gained a lot of insight in the brain, but then...
  • fast_forward01:22:29 - Given all this experience, what is Ebb's law that we should adhere to studying the brain?
  • fast_forward01:22:36 - Don't get caught up in conceptual preconceptions about what part of the brain does what.
  • fast_forward01:22:43 - I've developed a more empirical approach to investigating the brain.
  • fast_forward01:22:53 - This would be my advice to people getting into the field.
  • fast_forward01:22:58 - So you can have preconceived ideas about this part of the brain does this and
  • fast_forward01:23:05 - that part does that and design hypothesis-driven experiments based on that.
  • fast_forward01:23:11 - But usually that often winds up being problematic and unsuccessful and boring in some cases.
  • fast_forward01:23:21 - And so I think my take-home lesson of my life is that you have to just go in there and do it.
  • fast_forward01:23:28 - So, for example, this question of what happens if you connect motor cortex to
  • fast_forward01:23:36 - spinal cord stimulation.
  • fast_forward01:23:38 - When I was first proposed, the NIH study section was appalled that this was
  • fast_forward01:23:43 - way too complicated to be worth funding,
  • fast_forward01:23:49 - because we don't know what the outcome is and where's our hypothesis?
  • fast_forward01:23:54 - Fishing expedition, blah, blah, all those things. Well, geez,
  • fast_forward01:23:58 - I mean, that's how you make progress is to actually do some new stuff and not
  • fast_forward01:24:03 - be constrained by all your preconceptions as to how it would work.
  • fast_forward01:24:07 - So I basically had to reformulate,
  • fast_forward01:24:10 - motivation for that experiment to say we want to know what the behavioral adaptation
  • fast_forward01:24:16 - is to having this artificial connection in parallel with the biological connection
  • fast_forward01:24:20 - and we don't know what the outcome is but the fact that the monkey controls
  • fast_forward01:24:23 - both the artificial and the biological connections,
  • fast_forward01:24:26 - means that it's a meaningful questions to see how he integrates the two right
  • fast_forward01:24:30 - and the study section like that better so this is all good yeah but anyway don't
  • fast_forward01:24:35 - localize that's the key thing right And the last question, so five years from
  • fast_forward01:24:39 - now, we're going to come visit you there in Washington.
  • fast_forward01:24:42 - Okay. And we're going to say, look, you gave us this prediction in 2013,
  • fast_forward01:24:47 - and today we're going to come check whether it was true or false.
  • fast_forward01:24:52 - So what's this one prediction you would like to commit yourself to today?
  • fast_forward01:24:56 - That we still have a lot of interesting questions to investigate.
  • fast_forward01:25:02 - That's too easy. I know. Oh, well. All right, Epp.
  • fast_forward01:25:07 - Thank you very much for this conversation. You bet. Thank you. Thank you.
  • fast_forward01:25:12 - Music.
  • fast_forward01:25:18 - The CSN Podcast was produced by the Convergent Science Network of Biometrics
  • fast_forward01:25:23 - and Biohybrid Systems, a project funded by the European 7th Research Framework Program.
  • fast_forward01:25:31 - For more interviews, recorded lectures, or upcoming conferences in the field
  • fast_forward01:25:37 - of biometrics and biohybrid systems, go to csnnetwork.com.
  • fast_forward01:25:43 - Music.

Be the first to leave a comment

Leave a comment

Your email address will not be published. Required fields are marked *

Convergent PozitivLinia Logo

Exploring the convergence of neuroscience, robotics, and AI through conversations with leading researchers since 2010.

A project of the Convergent Science Network Foundation.

© CSN Podcasts. Developed by IMCreativeWEBC

0%

Login to enjoy full advantages

Please login or subscribe to continue.

Go Premium!

Enjoy the full advantage of the premium access.

Stop following

Unfollow Cancel

Cancel subscription

Are you sure you want to cancel your subscription? You will lose your Premium access and stored playlists.

Go back Confirm cancellation