Joseph Ayers on biomimetic robotics and lobster robot

  • cover play_arrow

    PLAY EPISODE


Season 2012
Season 2012
Description arrow_drop_down

Description

Can algorithmic control ever match the adaptability of a lobster navigating the ocean floor? Neuroscientist and roboticist Joseph Ayers reveals why DARPA abandoned traditional approaches and how chaos-based neural controllers are reshaping biomimetic robotics.

Subscribe for more from the Convergent Science Network podcast series.

In this episode, Ayers explains why conventional algorithmic robot control fails in unpredictable environments. Drawing on decades of studying lobster neurophysiology, he describes how animals use chaotic variations in their neural networks to escape situations no programmer could anticipate. The fundamental problem: you cannot pre-program escape strategies for every possible scenario an autonomous robot might encounter in the real world.

Ayers walks through four generations of robotic lobsters built since 1998, each informed by biological discoveries. The latest generation replaces state machines with true central pattern generators built from discrete-time map-based neurons developed by Nikolai Rukov. These phenomenological neuron models capture spiking, bursting, and chaotic dynamics using just two control parameters, enabling hundreds of neurons and synapses to run on a single DSP chip in real time. The coordination between six walking legs emerges from governing and governed oscillators maintaining proper phase relationships.

The conversation explores how building robots reveals gaps in biological knowledge. Ayers describes discovering that lobsters likely rely on simple bump sensing rather than sophisticated joint proprioception, and how accelerometry-based comparisons between expected and actual movement patterns can detect when the robot is stuck. He details the sensory architecture of the lobster brain, from Wiersma’s classification of visual interneurons to the layered reflex systems that process optical flow, hydrodynamic flow, and obstacle contact. The discussion reveals how the robot-biology feedback loop generates new hypotheses about corollary discharge and motor control that can be tested in living animals.

Tagged as:

About the author call_made

CSN Podcasts

Both the triumphs of humanity and its most evil deeds have resulted from collaboration. In a time where humanity is required to aspire to the former and minimize the latter, the question arises of how collaboration arises and why it fails. Surprisingly, this phenomenon, so central to who we are, is not well understood. Hence, a collaborative effort is required to understand collaboration in its full biological, psychological, sociological, cultural, and economic complexity and to translate this understanding into operational impact. This series of podcasts is one step toward achieving these complementary goals. The Collaboration Podcast presents interviews with people who are central orchestrators of collaboration in various domains including business, government, science, art, health, sustainability, and the military. The discussions were conducted by Prof. Dr. Paul F.M.J. Verschure and members of the Program Advisory Committee of the Ernst Strungmann Forum on Collaboration (https://www.esforum.de/forums/ESF32_Collaboration.html) during 2021 and had the goal to sketch a map of opportunities, challenges, and obstacles in human collaboration. The forum took place in May 2022, and now we would like to share this series of interviews with a broader audience. The full report of the Forum will be published in 2023 by MIT Press. The podcast was produced by the Convergent Science Network (https://www.convergentsciencenetwork.org/). Context: The stability of social systems depends critically on realizing sustainable methods of “collaboration,” yet how and by which means collaboration is achieved is not clearly understood; neither are the conditions or processes that lead to its breakdown or failure. Collaboration can be understood as cooperation between agents toward mutually constructed goals. Part of the reason for our lack of understanding is that the phenomenon of collaboration is, by nature, a highly multidisciplinary problem, and effective research into its complexities has been difficult to achieve across the broad range of scientific and technical disciplines involved. The need for a fundamental understanding of collaboration, however, has become increasingly important. Not only does humankind demand answers as it attempts to address critical challenges at multiple scales (e.g., climate change, migration, enhanced automation, social and economic inequality), but ever-increasing technological and economic means of interconnecting people and societies are disrupting long-established, familiar patterns of how we interact. Radical technological changes that are ongoing have the potential to reshape collaboration in ways that are currently hard to predict or influence (e.g., by altering configurations in interaction, information creation, and modes of communication). On one hand, such changes could disrupt hitherto stable forms of collaboration by affecting critical communication channels and traditional roles, as can be observed in the rapidly changing patterns in governance, commerce, and social interaction. Conversely, technology could lead to the emergence of novel, successful forms of collaboration that deviate from traditional “hierarchical” architectures. Evidence of this can be seen in areas as diverse as highly automated manufacturing plants, the open science movement, collaborative software repositories, user-centered services, and the sharing of economy-based modes of organization. Without a fundamental understanding of the mechanisms, processes, and boundary conditions of collaboration, it is not possible to evaluate or predict which of these possible scenarios are sustainable or even plausible. The Forum “How Collaboration Arises and Why it Fails” (May 8–13, 2022, Location: Frankfurt am Main, Germany) Chairs: Andreas Roepstorff and Paul Verschure Program Advisory Committee: Jenna Bednar, Julia R. Lupp, Bhavani R. Rao , Andreas Roepstorff, Ferdinand von Siemens, and Paul Verschure

More posts

Timestamp

  • fast_forward00:00:03 - This is the Convergent Science Network podcast. Leading researchers in the domain
  • fast_forward00:00:10 - of neuroscience, brain theory and technology are interviewed by Paul Vershoor and Tony Prescott.
  • fast_forward00:00:18 - Okay, so this is Paul Vershoor with the BCBT Summer School and the CSN podcast
  • fast_forward00:00:24 - here with Joe Ayers, one of our speakers.
  • fast_forward00:00:27 - And Joe, you started out telling us about a recent report of DARPA that was
  • fast_forward00:00:35 - telling us that algorithmic control actually doesn't really work very well.
  • fast_forward00:00:40 - So that sounds a bit surprising because most of the world is running on different
  • fast_forward00:00:45 - algorithms that are implemented in various ways on digital hardware. So what's the problem?
  • fast_forward00:00:49 - Well, the fundamental issue is that if you try to control a robot algorithmically,
  • fast_forward00:00:54 - you have to basically anticipate every possible situation it's going to be in
  • fast_forward00:00:59 - and program an explicit escape strategy for every situation.
  • fast_forward00:01:04 - If you can imagine what a lobster does in the bottom of the ocean,
  • fast_forward00:01:07 - I'd be impressed, or an octopus, etc.
  • fast_forward00:01:10 - And as a result, that's kind of a futile exercise.
  • fast_forward00:01:14 - And the robots end up
  • fast_forward00:01:17 - getting tested on very constrained environments and if
  • fast_forward00:01:20 - you remember the darpa challenge in the first year the
  • fast_forward00:01:24 - robots got a couple of hundred yards and that was it and then the second year
  • fast_forward00:01:28 - they made it to the finish well they changed the rules and so it was possible
  • fast_forward00:01:33 - to do that you know right and uh so but animals are not in a position to anticipate
  • fast_forward00:01:39 - what the world is going to be like in general, unless they're very territorial.
  • fast_forward00:01:43 - So any animal that migrates or goes into new places is going to have to have
  • fast_forward00:01:49 - an adaptive strategy that enables them to do that.
  • fast_forward00:01:52 - Now, if you watch most animals, when they get stuck, what they do is they wiggle and squirm.
  • fast_forward00:01:57 - And what they're doing is they're clearly exploring their full parameter space.
  • fast_forward00:02:02 - And we think that what they're doing is they're increasing the level of chaos
  • fast_forward00:02:06 - in the networks that would generate things like navigation and locomotion.
  • fast_forward00:02:10 - And the chaotic variations on the locomotion and navigation are the wiggling and squirming.
  • fast_forward00:02:17 - Okay, but now we've made quite some steps forward. This is really a fast forward, right?
  • fast_forward00:02:21 - Yeah, yeah. So now we have to sort of fast rewind and then revisit some of these
  • fast_forward00:02:25 - issues because in some sense you're saying, well, these algorithmic approaches
  • fast_forward00:02:29 - might be nice in a controllable world, but as soon as you start to think about
  • fast_forward00:02:32 - real world behavior as an animal behavior,
  • fast_forward00:02:35 - we face problems. Unpredictable environments.
  • fast_forward00:02:38 - Right. So this is the sticking point there, right? So then if we talk about
  • fast_forward00:02:43 - unpredictability in the world and the adaptation that sort of evolution has
  • fast_forward00:02:48 - generated to this kind of unpredictability,
  • fast_forward00:02:50 - what are the key tricks there?
  • fast_forward00:02:53 - So you say chaos, but what do you really mean? Well, I think another approach
  • fast_forward00:02:57 - is critter camps, where you put a camera on the back of the animal and see what
  • fast_forward00:03:01 - the world looks like for them.
  • fast_forward00:03:03 - Okay, what do you see? And that's going to give you a little more insight.
  • fast_forward00:03:06 - We do that on lobsters all the time. And what we're trying to establish is what
  • fast_forward00:03:11 - sense organs they're contacting the environment with and the patterns of contact
  • fast_forward00:03:16 - that exist so we can basically tune up our sensors,
  • fast_forward00:03:19 - which are things like antennae and bump sensors on claws and things like that.
  • fast_forward00:03:24 - So what kind of sensors do we have on a lobster? Because this is one of your
  • fast_forward00:03:28 - favorite preparations.
  • fast_forward00:03:29 - Yeah, yeah. Yeah. Well, most people have focused on joint receptors and joint
  • fast_forward00:03:33 - receptors tell the animal something about the angle of the joints and the action of the muscles.
  • fast_forward00:03:39 - But when you really see what happens, for example, on a claw of a lobster,
  • fast_forward00:03:44 - when it bumps into something, it moves all the joints at once.
  • fast_forward00:03:48 - So it's getting some signal from all the things, which I think is just a bump It's a lump response.
  • fast_forward00:03:55 - But if it comes from the side as opposed from the front, it's going to have
  • fast_forward00:04:00 - a slight variation, which is going to give the animal some subtle information
  • fast_forward00:04:04 - about where this insult is coming from. But for that to work, we need some…,
  • fast_forward00:04:10 - corollary discharge for an efference copy, because the animal must be knowing what it wants to do.
  • fast_forward00:04:16 - Then it detects some perturbation on its effectors that make these joints sort
  • fast_forward00:04:23 - of bent in different ways than it expects.
  • fast_forward00:04:25 - Yeah, yeah, exactly. And that is in the collision detection.
  • fast_forward00:04:27 - Would you agree with that?
  • fast_forward00:04:28 - Well, as to whether they're using a comparator with efference copy with a sensory input, I don't know.
  • fast_forward00:04:37 - I don't know of any real examples. But if they're walking, imagine here we have
  • fast_forward00:04:41 - a lobster, right? You're walking, you're a lobster. Yeah, yeah, yeah.
  • fast_forward00:04:44 - Okay, so I'm moving my legs, and now one of my legs hits a rock while I'm walking.
  • fast_forward00:04:49 - So part of the changes in my joint angles are due to my own walking movements I'm initiating.
  • fast_forward00:04:54 - Yeah, yeah. But now on a few of my
  • fast_forward00:04:56 - joints, this joint angle will come out differently because of an obstacle.
  • fast_forward00:05:00 - Mm-hmm. So how do we know this is an obstacle and not me walking?
  • fast_forward00:05:05 - Well, we don't deal with that. Yeah. I mean, I went through originally a real
  • fast_forward00:05:09 - plan to have joint receptors and having them feedback.
  • fast_forward00:05:14 - And I had a whole set of reflex pathways.
  • fast_forward00:05:17 - But before we put those on the robot, we actually tried the robot without them
  • fast_forward00:05:23 - over some very irregular substrates, cobble fields and stuff like that. And they do just fine.
  • fast_forward00:05:29 - And one of the things about a
  • fast_forward00:05:31 - lobster weighs about one-eighth its weight underwater as it would in air.
  • fast_forward00:05:37 - So a seven-pound lobster weighs just a few ounces underwater.
  • fast_forward00:05:42 - And the actual forces against gravity are very, very small, especially when
  • fast_forward00:05:48 - compared with a lateral hydrodynamic resistance to flow.
  • fast_forward00:05:51 - So when they get in surge, they're getting a lot more action from the surge than they are from.
  • fast_forward00:05:58 - Perturbations of their orientation relative to gravity and the
  • fast_forward00:06:01 - muscles tend to be pretty compliant so you
  • fast_forward00:06:05 - know right now on the newest generation
  • fast_forward00:06:09 - of lobster robot that we're building i have no plans to put limb proprioceptors
  • fast_forward00:06:13 - at all okay all right so this is interesting right so so what you've done why
  • fast_forward00:06:18 - don't you be studying a lobster in great detail yeah and then in order to let's
  • fast_forward00:06:23 - say understand that loves you've been building robot lobsters Yeah.
  • fast_forward00:06:26 - So how many generations of robot lobsters have you built? We're now on the fourth generation.
  • fast_forward00:06:31 - So when did you build the first one? How long ago was that? The first one was started in 1998.
  • fast_forward00:06:37 - Okay. So how did this system now progress? What's the difference between version
  • fast_forward00:06:42 - one versus version four?
  • fast_forward00:06:44 - Okay. Version one was the infinite power, infinite bandwidth variety.
  • fast_forward00:06:48 - Okay. And it was basically a set of legs on a hull.
  • fast_forward00:06:52 - It had a claw and tail on it. and then it was controlled by an external computer
  • fast_forward00:06:58 - through a serial interface.
  • fast_forward00:06:59 - And what we would do is send byte commands to some control boards and the byte
  • fast_forward00:07:06 - camp commands would say, turn on this muscle at a particular frequency or turn it off.
  • fast_forward00:07:13 - And we would be sending these byte commands over a serial line.
  • fast_forward00:07:18 - And that robot performed really quite well and that we got the basic patterns
  • fast_forward00:07:24 - of coordination working and stuff like that.
  • fast_forward00:07:27 - The second robot had an onboard computer, but it also had the ability to be
  • fast_forward00:07:33 - controlled through a serial line, so we could go either way.
  • fast_forward00:07:36 - And then it had onboard power.
  • fast_forward00:07:38 - So it was carrying the full mass.
  • fast_forward00:07:41 - And what we do is just compensate for mass with buoyancy.
  • fast_forward00:07:46 - So we would put syntactic foam on it so it had about the same mass as a normal
  • fast_forward00:07:51 - seven-pound lobster underwater.
  • fast_forward00:07:53 - And now version four? So version four is totally different.
  • fast_forward00:08:00 - It has basically the same mechanical system, except the basilar joint,
  • fast_forward00:08:06 - which used to be vertical, is now candid at a 45-degree angle.
  • fast_forward00:08:10 - And that gives the limb tip a rolling action like a wheel as opposed to a more rectilinear motion.
  • fast_forward00:08:17 - And that's also consistent with the morphology of the real lobster?
  • fast_forward00:08:21 - It's exactly like a real lobster. That's exactly the angle that a real lobster's
  • fast_forward00:08:24 - leg is at. And what's the advantage of that? The advantage of that is it lets the animal rear back.
  • fast_forward00:08:29 - So if it's trying to walk up a slope or down a slope, it has a more natural
  • fast_forward00:08:35 - angle of attack of the leg.
  • fast_forward00:08:37 - Okay. So now you emphasize very much just the legs.
  • fast_forward00:08:42 - Um what kind of sensors are you considering well we
  • fast_forward00:08:45 - have again from one to four yeah yeah right uh well
  • fast_forward00:08:49 - the basic sensor suite we have on is a compass we
  • fast_forward00:08:52 - have a pitch and roll inclinometer that is
  • fast_forward00:08:55 - also an accelerometer and that gives
  • fast_forward00:08:57 - us basically three axes of pitch roll
  • fast_forward00:09:01 - and yaw as well well as um the rate
  • fast_forward00:09:05 - of change of pitch roll in the off and then
  • fast_forward00:09:08 - we have bump sensors on the
  • fast_forward00:09:11 - claws and they basically have an accelerometer they full wave rectify the signal
  • fast_forward00:09:18 - low pass filter it and then create a square wave that indicates a bump and that
  • fast_forward00:09:24 - has a duration associated with it so it has a little bit of memory of having had bumped.
  • fast_forward00:09:30 - And then we have antennae, and the antennae have bin sensors in them.
  • fast_forward00:09:38 - And the antennae can be deployed at different angles, so they can be held straight
  • fast_forward00:09:43 - out in front of the animal or off to the sides laterally.
  • fast_forward00:09:46 - And their bin gives us a very quantifiable measure of the flow rate of the ocean around it.
  • fast_forward00:09:53 - And we're also putting optical flow sensors on the new robot.
  • fast_forward00:09:58 - So it can use optical flow information and compare that with a hydrodynamic
  • fast_forward00:10:02 - flow information and get a clearer picture of things that might be ambiguous with one sensor.
  • fast_forward00:10:09 - Okay, so if we talk about sensor processing now, are you saying that also lobsters use optic flow?
  • fast_forward00:10:15 - Oh, no question they do. Really? Yeah, I published a paper on that in Science in 1972.
  • fast_forward00:10:21 - I was barely born.
  • fast_forward00:10:26 - So they they often live in
  • fast_forward00:10:29 - rather murky and dark environments yeah so how much
  • fast_forward00:10:32 - help but their light their eyes are very are very sensitive in low light okay
  • fast_forward00:10:36 - so they can and viresma showed years ago that they have unidirectional optical
  • fast_forward00:10:42 - flow sensors and we work with jeff barrows from sentai on the RoboBee program,
  • fast_forward00:10:49 - and we're going to adapt the sensors from that program on the robot lobster.
  • fast_forward00:10:54 - Right. And on the RoboLamprey, for that matter. Right. But then we're skipping ahead, right? Yeah.
  • fast_forward00:11:00 - So now we have our lobster. We have the sensors. We have some of the sensors. Now we have the walking.
  • fast_forward00:11:09 - Now how about the control? Okay, there's where the big difference is.
  • fast_forward00:11:12 - So the new lobster, rather than being controlled by what was effectively a state
  • fast_forward00:11:19 - machine that mimicked the operation of central pattern generators,
  • fast_forward00:11:23 - it's now controlled by true central pattern generators that are formed from
  • fast_forward00:11:27 - what are called discrete-time map-based neurons.
  • fast_forward00:11:30 - And discrete-time map-based neurons were developed by Nikolai Rukov,
  • fast_forward00:11:35 - my colleague from the Institute for Nonlinear Science at UC San Diego.
  • fast_forward00:11:39 - And these are neurons that are a one-dimensional map that are a,
  • fast_forward00:11:44 - oh, what's the word for it? I'm blanking on this.
  • fast_forward00:11:52 - A phonological model of neurons. Right.
  • fast_forward00:11:55 - Okay, so they try to capture the dynamics of neurons, and they have two control
  • fast_forward00:12:00 - parameters that let us change the neurons from being either truly spiking neurons
  • fast_forward00:12:06 - that fire in tonic spiking patterns or bursting patterns,
  • fast_forward00:12:11 - or we can configure the two control parameters so they become chaotic.
  • fast_forward00:12:16 - Okay, but now can you control the spike frequency quite easily,
  • fast_forward00:12:21 - and the burst, and the bursting? Oh yeah, really. Yeah, there's another parameter,
  • fast_forward00:12:25 - which is the synaptic current.
  • fast_forward00:12:27 - So we can modify the synaptic current.
  • fast_forward00:12:31 - And we can modify that parametrically as would occur during neuromodulation,
  • fast_forward00:12:36 - or we can apply regular synaptic pulses from other neurons.
  • fast_forward00:12:41 - Okay, so how do you build a central pattern generator with that?
  • fast_forward00:12:44 - Well, the neurons are basically a...
  • fast_forward00:12:50 - Two equations, and the equations have some fuzzy logic over different ranges of membrane voltage.
  • fast_forward00:12:57 - And those equations represent the voltage in cycle n plus one as a function of voltage in cycle n.
  • fast_forward00:13:08 - And you loop through cycle by cycle and keep calculating the voltage in cycle
  • fast_forward00:13:15 - n plus one, and that turns out to be the voltage of the neuron,
  • fast_forward00:13:18 - the transmembrane voltage.
  • fast_forward00:13:20 - So by changing the rate, you can speed them up. So we need to speed them up,
  • fast_forward00:13:23 - for example, in RoboBee, but we can run them quite slowly in RoboLobster.
  • fast_forward00:13:28 - So these elements are intrinsically oscillating. Yeah. Okay,
  • fast_forward00:13:32 - so that's what you're exploiting then.
  • fast_forward00:13:33 - Well, by varying these two control parameters, alpha and sigma,
  • fast_forward00:13:37 - we can put them into a bursting regime.
  • fast_forward00:13:39 - Or for postural behavioral practice, we can put them in a spiking regime.
  • fast_forward00:13:43 - But now to, for instance, get coordinated movement of the six legs,
  • fast_forward00:13:47 - you have to also coordinate among these oscillators.
  • fast_forward00:13:51 - Yeah, and we have coordinating neurons that do that.
  • fast_forward00:13:55 - So the coordinating neurons pass information from a governing oscillator to
  • fast_forward00:13:58 - a governed oscillator so that the governed oscillator maintains the proper phase
  • fast_forward00:14:03 - with the governing oscillator to maintain a gate,
  • fast_forward00:14:06 - which is an attempt to create a pattern of footfall support to keep the thing
  • fast_forward00:14:11 - stable in the pitch and roll plane. Okay.
  • fast_forward00:14:13 - And how well is then such a control model validated in biological terms?
  • fast_forward00:14:20 - Do you have, let's say, analogs in the lobster nervous system somewhere that
  • fast_forward00:14:25 - would match, for instance, to this master oscillator that controls these sub-oscillators?
  • fast_forward00:14:29 - Do you have examples of that? Yeah, so the model is based initially off dynamical
  • fast_forward00:14:36 - analysis that I did from electromyograms in behaving lobsters in the 70s.
  • fast_forward00:14:41 - And then one of Al's, I mean, a Franco-Clarac student named Abraham Shrashri
  • fast_forward00:14:51 - did paired recording of many neurons in the thoracic ganglia,
  • fast_forward00:14:55 - worked out the synaptic network,
  • fast_forward00:14:58 - and we tried to capture that synaptic network in the network model that we built with the neurons.
  • fast_forward00:15:03 - So the neurons are connected by synapses, and the synapses are chemical synapses
  • fast_forward00:15:10 - that take into account the presynaptic voltage,
  • fast_forward00:15:13 - the postsynaptic voltage, and inject current appropriate to the difference between
  • fast_forward00:15:18 - pre- and postsynaptic voltages. Okay.
  • fast_forward00:15:20 - But then in some sense, I could argue, look, that's nice and well,
  • fast_forward00:15:24 - but it might just be a very phenomenological model that is sort of at the functional
  • fast_forward00:15:29 - end gives you something that is lobster-like walking.
  • fast_forward00:15:33 - But how do I kind of know more specifically that this is actually informing us about the lobster?
  • fast_forward00:15:38 - How does it help us understand what lobsters really do and how their brains work?
  • fast_forward00:15:43 - Well, for all intents and purposes, the neurons operate in the same patterns
  • fast_forward00:15:47 - that you would see from electromyograms.
  • fast_forward00:15:49 - So the timing, the patterns of output that we see are quite indistinguishable
  • fast_forward00:15:54 - from the patterns that you would see in behaving animals under the same circumstances.
  • fast_forward00:15:59 - They don't capture the details of the conductance mechanisms, okay?
  • fast_forward00:16:04 - But if we tried to use parallel conductance theory,
  • fast_forward00:16:08 - we'd have to use differential equations given the
  • fast_forward00:16:11 - fact that some of these neurons probably have five or six currents there
  • fast_forward00:16:15 - would probably be 10 or 12 differential equations for each
  • fast_forward00:16:18 - neuron and we wouldn't be able to model very many on a real-time processor right
  • fast_forward00:16:22 - we're using a model that's based on difference equations and as a result we
  • fast_forward00:16:27 - can do hundreds of neurons and synapses on a single digital signal processing
  • fast_forward00:16:32 - chip right but we don't program them as.
  • fast_forward00:16:38 - Whereas algorithmically, we program them by wiring up networks,
  • fast_forward00:16:41 - and we establish the dynamics of the neurons by putting these two control parameters,
  • fast_forward00:16:47 - alpha and sigma, in the appropriate range.
  • fast_forward00:16:49 - So now, have you found a correlate in the lobster nervous system of these two
  • fast_forward00:16:52 - control parameters? What would they be?
  • fast_forward00:16:56 - Again, these are phenomenological models, and there's no one-for-one mapping
  • fast_forward00:17:00 - of these phenomenological control parameters on any ionic conductance parameter.
  • fast_forward00:17:06 - And that's something that at UCSD they spent several years, and as Henry or
  • fast_forward00:17:12 - Bob and I would put it, there were a lot of bodies out in the hall trying to solve this problem.
  • fast_forward00:17:17 - And the bodies were decomposing.
  • fast_forward00:17:20 - Exactly.
  • fast_forward00:17:23 - Let this be a warning. So now we are at level thoracic ganglia, right?
  • fast_forward00:17:28 - Yeah. So we're controlling these legs, they're moving, we have a neural model
  • fast_forward00:17:32 - to do that. But actually, most of the bulk of the nervous system of this animal
  • fast_forward00:17:37 - sits above these thoracic ganglia.
  • fast_forward00:17:38 - Yeah, yeah. So what are they doing in your robot?
  • fast_forward00:17:42 - Okay, so we have a lot of exteroceptive reflexes that we've layered in what
  • fast_forward00:17:49 - would be the brain of this robot.
  • fast_forward00:17:52 - And, you know, there's been a lot of talk in this meeting about Breitenberg machines.
  • fast_forward00:17:57 - Well, I don't know of any alternative to a Breitenberg machine, you know.
  • fast_forward00:18:02 - Neurons receives input on one side or the other.
  • fast_forward00:18:06 - They project to one side or the other, and they can either project to the same
  • fast_forward00:18:11 - side or they can decussate to the opposite side.
  • fast_forward00:18:14 - There aren't a whole lot of alternatives in a bilaterally symmetrical nervous system.
  • fast_forward00:18:18 - So we have layered reflexes for optical flow.
  • fast_forward00:18:24 - We have layered reflexes for hydrodynamic flow, for bump, for deviations in
  • fast_forward00:18:30 - the pitch and roll plane and we have...
  • fast_forward00:18:35 - As we layer on more and more sensors, and very soon we'll have some very good
  • fast_forward00:18:39 - chemical sensors, we'll be able to begin to layer those sorts of reflexes on
  • fast_forward00:18:44 - top of these more fundamental reflexes. But now, which structures of that brain are you modeling?
  • fast_forward00:18:49 - Do you, for instance, approach the optical lobes of the lobster brain?
  • fast_forward00:18:54 - Are these also modeled at the neural level? We're not modeling the… So,
  • fast_forward00:18:59 - this is very interesting.
  • fast_forward00:19:02 - We operate at the level of what we call releasing mechanisms.
  • fast_forward00:19:07 - So our sensors are typical analog electrical sensors.
  • fast_forward00:19:12 - And then what we do with them is we create a spiking discharge,
  • fast_forward00:19:16 - typically range fractionated, where we take an input variable that might be
  • fast_forward00:19:22 - the amount of bending of the right antennae.
  • fast_forward00:19:25 - And then we create from that a set of interneurons that are recruited in order
  • fast_forward00:19:31 - of size that represent the magnitude of that input variable.
  • fast_forward00:19:36 - So if it were, say, if the antennae were held out to directly at right angles
  • fast_forward00:19:42 - to the long body axis, as flow came from the front, at low flow rates,
  • fast_forward00:19:48 - the antennae would be bent a little bit.
  • fast_forward00:19:51 - As the flow rates increased, they'd be bent more and more.
  • fast_forward00:19:54 - Sure. And we usually quantize these with about three levels of range fractionation,
  • fast_forward00:20:00 - which is fairly typical for the receptors we see in these animals.
  • fast_forward00:20:05 - Okay, but then we're really at the periphery of the animal, right? Yeah, yeah, yeah.
  • fast_forward00:20:10 - So what's the difference between the structure of, let's say,
  • fast_forward00:20:15 - a lobster brain and a drosophila brain?
  • fast_forward00:20:19 - If you look at the key structures, are these roughly similar?
  • fast_forward00:20:22 - Similar no no there are no mushroom bodies in the
  • fast_forward00:20:25 - lobster okay okay so so lobsters
  • fast_forward00:20:29 - have a series of four um um
  • fast_forward00:20:32 - there's a lamina ganglionaris a medulla externa medulla interna and a medulla
  • fast_forward00:20:39 - terminalis which are the four integrative layers going from the omentidia into
  • fast_forward00:20:44 - the optic nerve projecting in the central nervous system these are where virsma
  • fast_forward00:20:48 - used to be doing pin recordings in crayfish fish,
  • fast_forward00:20:51 - and crabs to establish all the six different types of interneurons that come in from the eye.
  • fast_forward00:20:58 - The thing that really distinguishes crustaceans from, say, humans is that we
  • fast_forward00:21:04 - just have one kind of, or two kinds of ganglion cells.
  • fast_forward00:21:07 - We have the on-center, off-surround, and the off-surround, on-center.
  • fast_forward00:21:11 - And these animals have, for example, fibers that Wiersma used to call,
  • fast_forward00:21:18 - oh, I'm blanking on these names now. So what did he call them?
  • fast_forward00:21:25 - Sustaining fibers which would respond to an
  • fast_forward00:21:28 - increase in illumination in one area there were
  • fast_forward00:21:31 - dimming fibers that would respond to a decrease in in
  • fast_forward00:21:34 - illumination in an area there were what he called unidirectional motion fibers
  • fast_forward00:21:40 - they're what he called space constant fibers and then he had a kind of fiber
  • fast_forward00:21:45 - he called seeing fiber in which these things would respond to things like movement
  • fast_forward00:21:50 - of the contralateral legs,
  • fast_forward00:21:51 - images in the contralateral eye.
  • fast_forward00:21:54 - He described their behavior as complex as the behavior of the whole animal.
  • fast_forward00:21:58 - And these would be like whole field. Yeah. And he also had another class of
  • fast_forward00:22:02 - fiber called jittery movement fibers, which we would call in a frog bug detectors.
  • fast_forward00:22:08 - Right. Okay. And so the thing about lobsters is that they have 30 or 40% of
  • fast_forward00:22:13 - their central neurons in their brain are out in their eye stalks doing optical processing.
  • fast_forward00:22:18 - Okay. And the number of central cell bodies within the brain itself are confined
  • fast_forward00:22:23 - largely to the motor neurons that go to the head appendages and then some releasing
  • fast_forward00:22:29 - mechanisms that respond to input from the statuses,
  • fast_forward00:22:33 - from the antennules, and from the antennae.
  • fast_forward00:22:36 - So that would mean that relatively little hardware is dedicated to olfaction?
  • fast_forward00:22:43 - In the lobster, you know, that's Barry Aki's world.
  • fast_forward00:22:48 - And chuck derby's world they focus on that and um you know we really haven't
  • fast_forward00:22:54 - gotten into that yet and as we get more of a capability of building sensors
  • fast_forward00:23:00 - using um synthetic biological approaches,
  • fast_forward00:23:04 - then we're going to start working in that area but that's that's right now something
  • fast_forward00:23:09 - i've just been funded to do and we really haven't started yet okay but now so
  • fast_forward00:23:14 - um so you started this work Or actually, you're a physiologist, right?
  • fast_forward00:23:20 - Yeah, I'm a systems neurophysiologist. Right. So at some point,
  • fast_forward00:23:24 - you decided to put this robot lobster together.
  • fast_forward00:23:28 - And I guess it was with the ambition to actually understand the real lobster.
  • fast_forward00:23:31 - Well, no. The way this all started is very interesting.
  • fast_forward00:23:36 - So I had spent a lot of time working on sea lamprey.
  • fast_forward00:23:42 - And we were interested in how they recover from spinal cord injuries.
  • fast_forward00:23:46 - And um we were successful at identifying how they recover the ability to turn on swimming.
  • fast_forward00:23:55 - And then the next grant review i put out um i got a review that said uh this
  • fast_forward00:24:01 - is very fundable work if you do it in a in a mammal and i don't do that you
  • fast_forward00:24:07 - know i'm you're not going to see me working on mice and rats, I guarantee you.
  • fast_forward00:24:11 - Uh, and so I decided to go back to the lobster and,
  • fast_forward00:24:17 - And this was at a time when people were really beginning to establish a really
  • fast_forward00:24:23 - incredible library of neuromodulatory substances in the stomatogastric ganglion.
  • fast_forward00:24:28 - And I think at that point, there were about 35 known substances that would alter
  • fast_forward00:24:36 - the motor output patterns generated by the stomatogastric ganglion. them.
  • fast_forward00:24:40 - So, um, working with Al Silverston and George Heinzel, um,
  • fast_forward00:24:47 - we decided to try to get a handle on which of those were really operating in
  • fast_forward00:24:53 - lobsters and which were artifactual because for them to be 35 different modulatory
  • fast_forward00:25:00 - substances was a little bit over the top.
  • fast_forward00:25:03 - So I got together with a company called Massa Products,
  • fast_forward00:25:09 - and we started developing a sonar biotelemetry system, which was a physiological
  • fast_forward00:25:14 - telemetry system, where we would record from the muscles that are controlled
  • fast_forward00:25:19 - by the stomatogastric ganglion.
  • fast_forward00:25:22 - And many of these muscles have only one motor neuron.
  • fast_forward00:25:25 - Or I think max, there's four motor neurons in a muscle.
  • fast_forward00:25:31 - And if you are studying feeding in
  • fast_forward00:25:34 - a lobster the lobster feeds when you feed it
  • fast_forward00:25:37 - what you feed it it doesn't have a chance to
  • fast_forward00:25:40 - select the chinese food or the italian food and it
  • fast_forward00:25:43 - doesn't have a chance to pick when it's going to eat so if you really want to
  • fast_forward00:25:46 - see what the normal patterns of operation of the stomatogastric ganglion are
  • fast_forward00:25:50 - you have to do it in freely behaving animals so the goal of this project was
  • fast_forward00:25:55 - to record for the muscles controlled by this ganglion in animals that were free
  • fast_forward00:26:00 - to run around in the world.
  • fast_forward00:26:02 - And the idea is that we took electromyographic recordings from these muscles,
  • fast_forward00:26:07 - did some signal processing, and then would transmit using sonar the on times
  • fast_forward00:26:13 - and off times of these different muscles.
  • fast_forward00:26:16 - And I created a webpage about this and got some funding from the Office of Naval Research.
  • fast_forward00:26:24 - And I got a phone call from a program officer at DARPA,
  • fast_forward00:26:28 - and he had seen this website and wanted to know if we could use this to take
  • fast_forward00:26:36 - control of a giant lobster to use the lobster for remote sensing purposes.
  • fast_forward00:26:43 - And he and I conversed for quite a while, and I told him that it was really
  • fast_forward00:26:48 - my belief that if you try to control an animal, it's going to do what it damn
  • fast_forward00:26:53 - well pleases. You're not going to have much luck doing this with a lobster.
  • fast_forward00:26:57 - And I personally thought it was easier to build a robotic lobster.
  • fast_forward00:27:01 - And I had been listening to Randy Beer talk about how they were building robotic insects.
  • fast_forward00:27:07 - And this looked like a pretty interesting endeavor to me.
  • fast_forward00:27:10 - Well, he took me very seriously and asked me how I would do this.
  • fast_forward00:27:15 - And I developed some ideas.
  • fast_forward00:27:18 - And then he asked me to write a proposal. And he gave me a considerable sum
  • fast_forward00:27:24 - of money to build a robotic lobster.
  • fast_forward00:27:26 - So I went out and hired a bunch of engineers.
  • fast_forward00:27:31 - And I had a very interesting experience at this point because I found that the
  • fast_forward00:27:38 - engineers that I was working with could be divided into two categories.
  • fast_forward00:27:43 - One were guys that knew how to make stuff work. And the other were what I called
  • fast_forward00:27:47 - experts on what's impossible.
  • fast_forward00:27:49 - And they turned out to be kind of lethal because no matter what you wanted to
  • fast_forward00:27:55 - build, they would find some reason why in principle it couldn't work.
  • fast_forward00:27:59 - And I found myself in situations where we would have something working and some
  • fast_forward00:28:05 - of the participating engineers would tell me that that couldn't possibly be happening.
  • fast_forward00:28:08 - And so I found it quite necessary to weed out this crew in order to be successful.
  • fast_forward00:28:15 - And that was the genesis of the lobster robot. Okay.
  • fast_forward00:28:18 - So here we have your neurophysiology, your system's neurophysiology.
  • fast_forward00:28:23 - Now we have sort of the robot lobster.
  • fast_forward00:28:26 - But now in retrospect, because you're doing this now for quite some time,
  • fast_forward00:28:30 - has it helped you in any way to understand the biological system?
  • fast_forward00:28:34 - Oh, very much so. Or if that was just a game, would it have been better to stick
  • fast_forward00:28:38 - to the neurophysiology? Yeah, yeah, yeah, yeah.
  • fast_forward00:28:41 - No, I think what really happens when you try to build a complete system is you
  • fast_forward00:28:46 - very quickly identify the lacunae in your knowledge.
  • fast_forward00:28:51 - And that really gives you some ideas on what you should really be looking for.
  • fast_forward00:28:55 - So for example i have right now a student
  • fast_forward00:28:58 - in my laboratory that's doing recordings from the
  • fast_forward00:29:02 - brain connectives going from the brain down to the lower
  • fast_forward00:29:04 - ganglia to look at the patterns of discharge of some of the the systems that
  • fast_forward00:29:10 - we can use to inform the way we drive the systems in our electronic nervous
  • fast_forward00:29:15 - system okay so that's really a clear finding from doing that but,
  • fast_forward00:29:23 - The other thing that building the robot lobster informed me on was this idea of bump sensing.
  • fast_forward00:29:31 - You know, if you look at the literature on crustaceans, everybody gets so worried
  • fast_forward00:29:35 - about what inner neurons come from what joint.
  • fast_forward00:29:39 - And I think a lot of these inner neurons are just responding to gross disturbances
  • fast_forward00:29:44 - of the sort that occur when the claw hits a rock, for example.
  • fast_forward00:29:48 - But how would that be achieved? I mean, is that a direct linking to some mechanoreceptor,
  • fast_forward00:29:54 - or is it more complicated?
  • fast_forward00:29:57 - We're not in a position to create cortitonal organs or bipolar neurons.
  • fast_forward00:30:02 - So we do this by using cheap analog sensors, and then we take either a small
  • fast_forward00:30:11 - PIC microcontroller or something like that, and create what we call a releasing mechanism.
  • fast_forward00:30:16 - And the output of that releasing mechanism are patterns of neuronal activity,
  • fast_forward00:30:21 - which form the input of our sensors.
  • fast_forward00:30:23 - Right. Okay. So then in your field tests, these robots have shown to be pretty
  • fast_forward00:30:28 - robust, which is actually remarkable because you could say, look,
  • fast_forward00:30:32 - this is in some of the minimal amount of control you could give them.
  • fast_forward00:30:37 - And then you give them this sort of chaos-based way to escape from local minima
  • fast_forward00:30:41 - when they're stuck in whatever, between obstacles or whatever.
  • fast_forward00:30:46 - And it seems to be sufficient, at least that's what it looks like,
  • fast_forward00:30:49 - to give you fairly robust behavior.
  • fast_forward00:30:52 - But where does it actually, what are the weaknesses? Are there weaknesses in
  • fast_forward00:30:56 - this approach? Does it really get stuck somewhere?
  • fast_forward00:30:58 - Are there problems it cannot solve? Yeah, yeah. Well, deciding how you're stuck
  • fast_forward00:31:03 - is one of the problems that we're working on right now.
  • fast_forward00:31:06 - And we do this with accelerometry. a tree. So what we do is we take a behaving animal,
  • fast_forward00:31:14 - And we look at the output of an accelerometer in real time, and we ask ourself,
  • fast_forward00:31:20 - what is the pattern of acceleration you would see when a lobster normally starts walking forwards?
  • fast_forward00:31:27 - Now, if you tell your robot lobster to walk forwards, and you see a different
  • fast_forward00:31:34 - pattern of acceleration, that's a pretty good indication you're stuck.
  • fast_forward00:31:37 - If you back up and you see the normal pattern of acceleration,
  • fast_forward00:31:42 - that's a pretty good indication that you aren't impeded in this direction.
  • fast_forward00:31:46 - So by comparing the patterns of movement in response to a command-initiated
  • fast_forward00:31:51 - behavior with the actual movement, we can determine whether we're stuck or not.
  • fast_forward00:31:56 - So that's in fact one of the areas of research that we're really going to get
  • fast_forward00:32:00 - into big time with the fourth generation robot.
  • fast_forward00:32:03 - Okay but but then then we are back at something like
  • fast_forward00:32:06 - a corollary discharge to make that happen there there's yeah
  • fast_forward00:32:09 - yeah yeah and in fact i think the the
  • fast_forward00:32:12 - you hit the nail right on the head that the in fact what we would see in response
  • fast_forward00:32:19 - to normal acceleration would be the corollary discharge exactly yeah yeah all
  • fast_forward00:32:25 - right but i think that would more reflect what the command neuron would be doing
  • fast_forward00:32:29 - rather than the output of the CPG.
  • fast_forward00:32:32 - But still, it would need some internal model of how its own body is acting.
  • fast_forward00:32:37 - I think, well, the way we actually plan to do this, and this is something that
  • fast_forward00:32:42 - we are fooling around with, is to have a leaky integrator that receives input
  • fast_forward00:32:47 - from the command neuron.
  • fast_forward00:32:48 - And by adjusting the rise and fall time of the leaky integrator so it mimics
  • fast_forward00:32:54 - what the normal pattern of acceleration would be.
  • fast_forward00:32:56 - And then we can compare that with the actual pattern of acceleration but this
  • fast_forward00:33:02 - leaky integrator you now propose you sort of.
  • fast_forward00:33:08 - Do not care whether at this stage you know whether the real lobster has that
  • fast_forward00:33:12 - same leaky integrator or not.
  • fast_forward00:33:14 - Well, again here, the robot's going to inform what to look for in the real animal. Exactly.
  • fast_forward00:33:20 - And then that's the kind of experiment we will continue to do in the real animals.
  • fast_forward00:33:24 - Right. So I'm always going to have with every model I work with somebody doing
  • fast_forward00:33:29 - biology and somebody doing robotics.
  • fast_forward00:33:31 - And they're going to inform each other. Yeah, well, it's interesting because
  • fast_forward00:33:33 - in analyzing all the history of the project, it's more that the robot is based
  • fast_forward00:33:38 - on, let's say, some informed imagination that you test in the biology.
  • fast_forward00:33:41 - Yeah. But not that the biology has made concrete suggestions of what to do on
  • fast_forward00:33:45 - the robot, except to give it six legs and a claw.
  • fast_forward00:33:48 - Yeah, well, I mean, I don't think we, before we started building robots,
  • fast_forward00:33:52 - that we really knew what to look for as well as we do after having built robots. Okay.
  • fast_forward00:33:56 - Okay, so it really helps you, right? Yeah, yeah. But now, in some sense,
  • fast_forward00:34:01 - this approach has been extremely successful because you have been now expanding
  • fast_forward00:34:05 - into many other projects, right?
  • fast_forward00:34:07 - And one of the issues, there's a great interest in these kinds of machines and
  • fast_forward00:34:11 - robots also because practical applications of, let's say, autonomous technology
  • fast_forward00:34:16 - are actually still fairly limited, right?
  • fast_forward00:34:19 - So you pointed out the reality of what it means to keep one of these drones up in the air, right?
  • fast_forward00:34:24 - So what's the problem there exactly? Exactly.
  • fast_forward00:34:28 - What's the problem with keeping drones going, and will we actually ever make them autonomous?
  • fast_forward00:34:33 - Well, most of the military robots right now are teleoperated.
  • fast_forward00:34:38 - And they're teleoperated primarily from the perspective that...
  • fast_forward00:34:47 - What's the best way to say this? This is complicated. First of all,
  • fast_forward00:34:51 - most of them are very large, expensive pieces of equipment. So,
  • fast_forward00:34:56 - a Predator, for example, is a big piece of equipment that you don't want falling
  • fast_forward00:35:00 - in somebody's backyard.
  • fast_forward00:35:01 - Nope. As a result, nobody attempts even to try to fly these autonomously.
  • fast_forward00:35:07 - And I don't really know what the details of this are.
  • fast_forward00:35:10 - I imagine we're hearing on the radio now that the pilot of a typical passenger
  • fast_forward00:35:17 - airplane is really only flying at about three minutes.
  • fast_forward00:35:20 - And the rest of the time it's on autopilot. So I guess the autopilot is autonomous
  • fast_forward00:35:26 - behavior that's very closely watched by a human operator.
  • fast_forward00:35:32 - But the risk management issues,
  • fast_forward00:35:36 - given the cost of these vehicles and the potential danger of them having a mishap,
  • fast_forward00:35:44 - has led the military to use teleoperation for almost every vehicle.
  • fast_forward00:35:49 - Now, once they get used to teleoperation, you know, they're not going to be.
  • fast_forward00:35:56 - So willing to play with autonomous behavior because they might lose a very expensive piece of equipment.
  • fast_forward00:36:02 - Now, the way we build these robots, probably the most expensive part outside
  • fast_forward00:36:08 - of the intellectual capital is the batteries.
  • fast_forward00:36:12 - So we can build vehicles where if you lose a few, it's all right.
  • fast_forward00:36:16 - And if we base the fabrication and our fourth generation robot,
  • fast_forward00:36:21 - is really designed for manufacture so it'll
  • fast_forward00:36:24 - be very cheap to produce these in quantity and and
  • fast_forward00:36:28 - we're already building four of them right now now when we used to our original
  • fast_forward00:36:34 - robots we built them one at a time right so we're just starting off with a mess
  • fast_forward00:36:39 - of them from the beginning so we won't feel so bad if one of them rides off
  • fast_forward00:36:43 - over the horizon yeah sure yeah so it's So the point is that,
  • fast_forward00:36:47 - so there's this great interest to have more,
  • fast_forward00:36:50 - let's say, a biomimetic approach to building these kinds of machines we can
  • fast_forward00:36:55 - use to identify or deal with agents of harm like landmines or missing nukes or what have you.
  • fast_forward00:37:03 - And a number of programs are underway in the US now to also realize these systems,
  • fast_forward00:37:08 - right? Like the MESS program from DARPA and so on.
  • fast_forward00:37:12 - So in that context you have now been expanding from let's say the lobster also into the insect right,
  • fast_forward00:37:20 - that was an artificial bee project so to what extent will the artificial bee
  • fast_forward00:37:26 - be a flying version of your lobster what's different to it.
  • fast_forward00:37:31 - I mean, I'm a comparative physiologist, and I don't think that we're ever going
  • fast_forward00:37:37 - to completely understand any organism.
  • fast_forward00:37:40 - But I think we're going to find general principles that are shared by broad
  • fast_forward00:37:45 - numbers of organisms that by using comparative physiology to figure out how
  • fast_forward00:37:51 - these things work in the most technically accessible species,
  • fast_forward00:37:55 - we can assemble a pretty complete library of control principles that that apply to all arthropods.
  • fast_forward00:38:02 - So, I mean, that's the sort of approach I take. You know, I don't think we're
  • fast_forward00:38:07 - gonna be able to record from very many neurons in behaving flies ever,
  • fast_forward00:38:12 - or behaving bumblebees, or behaving bees.
  • fast_forward00:38:15 - Their nervous system is quite small, the neurons are small, the electrodes we
  • fast_forward00:38:20 - use are still pretty big, you know.
  • fast_forward00:38:23 - There may be some advances using optogenetics where we are able to look at some
  • fast_forward00:38:28 - of these neurons using optical recording in the future that will give us better access.
  • fast_forward00:38:33 - But I think we're going to be forced to stick with this idea of find the general
  • fast_forward00:38:39 - principles by looking at animals where you get the best technical access and
  • fast_forward00:38:43 - try to build a more general model.
  • fast_forward00:38:47 - So in that regard, I think a lot of the control principles that apply to the
  • fast_forward00:38:51 - lobsters, certainly optical flow control.
  • fast_forward00:38:54 - Pitch and roll control, accelerometry, tree that sort of
  • fast_forward00:38:57 - thing will apply equally well to the bee okay so
  • fast_forward00:39:01 - so what are the principles that that stand out for you now for let's say the
  • fast_forward00:39:07 - control of the behavior of of these animals well certainly use of decussation
  • fast_forward00:39:15 - and and ipsilaterally projecting uh,
  • fast_forward00:39:19 - things give us both positive and negative feedback control that's proportional.
  • fast_forward00:39:24 - So, for example, we can use differences in the magnitude of an input stimulus
  • fast_forward00:39:29 - on the two sides to give us proportional control to maintain course,
  • fast_forward00:39:34 - say, in the yaw plane, for example.
  • fast_forward00:39:36 - So I think that's a principle that applies to all of our robots.
  • fast_forward00:39:41 - I think using range fractionation to fractionate sensory inputs into different
  • fast_forward00:39:48 - levels that we can use to apply different logical control projections over different
  • fast_forward00:39:54 - ranges of stimulus magnitude is another principle.
  • fast_forward00:39:58 - So say, for example, if you've got optical flow information going from front
  • fast_forward00:40:05 - to rear at low velocities, you might want that to project to the opposite side.
  • fast_forward00:40:10 - But as you approach a wall or an object, you might want it to project to the
  • fast_forward00:40:14 - same side to cause it to steer away, for example.
  • fast_forward00:40:18 - Right. But then you could argue if we go from lobster to the bee,
  • fast_forward00:40:23 - in some sense, the medium is changing. We go from water to air.
  • fast_forward00:40:27 - The scale is changing. We go from a pretty big animal to a pretty small animal.
  • fast_forward00:40:31 - So this has quite an impact on the dynamics of the behavior. Yeah.
  • fast_forward00:40:37 - And you could argue maybe these principles of the lobster will just fall down
  • fast_forward00:40:41 - or crumble in the face of the scale reduction and also this change in the relationship to the medium.
  • fast_forward00:40:48 - So the bee is following different principles, right? So what makes you hopeful
  • fast_forward00:40:53 - that this generalization will work?
  • fast_forward00:40:56 - Well, we're testing it right now. Okay.
  • fast_forward00:40:58 - So right now we're building helicopter-based robots to mimic the bee,
  • fast_forward00:41:04 - trying to work out these principles where we can use larger sensors and find
  • fast_forward00:41:10 - ways to get the coding right and then find ways to miniaturize that.
  • fast_forward00:41:15 - Right. And at that point, we'll sort of test the idea of whether scaling works or not.
  • fast_forward00:41:21 - And I don't think we can really do those tests until we have the hardware.
  • fast_forward00:41:25 - Work but then do that does it also mean that
  • fast_forward00:41:27 - you see the b brain as a miniaturized version of the lobster brain
  • fast_forward00:41:30 - i mean if to some extent yeah yeah this
  • fast_forward00:41:33 - should be true right yeah yeah yeah i think for certainly if we were going to
  • fast_forward00:41:37 - put chemoreception on the b and flow sensing etc i think those all are pretty
  • fast_forward00:41:43 - generally organized among the animal among the arthropods okay but but bees
  • fast_forward00:41:48 - have mushroom bodies and lobsters
  • fast_forward00:41:50 - do not yeah yeah yeah so where does that difference and come from Well,
  • fast_forward00:41:53 - I mean, we got to ask ourselves what the mushroom bodies are doing, you know.
  • fast_forward00:41:57 - Good question. Yeah, yeah. That's a very good question.
  • fast_forward00:42:00 - And I think my approach is to see what we're missing with the implementation
  • fast_forward00:42:07 - of the reflex pathways that we know about.
  • fast_forward00:42:11 - And that will certainly give us some indication of what we might want to look
  • fast_forward00:42:15 - to in the mushroom bodies. Okay, well, mushroom bodies in the insect literature
  • fast_forward00:42:20 - would be seen as a classification, a memory system, right? That's certainly
  • fast_forward00:42:25 - my impression. Of multimodal input.
  • fast_forward00:42:27 - So if you look at the chemical world, and I also would assume that for a lobster,
  • fast_forward00:42:31 - most of its interactions with the world are chemical.
  • fast_forward00:42:34 - Certainly when we talk about distal interactions with the world.
  • fast_forward00:42:37 - So is the complexity of the kind of compounds it's exposed to lower than what
  • fast_forward00:42:42 - you would expect from a bee?
  • fast_forward00:42:43 - Might that be a difference that explains the absence of a mushroom body?
  • fast_forward00:42:47 - Yeah. I mean, the lobsters really have two sets of, they have a smell receptor
  • fast_forward00:42:52 - in the antennules and a taste receptor on the walking legs.
  • fast_forward00:42:57 - And that's just one type of receptor. So there's no variable set of receptors.
  • fast_forward00:43:02 - Well, Chuck Gerby's done a lot of work on this and Beriaki, and they know exactly
  • fast_forward00:43:06 - what amino acids are responding to and in what proportion of the hair cells respond to that.
  • fast_forward00:43:13 - Again, in that we're just beginning to deal with that, I haven't paid the level
  • fast_forward00:43:18 - of attention that I should be paying and will be paying.
  • fast_forward00:43:22 - You'll be back. Yeah, exactly. No, that's very good. No, but this is interesting,
  • fast_forward00:43:26 - right? Because when you say, and this is completely plausible, right?
  • fast_forward00:43:29 - That these, let's say functional principles to generalize, then in some sense
  • fast_forward00:43:33 - we're committing ourselves to the implication which is, well,
  • fast_forward00:43:36 - these brains should also generalize, right?
  • fast_forward00:43:38 - We should find similarities between the structure because that,
  • fast_forward00:43:41 - in the end, gives you that function, one way or the other, right?
  • fast_forward00:43:44 - Yeah, yeah. But now one other principle that you...
  • fast_forward00:43:49 - Mentioned is the one of reflex chaining to to
  • fast_forward00:43:53 - let's say deal with with a bit
  • fast_forward00:43:56 - more complex behavior so what do you mean with reflex chaining
  • fast_forward00:43:59 - exactly well i mean i think the best example of reflex chaining is feeding in
  • fast_forward00:44:04 - the leech the work that was originally done by mike dickinson and charlotte
  • fast_forward00:44:07 - chuck lamb in which they found that at each stage uh of feeding that the animals
  • fast_forward00:44:15 - would encounter reflex feedback,
  • fast_forward00:44:17 - which would trigger the next phase of the behavior.
  • fast_forward00:44:20 - And that went from initially perceiving the water waves to swimming,
  • fast_forward00:44:25 - to contact with the leech, where they would then start crawling,
  • fast_forward00:44:29 - to contact with a warm spot, where they would then start biting,
  • fast_forward00:44:33 - to the flow of blood, which would then start them to do this suction paralysis, I mean, peristalsis.
  • fast_forward00:44:41 - So that was one of the best examples of reflex training. What we're looking
  • fast_forward00:44:45 - at with the RoboBee is when it leaves the hive, it's going to be given a search
  • fast_forward00:44:51 - vector, which is a compass heading and some odometry information.
  • fast_forward00:44:56 - We're going to have it fly out for a distance specified by the odometer,
  • fast_forward00:45:01 - at which point it will switch to looking ahead with UV ommatidia in a 3x3 array.
  • fast_forward00:45:08 - And in the horizontal plane, those ommatidia are going to cause yaw changes,
  • fast_forward00:45:13 - and in the pitch plane, they're going to cause pitch changes,
  • fast_forward00:45:16 - so that the robot will home in on a UV source, which would be a flower. Okay.
  • fast_forward00:45:22 - Once it, it will then use optical flow information to slow itself down as it
  • fast_forward00:45:29 - approaches the flower and then sort of bump around and get some pollen.
  • fast_forward00:45:33 - And at that point it will reverse heading and fly back to the hive using the
  • fast_forward00:45:41 - inverse of the odometry information that used to fly out.
  • fast_forward00:45:44 - And then we'll have a UV LED on the hive, which you can use to home in to a
  • fast_forward00:45:52 - docking station to recharge. Okay.
  • fast_forward00:45:53 - So that could work in a vacuum, right?
  • fast_forward00:45:56 - But if we now do it in an airflow, it will be drift.
  • fast_forward00:46:00 - So how is your autometer going to be corrected for drift?
  • fast_forward00:46:03 - Well, we're going to use optical flow information to respond to deviations of
  • fast_forward00:46:08 - rotation that are superimposed on the normal translation to get it to come back on heading.
  • fast_forward00:46:16 - Okay. And we'll probably periodically, if it exhibits a big deviation of that,
  • fast_forward00:46:23 - we'll probably get it to listen to its compass a bit and get back on heading.
  • fast_forward00:46:28 - Have you considered using a solar compass?
  • fast_forward00:46:30 - We're toying with that idea. So we have some people on the team that are involved
  • fast_forward00:46:35 - in the whole visual sense of the bee and the idea of a sun compass that uses
  • fast_forward00:46:41 - polarized light is something we're exploring.
  • fast_forward00:46:43 - All right. That would be a powerful solution. So now here we have the robot bee.
  • fast_forward00:46:47 - It's like a scaled down version of the lobster. Sure.
  • fast_forward00:46:51 - Okay, this is my claim. This is not your claim. Okay, but I'm sort of summarizing this.
  • fast_forward00:46:55 - So we have these sort of more uniform principles that we try to identify.
  • fast_forward00:46:59 - But the other thing, you also really want to deploy this.
  • fast_forward00:47:02 - So that means there's a lot of engineering involved to make such a system actually really work.
  • fast_forward00:47:06 - I mean, you can have these ideas based on neurothology, how we can control it.
  • fast_forward00:47:12 - But to get it done is actually not the story. We have to think about power,
  • fast_forward00:47:15 - sensing, integration, computation.
  • fast_forward00:47:18 - So how are you going to get that done? Well, there's a big team.
  • fast_forward00:47:21 - We have eight other investigators that are charged with addressing all those parts.
  • fast_forward00:47:28 - And my part is really to focus on the innate control of flight behavior.
  • fast_forward00:47:34 - There's another crew that's doing colony interactions. There are people that
  • fast_forward00:47:38 - are building the airframes.
  • fast_forward00:47:39 - There are people that are doing the wings, the airfoils. There are people doing the power supply.
  • fast_forward00:47:44 - There are people doing the and there are people doing the power electronics
  • fast_forward00:47:48 - to provide power to the actuators.
  • fast_forward00:47:50 - So this is not a one-man operation by any stretch.
  • fast_forward00:47:54 - Right. And the PI, Rob Wood, has really put together an extraordinarily elegant
  • fast_forward00:48:00 - plan for coordinating this.
  • fast_forward00:48:02 - And we're in our second year right now and and boy it's
  • fast_forward00:48:05 - looking good yeah okay yeah i mean i'm uh we just
  • fast_forward00:48:09 - did our our second year progress report to nsf we
  • fast_forward00:48:12 - had a site visit and and they were quite
  • fast_forward00:48:15 - happy with where we're at at this point uh-huh right oh that's
  • fast_forward00:48:18 - impressive so when are you going to see the first one fly well they
  • fast_forward00:48:22 - there are flying now but they're flying in sort
  • fast_forward00:48:24 - of open loop and they do crash and burn and we're we're we're flying the helicopters
  • fast_forward00:48:30 - that's where we're really try to do the ground truth experiments on all these
  • fast_forward00:48:35 - sensor systems so when will we see the first one fly autonomously for longer
  • fast_forward00:48:40 - than 10 minutes the b yeah.
  • fast_forward00:48:46 - The current expectation with the available power source rehab right now is on
  • fast_forward00:48:53 - the order of three minutes right okay um,
  • fast_forward00:48:58 - the whole issue of power for these things is an area of intense exploration.
  • fast_forward00:49:07 - We're looking at varieties of chemical batteries.
  • fast_forward00:49:11 - We're looking at solid oxide fuel cells, and we're also looking at supercapacitors.
  • fast_forward00:49:16 - So there's a broad variety of power supplies we might employ,
  • fast_forward00:49:20 - and this is why this is not a one-year project.
  • fast_forward00:49:25 - Right, exactly. you know but it's sort of humbling right
  • fast_forward00:49:27 - because we're sitting here having all these concerns about the brain but we
  • fast_forward00:49:31 - can give these things a super brain and still they won't fly because we just
  • fast_forward00:49:35 - have don't have the right power sources yeah so there are some other problems
  • fast_forward00:49:38 - and and how does actuation look how reliable is actuation really the wing the
  • fast_forward00:49:43 - wing control extraordinary and um.
  • fast_forward00:49:48 - Where my colleagues are in the process of writing up the fabrication process,
  • fast_forward00:49:52 - which is truly impressive.
  • fast_forward00:49:54 - And out of deference to their intellectual property rights, I think it's best
  • fast_forward00:50:02 - to not talk about that. Right. But we're just going to say it's fantastic.
  • fast_forward00:50:05 - Yeah, it's fantastic. I am a true believer right now.
  • fast_forward00:50:08 - There was a period of time where I was skeptical, but I really think this is going to happen.
  • fast_forward00:50:14 - And I think you hit the nail right on the head that getting the power to get
  • fast_forward00:50:20 - the appropriate duration of flight is one of the bigger challenges we face. Right, absolutely.
  • fast_forward00:50:26 - So here we have to be.
  • fast_forward00:50:29 - But now another challenge that you highlighted was this whole issue of chemical sensing, right?
  • fast_forward00:50:34 - So chemical sensing is a really interesting problem. I think it's also very
  • fast_forward00:50:37 - much an underestimated problem.
  • fast_forward00:50:40 - And you could also argue that this might be the oldest sense that we have from
  • fast_forward00:50:45 - an evolutionary perspective.
  • fast_forward00:50:46 - This is how cells will start to deal with the world, it's through chemical sensing.
  • fast_forward00:50:51 - So what you were sketching is that in terms of our technology for chemical sense,
  • fast_forward00:50:55 - if you want to understand this biological phenomenon and again build the technology,
  • fast_forward00:51:00 - right now this technology is not good enough in terms of sensitivity and robustness and so on.
  • fast_forward00:51:04 - And you are seeking to overcome this, taking a sort of a synthetic biology approach, right?
  • fast_forward00:51:10 - So what does it actually really mean with respect to chemical sensing?
  • fast_forward00:51:14 - Okay. With chemical sensing, there's basically three processes that we have to deal with.
  • fast_forward00:51:19 - One is the actual sensor itself, the receptor that binds an odorant molecule
  • fast_forward00:51:25 - that is going to cause some change in the cell, which might be an inward current.
  • fast_forward00:51:33 - It might be an enzymatic cascade that leads to a great amplification of the
  • fast_forward00:51:39 - response so that you might get a big cellular response to a single input molecule.
  • fast_forward00:51:45 - And then some sort of reporter that's going to report to us that,
  • fast_forward00:51:49 - in fact, this cell has contacted the odorant.
  • fast_forward00:51:53 - And then finally, some transduction mechanism by which we take the reporter
  • fast_forward00:51:59 - and activate of eight is sensory neuron and electronic nervous system.
  • fast_forward00:52:03 - So those are the three levels that we're working at.
  • fast_forward00:52:06 - Now, clearly for the actual receptor, we're going to use some sort of G protein coupled receptor.
  • fast_forward00:52:13 - And in some of these, they can be coupled with a calcium channel through a G
  • fast_forward00:52:17 - protein, such that when it binds an odorant, there'll be an inward current of calcium.
  • fast_forward00:52:22 - We can also have a G-protein coupled receptor that's coupled to nitric oxide
  • fast_forward00:52:27 - synthase so that the cell will generate nitric oxide, and then we can use a
  • fast_forward00:52:34 - nitric oxide electrode,
  • fast_forward00:52:35 - which is basically a Nafon membrane over a pair of silver and carbon electrodes
  • fast_forward00:52:42 - that would report with nitric oxide.
  • fast_forward00:52:46 - Okay. Okay, so then those electrodes,
  • fast_forward00:52:50 - which might be a photodiode that responds... Well, wait a minute.
  • fast_forward00:52:54 - When the calcium in the first case enters the cell, we would put an aqueorin
  • fast_forward00:53:00 - and silenzorime, which give off light.
  • fast_forward00:53:03 - Or we might put a luciferase, which gives off light.
  • fast_forward00:53:07 - And then our actual transduction mechanism would be a photodiode.
  • fast_forward00:53:11 - Right. Mm-hmm. But so, what will be the spectrum of compounds such a sensor can be sensitive to?
  • fast_forward00:53:20 - Well, there are a broad variety of things.
  • fast_forward00:53:24 - Pretty much anything you can smell has a G-protein coupled receptor.
  • fast_forward00:53:28 - And if you go to the library of biological parts at MIT, you can find a library of receptors.
  • fast_forward00:53:35 - And you can use those genes.
  • fast_forward00:53:39 - There are some two-component receptors, for example, that respond to RDX,
  • fast_forward00:53:44 - which is the explosive in C4.
  • fast_forward00:53:47 - And Jude Mulford has done some work with this where they, if you've seen these
  • fast_forward00:53:52 - experiments where they put a explosive receptor into grasses and the grasses
  • fast_forward00:53:58 - change color if they're planted over a mine.
  • fast_forward00:54:01 - So there's some really interesting tricks that can be put there.
  • fast_forward00:54:05 - There but then if you want to have multiple compounds you'll have
  • fast_forward00:54:07 - multiple diodes yeah right so you
  • fast_forward00:54:10 - might have a problem at your readout level then yeah yeah
  • fast_forward00:54:13 - yeah so you can have this reporting with light of different
  • fast_forward00:54:16 - wavelengths or with nitric oxide okay and and we can there's a uh a charge coupled
  • fast_forward00:54:23 - kind of photodiode uh that we can put filters over and then we can get different
  • fast_forward00:54:29 - filters to respond to different but then you would have a very so then in some of the very local cool.
  • fast_forward00:54:35 - But if you look at bees, it's true for your lobster, they have many chemoreceptors
  • fast_forward00:54:43 - distributed over their antenna, over parts of the body.
  • fast_forward00:54:48 - And this information arguably is also used for detection and localization.
  • fast_forward00:54:54 - So how do you scale up from let's say this very localized detection and readout
  • fast_forward00:54:59 - to this more global, spatially organized way of detection and reporting to some
  • fast_forward00:55:06 - controller what's going on out there. Yeah, yeah, yeah.
  • fast_forward00:55:08 - Well, I think we've got to create sense organs, and the sense organs might have a variety of sensors.
  • fast_forward00:55:14 - And one of our ways of increasing the scale of the kinds of features that we
  • fast_forward00:55:21 - can use is to use a new technology called EJET printing.
  • fast_forward00:55:26 - And EJET printing is a new technology that was developed by John Rogers and
  • fast_forward00:55:33 - Andrew Elwine at University of Illinois at Urbana-Champaign,
  • fast_forward00:55:37 - in which they use a sputter-coated microelectrode that's sputter-coated with gold.
  • fast_forward00:55:44 - And they put an ink in that and then
  • fast_forward00:55:47 - put that over a nanometer stage that can
  • fast_forward00:55:50 - move in x and y planes in nanometer increments
  • fast_forward00:55:53 - and then use very high voltages to put droplets to print very very small droplets
  • fast_forward00:56:00 - and they've been able to print protein features that are five microns in diameter
  • fast_forward00:56:04 - so then we can use this to create very very fine detail on an electrode system
  • fast_forward00:56:12 - that could be combined several photoreceptors on the same sense organ.
  • fast_forward00:56:17 - And the sense organ might be constructed on a glass cover slip, for example,
  • fast_forward00:56:24 - where we glue the bacterial cells that have the receptors we want on one side,
  • fast_forward00:56:30 - and then have the photodetectors on the opposite side of a piece of glass,
  • fast_forward00:56:35 - and we could be sniffing for several different things at once there.
  • fast_forward00:56:38 - Right. So that's one way we could approach that, yeah. Okay,
  • fast_forward00:56:42 - that's pretty impressive.
  • fast_forward00:56:44 - So in chemical sensing, another issue is that, for instance,
  • fast_forward00:56:49 - if you look at the moth, the male moth, the sensitivity at the periphery is
  • fast_forward00:56:54 - an order of magnitude or so lower as that at the neural end.
  • fast_forward00:56:59 - That's when you can look at changes in heart rate due to a pheromone,
  • fast_forward00:57:02 - and you can see that you can already detect changes in heart rate at concentration
  • fast_forward00:57:06 - levels that single receptors cannot detect reliably.
  • fast_forward00:57:09 - So there's a boost somewhere occurring in the neural processing of these signals.
  • fast_forward00:57:17 - So are you already looking into that issue or that's the future?
  • fast_forward00:57:21 - That's the future, definitely. Okay.
  • fast_forward00:57:23 - So now, so okay, we have the lobster, we have the bee project,
  • fast_forward00:57:28 - and this one was in the robotics domain, right?
  • fast_forward00:57:31 - So what's cooking in parallel to this in the neurophysiology?
  • fast_forward00:57:38 - At this point, we have proposals under review that I don't think I should be talking about.
  • fast_forward00:57:44 - But they're fantastic. They are fantastic. Exactly.
  • fast_forward00:57:49 - And in that they're under review
  • fast_forward00:57:51 - at this point, I just don't think it's appropriate to bring that up.
  • fast_forward00:57:54 - No, don't worry about it. We have plenty of ideas, believe me.
  • fast_forward00:57:57 - Oh, yeah, no, I'm sure about that.
  • fast_forward00:57:59 - So in a broader sense, right, outside of the specifics of your proposals,
  • fast_forward00:58:04 - where do you see this field go?
  • fast_forward00:58:06 - Go do you see that the field of biomedics as you
  • fast_forward00:58:09 - envision it is actually having an impact is it changing the way we do science
  • fast_forward00:58:13 - certainly so yeah yeah and i think really what's going on here is that we've
  • fast_forward00:58:18 - switched from analytical neuroscience to synthetic neuroscience and it's time
  • fast_forward00:58:23 - we start building things using principles that we established through analytical neuroscience.
  • fast_forward00:58:29 - Yeah but i could also argue that this might be true for you and very few of
  • fast_forward00:58:34 - your colleagues but that the majority of neuroscience has ended up in more sort
  • fast_forward00:58:38 - of a massive data-gathering exercise.
  • fast_forward00:58:40 - So will this have an impact to the majority of neuroscientists and how they
  • fast_forward00:58:46 - think about their discipline?
  • fast_forward00:58:48 - Well, I think it will depend on how many of them read our papers and take advantage
  • fast_forward00:58:54 - of the knowledge we've been able to create.
  • fast_forward00:58:57 - But I understand you're still optimistic. I'm very optimistic.
  • fast_forward00:59:01 - I think this is really where it's got to go.
  • fast_forward00:59:04 - I mean, this is the wide open frontier of neuroscience.
  • fast_forward00:59:08 - There is so much opportunity here. And I think that my colleagues that lag in
  • fast_forward00:59:14 - getting involved in this are going to miss out.
  • fast_forward00:59:18 - But so then the other issue is how do you see the scale up? Because I could
  • fast_forward00:59:22 - argue like, well, that's nice and well for the bee.
  • fast_forward00:59:25 - But in some sense, who cares about the bee? That's not really advanced organism.
  • fast_forward00:59:29 - Organism why don't you show me something around the
  • fast_forward00:59:32 - macaque or why don't you go up to humans or
  • fast_forward00:59:35 - even a rodent you know that would be more impressive so
  • fast_forward00:59:38 - how do you see the scaling up challenge i'm quite satisfied eating my experiments
  • fast_forward00:59:43 - and the lobsters are an extraordinary model for that and if i just uh create
  • fast_forward00:59:49 - a truly biomimetic lobster that does all the things that lobsters do I think
  • fast_forward00:59:56 - that will be quite an accomplishment,
  • fast_forward00:59:57 - you know, I, I.
  • fast_forward01:00:02 - I'm not particularly interested in working on mammals. I do enjoy some of the
  • fast_forward01:00:09 - technical advantages of working on lower, simple lower vertebrates like sea lamprey.
  • fast_forward01:00:15 - But in terms of the warm-blooded animals and consciousness and all that,
  • fast_forward01:00:21 - I'll leave that to others. Okay.
  • fast_forward01:00:23 - But you see this more as a personal idiosyncrasy or that's really also a very
  • fast_forward01:00:28 - clear scientific strategy?
  • fast_forward01:00:30 - In other words, you say, look, if I can crack this nut of the lobster or of
  • fast_forward01:00:34 - the bee, I can capture these general design principles, then I'm really close
  • fast_forward01:00:38 - to understanding any other brain that evolution has generated.
  • fast_forward01:00:42 - Yeah, but I think we need to figure out the lobster first, and that may be my
  • fast_forward01:00:47 - own personal idiosyncrasy.
  • fast_forward01:00:49 - Okay, right. So then to finish up, two questions.
  • fast_forward01:00:55 - So you came a long way in some sense.
  • fast_forward01:00:58 - I'm quite sure when you were postdoc, you weren't really thinking about ending
  • fast_forward01:01:02 - up talking about robots.
  • fast_forward01:01:04 - I'm not sure, but this is what I imagine. In fact, the whole robot business came up 10 years ago.
  • fast_forward01:01:10 - And if you had asked me 11 years ago if I was going to be working on robots,
  • fast_forward01:01:15 - I would have given you an odd look.
  • fast_forward01:01:18 - Exactly. But you tend to do that anyway, I realized.
  • fast_forward01:01:25 - But so on the basis of all this experience in terms of trying to understand
  • fast_forward01:01:29 - the brain, what's this one law of John Ayer you would like to give us?
  • fast_forward01:01:35 - The John Ayer law. The Ayer's principle. Ayer's principle, right?
  • fast_forward01:01:39 - The Ayer's principle is choose a model organism you can eat.
  • fast_forward01:01:45 - You can eat or you're allowed to eat. There's no problem eating lobsters. Okay.
  • fast_forward01:01:51 - All right. And I do have a recipe in my cookbook for honey smoked lobsters.
  • fast_forward01:01:57 - We're beginning to integrate all these lines of work.
  • fast_forward01:02:02 - So the last question is, five years from now, I'm going to trace you down and
  • fast_forward01:02:08 - I'm going to confront you with the prediction you're going to give me today.
  • fast_forward01:02:10 - So what's this one strong prediction you would commit yourself to today in this
  • fast_forward01:02:16 - domain of building lobsters, understanding lobsters, building bees, understanding bees?
  • fast_forward01:02:21 - What's the one prediction that you care about the most? Well,
  • fast_forward01:02:23 - I believe that we're going to be able to produce vehicles that will...
  • fast_forward01:02:30 - To realize what i call reactive autonomy uh at the scale certainly of a of a lobster,
  • fast_forward01:02:38 - uh within five years i have no doubt of that
  • fast_forward01:02:40 - and that that looks very very clear at this point all right and so where i think
  • fast_forward01:02:47 - i should define by what i mean by supervised autonomy well we actually we call
  • fast_forward01:02:53 - it supervised reactive autonomy right and the idea is that say if you take your dog for a walk.
  • fast_forward01:02:58 - Your dog is autonomously following you around.
  • fast_forward01:03:02 - If you chose to throw a stick out in the water and the dog swims out and retrieves
  • fast_forward01:03:09 - the stick, it's reactively autonomous under your supervision.
  • fast_forward01:03:14 - And it's doing that portion of the mission on its own.
  • fast_forward01:03:18 - And certainly my sense is that the people that would like to be operating robots
  • fast_forward01:03:25 - in the field would like to have that level of control over them.
  • fast_forward01:03:29 - I don't, I think the idea of these things just going off and dreaming up what
  • fast_forward01:03:33 - they want to do on their own is not the best idea.
  • fast_forward01:03:36 - And we wouldn't even want our dogs doing that, you know?
  • fast_forward01:03:40 - Right. So I think maintaining some modicum of supervision, uh,
  • fast_forward01:03:44 - is, is the best idea, but I think we can truly expect these robots to be able to perform,
  • fast_forward01:03:52 - small aspects of missions completely autonomously.
  • fast_forward01:03:56 - But okay, now that you went to qualify autonomous.
  • fast_forward01:04:01 - There's this issue that controls often also illusion, right?
  • fast_forward01:04:04 - We often believe we control our technology, and that we can have controlled
  • fast_forward01:04:08 - autonomy, and then in practice turns out actually our control was an illusion.
  • fast_forward01:04:13 - There was no real, also partially your dog might be conditioning you,
  • fast_forward01:04:16 - and just thinks, well, look, he likes to throw sticks, so I bring him another one.
  • fast_forward01:04:21 - So, to what extent can you actually
  • fast_forward01:04:25 - really quantitatively constrain and define this notion of control?
  • fast_forward01:04:31 - That's an interesting issue. Well, we're certainly going to have control over
  • fast_forward01:04:36 - the amount of power and the mission duration they're going to have.
  • fast_forward01:04:39 - So I think that's going to give us ultimately a degree of control that we can be confident in.
  • fast_forward01:04:47 - I think the general impression of the state of the art of robotics is that the
  • fast_forward01:04:56 - state of the art of autonomy is for a teleoperated vehicle to be able to recover
  • fast_forward01:05:04 - its teleoperative link if it loses it.
  • fast_forward01:05:07 - And I think there's this really interesting example that occurred last summer
  • fast_forward01:05:11 - where the Navy was doing an exercise in Chesapeake Bay with 13 Remuses,
  • fast_forward01:05:17 - and they lost four of them.
  • fast_forward01:05:20 - What are Remuses? Remuses are a torpedo-like robot.
  • fast_forward01:05:24 - It's the only autonomous vehicle in the Navy inventory right now.
  • fast_forward01:05:29 - And they were operating three of
  • fast_forward01:05:32 - them as a group and four of
  • fast_forward01:05:36 - them lost their teleoperative link and got lost
  • fast_forward01:05:39 - and they ended up recovering one of
  • fast_forward01:05:42 - the four that were lost was a marine mammal which is what they were intended
  • fast_forward01:05:46 - to replace in the in the beginning right so i think that will give you this
  • fast_forward01:05:50 - the good quantitative measure of the state of the art right exactly it's it's
  • fast_forward01:05:54 - sobering so joe ayers thank you very much for this conversation it's a pleasure
  • fast_forward01:05:58 - Pleasure. That's great.
  • fast_forward01:06:03 - The CSN podcast was produced by the Convergent Science Network of Biometrics
  • fast_forward01:06:09 - and Biohybrid Systems, a project funded by the European 7th Research Framework Program.
  • fast_forward01:06:17 - For more interviews, recorded lectures, or upcoming conferences in the field
  • fast_forward01:06:22 - of biometrics and biohybrid systems, go to csnnetwork.eu.
  • fast_forward01:06:29 - And thank you for listening.
  • fast_forward01:06:29 - Music.

Be the first to leave a comment

Leave a comment

Your email address will not be published. Required fields are marked *

Exploring the convergence of neuroscience, robotics, and AI through conversations with leading researchers since 2010.

A project of the Convergent Science Network Foundation.

© CSN Podcasts. Developed by IMCreativeWEBC

0%

Login to enjoy full advantages

Please login or subscribe to continue.

Go Premium!

Enjoy the full advantage of the premium access.

Stop following

Unfollow Cancel

Cancel subscription

Are you sure you want to cancel your subscription? You will lose your Premium access and stored playlists.

Go back Confirm cancellation