cover mitra j hartmann

Mitra Hartmann on active touch and whisker sensing

  • cover play_arrow

    PLAY EPISODE


cover mitra j hartmann
Season 2012
Season 2012
Description arrow_drop_down

Description

How does a rat build a three-dimensional picture of the world using nothing but hair? Mitra Hartmann unpacks the biomechanics of whisker sensing, the distinction between active touch and passive sensation, and her vision for a tactile paintbrush that could scan objects in 3D.

Subscribe for more from the Convergent Science Network podcast series.

Hartmann begins by carefully distinguishing active touch from active somatosensation, drawing on James Gibson’s original framework. Active touch requires purposeful exploration, not merely muscle engagement. A rat brushing its whiskers against objects to assess texture is performing active touch; a person grabbing a hot pan with a cloth is not. This distinction matters because it frames the central question of her research: how does an animal transform mechanical energy into a perception of the world?

The rat whisker system serves as Hartmann’s model for studying perception because its neural pathways are analogous to those carrying information from the human hand through the brain. Rats possess roughly 60 whiskers that sweep back and forth at speeds up to 1,000 degrees per second, with all sensory receptors located at the base of each whisker follicle rather than along the hair shaft. Multiple laboratories have demonstrated that rats can discriminate textures with whisker acuity comparable to human fingertips, navigate wall contours, and determine bar orientation. The open question remains why some animals actively whisk while others, like cats and dogs, do not.

As an engineer, Hartmann argues that characterizing the mechanical input is the essential first step before understanding neural processing. Her group investigates whisker material properties, single-whisker mechanics, and natural whisking behavior to determine what signals reach the follicle during real-world exploration. She describes her concept of a whisker paintbrush, a tactile 3D scanner consisting of an array of instrumented bristles that could reconstruct object geometry through mechanical contact. Building such a device could also answer fundamental biological questions about whether independent whisker movement provides information advantages over passive arrays.

Tagged as:

About the author call_made

CSN Podcasts

Both the triumphs of humanity and its most evil deeds have resulted from collaboration. In a time where humanity is required to aspire to the former and minimize the latter, the question arises of how collaboration arises and why it fails. Surprisingly, this phenomenon, so central to who we are, is not well understood. Hence, a collaborative effort is required to understand collaboration in its full biological, psychological, sociological, cultural, and economic complexity and to translate this understanding into operational impact. This series of podcasts is one step toward achieving these complementary goals. The Collaboration Podcast presents interviews with people who are central orchestrators of collaboration in various domains including business, government, science, art, health, sustainability, and the military. The discussions were conducted by Prof. Dr. Paul F.M.J. Verschure and members of the Program Advisory Committee of the Ernst Strungmann Forum on Collaboration (https://www.esforum.de/forums/ESF32_Collaboration.html) during 2021 and had the goal to sketch a map of opportunities, challenges, and obstacles in human collaboration. The forum took place in May 2022, and now we would like to share this series of interviews with a broader audience. The full report of the Forum will be published in 2023 by MIT Press. The podcast was produced by the Convergent Science Network (https://www.convergentsciencenetwork.org/). Context: The stability of social systems depends critically on realizing sustainable methods of “collaboration,” yet how and by which means collaboration is achieved is not clearly understood; neither are the conditions or processes that lead to its breakdown or failure. Collaboration can be understood as cooperation between agents toward mutually constructed goals. Part of the reason for our lack of understanding is that the phenomenon of collaboration is, by nature, a highly multidisciplinary problem, and effective research into its complexities has been difficult to achieve across the broad range of scientific and technical disciplines involved. The need for a fundamental understanding of collaboration, however, has become increasingly important. Not only does humankind demand answers as it attempts to address critical challenges at multiple scales (e.g., climate change, migration, enhanced automation, social and economic inequality), but ever-increasing technological and economic means of interconnecting people and societies are disrupting long-established, familiar patterns of how we interact. Radical technological changes that are ongoing have the potential to reshape collaboration in ways that are currently hard to predict or influence (e.g., by altering configurations in interaction, information creation, and modes of communication). On one hand, such changes could disrupt hitherto stable forms of collaboration by affecting critical communication channels and traditional roles, as can be observed in the rapidly changing patterns in governance, commerce, and social interaction. Conversely, technology could lead to the emergence of novel, successful forms of collaboration that deviate from traditional “hierarchical” architectures. Evidence of this can be seen in areas as diverse as highly automated manufacturing plants, the open science movement, collaborative software repositories, user-centered services, and the sharing of economy-based modes of organization. Without a fundamental understanding of the mechanisms, processes, and boundary conditions of collaboration, it is not possible to evaluate or predict which of these possible scenarios are sustainable or even plausible. The Forum “How Collaboration Arises and Why it Fails” (May 8–13, 2022, Location: Frankfurt am Main, Germany) Chairs: Andreas Roepstorff and Paul Verschure Program Advisory Committee: Jenna Bednar, Julia R. Lupp, Bhavani R. Rao , Andreas Roepstorff, Ferdinand von Siemens, and Paul Verschure

More posts

Timestamp

  • fast_forward00:00:03 - This is the Convergent Science Network podcast. Leading researchers in the domain
  • fast_forward00:00:10 - of neuroscience, brain theory and technology are interviewed by Paul Verscher and Tony Prescott.
  • fast_forward00:00:19 - This is Tony Prescott for the Convergent Science Network podcasts from the Barcelona
  • fast_forward00:00:25 - Cognition Brain and Technology Summer School 2012.
  • fast_forward00:00:28 - And I'm here with Mitra Hartman from Northwestern University where Mitra leads
  • fast_forward00:00:35 - a group looking at perception.
  • fast_forward00:00:38 - Mitra, in your talk this morning you gave us a definition of active touch which
  • fast_forward00:00:45 - comes from the psychologist James Gibson where he said that active touch was
  • fast_forward00:00:51 - what happens when we do touching rather than when we're being touched.
  • fast_forward00:00:55 - Can you explain a little bit more about what you understand about the idea of touch being active.
  • fast_forward00:01:03 - Well, I think we need to be careful with the definition of active touch.
  • fast_forward00:01:10 - And the reason is that you might think of a mechanical definition,
  • fast_forward00:01:16 - that is just that the muscles are active and the muscles are used to go touch
  • fast_forward00:01:22 - something thing, versus the muscles are not active and the human or animal receives
  • fast_forward00:01:29 - the data in and that's passive.
  • fast_forward00:01:31 - But the way that Gibson really meant the use of the word active touch is in
  • fast_forward00:01:37 - the sense of purposeful exploration.
  • fast_forward00:01:41 - And so one possible example is if you use a cloth to grab a hot pan off the
  • fast_forward00:01:50 - stove, you would typically say that you grabbed the cloth, you grabbed the pan.
  • fast_forward00:01:57 - You wouldn't say that you touched the cloth or you touched the pan.
  • fast_forward00:02:01 - You might say that you touched the cloth if you actively explored the texture of that cloth.
  • fast_forward00:02:08 - And that sort of exploratory movement would be the type of movement that Gibson
  • fast_forward00:02:13 - would have described as active touch.
  • fast_forward00:02:17 - But it's clear that in both cases, the muscles are active and you're making use of sensory data.
  • fast_forward00:02:24 - CB So you're restricting active touch to be behaviors where you're using touch
  • fast_forward00:02:32 - to actually sense the properties of the world around you.
  • fast_forward00:02:35 - That's the sense in which Gibson used the word active touch.
  • fast_forward00:02:39 - And so if we're going to be consistent with Gibson, that's how it would have to be defined.
  • fast_forward00:02:44 - I think it's also useful to talk about what one might call active somatosensation,
  • fast_forward00:02:51 - and there you could have a.
  • fast_forward00:02:54 - Purely mechanical definition in which the muscles are active in a particular sensory surface or not.
  • fast_forward00:03:02 - That is, the muscles are responsible for the sensory data that you receive or not.
  • fast_forward00:03:07 - And that would be independent of the intent of the movement.
  • fast_forward00:03:12 - So for example, as you're walking, you might get sensory data on your feet,
  • fast_forward00:03:18 - and that would be active somatosensation, sensation
  • fast_forward00:03:21 - but it would not be what gibson would have
  • fast_forward00:03:24 - called active touch yeah i
  • fast_forward00:03:27 - agree that it's it's complicated i suppose active sensing
  • fast_forward00:03:31 - is now a phrase that people use for many
  • fast_forward00:03:34 - mean many different things and it's probably best if individuals just say what
  • fast_forward00:03:39 - they mean by it when they start talking about it i think that's a really good
  • fast_forward00:03:43 - idea so in your group is it fair to characterize what you're doing as studying perception in general,
  • fast_forward00:03:52 - but through the route of looking at active touch.
  • fast_forward00:03:55 - I think that's a fair characterization. I'm very interested in the question of perception,
  • fast_forward00:04:02 - that is how an animal or a human can take energy in, in the form of a sound wave or a light wave,
  • fast_forward00:04:12 - and somehow turn that energy signal into a perception of the world.
  • fast_forward00:04:18 - That's something we have very little understanding of.
  • fast_forward00:04:22 - But also, we don't know that animals do form perceptions.
  • fast_forward00:04:27 - This is true. You're absolutely right. We do not know that animals form perceptions.
  • fast_forward00:04:32 - We infer that they form perceptions based on their behavior.
  • fast_forward00:04:38 - And so, what would be an example of a behaviour of, say, an animal,
  • fast_forward00:04:43 - from which you might infer that they've got a perception of an environment,
  • fast_forward00:04:47 - rather than just directly using sensing to control movement? Yeah.
  • fast_forward00:04:54 - Uh, some, some sort of tasks that you perform in the lab or where you ask them
  • fast_forward00:05:00 - to discriminate would be an example or.
  • fast_forward00:05:03 - Certainly in the laboratory, we can, uh, measure what we like to think of as perception.
  • fast_forward00:05:10 - Um, so for example, you can train a rat to, uh, go to the left if there's a
  • fast_forward00:05:19 - square object in its path and go to the right if there's a round object in its path.
  • fast_forward00:05:23 - And if the objects are in every other way identical and your task is appropriately
  • fast_forward00:05:28 - controlled, you might argue that the rat has had a perception of a cube or a sphere.
  • fast_forward00:05:35 - But in this case and in the studies you're talking about, the rat might well be using its whiskers.
  • fast_forward00:05:41 - The rat is very likely to be using its whiskers to perform such a tactile discrimination.
  • fast_forward00:05:49 - Rats have four paws, but they don't use them very much to explore the world.
  • fast_forward00:05:54 - Instead, they use their whiskers to explore the world. And people don't have whiskers.
  • fast_forward00:05:58 - So it's going to be quite different experience of the world or perceptions that
  • fast_forward00:06:04 - rats are going to have compared to people.
  • fast_forward00:06:06 - It does seem likely that rats will have very different perceptions of the world
  • fast_forward00:06:10 - than a person would have.
  • fast_forward00:06:12 - Humans are very visual animals. Rats are very tactually and olfactory-based creatures.
  • fast_forward00:06:19 - So you have some hypotheses about how a rat might use its whiskers to build
  • fast_forward00:06:24 - up a tactile perception.
  • fast_forward00:06:26 - Our laboratory has several different ideas about how the rat might use its whiskers
  • fast_forward00:06:32 - to build up perceptions of the world.
  • fast_forward00:06:34 - We are particularly interested in the mechanics of sensing.
  • fast_forward00:06:38 - Rats actively move their whiskers back and forth rhythmically as they explore objects.
  • fast_forward00:06:46 - Now, you may be familiar with the whiskers on a cat or a dog.
  • fast_forward00:06:50 - Cats and dogs do use their whiskers to explore different things,
  • fast_forward00:06:55 - but they can't actively move them in the same way that a rat does.
  • fast_forward00:06:58 - So a rat brushes its whiskers against objects rhythmically.
  • fast_forward00:07:03 - And we think that the forces that the whiskers exert on different objects tell
  • fast_forward00:07:12 - the rat something about the properties of those objects.
  • fast_forward00:07:16 - So the whisker is just a hair, isn't it? A particular kind of hair.
  • fast_forward00:07:19 - And so it's not sensing anything on the whisker itself, but at the base. That's right.
  • fast_forward00:07:25 - The whiskers are just like the hairs on your head in the sense that there are
  • fast_forward00:07:29 - no sensors along their length.
  • fast_forward00:07:31 - All of the sensors are at the base of the whisker in a follicle.
  • fast_forward00:07:35 - So we could, for example, trim the whiskers of a rat and it would not hurt the
  • fast_forward00:07:39 - rat at all, same way as you can get a haircut.
  • fast_forward00:07:42 - And so it has a whole bunch of these whiskers on each side of its face and it's
  • fast_forward00:07:47 - moving them quickly against objects and sensing that at the base of the whisker. That's correct.
  • fast_forward00:07:53 - There are about 60 whiskers in total on the rat, 30 approximately on each side,
  • fast_forward00:07:59 - and it moves the whiskers very rapidly, up to, say, 1,000 degrees per second.
  • fast_forward00:08:07 - And why do you think it moves its whiskers?
  • fast_forward00:08:12 - It's not entirely clear to the
  • fast_forward00:08:15 - field what the advantages are of active movements of the whiskers why for example
  • fast_forward00:08:22 - does the rat actively move its whiskers while the cat and dog don't one clear
  • fast_forward00:08:28 - advantage is that it's able to cover more volume but then you might easily ask well well,
  • fast_forward00:08:34 - why doesn't it just grow more whiskers to cover that same volume?
  • fast_forward00:08:38 - So I think it's a large open question that's certainly worth investigating about
  • fast_forward00:08:43 - why some animals perform this whisking behavior, as we call it,
  • fast_forward00:08:48 - and other animals don't.
  • fast_forward00:08:50 - So the whisking that the animal does against surfaces causes deflections of
  • fast_forward00:08:56 - the whiskers and that bending is picked up in the follicle, the base of the whisker.
  • fast_forward00:09:03 - What kind of things do you think the animal is able to extract from that signal?
  • fast_forward00:09:09 - What kind of properties of the world do you think it can detect?
  • fast_forward00:09:12 - So, a number of laboratories use the rat whisker system as a model to study the sense of touch.
  • fast_forward00:09:21 - There are maybe 20, 30 laboratories interested in the rat vibrissal or whisker system.
  • fast_forward00:09:29 - So, what I'm about to say comes from many different laboratories all working on this question.
  • fast_forward00:09:36 - There are a number of laboratories that have shown that rats can discriminate
  • fast_forward00:09:40 - different textures. with their whiskers.
  • fast_forward00:09:43 - There are some studies that show that rats can do as well with their whiskers
  • fast_forward00:09:48 - as we might be able to do with our fingertips in judging texture.
  • fast_forward00:09:53 - Rats can certainly use their whiskers to navigate along the contours of walls.
  • fast_forward00:09:59 - They're very good at scurrying along walls.
  • fast_forward00:10:01 - And from this, we infer that they can tell something about the way that the
  • fast_forward00:10:06 - wall is bending or curving back and forth.
  • fast_forward00:10:10 - So we infer that they're able to obtain information about the local curvature
  • fast_forward00:10:15 - of an object, at least far enough in advance to tell themselves how to locomote.
  • fast_forward00:10:20 - So certainly texture and the shape of a wall.
  • fast_forward00:10:25 - There's work by Ron Frostig showing that rats can determine the orientation
  • fast_forward00:10:30 - of a bar, whether it's vertical or horizontal.
  • fast_forward00:10:35 - And there's, I believe, some recent evidence showing they can do compliance,
  • fast_forward00:10:41 - but I'm not confident about that.
  • fast_forward00:10:45 - And is studying this system going to help us understand James Gibson's question
  • fast_forward00:10:51 - of active touch in people? Are there similarities?
  • fast_forward00:10:55 - One of the nice advantages of the rat vibrissal system,
  • fast_forward00:11:00 - and the reason that I've chosen it as a model for use in my laboratory,
  • fast_forward00:11:04 - is that the neural pathways that carry information from the vibrissae through
  • fast_forward00:11:11 - the rat's brain are in many ways analogous to the neural pathways that carry
  • fast_forward00:11:17 - information from the human hand through the human brain.
  • fast_forward00:11:22 - They're not entirely similar, but they are in many ways analogous,
  • fast_forward00:11:27 - and so we think that by better understanding the neural processing at different
  • fast_forward00:11:32 - stages of the rat whisker system,
  • fast_forward00:11:34 - we may gain insight into how sensory data from the hand is processed.
  • fast_forward00:11:41 - So what's your strategy for understanding this system? Where do you think is
  • fast_forward00:11:46 - the best place to focus the effort?
  • fast_forward00:11:49 - As an engineer, I argue that one of the most essential things we can do is to
  • fast_forward00:11:56 - characterize the input.
  • fast_forward00:11:58 - And what I mean by that is that we must know what each whisker is doing,
  • fast_forward00:12:04 - and the sensory data obtained by each of those whiskers, that is the mechanics
  • fast_forward00:12:08 - of those whiskers and the mechanical signals that each of those whiskers is receiving.
  • fast_forward00:12:14 - And then we need to know how those mechanical signals correspond to spikes,
  • fast_forward00:12:20 - or electrical activity, that is, in the very first stage of neurons in the system.
  • fast_forward00:12:27 - And if we understand that input, then we can go further on up the different
  • fast_forward00:12:33 - brain structures and look at how that input is transformed.
  • fast_forward00:12:37 - But we can't begin to understand the processor until we know what's going into the system.
  • fast_forward00:12:43 - And for that, you think we need to look at the material properties of the whiskers themselves?
  • fast_forward00:12:48 - We need to look at the material properties, and our laboratory has done some of that.
  • fast_forward00:12:54 - We need to look at the basic mechanics of single vibrissae, and we need to look
  • fast_forward00:12:59 - at how rats use their whiskers naturally during their normal, regular behavior.
  • fast_forward00:13:06 - How is the brain used as the rat is exploring its objects as it would in its natural environment?
  • fast_forward00:13:15 - So as well as being interesting for understanding the rat brain and potentially the human brain,
  • fast_forward00:13:21 - is there uses of vibrational sensing that you could imagine,
  • fast_forward00:13:27 - perhaps in technology, that we could develop by understanding the system better?
  • fast_forward00:13:32 - I think there are many technologies that we might develop that are inspired
  • fast_forward00:13:38 - by the rat whisker system, even if they're not 100% faithful to the way that
  • fast_forward00:13:45 - the rat uses its whiskers.
  • fast_forward00:13:47 - One example I can think of is what you might think of as a whisker paintbrush.
  • fast_forward00:13:55 - So imagine a paintbrush with lots of bristles coming out, and you can paint
  • fast_forward00:14:00 - this brush over objects, move this brush over objects.
  • fast_forward00:14:04 - And the idea is that each of its bristles picks up mechanical data that allows
  • fast_forward00:14:09 - us to reconstruct the three-dimensional features of that object.
  • fast_forward00:14:15 - Essentially a tactile three-dimensional scanner. And so you're planning to build one of these yourself?
  • fast_forward00:14:20 - We'd love to be able to build a whisker paintbrush, as we call it, a tactile paintbrush.
  • fast_forward00:14:26 - And you previously built a whisking robot with a few whiskers,
  • fast_forward00:14:31 - and this would have several.
  • fast_forward00:14:32 - We've built a limited whisking robot that can move in.
  • fast_forward00:14:37 - We only used it to move in one dimension and
  • fast_forward00:14:41 - it was able to uh to determine
  • fast_forward00:14:44 - uh an object's spatial features uh but this would be a high have more degrees
  • fast_forward00:14:51 - of freedom you would be able to move this in more ways than we could move the
  • fast_forward00:14:55 - previous version is the idea that the whiskers would move independently or they
  • fast_forward00:14:59 - would just be on the it would it be literally a paintbrush with.
  • fast_forward00:15:03 - I think in the initial prototype, it would literally be like a paintbrush where
  • fast_forward00:15:08 - the bristles don't move independently.
  • fast_forward00:15:09 - But a very interesting question we could answer that you're getting at is,
  • fast_forward00:15:16 - could we extract more information if each of the whiskers were allowed to have
  • fast_forward00:15:21 - its own degree of freedom and rotate independent of how the paintbrush itself was oriented?
  • fast_forward00:15:27 - And so building this paintbrush could help us answer some questions about the
  • fast_forward00:15:33 - biology, which is still unresolved, perhaps.
  • fast_forward00:15:36 - I could imagine potentially that investigating independent whisker movement
  • fast_forward00:15:43 - in the context of a robotic whisker array could help us answer some questions
  • fast_forward00:15:48 - about what advantages there are to active whisking movements.
  • fast_forward00:15:52 - And even suggest some hypotheses about how information from the viscous systems
  • fast_forward00:15:58 - process in the brain? I suppose that that's true, yes.
  • fast_forward00:16:02 - Okay, well, thanks very much for talking to us, Mitra. Thank you, Tony.
  • fast_forward00:16:08 - The CSN podcast was produced by the Convergent Science Network of Biometrics
  • fast_forward00:16:14 - and Biohybrid Systems, a project funded by the European Sevens Research Framework Programme.
  • fast_forward00:16:22 - For more interviews, recorded lectures, or upcoming conferences in the field
  • fast_forward00:16:27 - of biometrics and biohybrid systems, go to csnnetwork.com.
  • fast_forward00:16:34 - Music.

Be the first to leave a comment

Leave a comment

Your email address will not be published. Required fields are marked *

Convergent PozitivLinia Logo

Exploring the convergence of neuroscience, robotics, and AI through conversations with leading researchers since 2010.

A project of the Convergent Science Network Foundation.

© CSN Podcasts. Developed by IMCreativeWEBC

0%

Login to enjoy full advantages

Please login or subscribe to continue.

Go Premium!

Enjoy the full advantage of the premium access.

Stop following

Unfollow Cancel

Cancel subscription

Are you sure you want to cancel your subscription? You will lose your Premium access and stored playlists.

Go back Confirm cancellation