Guenther Knoblich on joint action and entrainment

  • cover play_arrow

    PLAY EPISODE


Season 2012
Season 2012
Description arrow_drop_down

Description

How do two tango dancers achieve millisecond-level coordination without a conductor? Guenther Knoblich decomposes joint action into five mechanisms, from unconscious entrainment to motor simulation, revealing that even speeding up is a sophisticated coordination strategy.

Subscribe for more from the Convergent Science Network podcast series.

Knoblich defines joint action broadly as any coordination between people in space and time, deliberately avoiding distinctions between intentional and unintentional, cooperative and competitive. This breadth allows him to identify shared mechanisms across seemingly different situations: table tennis opponents and dance partners may rely on the same low-level coordination processes despite having opposing goals. He identifies five mechanisms ranging from simple to cognitively demanding: entrainment, speeding, simulation, monitoring, and signaling.

Entrainment, borrowed from physics, describes how oscillating systems with perceptual coupling tend to synchronize automatically. People walking near each other converge on the same pace without intending to; rocking chairs in the same room align their rhythms. But Knoblich argues entrainment alone cannot explain most joint action. His group discovered speeding as an independent strategy: when asked to synchronize discrete responses with a partner, people speed up by about 50 milliseconds compared to individual performance. This is not competition or arousal. Correlation analysis reveals a causal chain: faster reactions reduce variability, and reduced variability decreases asynchrony between partners. The effect appears immediately and remains constant, suggesting a general mindset shift rather than a learned adjustment.

The discussion of motor simulation draws on EEG evidence from a bottle-passing task. The receiver shows motor preparation peaks time-locked to the giver’s action initiation, well before their own receiving movement begins, demonstrating that the motor system predicts a partner’s actions in parallel with planning one’s own. Knoblich proposes that the same forward models used for individual action planning are repurposed to simulate others, with expertise modulating simulation fidelity: an expert dancer simulates observed dance movements with greater motor activation than a novice. This framework connects individual motor control to social cognition through shared predictive mechanisms rather than requiring a separate theory-of-mind module.

Tagged as:

About the author call_made

CSN Podcasts

Both the triumphs of humanity and its most evil deeds have resulted from collaboration. In a time where humanity is required to aspire to the former and minimize the latter, the question arises of how collaboration arises and why it fails. Surprisingly, this phenomenon, so central to who we are, is not well understood. Hence, a collaborative effort is required to understand collaboration in its full biological, psychological, sociological, cultural, and economic complexity and to translate this understanding into operational impact. This series of podcasts is one step toward achieving these complementary goals. The Collaboration Podcast presents interviews with people who are central orchestrators of collaboration in various domains including business, government, science, art, health, sustainability, and the military. The discussions were conducted by Prof. Dr. Paul F.M.J. Verschure and members of the Program Advisory Committee of the Ernst Strungmann Forum on Collaboration (https://www.esforum.de/forums/ESF32_Collaboration.html) during 2021 and had the goal to sketch a map of opportunities, challenges, and obstacles in human collaboration. The forum took place in May 2022, and now we would like to share this series of interviews with a broader audience. The full report of the Forum will be published in 2023 by MIT Press. The podcast was produced by the Convergent Science Network (https://www.convergentsciencenetwork.org/). Context: The stability of social systems depends critically on realizing sustainable methods of “collaboration,” yet how and by which means collaboration is achieved is not clearly understood; neither are the conditions or processes that lead to its breakdown or failure. Collaboration can be understood as cooperation between agents toward mutually constructed goals. Part of the reason for our lack of understanding is that the phenomenon of collaboration is, by nature, a highly multidisciplinary problem, and effective research into its complexities has been difficult to achieve across the broad range of scientific and technical disciplines involved. The need for a fundamental understanding of collaboration, however, has become increasingly important. Not only does humankind demand answers as it attempts to address critical challenges at multiple scales (e.g., climate change, migration, enhanced automation, social and economic inequality), but ever-increasing technological and economic means of interconnecting people and societies are disrupting long-established, familiar patterns of how we interact. Radical technological changes that are ongoing have the potential to reshape collaboration in ways that are currently hard to predict or influence (e.g., by altering configurations in interaction, information creation, and modes of communication). On one hand, such changes could disrupt hitherto stable forms of collaboration by affecting critical communication channels and traditional roles, as can be observed in the rapidly changing patterns in governance, commerce, and social interaction. Conversely, technology could lead to the emergence of novel, successful forms of collaboration that deviate from traditional “hierarchical” architectures. Evidence of this can be seen in areas as diverse as highly automated manufacturing plants, the open science movement, collaborative software repositories, user-centered services, and the sharing of economy-based modes of organization. Without a fundamental understanding of the mechanisms, processes, and boundary conditions of collaboration, it is not possible to evaluate or predict which of these possible scenarios are sustainable or even plausible. The Forum “How Collaboration Arises and Why it Fails” (May 8–13, 2022, Location: Frankfurt am Main, Germany) Chairs: Andreas Roepstorff and Paul Verschure Program Advisory Committee: Jenna Bednar, Julia R. Lupp, Bhavani R. Rao , Andreas Roepstorff, Ferdinand von Siemens, and Paul Verschure

More posts

Timestamp

  • fast_forward00:00:03 - This is the Convergent Science Network podcast. Leading researchers in the domain
  • fast_forward00:00:10 - of neuroscience, brain theory and technology are interviewed by Paul Verschoor and Tony Prescott.
  • fast_forward00:00:19 - This is Paul Verschoor with the Convergent Science Network podcast.
  • fast_forward00:00:24 - And this episode that we also recorded as part of our
  • fast_forward00:00:27 - CSN Barcelona Cognition Brain Technology Summer School
  • fast_forward00:00:30 - I'm talking with Gunter Knoppleg and Gunter
  • fast_forward00:00:34 - um your the topic
  • fast_forward00:00:37 - of your presentation is joint action those are actually the
  • fast_forward00:00:40 - main topic of your of your research right and you decomposed
  • fast_forward00:00:43 - um joint action uh in a
  • fast_forward00:00:46 - number of let's say constituent components that you sort of
  • fast_forward00:00:49 - try to then again investigate in themselves
  • fast_forward00:00:52 - so why don't you can you explain to me how you see
  • fast_forward00:00:55 - joint action and how do you see this decomposed in its subcomponents right okay
  • fast_forward00:00:59 - so um i mean i'm i'm most interested in joint action where people actually perform
  • fast_forward00:01:05 - some sort of skilled action together like playing football or you know trapeze
  • fast_forward00:01:09 - artists and um uh one big question in this field is
  • fast_forward00:01:14 - how do they achieve temporal coordination?
  • fast_forward00:01:17 - How do they not miss each other? How do they pass the ball in time to score a goal and so on?
  • fast_forward00:01:25 - So there are different mechanisms to actually achieve this, psychologically speaking.
  • fast_forward00:01:32 - And it starts, and I was actually describing five different ones,
  • fast_forward00:01:38 - ranging from entrainment to providing signals for each other that it's of the coordination.
  • fast_forward00:01:45 - Okay, so you had entrainment, speeding, simulation, monitoring,
  • fast_forward00:01:50 - and then signaling. Exactly.
  • fast_forward00:01:54 - So it would be... But now let's backtrack a bit to the joint action issue itself.
  • fast_forward00:01:59 - So what makes joint action special as compared to,
  • fast_forward00:02:03 - Other forms of action. Right. So in individual action, the only thing you need
  • fast_forward00:02:10 - to do is get your own action right somehow.
  • fast_forward00:02:13 - Whereas in joint action, what you need to do is you need to coordinate your
  • fast_forward00:02:17 - action with somebody else in space and time.
  • fast_forward00:02:20 - And it's actually even in the most simple action, like handing a glass to somebody
  • fast_forward00:02:26 - else, it's actually from the point of view of the motor system.
  • fast_forward00:02:29 - That's a very tricky task because, for instance, you need to take the other's
  • fast_forward00:02:34 - actions into account in order to pass the glass on.
  • fast_forward00:02:38 - And the question is, how can this be achieved? And there are many,
  • fast_forward00:02:42 - many sort of real-time constraints where the motor system has to solve very
  • fast_forward00:02:46 - difficult problems in a very short sort of time window.
  • fast_forward00:02:50 - But then I could argue, okay, but the motor system is already tuned to deal
  • fast_forward00:02:56 - with a dynamic world. It might not be an intentional world, but a dynamic world.
  • fast_forward00:03:00 - So what then would make joint action different from, let's say, catching a ball?
  • fast_forward00:03:04 - Right. I mean, it's not necessarily qualitatively different all the time,
  • fast_forward00:03:10 - but it's certainly true that biological movement patterns are most of the time
  • fast_forward00:03:14 - different from simple physical movements.
  • fast_forward00:03:17 - I mean, they can also be described, of course, as physical movement,
  • fast_forward00:03:21 - but I think the intentionality behind it makes it more complex and also,
  • fast_forward00:03:27 - in a sense, there's more variety of movements, so to say.
  • fast_forward00:03:32 - And it follows different laws than simply sort of gravity.
  • fast_forward00:03:39 - Because the kinematics of human movement, for instance, they reflect actually
  • fast_forward00:03:42 - the working of a very complex apparatus that functions,
  • fast_forward00:03:49 - that produces, you know, kinematic patterns that are quite different than the
  • fast_forward00:03:55 - pattern that a ball has, you know, in a simple trajectory,
  • fast_forward00:03:59 - let's say. Okay, so can you give an example of that difference?
  • fast_forward00:04:02 - Yeah, I mean, think about two players in football, where actually,
  • fast_forward00:04:08 - you know, one is asked to anticipate where the other is going,
  • fast_forward00:04:12 - in order to place the ball in the right spot.
  • fast_forward00:04:17 - This could involve like, quite an intricate prediction,
  • fast_forward00:04:21 - you know, that requires knowing the player, which way will he turn,
  • fast_forward00:04:25 - there will be more sort of cognitive level, but it could It could also imply
  • fast_forward00:04:28 - something like how fast can he move and things like that.
  • fast_forward00:04:33 - And all of these things implicitly or explicitly could go into the kicking of the ball.
  • fast_forward00:04:39 - Right. Okay. So now let's turn to entrainment. So how should I think about entrainment
  • fast_forward00:04:46 - in this context of joint action?
  • fast_forward00:04:47 - Right. So entrainment is sort of perhaps the most simple way of getting people synchronized.
  • fast_forward00:04:53 - And that's actually a process that is not necessarily only restricted to humans
  • fast_forward00:05:01 - because it actually also works for clocks.
  • fast_forward00:05:03 - So if your entrainment is a principle that actually comes out of physics and
  • fast_forward00:05:09 - that just states that if you have an oscillation and if you have oscillating
  • fast_forward00:05:16 - devices like a metronome and they have a physical coupling,
  • fast_forward00:05:21 - they actually tend to go into phase with one another.
  • fast_forward00:05:26 - And dynamical systems psychologists have actually made the point that the same
  • fast_forward00:05:32 - sort of principles could hold for informational couplings, perceptual couplings between people,
  • fast_forward00:05:38 - so that in a sense there's a general pull towards synchrony
  • fast_forward00:05:43 - built into the systems so that people
  • fast_forward00:05:46 - have a tendency if they
  • fast_forward00:05:48 - perform sort of rhythmic movements to to be in
  • fast_forward00:05:51 - synchrony with one another so for instance you can observe that when people
  • fast_forward00:05:55 - many people cross a bridge or so that or walk close to each other they will
  • fast_forward00:06:01 - tend to walk in the same speed and assimilate at speed so that and this is a
  • fast_forward00:06:06 - very automatic process that people aren't aware of.
  • fast_forward00:06:10 - They're just pulled towards this synchronization. Right.
  • fast_forward00:06:12 - So do we know anything about the genesis of this? What's the generator of that synchronization?
  • fast_forward00:06:17 - The generator? You mean like an internal clock?
  • fast_forward00:06:21 - I don't know, yes. Well, look, earlier you said yourself, right?
  • fast_forward00:06:24 - Oscillators would do this, right? So where's this oscillator?
  • fast_forward00:06:28 - I think in a sense where this comes from is not… I mean,
  • fast_forward00:06:37 - there's not really an explicit theory about you know oscillations in the brain
  • fast_forward00:06:41 - that drive this sort of process,
  • fast_forward00:06:44 - it's more a particular view in psychology that wants to see sort of social interactions
  • fast_forward00:06:50 - between people partly driven,
  • fast_forward00:06:53 - by these sort of coupled oscillations so that in a sense the whole human who
  • fast_forward00:06:59 - is involved in this thing becomes the oscillator and the coupling is actually
  • fast_forward00:07:04 - between people So it's not a very cognitivist position,
  • fast_forward00:07:08 - it's more a sort of applying physical principles to the psychological domain.
  • fast_forward00:07:15 - Right, but at what point in human behavior does this kind of coupling break down?
  • fast_forward00:07:21 - Now, we are talking to each other, and at some level of description,
  • fast_forward00:07:26 - I could say, well, in some state space, I could say we are coupled because I
  • fast_forward00:07:30 - pose questions and you generate an answer and I try not to interrupt you and so on.
  • fast_forward00:07:34 - Well, let's say at the level of our physical actions, there might be not a single
  • fast_forward00:07:39 - signature of such coupling.
  • fast_forward00:07:42 - So how far can this really go? Where does it break down, this idea of coupling?
  • fast_forward00:07:46 - I think that this is sort of a very powerful way to look at interpersonal processing,
  • fast_forward00:07:54 - but it is only one possibility.
  • fast_forward00:07:58 - And I think that there are many types of social interactions and also joint
  • fast_forward00:08:01 - actions that cannot be explained by simple entrainment.
  • fast_forward00:08:05 - But I would still hold that even, so, you know, most of the time it's really
  • fast_forward00:08:11 - used to explain unintentional cases of synchrony, so that people synchronize
  • fast_forward00:08:17 - even though they don't want to,
  • fast_forward00:08:20 - like simulating velocities and rocking chairs, things like that.
  • fast_forward00:08:24 - Right. Or, you know, nodding their head like we do now in the same rhythm.
  • fast_forward00:08:28 - In the same sort of temporal frequency.
  • fast_forward00:08:33 - Mm-hmm.
  • fast_forward00:08:36 - But it can, of course, also be used, let's say, in dance, as something that
  • fast_forward00:08:41 - becomes part of an intentional joint action.
  • fast_forward00:08:44 - But in these cases, of course, the pure synchronization is not enough.
  • fast_forward00:08:50 - But it can actually help you to achieve what you want to achieve.
  • fast_forward00:08:55 - But it's only sort of part of the processing. and
  • fast_forward00:08:59 - in other cases I would say entrainment actually works against the coordination
  • fast_forward00:09:03 - that is required because we don't always need to be in the same rhythm or in
  • fast_forward00:09:09 - the same pattern exactly but the interesting thing is in some sense if it would
  • fast_forward00:09:14 - be entrainment in the way you now just define it it would be.
  • fast_forward00:09:19 - Actually invalidating the notion of joint action because action
  • fast_forward00:09:22 - would imply that there's an intentional component to
  • fast_forward00:09:25 - it while the entrainment would be unintentional right that's perhaps
  • fast_forward00:09:29 - a little bit of a problem with our our definition of
  • fast_forward00:09:32 - joint action which we want to be very broad so that
  • fast_forward00:09:36 - uh when i say action it doesn't necessarily imply
  • fast_forward00:09:39 - a uh an intention let's
  • fast_forward00:09:42 - say so it can include just let's say
  • fast_forward00:09:45 - behavioral patterns yes any kind
  • fast_forward00:09:48 - yes okay so i wouldn't i wouldn't want to make
  • fast_forward00:09:51 - a distinction between intention non-intentional action right away because there's
  • fast_forward00:09:56 - so many cases where it's very difficult to to really draw the boundary between
  • fast_forward00:10:01 - intentional unintentional action that um i think for our purpose which is sort
  • fast_forward00:10:06 - of trying to find out the nuts and bolts of joint action.
  • fast_forward00:10:09 - We rather work with a very broad definition of joint action,
  • fast_forward00:10:13 - which basically says that we want to talk of a joint action for anything that
  • fast_forward00:10:18 - involves coordination between people in space and time. So it's super broad.
  • fast_forward00:10:24 - It ignores perhaps also a little bit philosophical sort of basic distinctions,
  • fast_forward00:10:30 - but it's been so far very useful actually in trying going to sort of relate
  • fast_forward00:10:36 - processes that people would normally think of as very different.
  • fast_forward00:10:42 - Right, but it might also then lead to misclassifications, right?
  • fast_forward00:10:46 - Imagine we get thrown out of a plane and we're falling down,
  • fast_forward00:10:50 - free fall, flailing our arms and legs,
  • fast_forward00:10:53 - then a psychologist using your definition could say, ah, there are entrained,
  • fast_forward00:10:57 - but basically we're just both falling, racing towards earth and just in panic or flailing around.
  • fast_forward00:11:05 - Right. I think, but I mean, this example, my analysis would be that the joint
  • fast_forward00:11:10 - action stops in the moment when you've pushed me, for instance.
  • fast_forward00:11:15 - And then, you know, what I'm trying to do is to save my life.
  • fast_forward00:11:18 - And from then on, I would call it separate action.
  • fast_forward00:11:21 - But in a sense uh i would
  • fast_forward00:11:24 - perhaps even accept like you know an action like pushing somebody out as a plane
  • fast_forward00:11:28 - out of the plane as something that could be conceived of as a joint action perhaps
  • fast_forward00:11:34 - not with the same intended outcome but with different intended outcomes i know
  • fast_forward00:11:39 - it gets tricky in a way but i don't want to have you know sort of um,
  • fast_forward00:11:44 - Also, competition versus cooperativeness.
  • fast_forward00:11:48 - I also want to stay away from that distinction because for these sort of rather
  • fast_forward00:11:54 - low-level nuts and bolts coordination processes,
  • fast_forward00:11:57 - it might not matter so much whether you're in a competitive game or whether
  • fast_forward00:12:01 - you're trying to play cooperatively.
  • fast_forward00:12:04 - For instance, in table tennis, you can do both. And the coordination processes
  • fast_forward00:12:08 - might not be so different.
  • fast_forward00:12:10 - Exactly. This is very good, right? Because by using a broad definition of law,
  • fast_forward00:12:14 - you should look at, let's say, also the interaction of multiple factors.
  • fast_forward00:12:17 - And otherwise, you will be really pushed, let's say, drawing boundaries maybe
  • fast_forward00:12:20 - between them, which would not help you in understanding joint action.
  • fast_forward00:12:24 - So this is to understand.
  • fast_forward00:12:25 - But then, so we have entrainment. So now a lot of we were nodding and sort of
  • fast_forward00:12:30 - we're smiling at regular intervals. So we're really untrained.
  • fast_forward00:12:34 - But now we're speeding, right? So I'm speaking faster or slower.
  • fast_forward00:12:38 - Or apparently contributing to our joint action. So how should I think about that?
  • fast_forward00:12:46 - Yeah, I mean, entrainment is a process, you know, where we don't need to have
  • fast_forward00:12:50 - any internal representations about one another.
  • fast_forward00:12:53 - It just happens between us that we are pulled into something.
  • fast_forward00:12:58 - Speeding is a process where people actually try to sort of modify their own
  • fast_forward00:13:03 - task in order to make coordination possible.
  • fast_forward00:13:07 - And that is something that could come in handy when you don't know your partner's task very well.
  • fast_forward00:13:16 - And I think we have discovered this by accident because we were actually looking
  • fast_forward00:13:22 - for a different coordination process.
  • fast_forward00:13:25 - And we found in several different experiments that in situations where the task
  • fast_forward00:13:30 - was somehow ill-posed, people tended to just speed up.
  • fast_forward00:13:35 - And this could have been an instance of a very old sort of psychological phenomenon
  • fast_forward00:13:41 - that's called social facilitation.
  • fast_forward00:13:43 - You just get faster if you're doing stuff together with other people,
  • fast_forward00:13:48 - even if you do it not together, but also if you're just in the same room, let's say.
  • fast_forward00:13:52 - But then we notice that basically speeding
  • fast_forward00:13:55 - up also implies something statistic you know which is
  • fast_forward00:13:58 - just a statistic effect because it reduces variability if
  • fast_forward00:14:02 - you get faster you go to your limits you know
  • fast_forward00:14:05 - and you you can't become faster and that automatically means that your actions
  • fast_forward00:14:10 - timing becomes less variable so what you require for coordination is actually
  • fast_forward00:14:17 - in for instance in order to be synchronous or to engage in a certain rhythm.
  • fast_forward00:14:25 - Basically less variability helps you to be more synchronous, something like that.
  • fast_forward00:14:31 - So in some ways you're saying by finding maybe the right speed,
  • fast_forward00:14:35 - because it's not that you operate at the maximum speed, you just speed up as
  • fast_forward00:14:39 - compared when you act on your own.
  • fast_forward00:14:42 - Yes, but actually in our task most of the time that means going to your limits actually.
  • fast_forward00:14:48 - Okay like what's a task um uh in our
  • fast_forward00:14:51 - in our case it was a simple reaction time task but
  • fast_forward00:14:55 - it could also be you know any motor task i'm pretty sure
  • fast_forward00:14:57 - but but the point you to make
  • fast_forward00:15:00 - is okay by speeding up you become more predictable and this
  • fast_forward00:15:03 - would facilitate the joint action i mean it would
  • fast_forward00:15:06 - help the other to deal with you yeah it's
  • fast_forward00:15:12 - it's not even that i mean you're not trying to make
  • fast_forward00:15:15 - yourself predictable for the other you're just speeding up and you
  • fast_forward00:15:18 - know that in itself basically um
  • fast_forward00:15:22 - reduces your variability and so
  • fast_forward00:15:26 - basically it it increases the chances that you will be more synchronous with
  • fast_forward00:15:31 - the other person so for instance if you if you uh if you go faster and faster
  • fast_forward00:15:36 - and faster um basically you know if you if you have to clap a very slow rhythm together for instance.
  • fast_forward00:15:45 - And you need to stay one second apart or something like that.
  • fast_forward00:15:48 - It's pretty likely that if you try to do that now with clapping,
  • fast_forward00:15:54 - we would produce large asynchronies and our clapping wouldn't fall together.
  • fast_forward00:15:59 - If we clapped very fast, we'd have a pretty good chance actually that actually
  • fast_forward00:16:05 - these variabilities would be much lower.
  • fast_forward00:16:08 - And in the same sense basically, speeding up can can help you to be in time with one another.
  • fast_forward00:16:17 - So the speeding up effect is more closely related still to the entrainment we talked about before.
  • fast_forward00:16:26 - Or not? Because you're saying by speeding up, you increase the probability to
  • fast_forward00:16:31 - synchronize, so that would be in the form of entrainment or not.
  • fast_forward00:16:37 - I think we have to make a distinction between synchronicity and entrainment.
  • fast_forward00:16:43 - I think entrainment is one process that can produce synchronicity,
  • fast_forward00:16:47 - but there are other processes that can produce synchronicity too.
  • fast_forward00:16:52 - I think what I'm saying is that speeding can also produce use synchronicity
  • fast_forward00:16:56 - all right understand so for your synchronicity is in some sense then,
  • fast_forward00:17:00 - synonymous to a joint action in a broad sense like joint
  • fast_forward00:17:03 - action by definition must show some synchronicity
  • fast_forward00:17:07 - many joint actions need to show that
  • fast_forward00:17:10 - i mean some other joint actions would actually require turn taking okay fair
  • fast_forward00:17:15 - enough yes okay very good yeah you're right so but for these for this subclass
  • fast_forward00:17:19 - of joint actions because you know i mean that's sort of there was also one one
  • fast_forward00:17:24 - main message of the talk to say that, in a sense,
  • fast_forward00:17:28 - in order to solve different coordination problems that you have in order to perform joint actions,
  • fast_forward00:17:36 - you can have different processes for solving these coordination problems.
  • fast_forward00:17:40 - And it's not a single one, so to say. And speeding is one that can actually
  • fast_forward00:17:44 - help you to be in synchrony with another person if you don't know a lot of other
  • fast_forward00:17:49 - stuff about that person.
  • fast_forward00:17:50 - And it is a strategy because we were only observing it in the experiments when
  • fast_forward00:17:55 - we asked people to be synchronous with one another and not when they sort of
  • fast_forward00:18:00 - performed the task independently.
  • fast_forward00:18:03 - So in a sense, speeding can be an intentional strategy that can actually lead
  • fast_forward00:18:12 - to achieving synchronicity.
  • fast_forward00:18:14 - But now tell me, if in the speeding, in this component of speeding,
  • fast_forward00:18:19 - it seems to imply that the behavior itself is again oscillatory in some way.
  • fast_forward00:18:26 - It has some frequency component and I'm sort of speeding or slowing that frequency element.
  • fast_forward00:18:33 - This now helps me to catch up with this other oscillator, which is another human doing something.
  • fast_forward00:18:37 - And then once I caught up, I can now set my frequency equal to the other.
  • fast_forward00:18:42 - Right. Okay, so maybe speeding is not the best way to name this.
  • fast_forward00:18:49 - Okay. Because we don't mean sort of speeding up in a task.
  • fast_forward00:18:54 - We mean sort of speeding up compared to individual performance. Mm-hmm.
  • fast_forward00:18:58 - So that basically what we had was actually individual reactions to a stimulus.
  • fast_forward00:19:04 - So it's not a rhythmic action like, you know, in a rocking chair.
  • fast_forward00:19:08 - Right. But it's sort of discrete responses to stimuli.
  • fast_forward00:19:11 - Sure. So if you need sort of this, for these discrete responses to stimuli,
  • fast_forward00:19:17 - speeding up would be a strategy to be synchronous.
  • fast_forward00:19:21 - But then if I understand it well, then it's like.
  • fast_forward00:19:25 - If you compare the joint case versus individual case, you see that in processing
  • fast_forward00:19:29 - discrete stimuli, like my reaction time to the stimulus, there I'm faster,
  • fast_forward00:19:33 - right? So this is the data.
  • fast_forward00:19:37 - But then in your interpretation of something like, oh, but by speeding up,
  • fast_forward00:19:41 - I make it more likely to synchronize with the other.
  • fast_forward00:19:44 - That's something that is supposedly going to happen after you saw the stimulus
  • fast_forward00:19:48 - because we only know now we're speeding up to see the stimulus.
  • fast_forward00:19:51 - No, I think it's a general strategy, really. I mean, it's sort of a different
  • fast_forward00:19:55 - mindset of approaching the task because what we saw in our experiments is that
  • fast_forward00:20:00 - people actually didn't speed up during the experiment.
  • fast_forward00:20:02 - So there's a constant difference between the individual and joint condition
  • fast_forward00:20:06 - right from the start. Okay.
  • fast_forward00:20:08 - So that it just means when you are asked to do it synchronously with me, you're going faster.
  • fast_forward00:20:18 - And you do that right away, so to say. And that modulation is an order of magnitude, I guess not.
  • fast_forward00:20:26 - It is pretty much. So it's a task where people would normally,
  • fast_forward00:20:30 - in the individual condition, would normally require about 400 milliseconds to respond.
  • fast_forward00:20:36 - The joint conditions, they are 50 milliseconds faster.
  • fast_forward00:20:40 - So it's quite substantial. I mean, it's only 50 milliseconds.
  • fast_forward00:20:46 - Seconds but uh in experimental psychology in that sort of time window it means
  • fast_forward00:20:50 - a lot exactly sure so i mean it's a huge it's a huge difference so um.
  • fast_forward00:20:58 - I mean, I think our most convincing data is in analyzing the correlations between
  • fast_forward00:21:03 - reaction times, variability, and asynchrony. Okay, because what people need
  • fast_forward00:21:11 - to achieve is asynchrony.
  • fast_forward00:21:12 - And what you can show is that if you ask people to synchronize intentionally,
  • fast_forward00:21:17 - basically, people, you have a big relationship between reaction times and variability.
  • fast_forward00:21:25 - And again, between variability and asynchrony, and no relationship between reaction
  • fast_forward00:21:31 - time, direct relationship between reaction times and asynchrony.
  • fast_forward00:21:36 - Whereas in an unintentional condition, the main relationship is actually a direct
  • fast_forward00:21:42 - relation between the reaction times and the asynchrony.
  • fast_forward00:21:45 - And variability doesn't play a role. Which really means that the relationship,
  • fast_forward00:21:52 - that people are really sort of speeding up in order to reduce variability,
  • fast_forward00:21:57 - which in turn actually reduces the asynchrony.
  • fast_forward00:22:03 - That at least is our interpretation. Right, exactly. Because now you see there's a causality here.
  • fast_forward00:22:09 - Right. And it's only correlations. I have to admit that.
  • fast_forward00:22:13 - Okay. Okay, so I want to ask whether you had any way to really now dissect this
  • fast_forward00:22:18 - assumed causal structure. Have you made progress on that?
  • fast_forward00:22:22 - Well, I think our argument would be that you have to do basically exactly the
  • fast_forward00:22:29 - same, at least primary task, in an unintentional condition and in the coordination condition.
  • fast_forward00:22:36 - And just the requirement for coordination completely changes the pattern of
  • fast_forward00:22:43 - correlations between the three parameters.
  • fast_forward00:22:45 - But again, it's only correlations, and I think we have to find manipulations that allow us to add,
  • fast_forward00:22:57 - for instance, variability in order to actually
  • fast_forward00:23:01 - provide further evidence right i was wondering about
  • fast_forward00:23:04 - how this would generalize right because imagine it's not
  • fast_forward00:23:07 - let's say an image uh detection task
  • fast_forward00:23:10 - or something like that but it is a task where you
  • fast_forward00:23:13 - use different modalities or maybe it's even more an active task which you also
  • fast_forward00:23:17 - believe or do you have information that the same speeding occurs um i think
  • fast_forward00:23:23 - it's a very general strategy so if you um because it It just means that as soon as you don't know,
  • fast_forward00:23:32 - when you know you need to be synchronous,
  • fast_forward00:23:34 - and if you and another person have roughly the same task, basically when you
  • fast_forward00:23:41 - speed up, it's likely to work better.
  • fast_forward00:23:45 - But I could also argue, well, maybe it's a form of competition.
  • fast_forward00:23:48 - You want to just be better than the other. That's why you speed up.
  • fast_forward00:23:52 - Is that an alternative explanation? It is an alternative explanation.
  • fast_forward00:23:56 - And it would be, I think, you know, many people would be sort of more interested.
  • fast_forward00:24:01 - Also in social facilitation, you know, heightened arousal is the explanation.
  • fast_forward00:24:06 - But I think our only point is, what we want to point out is,
  • fast_forward00:24:11 - here's this very simple strategy.
  • fast_forward00:24:13 - How, you know, changing the performance on your own task could actually help
  • fast_forward00:24:17 - you to achieve synchronization with somebody else.
  • fast_forward00:24:20 - Where you know most of the time people would think
  • fast_forward00:24:22 - well in order to to achieve that you need to decode the
  • fast_forward00:24:25 - other's intention or you know simulate the other's actions
  • fast_forward00:24:28 - right and we want to propose this you know
  • fast_forward00:24:31 - as a computationally or so cheap uh way of actually
  • fast_forward00:24:35 - getting um things synchronized uh
  • fast_forward00:24:38 - between people so for you this is like a joint action
  • fast_forward00:24:42 - reflex or joint action heuristic or
  • fast_forward00:24:45 - something like that but completely it's a it's innate in the
  • fast_forward00:24:48 - system it will always do this you don't need to learn this no okay
  • fast_forward00:24:51 - people just do it right i mean that's what we saw
  • fast_forward00:24:53 - in our experiments they didn't have to acquire this they just do
  • fast_forward00:24:56 - it did have you looked at how how this appears in development
  • fast_forward00:24:59 - no uh but this was i think it would be really interesting if if kids showed
  • fast_forward00:25:05 - uh the same tendency right exactly yeah all right so so now we have let's say
  • fast_forward00:25:11 - entrainment the most simple form of sort of synchronization now we have speeding where we Indeed,
  • fast_forward00:25:17 - I'll start looking at, let's say, behavioral strategies to facilitate synchronization.
  • fast_forward00:25:22 - And the next one is now really like a big jump up in, let's say,
  • fast_forward00:25:25 - the cognitive ladder, because now we're going to talk about simulation. Right. Right. So-
  • fast_forward00:25:30 - So how does simulation help me now in joint action? What am I simulating?
  • fast_forward00:25:35 - What you're simulating is the other's task.
  • fast_forward00:25:38 - The task or the intentions? Actually, not the intentions. It's aspects of the other's performance.
  • fast_forward00:25:46 - And, of course, you need to sort of represent somehow the other's task,
  • fast_forward00:25:51 - at least to some detail, and to which detail is actually unknown.
  • fast_forward00:25:55 - And that can also vary a lot, but that will be a different talk.
  • fast_forward00:25:59 - But you have to represent that the other person has a task and you have to represent
  • fast_forward00:26:03 - some details of this task.
  • fast_forward00:26:05 - And the idea then is that your own action knowledge allows you to actually predict
  • fast_forward00:26:10 - how difficult it will be for the other person to perform this task or how long
  • fast_forward00:26:16 - it will take for the other person to perform this task.
  • fast_forward00:26:19 - Am I modeling the user, the other, and the other's performance?
  • fast_forward00:26:24 - Or am I really modeling the task? If I model the task... I think you model the
  • fast_forward00:26:28 - other's performance. Okay, okay.
  • fast_forward00:26:30 - Because then the question is, is now my modeling of the other's performance,
  • fast_forward00:26:36 - some sort of modulation of my own model, of my own performance,
  • fast_forward00:26:41 - or is it something that is really separated from that?
  • fast_forward00:26:45 - Right. So actually, I mean, the idea is that you use your own motor system,
  • fast_forward00:26:49 - and it sounds perhaps a bit funny,
  • fast_forward00:26:51 - that you use your own motor system to plan your own actions,
  • fast_forward00:26:55 - and at the same time you use the same motor system to actually also predict
  • fast_forward00:26:59 - what the other person will do so the idea is really of.
  • fast_forward00:27:05 - In a sense simulating what the other does with your own action knowledge actually biases,
  • fast_forward00:27:13 - how you perform your own action so in
  • fast_forward00:27:16 - a sense and this would also involve a predictive of the
  • fast_forward00:27:19 - prediction of the consequences of the action so in a
  • fast_forward00:27:22 - sense it's an idea that there could be something
  • fast_forward00:27:24 - like a parallel prediction for your own action and the
  • fast_forward00:27:27 - other section so that means if i'm clumsy if i'm
  • fast_forward00:27:31 - a clumsy person then i will have a tendency to also model you as
  • fast_forward00:27:33 - a clumsy person but we have to perform a task together yes so
  • fast_forward00:27:37 - in a sense it would also depend on your experience for
  • fast_forward00:27:40 - instance so for instance if if you're a very good
  • fast_forward00:27:43 - dancer and you you perform dancing you will
  • fast_forward00:27:46 - have more motor activation in your system because you you can sort of simulate
  • fast_forward00:27:52 - better what you're observing um than when you're not an experienced dancer because
  • fast_forward00:27:57 - you will simulate a dance movement more like a walking movement let's say or
  • fast_forward00:28:02 - something that's familiar to you you can still sort of simulate aspects of it,
  • fast_forward00:28:06 - but not in a very fine-grained detail.
  • fast_forward00:28:11 - So that means you're sort of in this Merleau-Ponty school there of a perception
  • fast_forward00:28:16 - that my model of you is grounded in my model of myself.
  • fast_forward00:28:23 - Yeah, but not in a phenomenological sense, actually.
  • fast_forward00:28:27 - It's more in a functional sense, sense really um because
  • fast_forward00:28:30 - um uh i the assumption would
  • fast_forward00:28:33 - be that the the simulation itself um actually
  • fast_forward00:28:37 - is sort of completely unconscious it's not you know it's it's not a particular
  • fast_forward00:28:40 - experience connected to it it's more like um um a parallel prediction of what
  • fast_forward00:28:46 - could happen in the outside world so um it's i think it's more closely related
  • fast_forward00:28:51 - to you know the uh mirror system sort of theories or
  • fast_forward00:28:56 - common coding theories that sort of postulate a close link between perception and action. Mm-hmm.
  • fast_forward00:29:04 - And I think it's sort of, you know, you can link both action planning perception to external events.
  • fast_forward00:29:11 - And then basically, once you make that move, the motor system can actually contribute
  • fast_forward00:29:17 - to action understanding, to action simulation for other people too.
  • fast_forward00:29:21 - Okay. But then your simulation model is a bit more like a forward model in some
  • fast_forward00:29:29 - sense, right? Yes, that's true. I'm just predicting states of the world.
  • fast_forward00:29:32 - And the other is just another aspect of that world who then also has an impact
  • fast_forward00:29:38 - on that world. So changing its property.
  • fast_forward00:29:40 - So I can just make now forward predictions about what this impact will be.
  • fast_forward00:29:44 - That is true. But, you know, in some situations, I mean, one task that we use
  • fast_forward00:29:48 - in an EEG study was, you know, I'm handing a bottle to you. That was basically the task.
  • fast_forward00:29:54 - And then we were looking at preparation components for motor actions, for motor initiation.
  • fast_forward00:30:00 - And the interesting finding there was that the person who was actually sort
  • fast_forward00:30:05 - of receiving the bottle was showing a sort of motor peak that you normally see
  • fast_forward00:30:12 - for processes of motor simulation at the point where the person who was giving
  • fast_forward00:30:17 - initiated their action.
  • fast_forward00:30:19 - Action, so much before they actually initiated their own receiving action,
  • fast_forward00:30:24 - which was sort of very clear evidence that the motor system was actually making a contribution,
  • fast_forward00:30:30 - was sort of simulating the other's actions in addition to one's own action. Right.
  • fast_forward00:30:38 - So now, in simulation, it's very operational.
  • fast_forward00:30:44 - You define it in very operational terms, right? So you're not including anything
  • fast_forward00:30:47 - about making inferences about the goals of the other.
  • fast_forward00:30:52 - So you think that is not really a required component to account for joint action?
  • fast_forward00:30:59 - Oh, it's absolutely a required component.
  • fast_forward00:31:03 - That's sort of the aspect of joint action I didn't talk so much about,
  • fast_forward00:31:09 - which is sort of you have to have a goal representation that allows you to define
  • fast_forward00:31:13 - different roles in the joint action and also the joint outcome that you want to achieve.
  • fast_forward00:31:18 - And the idea is that this sort of goal representation actually drives the simulation
  • fast_forward00:31:24 - and reaches into the motor system.
  • fast_forward00:31:30 - And so that basically, you know, this sort of top-down modular,
  • fast_forward00:31:35 - this sort of high-level planning structure, let's say, actually sort of ties,
  • fast_forward00:31:41 - gets tied into the motor system.
  • fast_forward00:31:43 - Them and then does the appropriate simulation also takes care of the integration
  • fast_forward00:31:48 - of the information that comes out of the motor system.
  • fast_forward00:31:52 - Okay. But then the specific task that you described here was this musical task,
  • fast_forward00:31:58 - right? We have two people playing on a piano.
  • fast_forward00:32:00 - Right. It was under monitoring, though. Oh, sorry. Okay. I'm sorry.
  • fast_forward00:32:03 - Yeah. This is one step further. But it was interesting to see that also in that
  • fast_forward00:32:07 - task, actually, it had an impact on, let's say, with respect to predictions
  • fast_forward00:32:13 - pertaining to the other or the self.
  • fast_forward00:32:16 - Right. But let's discuss that experiment when we hit the monitoring.
  • fast_forward00:32:20 - Maybe I'm moving too fast here.
  • fast_forward00:32:22 - But now for the simulation, to what extent?
  • fast_forward00:32:27 - So you could imagine that I have a generic simulation system in my brain,
  • fast_forward00:32:32 - like a mirror mechanism and with all sorts of bells and whistles attached to it.
  • fast_forward00:32:38 - Would a generic modeling system or simulation system be sufficient to support
  • fast_forward00:32:43 - joint action or should it have some special ingredients that make joint action
  • fast_forward00:32:47 - then standing out from just predicting other aspects of the world?
  • fast_forward00:32:52 - I think it would capture different regularities. So I don't think that it would
  • fast_forward00:32:57 - necessarily need to be completely different in that it would also,
  • fast_forward00:33:07 - of course, capture regularities.
  • fast_forward00:33:09 - But the regularity that it captures would be very different.
  • fast_forward00:33:13 - And especially in those cases where people perform highly coordinated actions.
  • fast_forward00:33:20 - So for instance you know if you
  • fast_forward00:33:23 - take two dancers uh by coordinated tango dances
  • fast_forward00:33:26 - and so on um so that in
  • fast_forward00:33:29 - a sense uh you know the specifics of the other's body and movements would become
  • fast_forward00:33:35 - part of your own prediction models so that and so you know one way of thinking
  • fast_forward00:33:41 - about this is would be saying it's not different because it also captures regularities
  • fast_forward00:33:46 - just like any other forward model Yeah.
  • fast_forward00:33:50 - The other way of thinking about it would be that here we have a system that
  • fast_forward00:33:57 - captures a different type of regularity that doesn't only depend on my body,
  • fast_forward00:34:02 - but also on another body.
  • fast_forward00:34:03 - And could that perhaps capture regularities that are then interesting to use
  • fast_forward00:34:10 - for other functions in the system so that being in close interactions with others
  • fast_forward00:34:16 - could perhaps even have, let's say,
  • fast_forward00:34:18 - something like implications for your bimanual coordination, something like that.
  • fast_forward00:34:23 - And the other way around would be that perhaps in order for joint action,
  • fast_forward00:34:33 - you could, and that would be the similarity again,
  • fast_forward00:34:37 - could I use by manual models where I have to have some similar coordination
  • fast_forward00:34:45 - problems as when coordinating with other people?
  • fast_forward00:34:48 - So if you're shaking hands, I can do that with my two hands too.
  • fast_forward00:34:52 - Can I use the bimanual models as a proxy to bootstrap my joint action model from? Things like that.
  • fast_forward00:35:01 - Okay. But you do see a continuity there. There's not… Yeah. Okay.
  • fast_forward00:35:07 - It's not fundamentally different, but I could say, imagine, for instance,
  • fast_forward00:35:12 - if you wanted to talk about modules or separation of function for a bit.
  • fast_forward00:35:18 - Because of the different type of regularity that these forward models capture,
  • fast_forward00:35:24 - because there's perhaps more dynamics or more complex dynamics,
  • fast_forward00:35:27 - or because the biological movement has invariant characteristics somehow,
  • fast_forward00:35:33 - it could still be a specified way of modeling.
  • fast_forward00:35:39 - But the capturing of regularities would be the same. Right, exactly.
  • fast_forward00:35:45 - But then how quickly in development do you see this?
  • fast_forward00:35:50 - This ability to capture this and to make predictions. Right.
  • fast_forward00:35:54 - I don't think we know anything about this actually empirically,
  • fast_forward00:35:58 - but my hunch would be that in a sense these sort of low-level components are
  • fast_forward00:36:04 - in place before the sort of planning structures.
  • fast_forward00:36:08 - So that you have sort of forward models which are able to actually deal with,
  • fast_forward00:36:16 - let's say, combined human motion in place.
  • fast_forward00:36:21 - Before children start to perform intentional joint actions, where they sort
  • fast_forward00:36:30 - of take the initiative in the joint action.
  • fast_forward00:36:32 - Because, of course, in the broad definition of joint action.
  • fast_forward00:36:36 - Infants are involved in it right from the first day of birth because they're
  • fast_forward00:36:40 - being carried around and so on.
  • fast_forward00:36:42 - They make posture adjustments adjustments that
  • fast_forward00:36:45 - make it easy for their parents to to carry them
  • fast_forward00:36:49 - around and so on so i would expect that you
  • fast_forward00:36:52 - know a lot of this sort of motor stuff is in place and
  • fast_forward00:36:55 - that sort of the developmental boundary they
  • fast_forward00:36:58 - have to cross really is more in the development of um
  • fast_forward00:37:02 - of uh task representations that allows
  • fast_forward00:37:05 - them to keep you know self and other together assign different
  • fast_forward00:37:08 - roles to different actors things like that right okay so
  • fast_forward00:37:12 - now now I can simulate and so
  • fast_forward00:37:17 - that means I have more power in let's say
  • fast_forward00:37:19 - predicting and adjusting but now
  • fast_forward00:37:24 - now I must be able to let's say then monitor the quality of
  • fast_forward00:37:27 - my performance and the quality of the performance of the other okay
  • fast_forward00:37:30 - so so how does
  • fast_forward00:37:34 - this exactly work right i mean i
  • fast_forward00:37:37 - think we know that um people are very good at
  • fast_forward00:37:40 - monitoring the outcomes of their own actions and there's
  • fast_forward00:37:43 - also some previous research that shows that surprisingly uh when
  • fast_forward00:37:47 - we observe others performing actions you know we also monitor whether they make
  • fast_forward00:37:51 - an arrow or not and that uh the brain seems to do pretty much the same thing
  • fast_forward00:37:56 - in both cases so that you can can actually observe the same ERP-EG patterns.
  • fast_forward00:38:04 - ERP components in both cases.
  • fast_forward00:38:08 - Now, one question is whether.
  • fast_forward00:38:12 - In monitoring, you know, you have a separate monitoring mechanism for the joint
  • fast_forward00:38:17 - outcome of the actions, because in a piano duet,
  • fast_forward00:38:20 - basically what counts is the coordinated actions of both of us and not the individual playing.
  • fast_forward00:38:28 - So, you know, we can do a perfect performance of our part, of our individual
  • fast_forward00:38:33 - parts, and it can sound very terrible.
  • fast_forward00:38:36 - Right, exactly. And so the question is, you know, how do we represent this destroyed
  • fast_forward00:38:44 - outcome and are we able to monitor it?
  • fast_forward00:38:47 - So in this experiment, there's two piano players.
  • fast_forward00:38:50 - They play a fixed score, two notes at a fixed rhythmicity.
  • fast_forward00:38:56 - And then you started to sort of perturb the sound that was produced. Right.
  • fast_forward00:39:02 - So basically what we do is they have, these are expert pianists and they know
  • fast_forward00:39:07 - how to play pieces very well and basically plant errors in their performances while they're doing it.
  • fast_forward00:39:13 - So they play the right note, but they hear the wrong tone.
  • fast_forward00:39:16 - And uh we could so we were measuring eg from from two people at the same time
  • fast_forward00:39:23 - while they were playing a duet and uh we planted mistakes for the person themselves
  • fast_forward00:39:28 - for the other person and then we
  • fast_forward00:39:31 - basically uh planted two types of mistakes one mistake would basically affect
  • fast_forward00:39:37 - the joint performance a lot and the other mistake would affect it less and so
  • fast_forward00:39:44 - it would sort of either change or not change,
  • fast_forward00:39:46 - jointly produce harmonies.
  • fast_forward00:39:50 - And what we found is that early sort of monitoring components and error components
  • fast_forward00:39:56 - reacted in exactly the same way to self errors, other errors,
  • fast_forward00:40:02 - and it also didn't make a difference of whether to which extent a joint result was affected.
  • fast_forward00:40:08 - Whereas later components that are interpreted as signals of conscious error processing.
  • fast_forward00:40:18 - Were actually scaled with self-relevance.
  • fast_forward00:40:21 - So these components, so there was sort of more of an error signal when the person
  • fast_forward00:40:28 - made a mistake themselves than when the other person made the mistake.
  • fast_forward00:40:32 - But the second factor, and that's more interesting, is that these signals were
  • fast_forward00:40:37 - larger for self and other when the joint outcome was more affected.
  • fast_forward00:40:41 - So which really indicates that, you know, people have to do this monitoring of the joint outcome.
  • fast_forward00:40:49 - But how was the modulation factor there?
  • fast_forward00:40:52 - Because in that sense, now you had really parametric control over the error
  • fast_forward00:40:56 - that you could induce, or not?
  • fast_forward00:40:59 - No, I mean, it's not a parametric simulation because it's not on a sort of continuous pitch dimension,
  • fast_forward00:41:08 - because the music harmonies sort of constrain a lot what you can do there, so to say.
  • fast_forward00:41:15 - So it's not really true that we could have had parametric violation.
  • fast_forward00:41:25 - So we could only sort of come up with, and that was tough enough actually,
  • fast_forward00:41:30 - experimental manipulations where we had, we actually changed the joint harmony or not.
  • fast_forward00:41:38 - Okay. So, so either there was an error or there was no error. This is, this is.
  • fast_forward00:41:43 - I mean, there wasn't, okay, so in all four cases, we planted errors.
  • fast_forward00:41:52 - But these errors could be sort
  • fast_forward00:41:55 - of individual in the sense that they didn't affect the overall harmony.
  • fast_forward00:41:59 - They still fit into the jointly produced harmony.
  • fast_forward00:42:03 - So it was just, you know, we replaced one tone that belongs to a particular
  • fast_forward00:42:07 - chord, let's say, with another tone that also could belong to that chord.
  • fast_forward00:42:12 - So the chord remains unchanged.
  • fast_forward00:42:14 - Whereas in the other case, we actually change the pitch so that the chord actually flips.
  • fast_forward00:42:24 - And that changes the joint result more. Right.
  • fast_forward00:42:28 - But now, if you look at this P300 response, so these late components for error monitoring,
  • fast_forward00:42:34 - monitoring um how does it what does
  • fast_forward00:42:37 - it look like if we compare let's say own errors
  • fast_forward00:42:40 - versus the errors of the other right so
  • fast_forward00:42:45 - for the for the for the later component uh um basically uh it um it matters
  • fast_forward00:42:52 - more you know it's the component is much higher when you make a mistake yourself
  • fast_forward00:42:56 - okay but But there's still an error signal when the other person makes the mistake,
  • fast_forward00:43:02 - but only when it has consequences for the joint result.
  • fast_forward00:43:09 - In other words, sorry, sorry to interrupt, but just to clarify,
  • fast_forward00:43:12 - it has no impact on the joint outcome if the harmony is not affected,
  • fast_forward00:43:18 - is that what you're saying?
  • fast_forward00:43:22 - Okay so the error signal so if
  • fast_forward00:43:26 - i'm if if uh if you if i
  • fast_forward00:43:29 - planted an error in your playing you you
  • fast_forward00:43:32 - would always find the error component yeah exactly and
  • fast_forward00:43:35 - if i'm your partner in the in the piano duet um
  • fast_forward00:43:39 - we would only find uh an
  • fast_forward00:43:42 - error component if my arrow
  • fast_forward00:43:46 - affects our harmony but not if
  • fast_forward00:43:49 - it doesn't affect our harmony okay all right and your
  • fast_forward00:43:52 - arrow component for your own error is is
  • fast_forward00:43:55 - also higher when the individual when
  • fast_forward00:43:59 - the sorry when the joint harmony is affected so even
  • fast_forward00:44:02 - for your own mistake you know the job uh it's
  • fast_forward00:44:05 - it's sort of worse it's there's a bigger error signal
  • fast_forward00:44:08 - when it affects the joint reside more so
  • fast_forward00:44:11 - the error you know that uh in this component so to say the error there's less
  • fast_forward00:44:17 - of a of an error signal when you make an individual mistake that doesn't affect
  • fast_forward00:44:22 - our joint harmony so that means um errors detected in my own play yeah are.
  • fast_forward00:44:30 - Sort of lead to more vigorous responses yes
  • fast_forward00:44:33 - than those in the play of others right of
  • fast_forward00:44:36 - the other and um how is this
  • fast_forward00:44:38 - expressed really in the at the erp level is it like really
  • fast_forward00:44:41 - an amplitude difference or is it so okay so it's
  • fast_forward00:44:44 - a it's a difference in the p300 and the
  • fast_forward00:44:47 - p p300 is really scaled in our four
  • fast_forward00:44:50 - conditions so it's highest if if it's
  • fast_forward00:44:53 - your own error and it affects the joint actions it's second highest and significantly
  • fast_forward00:44:58 - different if it's your own error and if it uh if it doesn't affect the joint
  • fast_forward00:45:04 - outcome it's third highest if it's my error the other's error and it affects the joint outcome,
  • fast_forward00:45:11 - and there's no error component whatsoever if I make the error and it doesn't
  • fast_forward00:45:17 - affect the joint outcome. Okay.
  • fast_forward00:45:19 - But then in case of monitoring, so here we see that monitoring affects both
  • fast_forward00:45:27 - the joint action and the own action.
  • fast_forward00:45:29 - But now, isn't that a bit in a contradiction with the idea of the simulation
  • fast_forward00:45:35 - where you would say, They say, well, the simulation runs on a more generic infrastructure.
  • fast_forward00:45:39 - So then you would expect that self and other monitoring would lead to roughly similar responses.
  • fast_forward00:45:46 - I would guess, okay? I mean, you know better than me. But isn't it surprising
  • fast_forward00:45:50 - to see that there's such a big difference between these two? Yeah.
  • fast_forward00:45:55 - Yeah, I think it depends a lot on the domain again, really.
  • fast_forward00:45:59 - And I think for the type of continuous action that's required for piano playing,
  • fast_forward00:46:04 - where you also have sort of, you know, normatively, actually,
  • fast_forward00:46:07 - you know, in the notation, to play two different pieces,
  • fast_forward00:46:15 - I'm not exactly sure how much simulation could do.
  • fast_forward00:46:18 - But I also have to say, it has to do something,
  • fast_forward00:46:23 - thing because we actually know that you know piano players who
  • fast_forward00:46:26 - actually hear piano playing do have a lot
  • fast_forward00:46:28 - of the motor activations so um
  • fast_forward00:46:32 - perhaps it you know depends a
  • fast_forward00:46:35 - little bit on which aspect so that you know in the piano playing when
  • fast_forward00:46:38 - it when it comes to pitches and harmony you actually
  • fast_forward00:46:41 - have to keep it separate for safe and other and you know the joint harmony has
  • fast_forward00:46:44 - to govern stuff but if you looked at the expressive dynamics perhaps you know
  • fast_forward00:46:50 - perhaps you would still apply aspects of your own actions to these expressive timing aspects.
  • fast_forward00:47:01 - There's no reason to believe that only one or the other process can be active
  • fast_forward00:47:05 - at a time in a joint action. Right.
  • fast_forward00:47:10 - So now we have a monitoring system. And then our other element is then signaling.
  • fast_forward00:47:25 - So how does signaling now play a role in this?
  • fast_forward00:47:28 - Because so I can simulate, I can monitor, I can use this to sort of synchronize my actions.
  • fast_forward00:47:34 - So why do
  • fast_forward00:47:37 - i need signaling now to keep on going right i
  • fast_forward00:47:40 - mean i think that's i think uh
  • fast_forward00:47:43 - you know the uh the the simulating business
  • fast_forward00:47:46 - uh uh and perhaps also the monitoring business if you don't have a lot of feedback
  • fast_forward00:47:52 - about one another can be difficult so for instance the simulation mechanism
  • fast_forward00:47:57 - i talked about is uh you can actually you actually have yourself to use as a
  • fast_forward00:48:02 - proxy for or the other person and go from there.
  • fast_forward00:48:04 - And sometimes that might not be the best thing to do.
  • fast_forward00:48:07 - Or perhaps you don't have the skill that you would need to be simulating if
  • fast_forward00:48:11 - you want to perform a particular joint action with another person.
  • fast_forward00:48:16 - So that signaling is another way of actually achieving coordination.
  • fast_forward00:48:24 - For instance, when you don't share a skill, or it might just be the more effective
  • fast_forward00:48:29 - method in a particular situation.
  • fast_forward00:48:31 - So, for instance, when you can give each other signals, when you can sort of, how should I say,
  • fast_forward00:48:42 - when a discrete way of sort of affecting the action planning is actually more important than...
  • fast_forward00:48:52 - So let me give you an example. I think that would make it better to understand.
  • fast_forward00:48:57 - So if you're carrying a box together, I could probably simulate your behavior,
  • fast_forward00:49:02 - but you know, we have different perspectives, you walk in a different way than me, and so on.
  • fast_forward00:49:07 - So if I'm sort of shoving the box in a particular direction.
  • fast_forward00:49:11 - This communicative signal might actually be much more effective in giving a, let's say,
  • fast_forward00:49:18 - unambiguous coordination signal than all of the simulations and the monitoring that I'm doing.
  • fast_forward00:49:27 - So that this is sort of our attempt to now link up to a more sort of symbolic
  • fast_forward00:49:37 - way of achieving coordination but also retaining the link to the motor system
  • fast_forward00:49:44 - because in joint action,
  • fast_forward00:49:46 - you say when we carry the box, we need to coordinate our action in space and
  • fast_forward00:49:52 - time and we want to provide this signaling
  • fast_forward00:49:56 - as a let's say um interface perhaps
  • fast_forward00:50:01 - uh for you know what would could also be
  • fast_forward00:50:04 - a language input you know you could also bias things in
  • fast_forward00:50:07 - a language way but people actually um often also do it in a more continuous
  • fast_forward00:50:13 - manner with these little sort of shoving signals because you know sometimes
  • fast_forward00:50:16 - it's hard to talk about exactly the actions that you need to do and you know
  • fast_forward00:50:20 - a haptic push or sort of a proprioceptive signal for the other person might
  • fast_forward00:50:24 - be more effective than talking a lot.
  • fast_forward00:50:27 - Right, exactly. But now the task you looked at was this pendulum task.
  • fast_forward00:50:31 - Actually, it's a box, there's a pendulum, then you have to pull a string left
  • fast_forward00:50:34 - or right which you can sort of move the pendulum up and down,
  • fast_forward00:50:38 - and then the instruction is to keep the pendulum within a certain range,
  • fast_forward00:50:42 - and what you looked at was, you know, when a single person performs this task,
  • fast_forward00:50:46 - pulling the two strings,
  • fast_forward00:50:47 - or a dyad, so two people will perform this test, each pulling one string. Yes.
  • fast_forward00:50:52 - So what's the difference between these two conditions? What do you observe?
  • fast_forward00:50:57 - Well, I mean, the interesting difference between these conditions is that people
  • fast_forward00:51:00 - do it in a completely different way.
  • fast_forward00:51:03 - So you can actually, what individuals do when they're confronted with balancing
  • fast_forward00:51:08 - this pendulum is that they apply forces in turn to the right and left side to keep it stable,
  • fast_forward00:51:14 - which means that the forces don't overlap very often on both sides,
  • fast_forward00:51:22 - which sort of saves them energy.
  • fast_forward00:51:23 - And it also means that the one hand doesn't get a lot of proprioceptive signals
  • fast_forward00:51:30 - about what the other hand is doing. Okay.
  • fast_forward00:51:33 - In a sense, if you perform this individually, you don't need these signals,
  • fast_forward00:51:37 - because you know what your other hand is doing, you know, within the system.
  • fast_forward00:51:42 - Whereas in the case where we are doing this together, and, you know,
  • fast_forward00:51:47 - we have to really control this pendulum movement in real time,
  • fast_forward00:51:51 - I don't know exactly, you know, how your action will unfold in real time.
  • fast_forward00:51:58 - So it's actually a good idea to pull at the same time so that we have a lot
  • fast_forward00:52:04 - of force overlap because then we can feel each other on both sides.
  • fast_forward00:52:09 - And that is how people achieve coordination.
  • fast_forward00:52:11 - And we can observe this in two different strategies to move the pendulum because
  • fast_forward00:52:16 - people sort of in the joint action condition pull at the same time and then
  • fast_forward00:52:22 - they lose at the same time and they start pulling at the same time again and so on.
  • fast_forward00:52:26 - Yeah, but as was pointed out by
  • fast_forward00:52:30 - Andreas Engel in your talk, people have different forms of feedback here.
  • fast_forward00:52:35 - It's not only the proprioceptive feedback or the force they might feel on the
  • fast_forward00:52:38 - string like the haptic feedback, but they also see what the other is doing.
  • fast_forward00:52:48 - So how would that translate in the specific synchronization effect that you observe?
  • fast_forward00:52:52 - It seems very odd, right? Because what you observe is that the diator,
  • fast_forward00:52:55 - two people basically start to pull simultaneously, which your device allows,
  • fast_forward00:53:00 - while the single user is sort of pushing or pulling alternately left or right. Right, yes.
  • fast_forward00:53:06 - And in both conditions, does the pendulum end up swinging at the same frequency?
  • fast_forward00:53:11 - Well, we instructed them to achieve a certain frequency and a certain amplitude.
  • fast_forward00:53:17 - So, and we observed the same difference across, you know, all sorts of different task conditions.
  • fast_forward00:53:25 - So, you know, whether you had to move the pendulum in the frequent,
  • fast_forward00:53:30 - let's say three times per second from one side to the other,
  • fast_forward00:53:33 - or let's say two times, didn't matter.
  • fast_forward00:53:37 - And also not how far you had to move it, whether you had to let it fall down
  • fast_forward00:53:43 - almost completely or whether you had to turn it sort of only in a small amplitude.
  • fast_forward00:53:51 - So across all conditions that we observed, it was pretty much all that the device
  • fast_forward00:53:58 - allowed, we actually observed the same difference.
  • fast_forward00:54:00 - And your point about vision, of course, vision was involved all of the time.
  • fast_forward00:54:06 - But you had equal visual information in the visual case and in the joint case.
  • fast_forward00:54:11 - So that in both cases, in a sense, you need visual control of the pendulum in
  • fast_forward00:54:17 - order to do the task at all.
  • fast_forward00:54:19 - But I think the signaling went through the haptic channel.
  • fast_forward00:54:22 - Because it would have been hard to observe, to create a false overlap in vision.
  • fast_forward00:54:28 - I think that's also really an interesting aspect of proprioception,
  • fast_forward00:54:33 - because it's sort of less this in a sense.
  • fast_forward00:54:38 - Perhaps it has different advantages compared to the visual system as an information channel.
  • fast_forward00:54:49 - Well, but in some sense it was also biased by the task. I mean,
  • fast_forward00:54:52 - people couldn't do anything else than just pull the string.
  • fast_forward00:54:55 - Yes, absolutely. But that's what we have to do as experimental psychologists.
  • fast_forward00:54:58 - Of course. No, this is clear.
  • fast_forward00:54:59 - But now, the alternative could be that, let's say, people just prefer to move
  • fast_forward00:55:04 - with a certain movement frequency.
  • fast_forward00:55:06 - Yes. Right. So, that means the individual user alternating effectively has the
  • fast_forward00:55:13 - same, let's say, effort movement frequency as the dyad that's sort of pulling simultaneously.
  • fast_forward00:55:20 - Right. So, with one swing of the pendulum, you still move an equal amount of
  • fast_forward00:55:25 - pull, an equal amount of times.
  • fast_forward00:55:27 - Right. Can you exclude that, that sort of movement frequency?
  • fast_forward00:55:30 - We can exclude that with the tempo manipulation that we did.
  • fast_forward00:55:35 - Because in a sense we were going from, I think it was like one turn per second,
  • fast_forward00:55:43 - or actually two turns per second, to up to five turns per second.
  • fast_forward00:55:47 - And that is a wide range of tempo, basically, and we observe the same differences
  • fast_forward00:55:55 - in all of these conditions,
  • fast_forward00:55:57 - whether you were going first, you know, whether individuals and groups were
  • fast_forward00:56:01 - going fast or small, the difference was always there.
  • fast_forward00:56:04 - But now the other thing about signaling is that in some sense you could argue
  • fast_forward00:56:08 - this haptic feedback is more like an implicit signal because if I'm part of the diet,
  • fast_forward00:56:14 - I'm not really sort of deciding to, okay, let's inform the other about what
  • fast_forward00:56:18 - I'm doing. You get it for free because I'm pulling the string.
  • fast_forward00:56:22 - I think that's a really interesting aspect of it.
  • fast_forward00:56:26 - And I'm not sure whether I would actually want to talk a full-blown communication here.
  • fast_forward00:56:30 - But I find it even more interesting than if it was full-blown communication,
  • fast_forward00:56:35 - because here seems to be a case where perhaps you have something in between
  • fast_forward00:56:39 - intending to communicate, and actually something that the task requires.
  • fast_forward00:56:45 - And my hunch is that if you talk about evolution development of these things,
  • fast_forward00:56:53 - this could be sort of an interesting in-between case.
  • fast_forward00:56:55 - And I think in general, haptic coupling in these sort of practical joint actions
  • fast_forward00:57:00 - is one way of trying to bootstrap communication, intentional communication,
  • fast_forward00:57:06 - from something simpler.
  • fast_forward00:57:08 - Right. It's more like implicit communication in this case, yeah? Yeah. Okay.
  • fast_forward00:57:12 - That's very good. Good. Okay, so now we have this list going from entrainment to signaling.
  • fast_forward00:57:19 - So how many steps are still missing in the list?
  • fast_forward00:57:23 - I don't know. I mean, I think we will find out in the future.
  • fast_forward00:57:27 - I'm pretty sure that there are still steps missing.
  • fast_forward00:57:30 - And I think it will also be interesting to see how the different processes can work together there.
  • fast_forward00:57:37 - And then also, you know, which kind of task representations and separations
  • fast_forward00:57:41 - between task representations. you need and how specifically your representations
  • fast_forward00:57:46 - of the other staffs need to be for these different processes to be useful.
  • fast_forward00:57:51 - So for entrainment, for instance, you don't need any representation of the other
  • fast_forward00:57:55 - person, whereas for simulation you absolutely need it.
  • fast_forward00:58:00 - So then, if I now want to build a robot that can pull the string of the pendulum
  • fast_forward00:58:05 - together with you, where would I start?
  • fast_forward00:58:08 - Would I really immediately have to include all of these five?
  • fast_forward00:58:11 - Or would you have one or two preferred elements where you say,
  • fast_forward00:58:14 - no, it should really start here, it's really bootstrapped from these two or three?
  • fast_forward00:58:18 - I think i you should implement
  • fast_forward00:58:21 - all of them okay it's sort
  • fast_forward00:58:24 - of a and i think you know some of i mean it's a cookbook um
  • fast_forward00:58:27 - i i really i mean at least that was the idea sort of and i think in terms of
  • fast_forward00:58:33 - of uh implementing them uh there's huge differences in how difficult it would
  • fast_forward00:58:38 - be because i think the the entrainment one and the speeding for instance could
  • fast_forward00:58:43 - be very easy um there's uh you You know,
  • fast_forward00:58:46 - everything that has to do with simulation will probably be very costly.
  • fast_forward00:58:51 - And I'm not exactly sure about the signaling because it would require something
  • fast_forward00:58:58 - like a very close crosstalk between modules that do perception action links
  • fast_forward00:59:06 - and communication modules or task representation modules.
  • fast_forward00:59:11 - And I think, to my knowledge, they are mostly kept separate in current architectures in robotics.
  • fast_forward00:59:18 - But that's also sort of the challenge we want to pose in robotics,
  • fast_forward00:59:21 - to think about the closer links between the different… But if I would go to
  • fast_forward00:59:26 - the other extreme, and I would look at other kinds of, let's say,
  • fast_forward00:59:28 - social animals, if I go to ants, for instance.
  • fast_forward00:59:31 - Would you qualify ants having joint actions?
  • fast_forward00:59:35 - It depends on your definition of joint action, again.
  • fast_forward00:59:38 - And I think, I mean, perhaps with our current definition, they would even count.
  • fast_forward00:59:45 - You know, in our current definition, it all depends on what the social interaction is, really.
  • fast_forward00:59:52 - And they can do coordination space and time. So I would think that,
  • fast_forward00:59:56 - you know, certainly training would exist on a level between ends.
  • fast_forward01:00:02 - And then I think what they do, and we haven't explored that at all,
  • fast_forward01:00:06 - is also use the environment a lot to actually achieve coordination.
  • fast_forward01:00:10 - So yes, I think I wouldn't be completely opposed to call some of what ends through joint actions,
  • fast_forward01:00:19 - but I would also try to be clear about exactly what coordination mechanisms
  • fast_forward01:00:27 - that involves and what it doesn't involve.
  • fast_forward01:00:30 - And they would certainly not have very elaborate task representation
  • fast_forward01:00:34 - right but you would see a continuum there
  • fast_forward01:00:37 - in some sense absolutely okay absolutely so I wouldn't
  • fast_forward01:00:40 - I mean that's sort of our general research strategy
  • fast_forward01:00:44 - not to make any sort of top you know tough
  • fast_forward01:00:47 - delineation on the way and just
  • fast_forward01:00:50 - see you know whether that can improve our
  • fast_forward01:00:53 - understanding okay very good so then to finish up two questions so you are studying
  • fast_forward01:00:59 - this whole domain of let's say social interaction or joint action intensely
  • fast_forward01:01:03 - for many years so in our in our study of this phenomena what would be gunter's law gunter's law yeah,
  • fast_forward01:01:12 - oh that's that's a tricky one.
  • fast_forward01:01:18 - Gunter's law,
  • fast_forward01:01:23 - um would be not to postulate too much internal representations and joint intentionality,
  • fast_forward01:01:33 - for social interaction and to consider what the sensory motor system can actually
  • fast_forward01:01:40 - do for social cognition quite in general.
  • fast_forward01:01:43 - Okay. And then last question. So five years from now, I'm going to visit you
  • fast_forward01:01:46 - there in Budapest and remind you of the hypothesis you're going to generate for me now,
  • fast_forward01:01:54 - which is, so what's the hypothesis or the prediction that you're that you're
  • fast_forward01:02:00 - most committed to today that you would like to remind you of five years from
  • fast_forward01:02:04 - now to see if it really came came true or not a prediction yeah,
  • fast_forward01:02:10 - about joint action about joint action a specific prediction on on the research you do.
  • fast_forward01:02:21 - Okay, maybe I should rephrase that as a hope, but I will predict that we will
  • fast_forward01:02:28 - have increasing impact on sciences that are interested in culture,
  • fast_forward01:02:32 - and at least we will try to make some links that allow anthropologists to understand
  • fast_forward01:02:40 - how sensory-motor processing coordination can actually contribute to the development of rituals.
  • fast_forward01:02:50 - And more sort of, how can I say, more sort of encultured actions, high-level actions,
  • fast_forward01:03:02 - and how there's a continuity between joint action, ritualized joint action,
  • fast_forward01:03:10 - and then sort of full-blown communication.
  • fast_forward01:03:12 - And that's sort of a path that we want to explore, actually together with developmentalists.
  • fast_forward01:03:17 - Also, we want to understand much more about the development of this phylogenetically
  • fast_forward01:03:23 - and autogenetically. That's beautiful.
  • fast_forward01:03:26 - Well, Gunther Knopple, thank you very much for this interview. Okay, thank you.
  • fast_forward01:03:33 - The CSN Podcast was produced by the Convergent Science Network of Biometrics
  • fast_forward01:03:38 - and Bio-Hybrid Systems, a project funded by the European 7th Research Framework Programme.
  • fast_forward01:03:45 - Music.

Be the first to leave a comment

Leave a comment

Your email address will not be published. Required fields are marked *

Exploring the convergence of neuroscience, robotics, and AI through conversations with leading researchers since 2010.

A project of the Convergent Science Network Foundation.

© CSN Podcasts. Developed by IMCreativeWEBC

0%

Login to enjoy full advantages

Please login or subscribe to continue.

Go Premium!

Enjoy the full advantage of the premium access.

Stop following

Unfollow Cancel

Cancel subscription

Are you sure you want to cancel your subscription? You will lose your Premium access and stored playlists.

Go back Confirm cancellation