Listeners:

Top listeners:

00:00 00:00

Go to album

Paul Verschure on consciousness and distributed adaptive control CSN Podcasts
Edvard Moser on grid cells and entorhinal cortex CSN Podcasts
Giacomo Rizzolatti on mirror neurons and action understanding CSN Podcasts
Robert Axelrod on game theory and prisoner's dilemma CSN Podcasts
Adrian Owen on disorders of consciousness and vegetative state CSN Podcasts
Jonathan Whitlock on markerless motion capture and posterior parietal cortex CSN Podcasts
Luis Puelles on neuroanatomy and prosomeric model CSN Podcasts
Zoltan Molnar on subplate neurons and cortical development CSN Podcasts

Etienne Koechlin on prefrontal cortex and cognitive control

Episode 10 15.03.2013

PLAY EPISODE

Season 2013

0 Followers 10 Episodes

Description

Why does the prefrontal cortex prefer not to be involved, and how does a cascade of cognitive control from premotor cortex to frontal pole organize human decision-making? Etienne Koechlin maps the hierarchical architecture of executive function.

Subscribe for more from the Convergent Science Network podcast series.

Etienne Koechlin presents a model of prefrontal cortex function centered on the idea that action imposes a fundamental constraint on cognition: it forces the brain to collapse multiple interpretations into a single committed choice. Rather than viewing the prefrontal cortex as a repository of complex representations, Koechlin argues its primary role is to introduce seriality and decisiveness into cognitive processing, excluding alternative interpretations so the organism can act. The system’s default state is automated behavior driven by premotor and posterior associative regions; the prefrontal cortex engages only when these routines fail.

The hierarchical organization follows a posterior-to-anterior gradient with three distinct levels. The premotor cortex stores basic stimulus-response associations. When these are ambiguous, the posterior prefrontal cortex incorporates immediate contextual cues from the present environment. When context is insufficient, more anterior regions access episodic information from the past. At the apex, the frontal pole enables the consideration of multiple alternative strategies simultaneously, breaking the pure seriality that characterizes lower levels. Koechlin emphasizes that these levels operate concurrently rather than sequentially, with the system recruiting more anterior regions only as needed.

A key function Koechlin attributes to the prefrontal cortex is monitoring, specifically judging whether a current behavioral strategy remains reliable based on its ability to predict action outcomes. He distinguishes between relative monitoring, which compares alternatives against each other, and absolute monitoring, which evaluates each strategy independently against a reliability criterion. The absolute approach avoids the trap of being locked into a limited set of alternatives and enables the critical decision of whether to persevere with learning or abandon a strategy entirely.

The episode reveals an intriguing limitation: humans can suspend one task to perform a subtask, but attempting a second level of recursive suspension produces severe performance deficits, suggesting the monitoring system operates at only one level without true recursion. Koechlin connects this to the broader question of how automatization transfers complex behaviors from prefrontal control to encapsulated routines in premotor and posterior cortical regions.

Tagged as:
cognitive control Frontal Pole Monitoring prefrontal cortex Premotor Cortex Regions

About the author

CSN Podcasts

Both the triumphs of humanity and its most evil deeds have resulted from collaboration. In a time where humanity is required to aspire to the former and minimize the latter, the question arises of how collaboration arises and why it fails. Surprisingly, this phenomenon, so central to who we are, is not well understood. Hence, a collaborative effort is required to understand collaboration in its full biological, psychological, sociological, cultural, and economic complexity and to translate this understanding into operational impact. This series of podcasts is one step toward achieving these complementary goals. The Collaboration Podcast presents interviews with people who are central orchestrators of collaboration in various domains including business, government, science, art, health, sustainability, and the military. The discussions were conducted by Prof. Dr. Paul F.M.J. Verschure and members of the Program Advisory Committee of the Ernst Strungmann Forum on Collaboration (https://www.esforum.de/forums/ESF32_Collaboration.html) during 2021 and had the goal to sketch a map of opportunities, challenges, and obstacles in human collaboration. The forum took place in May 2022, and now we would like to share this series of interviews with a broader audience. The full report of the Forum will be published in 2023 by MIT Press. The podcast was produced by the Convergent Science Network (https://www.convergentsciencenetwork.org/). Context: The stability of social systems depends critically on realizing sustainable methods of “collaboration,” yet how and by which means collaboration is achieved is not clearly understood; neither are the conditions or processes that lead to its breakdown or failure. Collaboration can be understood as cooperation between agents toward mutually constructed goals. Part of the reason for our lack of understanding is that the phenomenon of collaboration is, by nature, a highly multidisciplinary problem, and effective research into its complexities has been difficult to achieve across the broad range of scientific and technical disciplines involved. The need for a fundamental understanding of collaboration, however, has become increasingly important. Not only does humankind demand answers as it attempts to address critical challenges at multiple scales (e.g., climate change, migration, enhanced automation, social and economic inequality), but ever-increasing technological and economic means of interconnecting people and societies are disrupting long-established, familiar patterns of how we interact. Radical technological changes that are ongoing have the potential to reshape collaboration in ways that are currently hard to predict or influence (e.g., by altering configurations in interaction, information creation, and modes of communication). On one hand, such changes could disrupt hitherto stable forms of collaboration by affecting critical communication channels and traditional roles, as can be observed in the rapidly changing patterns in governance, commerce, and social interaction. Conversely, technology could lead to the emergence of novel, successful forms of collaboration that deviate from traditional “hierarchical” architectures. Evidence of this can be seen in areas as diverse as highly automated manufacturing plants, the open science movement, collaborative software repositories, user-centered services, and the sharing of economy-based modes of organization. Without a fundamental understanding of the mechanisms, processes, and boundary conditions of collaboration, it is not possible to evaluate or predict which of these possible scenarios are sustainable or even plausible. The Forum “How Collaboration Arises and Why it Fails” (May 8–13, 2022, Location: Frankfurt am Main, Germany) Chairs: Andreas Roepstorff and Paul Verschure Program Advisory Committee: Jenna Bednar, Julia R. Lupp, Bhavani R. Rao , Andreas Roepstorff, Ferdinand von Siemens, and Paul Verschure

Timestamp

00:00:03 - This is the Convergent Science Network podcast. Leading researchers in the domain
00:00:10 - of neuroscience, brain theory and technology are interviewed by Paul Verschoor and Tony Prescott.
00:00:27 - This is Paul Fouchard with the Convergent Science Network. And today I'm speaking
00:00:32 - with Etienne Kuchla, who was also a speaker in our summer school.
00:00:37 - And Etienne, you focus very much on the human prefrontal cortex.
00:00:43 - Not only your talk, also in your work in general. So, what is so special about this part of the brain?
00:00:53 - It's a huge question. First,
00:00:57 - why I'm interested in the prefrontal cortex and more generally in the frontal
00:01:01 - lobe function is that I think part of what makes us really humans compared to
00:01:08 - other primates lies especially in the prefrontal cortex.
00:01:14 - So, for me, it's a curiosity. It's a question of curiosity.
00:01:20 - And it's also a huge region, and it's a region that is really involved in,
00:01:25 - I think, in how we feel we are the actor of our own actions and behavior.
00:01:34 - And it's related to consciousness, to many different things that I'm interested in.
00:01:39 - Right. But now, you have sort of summarized your view on prefrontal cortex also
00:01:49 - in a very formalistic way.
00:01:51 - You had actually one very simple equation that you thought captured most of
00:01:56 - its function around sensory states and actions and so on and so on.
00:02:00 - So how can you then characterize the function of this complex structure in a
00:02:05 - simple equation? What is that equation exactly?
00:02:08 - Your question is how it is possible to simplify both actually what's the equation
00:02:15 - and how did you get there So first I like simple models because I think a model should be the.
00:02:24 - The most simple is the model, the most explanatory, I think it is, in one sense.
00:02:31 - So I really try to first find simple models.
00:02:36 - And I think also simple models are more intelligible.
00:02:41 - And also I develop simple models because I try to develop models that can be
00:02:46 - testable in experiment. I mean, and even simple models are not so easy to test
00:02:53 - and to confirm or infirm in experiment.
00:02:57 - And that's why I do simple models, I develop simple models, because I know that
00:03:03 - these models have some straightforward predictions.
00:03:06 - They might be a bit simplistic in one sense, but still with these models you
00:03:11 - can test them and you can tease apart some quite deep conceptual differences
00:03:17 - between different hypotheses.
00:03:20 - Right. So now prefrontal cortex is essentially in some way bringing together,
00:03:26 - perceptual states, states of
00:03:28 - the world, actions, and a sense of value or utility of their combination.
00:03:36 - Do you see those as the key ingredients upon which these areas operate Or is
00:03:42 - there another element to that?
00:03:44 - I think one of the most important key elements of the prefrontal function is action.
00:03:53 - Action has very specific constraints. I mean, first, action requires choosing.
00:04:00 - You cannot say that I think I will do this 80% of time and this 20% of time.
00:04:08 - When you really act in the world, you do one action, another one.
00:04:12 - So it's required making decision, making choice.
00:04:15 - And this is a huge consequence, because making choice engage you and is in one
00:04:21 - sense suboptimal to make choice.
00:04:24 - Better wait forever. No, I mean, to be optimal is always to have some multiple
00:04:30 - interpretation of the words and to continue with this kind of multiple representation.
00:04:37 - And when you do an action, basically you stick with an interpretation.
00:04:42 - So action is a very specific constraint. And I think one of the role of the
00:04:48 - prefrontal cortex on a very general view is to introduce this constraint into internal processing.
00:05:01 - So action is about choosing. It's about seriality. It's a reality,
00:05:06 - and basically the prefrontal function introduces all these constraints in the
00:05:10 - way the mind or cognitive process occurs in the brain.
00:05:15 - So action is very important. And second, of course, utility and values,
00:05:21 - if you want, are important.
00:05:24 - But, of course, they're important because you need value to know what is good for you or not.
00:05:32 - But value is very archaic. And I don't think that value is actually one of the
00:05:37 - key components of the prefrontal function.
00:05:39 - It's a key component of action and decision-making, of course.
00:05:43 - But I think the prefrontal...
00:05:46 - Function is more related to understanding and learning what seems to be a true
00:05:57 - representation of the world than really what is good or bad.
00:06:02 - Because I think even the most simple insect, the brain of the most simple insect,
00:06:07 - knows in one way what is good and bad for the organism.
00:06:12 - Right. So, what is specific to the human prefrontal function is that we have
00:06:17 - all this kind of, we can say, reasoning process that are not so much interesting in values,
00:06:25 - but are also interesting in what is true, what can be predicted,
00:06:30 - what is reliable, and so on and so on. Right.
00:06:33 - But now if I combine it, because on the one hand you're saying it's action and
00:06:38 - action is unitary at each point in time, I can execute only one.
00:06:42 - I have one body to act with. But on the other hand, you talk about,
00:06:46 - let's say, modeling the world context and so on.
00:06:50 - So these would be two functions. So, the action-selection component of this,
00:06:55 - where you actually are collapsing all of these possibilities that you can engage
00:07:01 - with, to collapse that into one interpretation,
00:07:04 - one action, you see both of these things reside in frontal areas,
00:07:08 - or is that in synergy with other areas?
00:07:11 - No, of course. I mean, the prefrontal
00:07:15 - cortex is in synergy with most other associative areas in the brain.
00:07:22 - I mainly focus on the prefrontal cortex because in one sense I think it's simpler
00:07:28 - because this, usually other associative regions are actually I think at the interface between,
00:07:39 - Peripheral current system like sensory system, like the visual system.
00:07:44 - Let us talk about the parietal region.
00:07:46 - The parietal cortex is very complex, actually, because it's at the interface
00:07:52 - between all this low-level sensory system, like vision and whatever.
00:08:00 - And interfacing this system with this internal cognitive system,
00:08:06 - which is the prefrontal cortex.
00:08:08 - So you have a two level of complexity within this region.
00:08:11 - Whereas in the prefrontal cortex, it's far away from this peripheral system.
00:08:18 - And I think, for me at least today, I think it's easier to understand what's
00:08:24 - going on in the prefrontal cortex and what's going on in this other associative
00:08:29 - area like temporal associative regions.
00:08:33 - They are very complex. I'm not sure nobody has a very good idea about what's going on in this region.
00:08:39 - In the parietal cortex, we have some cue about some specific things.
00:08:43 - But this is a huge region. There are so many things going on that I think that's
00:08:49 - what I like with the prefrontal cortex.
00:08:50 - I have the feeling that I understand something about the parietal cortex.
00:08:55 - But now, if you had to choose, right? Because, like I said earlier,
00:08:58 - you emphasize both action, unitary action, and you emphasize something like
00:09:04 - context. an internal model.
00:09:06 - You see them both as a function of prefrontal cortex, or do you see prefrontal
00:09:11 - cortex more as maintaining, let's say, these representations of what is possible?
00:09:16 - Or you see it as together, that there is both representations of possible together
00:09:23 - with... No, I think this is exactly the converse.
00:09:26 - The prefrontal cortex forces the mind, the human mind, the cognitive system
00:09:33 - everywhere in the brain,
00:09:35 - Not to multiply many possibilities. Okay.
00:09:42 - And it forced to make a choice, in one sense, to say, okay, the most probable
00:09:48 - interpretation of what I see in the scene is this, so I am going to act like that. And then?
00:09:55 - It's not necessary guys, other guys, to represent other alternatives.
00:10:01 - Because we decide to do that. So now we go for that.
00:10:04 - And at some point you need this, you need to simplify the representation.
00:10:08 - Otherwise, I mean, the system saturates very easily and very fast and very rapidly
00:10:13 - in making an inference about what is possible.
00:10:18 - So I think this is a region that, which means that it's a decision region.
00:10:23 - That really make decisions in the sense that exclude alternative interpretation. Okay.
00:10:30 - But then, so we can later look at how many alternatives you might want to consider.
00:10:36 - But then what you emphasize is that there are actually three sources of information
00:10:42 - in prefrontal cortex, right?
00:10:43 - You talked about context, episodic events or memory, and expected rewards.
00:10:50 - Words so how what are the boundaries of of these notions right so what's the
00:10:58 - difference exactly between context of action and episodic memory for instance uh so yeah,
00:11:08 - so uh the idea is that the context is something that is present when you make the selection,
00:11:16 - So in a basic way, there is no memory involved.
00:11:22 - The context is present here. Of course, it involves some memorized representation
00:11:27 - about how the context is connected to your action.
00:11:31 - But the context is present, basically. It's part of your environment where you make the selection.
00:11:39 - Episodic events is the past, basically. It's everything that happened in the
00:11:43 - past that, of course, you can memorize or not. and that may influence your actions.
00:11:49 - And I would say the opposite thing is expected reward, which is in the future,
00:11:55 - or expected outcome, more generally, is about the future, and it's just the
00:12:00 - symmetric of episodic events.
00:12:03 - So basically, the idea is very simple. I mean, you have the past,
00:12:07 - the present, information from the past, episodic event.
00:12:10 - The context is information from the present. and the future. Very simple.
00:12:17 - But this is, of course, in its generality, this also becomes,
00:12:21 - again, problematic, right? Yeah, I agree.
00:12:24 - Because each of these will be bounded in some way.
00:12:27 - So if you say, look, the prefrontal or the frontal lobes have access to past,
00:12:33 - present, and future, the question arises like, okay, if we imagine that these
00:12:38 - are not of infinite capacity, there must be boundaries on this.
00:12:42 - There must be aspects of past, present, future that you are considering because
00:12:47 - they're highly relevant.
00:12:48 - And there will probably be many aspects of it that you fully have to neglect to stay operational.
00:12:54 - So where would you draw that line?
00:12:57 - I think the line is drawn by your internal representation, by your learning, by your experience.
00:13:02 - So your internal model, what is very important is that, and I think this is
00:13:06 - one of the role of other associative regions, is to memorize and to implement
00:13:14 - and to code internal model of the world.
00:13:18 - So according to your internal model, I mean, a past event, even very close to
00:13:24 - your action, could be totally irrelevant and not deserving to be memorized.
00:13:29 - Or this kind of event could happen, I mean, a long time ago and could be very
00:13:35 - informative because your internal model tells you actually what is important in the world and not.
00:13:42 - And this is to be learned. And this is why I think this other associative region
00:13:47 - that is the parietal or the temporal cortex are very complex because this is
00:13:51 - probably where all these internal models that allow to capture information from the world,
00:14:00 - to make your selections are encoded.
00:14:04 - And the prefrontal cortex is organized in a way that it can make a difference between.
00:14:12 - What is,
00:14:15 - what is part of the immediate context.
00:14:19 - So there are some specific regions in the prefrontal cortex that allow you to
00:14:23 - include immediate information in your choice.
00:14:27 - But the prefrontal context by itself doesn't know what or which immediate information
00:14:33 - is useful in this situation.
00:14:35 - It just can say, okay, it just can include this in the selection process.
00:14:40 - So its role is really at the very end of the selection process to make the selection
00:14:46 - and to be able to include as many as information that can be processed by your
00:14:53 - internal models to elsewhere in the brain in the action selection process.
00:14:59 - Okay, so then in some way you're saying the magic resides in these areas at
00:15:06 - the interface between the sensory systems and the frontal lobe where in some
00:15:10 - way these internal models are constructed.
00:15:12 - But then, I guess I would expect that the frontal area would also add some intrinsic
00:15:22 - aspects to that process.
00:15:24 - They cannot just be, let's say, a selector driven and enslaved by information
00:15:29 - provided by other systems.
00:15:31 - So what would then be this added value? So one of the added value is that it's
00:15:36 - a monitoring system. This is what usually other people call metacognition in one sense.
00:15:43 - It means that it's a system that monitors all the time the processing,
00:15:48 - the behavior, and is able to make some important switch.
00:15:54 - So, for example, you may have an internal system, a very sophisticated system
00:16:00 - in the parietal cortex that you use to behave.
00:16:03 - And what the added value of the prefrontal cortex is to monitor all the time
00:16:10 - whether I should perseverate with this very complex strategy or possibly adjusting it or,
00:16:17 - should I something wrong with this strategy and I need to switch to something
00:16:23 - else so this is really the added value of the prefrontal cortex to have this
00:16:27 - kind of meta cognitive role in
00:16:31 - judging whether I should persevere,
00:16:35 - continue to learn, or to switch to something else, and just to give up with
00:16:41 - that and with this behavior and to try something else.
00:16:44 - This is exactly what's the problem in learning.
00:16:48 - When you learn something, of course you make errors or you get some negative feedbacks.
00:16:53 - At some point, there is a system that needs to say, okay, you make this error,
00:16:59 - but perseverate, learn.
00:17:03 - Or,
00:17:04 - Too many errors, it's no more valuable to learn this. You should change and
00:17:09 - give up and do something else.
00:17:10 - And this is the added value of the prefrontal cortex. Should I persevere in
00:17:14 - what I am doing, in what I am learning, or should I give up and switch to something else? Right.
00:17:20 - But then, so this is clear, right?
00:17:23 - So now we have sort of a functional understanding of this frontal area.
00:17:29 - And then already in your early work on this area, You seem to have identified
00:17:34 - a fairly clean mapping, if you want, of these functional components onto specific
00:17:41 - structures in the frontal lobe.
00:17:44 - So could you explain that in a bit more detail?
00:17:48 - So the idea is that, because as I said before,
00:17:52 - action is very important, the idea is that the prefrontal cortex is organized
00:17:55 - on the basis of the motor system, motor-premotor system.
00:18:04 - And the more you go more interiorly, the more you added some layers that allow
00:18:10 - you to add some additional information in the decision process.
00:18:13 - So this is the general idea of the organization.
00:18:17 - And the general idea is that the best, I would say, the goal of the prefrontal
00:18:23 - system is not to be involved, in one sense.
00:18:26 - That is that the more you are able to use routine or to routinize your action, it's good.
00:18:35 - So it means that you recruit additional layers when the routine in lower layer
00:18:41 - are not enough to resolve ambiguities in your actions.
00:18:45 - So in that way you recruit more and more entirely.
00:18:50 - Regions in the prefrontal cortex to solve the decision problem because decision
00:18:57 - is a problem first rather than a solution.
00:19:02 - But then if you recruit more areas what's the criteria to do that and how deep can you go?
00:19:10 - Yeah, so the idea is that first of course you have the basic stimulus that triggers
00:19:18 - the actions and this kind of stimulus response association are stored in the premotor cortex.
00:19:24 - So this is a very basic level. Then,
00:19:28 - If you have some ambiguities at this level, the first thing you want to know
00:19:33 - is whether in the immediate context, in the present context,
00:19:36 - there are some cues that help to disambiguate this.
00:19:40 - And this is the role of the posterior prefrontal region that just lie next to the premotor cortex.
00:19:48 - So this is the first layer to disambiguate actions, selection.
00:19:52 - Then if in the immediate context I mean there are no cues that help you to know
00:19:58 - which action you should select then you go more anteriorly and in this layer you have regions,
00:20:08 - that have access to more distant information more temporarily distant information and as we said mainly.
00:20:16 - Episodic information in the
00:20:17 - past events that occur maybe one minute ago that may provide some cues.
00:20:26 - And then you have the frontal pole which has a specific role which allows you
00:20:31 - to consider multiple alternatives.
00:20:34 - And which is important just to break the pure seriality of actions and to be
00:20:41 - able to consider several alternatives in the selection process.
00:20:49 - And so this is more or less the way I think, and of course we have data that
00:20:53 - provide evidence about this organization, that I think the prefrontal cortex is organized.
00:21:00 - So you would see it as a three-step process?
00:21:06 - Yes. Yeah, I would say that basically I think these are three steps.
00:21:11 - So there is a sensorimotor level, then there is a contextual level that allows
00:21:16 - you to select appropriate sensorimotor associations according to the present context,
00:21:21 - and then an additional layer that provides you information about the past, episodic events.
00:21:27 - And then there are these specific regions, so it's a top layer,
00:21:30 - which is a frontopolar cortex that enables you to process different alternatives at the same time.
00:21:41 - To consider different alternatives that might be influence the selection process. Right.
00:21:46 - But now, so if I face a certain problem-solving task, who decides that I switch processing level?
00:21:55 - Or which system or what criteria would switch between these levels of processing?
00:22:02 - So you can see a problem like...
00:22:07 - An environment that you don't know and that you travel within.
00:22:13 - Like a city, you arrive in a new city, you have no plan, and basically you travel within this city.
00:22:21 - And so I think that every problem,
00:22:27 - can be reduced to navigating into an unknown space and find a way, a path within this.
00:22:38 - And so it means that at the end, at the beginning, you first start by some very
00:22:45 - basic routine, maybe on store in the premotor cortex.
00:22:49 - And at some point, this routine will fail because this is a new situation, a new city.
00:22:55 - And then you start to see whether in the immediate environment,
00:23:01 - There are some cues that will trigger in your memory some other basic routine
00:23:09 - you learn somewhere else.
00:23:11 - Some system must be monitoring this, right? Some system must be monitoring like,
00:23:15 - okay, sensory cues are not helping me now.
00:23:19 - So there must be some integrator somewhere with some threshold that says,
00:23:23 - okay, we're lost at the level of sensory cues.
00:23:26 - Let's move on. Let's try context.
00:23:31 - So I guess it's really that sequential and that scheduled or are these systems…
00:23:35 - No, of course, everything is combined.
00:23:37 - It's just an easy way to describe things.
00:23:41 - Okay, so in your mind, this all runs concurrently. All these systems run in
00:23:45 - parallel at the same time generating solutions. Yes, of course,
00:23:48 - there is no reason that I switch on or switch off.
00:23:53 - Always, I mean, the context, context where you are integrated within the process.
00:23:59 - Yeah, but so what I'm asking for, on the one that you were saying earlier,
00:24:03 - this frontal area has this intrinsic property to monitor and to regulate if you want.
00:24:09 - But now, if we look at how this system is deployed in a task,
00:24:14 - where it actually performs multiple functions in parallel, this in itself would
00:24:19 - require some form of monitoring.
00:24:21 - So that raises then this question, okay, where is that coming from?
00:24:25 - Because it's monitoring other areas taking that into account in its own processing
00:24:32 - but now we need a monitor that monitors the monitor so how is that done.
00:24:38 - Yeah, I see what you mean, but there is only one type of monitoring.
00:24:44 - I don't think, but this is an interesting question by itself.
00:24:49 - Your question is whether there are some monitoring of the monitoring process,
00:24:54 - because in one sense we can go with no limit. Infinite regress, exactly.
00:24:59 - My view is that there is only one level of monitoring.
00:25:03 - So basically you have the basic process and then you have the monitoring process,
00:25:06 - And the prefrontal cortex is about this level of monitoring.
00:25:10 - And they don't have systems that don't, there is no system that basically monitor what,
00:25:17 - there is no recursive way of monitoring, I think, in the prefrontal.
00:25:26 - We have some evidence about that, some tiny evidence about that.
00:25:30 - When you ask people to, you know,
00:25:33 - we know that the prefrontal cortex is important in suspending a task you are
00:25:43 - performing for performing another task.
00:25:47 - What we notice is that people are very bad in doing this process recursively twice.
00:25:54 - That is, you interrupt the first
00:25:57 - task to perform a subtask. When I say interrupt, I don't say stopping.
00:26:01 - I just say you interrupt, you suspend it. So you have to keep some information about the task.
00:26:05 - So you suspend it, then you switch to a subtask to perform it.
00:26:10 - And so people can do that very easily.
00:26:14 - But when you ask them to suspend this secondary task to perform a tertiary task,
00:26:20 - then they got real problems.
00:26:23 - So it seems that they don't have the ability to,
00:26:29 - To have a two-level monitoring system. Right. It's tiny evidence,
00:26:34 - but it's some evidence. It's an interesting prediction, right?
00:26:37 - So, but are you, in some sense you're saying, look, overall the brain is organized
00:26:44 - in such a way that it hopes the frontal areas don't get involved because it
00:26:48 - means it knows what to do automatically.
00:26:52 - So, how is that then linked to this whole debate on controlled versus automated
00:26:57 - processing Because it's not only now about the decision making,
00:27:00 - it's about also the discovery of the structure in a decision making problem
00:27:05 - so that you can automate it.
00:27:08 - So how does that play out?
00:27:11 - Um, I far as understand your question, I, for me, automation is,
00:27:17 - uh, is a process by itself.
00:27:21 - And when I say that, uh, the goal of the prefrontal function is not to be involved,
00:27:25 - it means that when it is involved, it means basically that you face a situation
00:27:30 - that, uh, you don't really know what to do, uh, about.
00:27:36 - And, uh, but I'm not sure there's a, there is no, there is no process control
00:27:43 - process that controls the auto automatization.
00:27:45 - If you don't automatization is by default, what's occur, but it fails when it
00:27:53 - fails, the prefrontal cortex is engaged. Okay.
00:27:57 - But this is a default. There is a default. I think this is this notion of default.
00:28:01 - I mean, you are involved in robots.
00:28:05 - I think the notion of default behavior is very important.
00:28:12 - You don't think this is the same in robots? Of course. That you need to have some default.
00:28:16 - Sure. If everything goes wrong, so I do that by default. Yes.
00:28:21 - What I'm after is, so now we have this, you also call this cascade of cognitive
00:28:28 - control, right? This is really what you described.
00:28:31 - Previously but now if i if
00:28:34 - i engage in a certain task like talking to you um if
00:28:37 - we would do this 20 times over and i would say the same things at
00:28:40 - some point in time i don't have to invent questions anymore because i know them
00:28:44 - by heart i've automated this task so but do you see that automation as as an
00:28:50 - active process that is regulated by by this frontal area or do you see this
00:28:55 - as being a concurrent process dependent on other neural structures that is just picking
00:29:00 - up these regularities again, and automates them.
00:29:03 - I think it's even simply a sensorimotor model, internal model that is stored
00:29:11 - in probably a premotor region, some basal ganglia, and some posterior associative regions.
00:29:19 - It's become more or less encapsulated in this system, and it can be just triggered
00:29:24 - or stopped as a wall by the prefrontal system.
00:29:28 - But then it can be processed, I mean, by itself. It's run by itself.
00:29:34 - That's what, you see what I mean?
00:29:39 - It's become a fully consistent representation driving behavior.
00:29:45 - But do you think that this distinction then, controlled automatic,
00:29:50 - is actually helpful to look at this system?
00:29:56 - I think so. At least, yes, I think it's an important distinction because for
00:30:02 - the prefrontal function,
00:30:03 - I really think that prefrontal functions work above what's going on in an automatic system.
00:30:14 - So a task could be as complex as possible.
00:30:18 - As far as it is automatized, it is stored in this wonderful area,
00:30:24 - which are the premotor cortex, the parietal cortex, the temporal cortex,
00:30:28 - which have impressive representational power,
00:30:33 - and it could run automatically.
00:30:36 - And the prefrontal cortex is not concerned by this. There are other regions
00:30:42 - that do this job perfectly.
00:30:45 - Just the prefrontal cortex wants to know when this should be activated and when
00:30:50 - it should not be activated.
00:30:52 - So this is the monitoring part.
00:30:54 - And, of course, the monitoring part is a way of controlling things.
00:31:01 - Right. This is a notion of control. Control is also a notion of,
00:31:05 - I mean, acting a little bit on things.
00:31:09 - But monitoring, of course, sounds easy, but it does imply that you have norms of monitoring.
00:31:18 - You have to have criteria on the on
00:31:21 - the grounds of we say like oh wait this is now a relevant exception that we
00:31:25 - have to deal with right so just just say monitoring is yeah so there is there
00:31:32 - are two view for two general views which are one view is that everything is relative,
00:31:40 - that is that you always monitor different alternatives,
00:31:48 - and you are interested in selecting the most relevant alternative within the one you monitor.
00:31:58 - It's fine. The problem with this view is that you are always stuck within this
00:32:04 - collection of alternative view monitors.
00:32:09 - You have no systems that allow you to say, okay, I should look elsewhere.
00:32:14 - Well, not just within this small collection of alternatives I can collect,
00:32:23 - to know, maybe to select an even more relevant alternative.
00:32:29 - So the other notion is that rather to compare alternatives together,
00:32:35 - it's just for each alternative you monitor, you try to have a measure whether
00:32:41 - this alternative remains reliable or relevant or not,
00:32:46 - to try to have an absolute measure of whether this alternative is,
00:32:52 - let us say, relevant, a quite elusive term.
00:32:58 - And when you have, and how do you say an alternative is relevant?
00:33:03 - So there are, I think, probably different factors that can contribute to judge
00:33:09 - an alternative as relevant or irrelevant, but one important is its ability to predict,
00:33:18 - action outcome. I use, for example, I use this...
00:33:26 - Let us say that a behavioral strategy is like a map with some paths.
00:33:34 - So you use a map with this path, what you expect first is that when you follow
00:33:38 - the path, you expect to see in the real world what you expect on the map.
00:33:46 - So the first important criteria for relevance is the ability to predict the result of your actions.
00:33:57 - There might be others, but I think this is probably one of the most important.
00:34:02 - And this is the way I think
00:34:06 - the prefrontal functions solutions monitor strategies
00:34:11 - that is many in
00:34:16 - an absolute way for each strategy try to figure out whether the strategy is
00:34:20 - relevant or not but this has interesting consequences right because then although
00:34:24 - with respect to action you might want to say i want to go to one then if you
00:34:31 - want to be able to monitor its outcome,
00:34:33 - you must actually be able to load in memory any reference for future consultation.
00:34:41 - So, I mean, that basically means a whole set of possible outcomes must now be
00:34:47 - considered because any action in a complex world can have a quite wide range
00:34:52 - of consequences. So, this world is dynamic.
00:34:59 - First, I mean, when I talk about behavioral strategy,
00:35:07 - I think about a set of internal representations that include representation
00:35:15 - about what kind of outcome I expect when I do this action in this situation.
00:35:24 - So uh of course you uh and this is what you have in your memory basically you know that,
00:35:33 - if for example i i am at home and i press on this interrupter that i will get some lights,
00:35:41 - so you you learn this this is part of your strategy what yeah but for them all
00:35:45 - i'm saying is for monitoring to work effectively,
00:35:48 - it must consider a set of possible outcomes. It's not only one.
00:35:54 - Yeah, of course. I mean, maybe in this example, there is only one,
00:35:59 - but you may have several.
00:36:02 - Yeah, I see what you mean, which you mean, for example, I do an action and I
00:36:07 - may have a chain of consequences. That's right. This is what you mean.
00:36:12 - Yeah, but I think in one sense.
00:36:17 - There is no reason to believe that in principle you can code all the consequences.
00:36:24 - But of course there are some problems of dimensionality.
00:36:27 - So it's possible that at some point there are some criteria that allow you to
00:36:34 - identify some landmarks, specific outcomes and landmarks.
00:36:41 - And it's part Part of the complexity of the system, of course.
00:36:49 - It is an interesting counterpoint because you could say, well,
00:36:54 - one thing what I'm doing, I'm pruning away all less preferable alternatives.
00:37:00 - So I have my one interpretation of the task and the action I have to execute.
00:37:05 - But you could say conversely, the more complex the task, the more pruning I
00:37:09 - have to do to get to my action, the more outcome alternatives I have to consider for my monitoring.
00:37:18 - Yes. As an example, we can walk out of the studio, we can go through that door,
00:37:25 - but maybe Giovanni, our sound engineer, stands there with a baseball bat to
00:37:31 - chase us down the whole wheel. We don't know.
00:37:34 - Or maybe the building has disappeared, etc. So these are all consequences,
00:37:40 - all future states of the world that we must be able to consider.
00:37:45 - Yeah, but you don't consider it. Okay. You know, I mean, this is because when
00:37:50 - you are in the studio, you know the studio, you are used with the studio.
00:37:54 - So you know that in 99% of time when you use it, nobody wait for you with a baseball bat.
00:38:05 - So you don't code that. Baguette, maybe, the French version. Baguette.
00:38:10 - I mean, you know, every situation, this is part, every situation.
00:38:15 - I mean, if you go to the airport, you have some expectations, okay?
00:38:21 - Of course, if you go to a place where nobody looks like what you experienced
00:38:27 - before, I think you start to be very scary. But it never really happened.
00:38:33 - Right. Okay, now this is resolved, right? So you're saying,
00:38:36 - no, monitoring acts upon a rather explicitly defined world model,
00:38:42 - Which is the same one that feeds into the action you generate and then also
00:38:48 - the monitoring of its outcomes.
00:38:50 - This is roughly what you would say. So this frontal area is really compressing
00:38:56 - everything down into just one unitary interpretation of what you're doing. Yeah.
00:39:02 - Yes, I think, and this is an important point, what you say that,
00:39:05 - at least for myself, is that there are discrete entities, which are different
00:39:13 - world, I call that strategy or behavioral strategy.
00:39:17 - I mean, psychology, they say task set, but it's the same concept.
00:39:21 - Set, that is, you have discrete sets which are consistent.
00:39:26 - Collection of, each set is a consistent collection of internal world representation,
00:39:33 - and that can be selected by it independently.
00:39:39 - And this is the role of the prefrontal cortex to select them independently,
00:39:43 - to monitor them independently,
00:39:44 - to possibly perseverate with one set in order that this set develop and learn better the word.
00:39:54 - And that's it. So this is a basic unit that the prefrontal cortex manipulates.
00:39:59 - This is this discrete set.
00:40:00 - So there is this notion of discreteness, which I think is important.
00:40:04 - That's an important point, because also in your experimental work,
00:40:07 - I think this is really one of the elements you emphasize a lot.
00:40:10 - So one set of experiments you described was about a comparison between,
00:40:15 - let's say, rule-free tasks and And rule-based tasks, right?
00:40:21 - So why is that an important manipulation for understanding what this frontal area is doing?
00:40:30 - So there are several, I think, important things related to this issue.
00:40:39 - First, there is a general, I would say, questions. questions,
00:40:43 - this very general question, why do we follow rules?
00:40:51 - I mean, uh, we follow rules all the time.
00:40:55 - I mean, uh, especially when you behave in a group, there are some rules,
00:41:01 - you follow rules and often at your expense of your own preferences.
00:41:05 - So there is, for me, it was one of the important things to understand why basically
00:41:10 - we follow rules and especially in human groups.
00:41:13 - I mean, there are some, um, many rules are about cooperative rules and,
00:41:18 - um, coordination rules.
00:41:21 - And especially coordination rules, that are very sensitive to deviations from others.
00:41:28 - I mean, a coordination rule is meaningful if everybody follows the rules, okay?
00:41:35 - Like driving on the right.
00:41:41 - So the idea was to, okay, if rules are very important to follow in groups,
00:41:51 - It means that there might be some specific process that allow rules to prevence
00:41:57 - on subjective values or subjective preferences.
00:42:01 - So I was interested in this question, this general question.
00:42:04 - So it's more a question about how it is possible that rules that are very sensitive
00:42:10 - to individual variation develop in human groups.
00:42:14 - So there might be some very specific mechanisms or functional architecture in
00:42:19 - the brain that make it possible.
00:42:21 - So a very general evolutive question.
00:42:24 - The second question was about, it's related to the notion of context.
00:42:30 - So the rule is basically you have cues, and these cues trigger some specific behavior.
00:42:39 - And you have rewards expected rewards that can drive some behavior so the question
00:42:47 - is exactly how these two process what we can identify independently interact mm-hmm.
00:42:57 - And this is related to what I said. There is, of course, the notion of values,
00:43:01 - which is important to select action.
00:43:02 - But the rule seems not to be about values, but about relevance.
00:43:07 - What is relevant in this situation?
00:43:09 - So that's why I was interested about this issue,
00:43:12 - is whether this notion of relevance or reliability is really relevant,
00:43:23 - or whether a rule is simply some representations that at some point are transformed into values,
00:43:31 - subjective value, or modulate what I can expect as a reward in the future,
00:43:37 - so that every selection ends up as a choice between two options with different values.
00:43:46 - And what we found is that actually this is not the case.
00:43:51 - We found that, according to our data, the selection process at the end occurs in the rule space.
00:44:04 - And preferences or expected reward are just some additional information that
00:44:10 - is provided to this rule-based space to make the selection.
00:44:17 - Right. but the rule the rules prevail on the selection that is if you have rules
00:44:26 - that another way to say things maybe more explicitly that as long as you have rules that allow you to,
00:44:35 - to decide what to do, the system doesn't care about your preferences.
00:44:40 - In the selection process, of course, it cares when it evaluates the result of
00:44:46 - the action, action outcome.
00:44:48 - But in the selection process, it doesn't care. The subjective preferences or
00:44:52 - expected reward start to influence selection as long as the rules become ambiguous.
00:44:58 - But now, so underlying this is like a two-dimensional space that also maps onto
00:45:05 - the anatomy of a frontal cortex.
00:45:09 - This was the idea you were sort of investigating here, that along a medial axis,
00:45:15 - it's more value-oriented, and along a lateral axis, so towards the outside,
00:45:20 - it's more rule-oriented.
00:45:22 - And so what you're saying in your experiment with the fMRI you did,
00:45:28 - it gave you the impression that the real action selection, like the dominant
00:45:32 - axis here, would then be more this lateral axis where the rules reside.
00:45:36 - But is it really that discrete? I mean, on the grounds of which can you really say that?
00:45:44 - So there are many evidence that there is this dual system.
00:45:50 - So the first is that the medial system, as you said, that is related to processing
00:45:56 - expected reward, values, subjective preference, whatever you call that.
00:46:03 - And this processing is implemented mainly in the medial prefrontal system.
00:46:12 - Then you have this lateral prefrontal system that seems to be involved whenever
00:46:17 - you have some rules, some instructions, some internal model that,
00:46:26 - drives the selection.
00:46:29 - And we know also, of course, from anatomy that these two systems are tightly connected.
00:46:37 - So
00:46:42 - So the idea is that you have, I think this is part of, I think there are two possible views.
00:46:51 - One view is the homogeneous view. That is, there is no real specializations
00:46:57 - and preferences and roles are mixed in the interaction between the two systems.
00:47:03 - At the end, the selection is made by the system as a role.
00:47:07 - Okay. The system relax to a given state and it makes a selection.
00:47:13 - It's a possible view the other view is that,
00:47:17 - there is one of these subsystems the medial the preference system or the lateral
00:47:22 - the whole system that actually,
00:47:27 - is the system that makes the final decision that commit behavior that is the
00:47:32 - information coded this is actually what the system what the system or what the
00:47:37 - organism is going to do as action and.
00:47:46 - We found evidence about this second interpretation, this second hypothesis,
00:47:52 - that is that the lateral system make final selections that commit the organism
00:47:58 - and in my view and the question is why it's like that,
00:48:03 - in my view is that,
00:48:07 - you know in human the lateral prefrontal cortex developed a lot. And,
00:48:15 - this is a rule system. So I think what is very specific to human is that we
00:48:21 - have the ability to build rules all the time.
00:48:26 - And I think the selection is moved to the lateral system where basically there
00:48:32 - is all the process that allow to learn rules.
00:48:36 - And,
00:48:39 - And promoting, therefore, all this, the learning of rules and the use of rules
00:48:44 - in the selection process, because we are social organisms.
00:48:48 - And in groups, you need rules.
00:48:51 - Right. It's very important. And especially, as I said at the beginning,
00:48:55 - coordination rules are critical in groups.
00:48:59 - So, in so much you're describing here, you described in terms of a utilitarian
00:49:05 - model versus a normative model, right? So the utilitarian one would be more
00:49:09 - value-dominated, and the normative one is more rule-dominated.
00:49:13 - And you're saying, look, your data is pointing you in this direction,
00:49:16 - that this normative model is, if you want, more dominating the action outcome
00:49:21 - than by this utilitarian model.
00:49:25 - But now, so the experiments on which you base this, which is sort of human subjects
00:49:29 - performing different decision-making tasks, and you do fMRI on them,
00:49:33 - so you look at the brain activity in these areas, There are,
00:49:38 - of course, a number of caveats, if you want, because imagine I interpret your
00:49:44 - statement in a very categorical sense.
00:49:46 - You would say, look, that would have the implication that I have a utilitarian
00:49:50 - module that just worries about value.
00:49:52 - This is more in the medial prefrontal cortex.
00:49:56 - Then I have this normative rule-based system sitting more lateral as a well-delineated,
00:50:02 - again, module. and they exchange well-defined information chunks, if you want.
00:50:09 - One is informing the other about
00:50:10 - the value, and the other one is informing back about the rules, right?
00:50:13 - But now I could argue, well, that's great.
00:50:16 - That's a really nice way to interpret the data, and it is consistent with the
00:50:20 - experiments you performed. There's no doubt about it.
00:50:23 - But for starters, the signals you interpret become significant only at a scale
00:50:30 - of seconds. while the performance is occurring at the scale of hundreds of milliseconds.
00:50:38 - So it's possible that the neural process that really is driving this action
00:50:43 - selection is really below the radar of your fMRI evaluation and that what you're
00:50:49 - analyzing is maybe more,
00:50:50 - let's say, how you process decisions in memory after the decision has been made
00:50:57 - than the real-time performance of the subject. Thank you.
00:51:08 - Yes. Of course, this is a problem with fMRI data, that we don't have access
00:51:13 - to the millisecond time scale, which is important for neural processing.
00:51:20 - But I can tell you that if you look at neural data on these regions,
00:51:29 - I mean, they are consistent with what we found.
00:51:32 - I mean, we know, for example, that in the dorsomedial prefrontal cortex,
00:51:40 - we have neurons that encode action outcomes associations,
00:51:44 - and more in larger proportions than in the lateral prefrontal regions.
00:51:53 - And we know also that during a decision, these regions activate first in medial
00:52:00 - regions and before neurons in lateral prefrontal regions.
00:52:06 - So we have neurons in the lateral prefrontal region activate closer to the decision
00:52:13 - time than neurons in the medial prefrontal cortex.
00:52:16 - So it's consistent with what I am saying. Right, okay. Okay.
00:52:20 - But then there's still another missing link, which is that I could argue,
00:52:24 - but look, if we look at these cortical areas,
00:52:26 - if we just, I give you a cubic millimeter of this medial prefrontal cortex,
00:52:31 - and I give you another cubic millimeter of lateral prefrontal cortex,
00:52:34 - and I don't tell you where they came from, you will have a hard time on morphological
00:52:38 - grounds to tell me what's what.
00:52:40 - So, I mean, there's huge similarity between these circuits.
00:52:43 - So what makes them so and so different in their functional contribution in decision-making.
00:52:54 - I think apparently what is different is that they are located in different positions
00:53:01 - in the network so that each have access to different type of information,
00:53:08 - and what is important at the end to describe the function of every regions,
00:53:16 - ideally is to be able to describe the inputs outputs and the output of these regions.
00:53:22 - And of course, we cannot do that comprehensively for a region because there
00:53:33 - are many connections from everywhere.
00:53:35 - But at least if I take this example about medial and lateral prefrontal region,
00:53:44 - I mean, I mean, if we try to understand what kind of information the media regions
00:53:51 - send to the lateral regions, and conversely, what kind of information the lateral
00:53:55 - regions send to the media regions,
00:53:58 - we see that it's not the same.
00:54:02 - Well, but in some sense, with your data, you don't really know what travels
00:54:08 - over these axons, right?
00:54:09 - You only know something about the covariance of their activity under certain task conditions.
00:54:15 - And the information exchange could possibly be regulated through another structure.
00:54:23 - Yes, it might be not direct for sure, but this data tells you that in one direction
00:54:29 - there is something happening which is not the same as in the other direction.
00:54:33 - Okay. This is what it means.
00:54:35 - But... That is the information that is shared between the two regions differ
00:54:39 - in one direction and in the other direction.
00:54:42 - So it provides some cues about the function of every region.
00:54:46 - Because my deep belief is that I think every region has a specific operation,
00:54:56 - implements a specific operation,
00:54:57 - which is quite abstract and every region is like an operator,
00:55:08 - if you want, an information processing operator.
00:55:10 - And of course, at least what I try to know is which operator is implemented
00:55:16 - in this given region and so on. Sure.
00:55:18 - I understand that. But now what we see is that if the local circuit is fairly
00:55:24 - uniform, right, the information exchange we cannot directly assess.
00:55:30 - So that means possibly there are actually other areas that are dictating this
00:55:35 - function, right? For instance, let's say it might be an interaction with basal
00:55:39 - ganglia-related structures or so. We don't know.
00:55:42 - But then I could argue, hey, but wait one moment, Etienne.
00:55:46 - This might also imply that it's actually these tertiary structures that we have
00:55:49 - not identified that are really doing the decision-making.
00:55:53 - And these other guys in this prefrontal area you're measuring from are just
00:55:57 - echoing in a way you can detect with your system, with your device, right?
00:56:03 - The result of that decision that has been made.
00:56:10 - Yes. You're perfectly right. I mean, but I think it's part of the scientific process.
00:56:20 - It's first to start with the most simple hypothesis.
00:56:23 - Okay. We found correlation or some information transfer from one region to another.
00:56:29 - We know that this region are deeply and densely connected.
00:56:33 - Okay, if we observe some influence, the first most simple interpretation is
00:56:40 - to consider that it goes directly from one region to the other.
00:56:44 - It might be wrong at some point, but I think we should always first by the most
00:56:48 - simple interpretation.
00:56:49 - As I said at the very beginning, with the most simple model.
00:56:52 - And then you complexify the model gradually if you need.
00:56:58 - And I'm sure it's a simplistic view in one sense.
00:57:02 - I don't say that this is, it's probably much more complex, but it's always useful
00:57:08 - to start from very simple principle and to refine this progressively. Absolutely.
00:57:16 - And I am sure that in the selection process, basal ganglia, of course,
00:57:26 - involves many loops. Mm-hmm.
00:57:30 - It's true, but you need to start from one point. No problem.
00:57:38 - But it's true that it's good to have a simple model, but you have to never forget
00:57:47 - that your models are just models and simple models.
00:57:51 - Sure, exactly. These are probably wrong at some point, and for sure they are wrong at some point.
00:57:55 - But what I'm challenging you on is that sometimes I'm saying,
00:57:58 - well, maybe your model is not as simple as it could be because you are already
00:58:02 - assuming that these are like two distinct modules with distinct functions that
00:58:06 - are exchanging well-defined information.
00:58:09 - Well, these are actually really pretty strong assumptions.
00:58:13 - That's why I used anatomy as an example. If I go to the anatomy,
00:58:16 - it will be really difficult to distinguish these circuits, medial, lateral.
00:58:21 - They will look rather similar, and their differences will be in,
00:58:24 - let's say, fairly subtle differences in how other structures project into them
00:58:30 - and receive information from them. But these are all really minute.
00:58:33 - These will be minute variations.
00:58:36 - So I'm sort of challenging your idea. I completely buy the method,
00:58:40 - but I would say, well, maybe this is not a minimal interpretation.
00:58:44 - Yeah, you said that it's already quite complex. But why is it complex?
00:58:50 - It's complex because simply we consider a coupling playing system with reciprocal connections.
00:58:57 - And our mind doesn't seem to be very well adapted to understand reciprocal interaction.
00:59:04 - As long as you have reciprocal interaction, you need mathematical model to make sense. Fair enough.
00:59:11 - However, you do assume that within each of these modules, a very specific function
00:59:17 - is performed because one does value-based operations and the other one does
00:59:20 - rule-based operations. And that's not necessarily fairly simple.
00:59:24 - Yes. Yes. But this is what I said at the very beginning.
00:59:28 - Imagine that, okay, let us, if you raise the question, okay,
00:59:32 - we have this media region, we have this data region, what could be the difference
00:59:36 - between the two regions, functionally?
00:59:39 - If you start to think, you look at the literature, you look at all the data,
00:59:44 - many labs and people collected, and you say, okay, what could be the differences,
00:59:51 - the functional differences, the functional segregation between these two regions, if it exists,
00:59:55 - you may end up with a conclusion that, okay, they're quite similar.
01:00:01 - And this is, as I said at the beginning, this is a homogeneous view.
01:00:05 - But you say, okay, let us really think that they make different functions.
01:00:09 - And you cannot end up with a,
01:00:12 - So many different assumptions, you know. There are a few, but not so much.
01:00:16 - Right. And because still we have data and there is a consistency also you need
01:00:22 - to look at when you build this kind of hypothesis.
01:00:26 - It's not consistent to imagine that, for example, I don't know, but...
01:00:33 - Yeah, this is an important point, right? Because if you look at your more recent
01:00:36 - experiments, you actually have found only further support for this way of thinking
01:00:41 - about the system as opposed to falsification of it.
01:00:46 - So this, I think, would argue for that.
01:00:49 - And so that had a lot to do with these experiments where you looked at the transfer
01:00:56 - of value in different tasks of varying complexity.
01:01:03 - And also in these tasks, what you did, which I think was extremely interesting,
01:01:07 - you really looked at how different models of decision-making actually scaled
01:01:12 - on these tasks, what their problems were, and on the basis of that,
01:01:16 - you have formulated an alternative where you say, look, actually,
01:01:19 - all these other models that have been very popular in literature,
01:01:22 - they might be very interesting, but they fail on really explaining what this
01:01:26 - frontal area is doing. So what was the trajectory there exactly?
01:01:30 - You mean okay the way we you mean your question is about the way we end up with
01:01:38 - this monad yeah exactly exactly,
01:01:42 - yeah this is always this is still the same process starting with very simple,
01:01:49 - stuff and try to explain things with very simple monads so we have and,
01:02:03 - It's difficult to explain. I mean… Well, maybe as a hint, you know.
01:02:07 - So the point was basically we're saying, okay, we look at the prefrontal cortex, right?
01:02:12 - So what can it really do in a task?
01:02:14 - Well, it can decide to stay. It just keeps on executing, following the same
01:02:20 - rule because it has been successful.
01:02:22 - Secondly, it might decide to switch rule because things are failing.
01:02:27 - Or it can decide, okay, I don't know what to do.
01:02:30 - I better explore. something new has to happen right so and I think it was it
01:02:36 - was that consideration that really gave rise to
01:02:38 - the model that that yes so at the very beginning this is an intuition about that,
01:02:46 - something that was largely overlooked in the literature is this exploration
01:02:51 - process there are models there are a few models that explain how you switch in exploration.
01:03:00 - Which means that basically for example you learn
01:03:03 - something and at the point the system the monitoring system tells that you need
01:03:06 - to switch and in this model you just reset everything and you will start from
01:03:10 - scratch but in the recharge there was no model explaining how you switch out
01:03:21 - from exploration and uh
01:03:25 - that is
01:03:27 - at some point uh you may
01:03:30 - explore but you may when you explore
01:03:33 - you suddenly uh you
01:03:38 - suddenly uh notice that you can
01:03:43 - switch back to return to something you know so you
01:03:47 - quit exploration and there was no model
01:03:49 - on this so that was the basic idea that the basic intuition about okay we really
01:03:56 - need a model that explain how we decide to to explore something new and how
01:04:02 - we decide to quit exploration to re-explore what you already know.
01:04:09 - This was intuition at the very beginning. The second intuition that we also need to understand,
01:04:18 - the second intuition was about learning. So learning means, I already said a
01:04:22 - little bit about that before, learning means experiencing negative feedback,
01:04:29 - which means that you need to persevere it.
01:04:33 - Also, you get negative feedback and you want to switch.
01:04:37 - So it means that there needs to be a system that pushes you to learn,
01:04:42 - and and conversely this same system can,
01:04:50 - consider that at some point it's no more valuable to perseverate,
01:04:54 - too much negative feedback or whatever and then you need to switch,
01:04:58 - so this is this intuition at the very beginning that helps us to,
01:05:05 - at least to.
01:05:09 - To set the problem, to, to, to set the problem. I mean, to, to raise the problem.
01:05:16 - And I think in science, it's very important to be able to raise problem.
01:05:20 - Right. And to, and to, to delimit, to circumvent a given problem,
01:05:29 - to raise this problem. And then after,
01:05:34 - given this wish, issue, how you explore and you quit exploration,
01:05:37 - how you persevered to learn or switch when it's no more valuable to learn.
01:05:43 - I mean, we start developing a model and at the same time an experiment and.
01:05:54 - So we develop a model based on our ideas, our intuitions and then we test it
01:06:00 - with this experiment and of course on this experiment we tested whether some
01:06:05 - more simple or more regular monad were able to explain the performance.
01:06:11 - And as our intuition,
01:06:16 - I mean,
01:06:20 - provided us some cues, I mean, we were able to show that in this kind of experiment,
01:06:26 - I mean, regular monads that have no exploration.
01:06:32 - Capabilities or no I mean cannot explain the data and if you and conversely
01:06:43 - if you consider very sophisticated model I mean normative model in the sense
01:06:47 - that statistical learning model very sophisticated model,
01:06:52 - they perform the task of course but they outperform human performance they don't
01:06:57 - explain human performance What's the difference there?
01:07:00 - In what sense do they outperform humans?
01:07:04 - So first, they are able to adjust to uncertainty and to the viability of their
01:07:13 - environment much faster than humans.
01:07:16 - And they basically are able to use every kind of, every piece of information
01:07:26 - to inform about what should be learned and what should be, when to switch.
01:07:31 - Which in a way, which is, of course, impossible.
01:07:36 - So this was this Dirichlet optimal agent, right? This was your criterion.
01:07:41 - But does it have access to other information that humans don't have access to?
01:07:45 - No, no, no. Where does the difference come from?
01:07:48 - One of the major differences is that this kind of monar.
01:07:53 - They will, for example, explore at some point, create a new strategy or a new task set.
01:07:59 - But then later on, they will get a feedback.
01:08:03 - And whenever they get a new feedback, they revise all the history of creating a set, new strategy.
01:08:12 - So every time they get new information, they revise the entire history.
01:08:15 - So they memorize the entire history and they try to find, given any new information,
01:08:21 - what would be the best history. Right. Right.
01:08:26 - And that's why they are very powerful. Sure. But what I found interesting,
01:08:30 - though, if you compare their performance to the human, indeed,
01:08:35 - in most task conditions, they were better than humans.
01:08:39 - But there were other task conditions where humans actually outperformed this optimal agent.
01:08:47 - I'm not sure you're right. Okay. No, they don't really outperform. I'm not sure.
01:08:55 - Yeah, I remember this question.
01:08:59 - I saw it in one of your plots, right? To me, you have these two conditions.
01:09:03 - There's sort of a recurrent task and there's an open task.
01:09:09 - And then in one of these, even though the optimal agent can always adjust its
01:09:15 - full policy space to any outcome, come,
01:09:18 - you still saw that humans with their assumingly more restricted capabilities
01:09:25 - would still outperform these optimal agents.
01:09:28 - So I was wondering whether this had to do with, for instance,
01:09:30 - perceptual capabilities that such agents don't have and humans do have.
01:09:37 - I'm sorry, but I think you are wrong. I don't see this in my graphs.
01:09:41 - Maybe we should have the graph. Yeah. Okay. We'll look at that later.
01:09:45 - That's very good. But then you also compare two standard algorithms like reinforcement
01:09:50 - learning models where you would basically learn policies given the feedback
01:09:56 - that you receive from your environment.
01:09:58 - So why does a standard reinforcement learning model fail in this task?
01:10:04 - So it fails because this model is just adjusted continuously to the new contingency.
01:10:11 - So in this model, there is no memory. Mm-hmm.
01:10:16 - It's just you learn something, it works, and when it no more works,
01:10:20 - you just unlearn this and relearn something new.
01:10:23 - But you never store some specific mapping you learn.
01:10:31 - Right. But if I would have a reinforcement learning model that would just store
01:10:35 - its different mapping so it knows when to switch tasks, it would be appropriate
01:10:39 - possibly for this task, for this problem.
01:10:43 - Of course but then you need a monitoring system to know
01:10:46 - when to switch okay right and this
01:10:49 - is exactly the point so you start to need a monitoring system and
01:10:53 - uh to monitor these different mapping you store and then you need a system also
01:10:59 - that decides that uh okay you have learned all this mapping but you need to
01:11:03 - maybe learn now a new one right and and explore a new one and we end up with our model, basically.
01:11:11 - Right. Of course, it raises the question whether it was a completely fair comparison
01:11:15 - or whether it was more like a straw man because you knew a priori that that
01:11:21 - model would fail given the task conditions.
01:11:24 - Of course, of course. And the task was built to make this model fail.
01:11:27 - Otherwise, we would not have developed this model. Right. But it made the point, right?
01:11:32 - Yeah, it makes the point. It's just a pedagogical way to show.
01:11:38 - But what was interesting with the reinforcement I don't know whether you notice
01:11:42 - that it captures the overall dynamics you see I mean it doesn't capture qualitatively
01:11:49 - the difference between conditions,
01:11:51 - when there is a recurrent mapping that can be used or no recurrent mapping open
01:11:57 - condition but still captures the overall dynamic of adaptation.
01:12:03 - Which and I think it's interesting it shows that Uh...
01:12:10 - The very important things about discriminating between models are in the details.
01:12:18 - Small difference at specific points that are very informative about what subjects do or not.
01:12:26 - Because the basic reinforcement model captures the overall dynamics,
01:12:31 - which is actually an artifact of averaging across episodes. Zs. Right.
01:12:37 - But now in the model you proposed, that was your alternative,
01:12:41 - you have, let's say, a Bayesian inference process that sort of is trying to
01:12:49 - figure out what's going on in the world.
01:12:50 - How well do my policies probably match to this, right? Because also my policies
01:12:55 - are tied to states of the world.
01:12:57 - Then you have a hypothesis testing component and you have an exploration component,
01:13:03 - right? So how do these three components really work together in your model?
01:13:09 - If you want to isolate components, there are three components.
01:13:14 - The first component is the inferential buffer, the hypothesis testing component,
01:13:22 - and the way we built new strategy from long-term memory.
01:13:29 - This is the third component. Of course, they are intrinsically linked,
01:13:32 - but so the idea is that you can monitor only a small number of concurrent strategy or policies.
01:13:46 - So this is a constraint we put on the model.
01:13:55 - Then you need to update this small collection.
01:13:58 - And so to update this small collection, we consider that the best way is to
01:14:05 - have an hypothesis system.
01:14:07 - I test a new policy.
01:14:11 - I start monitoring this new policy. If it seems to be reliable at the end,
01:14:17 - I continue to monitor it.
01:14:19 - Or if it's not competitive compared to other strategy, or I just don't need
01:14:28 - to monitor it, I can discard it.
01:14:31 - So this was the idea that hypothesis testing is important to update this monitoring
01:14:38 - buffer because it cannot monitor everything.
01:14:42 - And then, of course, when you decide to go for a strategy which is not in the
01:14:49 - monitoring buffer for a policy, you see, you just need to go first.
01:14:54 - I mean, you go to your long-term memory, but you don't monitor things here.
01:14:58 - So the only way you can build something from your long-term memory is to have
01:15:03 - a weighted mixture of this long-term memory, weighted by some queues,
01:15:12 - given some queues, and given, of course, some internal models.
01:15:16 - And you build a new strategy from your long-term memory like that, and you try it.
01:15:22 - And all these three components are important for the author,
01:15:29 - so to build a new strategy is like I have a long term memory system,
01:15:34 - in this long term memory system I have different policies which effectively means I have,
01:15:40 - stages in every policy where I have a certain sensory state and a certain action
01:15:44 - that goes with it and then I say and therefore and the future should look like
01:15:48 - this and if I keep on doing this I follow this chain I get some reward, right?
01:15:52 - And then you say, well, but what I can do, I can actually now just cut out bits
01:15:57 - and pieces of all these different policies of long-term memory and build a new one, try a new one.
01:16:03 - So what would be the key criterion to perform a selection on the pool of segmented, if you want, policies?
01:16:13 - So the way it works in our model is that, But first, it's not primarily a model
01:16:20 - of long-term memory, the model I described.
01:16:24 - So, of course, it's processed or is described in the model in a quite simplistic
01:16:29 - way. But still, there are some important ingredients.
01:16:35 - So, one is that actually a policy which is stored in long-term memory is always stored with some,
01:16:43 - internal representations that link this representation to external queue.
01:16:49 - Some contextual queue, if you want.
01:16:52 - Which means that in the process of mixing strategy from long-term memory,
01:16:58 - it's always weighted by the contextual queue.
01:17:01 - So it means that in a given context, when you are in a given context and you create a new strategy,
01:17:07 - the way the strategy combines in this mixing process may be different from a
01:17:14 - mixture done in a different context.
01:17:21 - In other contexts. I don't know whether you see what I mean.
01:17:25 - Because every strategy is taught in long-term memory with some contextual model.
01:17:31 - That encodes the relevance of the strategy within this context.
01:17:35 - When you are in a given context, context, your marginalization of your strategy
01:17:41 - in long-term memory could be rather different in one context than another one.
01:17:46 - To me, it's clear, right? Because I have different tasks.
01:17:49 - Let's say one can be playing football and the other one can be playing tennis, right?
01:17:53 - So these are different tasks, different rules. And now, dependent on the context
01:17:56 - I'm in, I'm either playing tennis or football, there's a different subset of
01:18:01 - policies I should now start to worry about to improve my football game.
01:18:06 - But, yeah, of course, but it could be more subtle than that.
01:18:11 - Of course, I mean, if you are trying a new recipe in your kitchen,
01:18:16 - you are not going to use a policy on the football.
01:18:19 - Exactly right. For sure. Exactly right. This is, I would say,
01:18:22 - the most evident way, but it could be more subtle because the system can have some links.
01:18:31 - For example, if you have some maze, for example, and there are some queues in
01:18:38 - the maze that are related to some strategy, when you combine them,
01:18:42 - then this internal model linking external queue or contextual queue to some
01:18:47 - strategy could make the combinations at the end when you create a new strategy quite subtle.
01:18:55 - Right, I understand. And quite unexpected. Sure. Sure, but what I found interesting
01:18:59 - is that on the one hand you're saying, well, so now in my long-term memory,
01:19:03 - I have this enormous space of policies, and this is stuff I did in the past, right?
01:19:07 - So this is how I have acquired these.
01:19:10 - Yeah, along my lifetime. Exactly.
01:19:13 - And then you said, but now if I invent or create a new sequence,
01:19:19 - a new policy or hypothesis on a policy, I actually want to equalize the outcome expectation.
01:19:26 - I just, this is what you call the dumb strategy. You say, look,
01:19:29 - if I now invent a bunch of new policies I can try out,
01:19:34 - The predicted outcomes that I associate with them, I just set to the same uniform
01:19:38 - level, as if I'm randomizing this outcome space.
01:19:43 - So why are you doing that? Why is that necessary?
01:19:48 - Yeah, so the way we describe things are not exactly correct. Okay.
01:19:55 - You need to have, so as you said before, the key point is what are the criteria
01:20:05 - for monitoring the relevance of strategy.
01:20:08 - And of course, in a situation where you know all possible strategy,
01:20:14 - your criteria is quite simple.
01:20:16 - For example, you just compare, let us say, how well each strategy predicts the next state,
01:20:25 - and you will, at the end, find what is the strategy appropriate to this situation,
01:20:31 - because you know all possible strategies,
01:20:34 - which is, of course, never happened in real life.
01:20:38 - And this means that it's very difficult to judge the relevance of a strategy
01:20:45 - because you don't know even all the alternatives.
01:20:49 - So you need to have at some point.
01:20:55 - An estimation about in the different strategy I am monitoring which is the probability
01:21:03 - that actually the the true strategy,
01:21:09 - doesn't belong to the one I am monitoring and to be able to compute this probability
01:21:17 - probability, exactly, you cannot.
01:21:22 - It's an intractable problem.
01:21:25 - But you can estimate this probability as saying that, okay, the probability
01:21:30 - that actually the right strategy doesn't belong to the monitor strategy,
01:21:34 - you can estimate it by using, as you said in your question,
01:21:39 - this do-me strategy that if I perform randomly, I will get this outcome.
01:21:45 - And monitoring the relevance of this random strategy is an estimation of the
01:21:54 - priorities that the true strategy is not in your monitoring strategy. Mm-hmm.
01:21:59 - But there's an alternative. It's a trick. It's a trick. I understand.
01:22:03 - And also it keeps things simple in some sense. Very simple. But I could also
01:22:08 - argue, look, let's take the cooking scenario.
01:22:11 - So here we have the policy space or the strategy space of cooking.
01:22:16 - And now certain sequences of actions have been more successful in the past than
01:22:22 - in your kitchen activities and others.
01:22:24 - So now if I assemble a new policy, taking elements of these other sequences,
01:22:32 - by picking them out of an existing policy that has been tested,
01:22:36 - I can make an inference about their probability to have a certain impact on outcome.
01:22:41 - Outcome for instance let's say you have a long sequence
01:22:44 - and have something an event sitting real
01:22:47 - really far away from the end state of that sequence i could
01:22:50 - say well the probability that that event is is
01:22:53 - really having a big impact on outcome is probably low however if having an event
01:22:58 - that is close to this endpoint you could say well that probability is higher
01:23:01 - i think it depends on the level of automatization okay even if you have a very
01:23:06 - complex strategy but every time you start with the first action,
01:23:11 - you will get at the very end this reward, this outcome.
01:23:14 - Then probably in your system I mean the link between the first action and the
01:23:21 - outcome would be very strong.
01:23:23 - Can't I exploit that information in assembling a new policy and making at least
01:23:31 - an estimate of its outcome?
01:23:35 - For instance, I can take elements of very successful policies,
01:23:39 - so the probability that the new policy will be successful is high,
01:23:42 - or it can take elements of really policies that are not that great.
01:23:48 - You see the point? Yeah, I see exactly your point.
01:23:51 - I think it's a very complex issue, especially.
01:23:58 - I think this process is reasonable. I mean, it's tractable. if you can project
01:24:07 - your strategy on some topological space. Mm-hmm.
01:24:13 - It will work. For example, we know that many spaces have a very dedicated system
01:24:21 - for spatial navigation.
01:24:24 - And it's possible that within this topological system, this very specific system
01:24:30 - representing the space,
01:24:32 - because of the topology, you may somehow combine things according to this topological space.
01:24:41 - Right. But it could be also for audition, for other system.
01:24:48 - But in general, I don't think there is a system outside this topological,
01:24:54 - I would say sensory space.
01:24:58 - That allow you to recombine in a clever way strategies. Okay.
01:25:05 - So this would be an empirical hypothesis we could test. Yeah.
01:25:08 - But then, so with your model, which gives a very specific prediction on how you can form,
01:25:16 - let's say, a new hypothesis so that you can deal with this task condition of
01:25:21 - staying or switching on the basis of exploration, because this is the problem
01:25:24 - you want to solve, right?
01:25:26 - You found actually an amazing close match with human performance.
01:25:31 - So but now so also humans
01:25:34 - were exposed to a task but they follow rules but the rules were
01:25:37 - switched so they had to sort of figure out what the new rule was but how did
01:25:42 - you assess this consistency between the model and human performance which were
01:25:46 - the aspects of human performance that were most now predictive if you want of
01:25:51 - the consistency with the model so yes.
01:25:57 - So So first, yeah, it's a very important question.
01:26:03 - It's basically how you fit a monor and how you compare model fit. Mm-hmm.
01:26:12 - It's a complex issue. At the end, we end up with this model.
01:26:18 - This model has a nice feature, is that it predicts some specific events,
01:26:25 - algorithmic events, which is predicted by the algorithm.
01:26:28 - That is, at some point, the model will create a new strategy from long-term
01:26:33 - memory. In this trial, the model predicts this.
01:26:37 - And of course, it implies a given profile in the response given by the model
01:26:44 - following this time point.
01:26:50 - It also predicts other types of events, very specific events,
01:26:55 - that is, at some point in a given trial,
01:27:00 - the algorithm switches out from exploration to return to exploitation by confirming
01:27:05 - the hypothetical strategy.
01:27:09 - And of course, it's associated with a given profile of response.
01:27:14 - The model also predict another kind of event that the hypothetical strategy needs to be rejected.
01:27:23 - So the algorithm, there are some specific algorithmic events,
01:27:28 - and we can check whether this, how the model perform around these algorithmic events.
01:27:38 - Predict what subject how subject perform around these events.
01:27:45 - Right. But it's important to know that these events are pure theoretical construct.
01:27:52 - Right. They are not in the experiment. They are not manipulated by the experimenter. Right, exactly.
01:27:58 - Yes. The model, the algorithm tells you, okay, in this triangle subjects should
01:28:03 - have set a new hypothetical strategy.
01:28:08 - Right, exactly. And the model behaves in this way around this event.
01:28:13 - So the way we check is how subject perform in the way the model perform around this predicted trial.
01:28:20 - And we found that it was reasonably well the case. Right.
01:28:27 - And this is the way, I think this is a very good test about the model.
01:28:31 - The model predicts that in this event, this is what should happen and you observe
01:28:36 - it right but now what you found is that also these hypothetical,
01:28:44 - no the theoretical constructs that you use now to interpret the performance
01:28:49 - of the human subjects actually matched again amazingly well on your fMRI signatures,
01:28:56 - so what were the outstanding the most salient effects that you found there.
01:29:02 - So for me, the most exciting effect and the most salient was about a specific
01:29:10 - type of algorithmic events, which is confirmation events.
01:29:16 - What is a confirmation event? A confirmation event is when the model,
01:29:20 - after creating a new strategy, decides that it continues with it.
01:29:31 - It confirms it. It confirms the assumption, the hypothesis.
01:29:35 - And because it confirms, it just means that it proceeds with it and continues
01:29:40 - to proceed, to use it for behaving.
01:29:42 - Which means that in the behavior, you cannot see this event.
01:29:45 - There is no marker, no signature in the behavior about this event.
01:29:50 - But of course, if this algorithm is really implemented in the brain,
01:29:54 - There should be an event in the brain that marks this time,
01:30:02 - this trial, when this strategy is confirmed. Mm-hmm.
01:30:07 - And we found actually a correlate of this event, of this algorithmic event,
01:30:12 - in the ventral striatum.
01:30:14 - So in the region that is really known to process rewards.
01:30:21 - So we have, and I think this is what could be nice with fMRI,
01:30:25 - that you have a purely internal neuronal event that is predicted by the model,
01:30:31 - at least the time it should happen.
01:30:34 - And that you cannot see in the behavior.
01:30:43 - But then your interpretation would be that this confirmation event triggers
01:30:48 - a ventral striatum to modulate, let's say, neuromodulatory signals to modulate memory.
01:30:55 - To cortical areas that memorize the internal. Right, exactly.
01:30:58 - So this is your interpretation of the signal. Right, exactly.
01:31:01 - So basically we know because the reliability of strategy are monitored within
01:31:09 - the ventromedial prefrontal context.
01:31:11 - And we know that this region projects to the ventral striatum. Right, exactly.
01:31:15 - So what is a confirmation event? It is when the probe actor,
01:31:18 - the hypothesis, passes from an unreliable to a reliable state.
01:31:25 - So it's a transition. It's a transition from unreliability to reliability.
01:31:31 - And probably my interpretation is that this transition is conveyed to the ventral
01:31:36 - striatum, and the ventral striatum use it as,
01:31:39 - or transform it as a reinforcer signal that is dispatched to regions that memorize the strategy.
01:31:49 - But this also means that the subject is performing the task.
01:31:53 - So it just tries out something. Now it's a hypothesis. There's no idea about the outcome.
01:31:59 - The action is successful. So there's positive reinforcement.
01:32:03 - Now we have an unexpected reward.
01:32:05 - So there we go with a reward signal. Is that roughly a correct interpretation as well?
01:32:15 - No because we can we can show that uh
01:32:18 - this effect of confirmation is goes
01:32:21 - in addition to the effect of having a positive or
01:32:24 - negative reward okay so it's an additional effect okay it's a with their own
01:32:30 - sources you're saying so it would be like an externally triggered event which
01:32:35 - is whatever happens in the task and there's an internally more cognitively dependent
01:32:39 - effect yes so basically Basically, there is an external signal,
01:32:43 - the external reward that is, of course, processed in the ventral three atoms
01:32:47 - as positive, negative, expected or not expected.
01:32:50 - And there is this internally driven feedback.
01:32:55 - It's an internal feedback. It's a cognitive feedback from the monitoring system,
01:33:01 - which is the anterior prefrontal regions, that provides this internal feedback,
01:33:09 - this cognitive feedback.
01:33:10 - I understand. But now, in some sense, to sort of close a little bit this part
01:33:15 - of the discussion, I could argue that, well, your tasks are cognitively encapsulated.
01:33:23 - That means, in some sense, you focus very much on this frontal area,
01:33:27 - almost in disconnection from everything else, right?
01:33:30 - So that means in some sense you are forced to think about hypothesis development
01:33:36 - and hypothesis testing as a pure internal cognitive act.
01:33:40 - But that's why I emphasized earlier this operational aspect.
01:33:44 - If I'm a behaving agent, this is how we are modeling problem solving and so
01:33:48 - on, actually I'm also testing hypotheses out in the world.
01:33:52 - And that in itself would allow me to build new policies from long-term memory,
01:33:58 - not by internal recombination,
01:34:00 - but by playing it out in the real world and building new memories following
01:34:05 - the standard procedures of memory.
01:34:07 - So without having to assume an additional layer… You mean in a totally unsupervised way?
01:34:13 - Well, without having to rely on internal cognitive hypothesis generation.
01:34:18 - I could just say, look, I'm here in the world, I want to explore stuff,
01:34:22 - so I'm going to allow different policies to play out or dominate my actions in some sequence.
01:34:29 - So now, as an end result, I have constructed a new policy.
01:34:33 - I've built a recombination, a recombinant of all of them, which you can now…
01:34:37 - Yeah, you mean, Parfait, if you behave randomly, you build a new… Well,
01:34:43 - it would be a very extreme way to do it. Yeah, it's extreme.
01:34:45 - But yes. Um...
01:34:51 - What I think you always monitor your own behavior.
01:35:01 - The monitoring system that monitors your behavior is always active.
01:35:07 - It seems to be very important for the organism.
01:35:12 - So I think this system is always there. The question is whether you monitor
01:35:17 - alternative strategy all the time.
01:35:19 - And explicitly, right? And explicitly, yeah.
01:35:25 - Explicitly is a good question. It's not sure that it's explicit for the subjects.
01:35:33 - It's an interesting question because as we found, we found that actually this, you are able to monitor,
01:35:42 - two, three alternative strategies in addition to your strategies you are using to act.
01:35:50 - I'm not sure that subjects are aware about or explicitly aware about having
01:35:55 - these three strategies in mind.
01:36:00 - It's a good question. I have no answer about that. But still,
01:36:04 - let us say that it's explicit.
01:36:08 - And I think you're right. We are not always monitoring alternative strategy. Right.
01:36:16 - Because it's probably quite effortful. Exactly right.
01:36:20 - Right, so you could also, this might be a method of last resort because you
01:36:24 - also could have, let's say, a more situated form of monitoring where you just
01:36:28 - say, look, let's take the football example.
01:36:30 - You could say, okay, I tried this in the past, it worked half,
01:36:34 - so I use only part of this policy and I just try it out.
01:36:37 - Okay, you just try it out in the world and now the world responds to whatever you did.
01:36:42 - So I have been monitoring if you want, but in a situated fashion. Right.
01:36:49 - By performing an experiment effectively.
01:36:54 - I think what you are describing here is just learning.
01:37:03 - It's just learning. It's just monitoring. You know?
01:37:08 - But it could give rise. But the funny thing is it can solve a problem you were
01:37:12 - solving with your monitoring.
01:37:13 - Because I can now have invented a new policy, building on known policies,
01:37:19 - but I have not relied on a complex internal memory-heavy process.
01:37:25 - I just played it out in the real world and indeed I learned and picked it up.
01:37:31 - Yes. Yeah, I think learning could be quite sophisticated. I mean,
01:37:40 - it's an embedded system that can be quite sophisticated.
01:37:49 - So as an alternative interpretation, you would leave that option open?
01:37:53 - No, I think this is too complementary system.
01:37:59 - But to explain the behavior of your subjects? No, you cannot.
01:38:05 - I mean, we have evidence in our experiments that we cannot explain our subject behavior using just,
01:38:19 - I mean, eliminating a monitoring system.
01:38:22 - Okay. So would you claim that all action depends on monitoring and rules?
01:38:32 - I think all your behavior is continuously monitored by your medial prefrontal system.
01:38:42 - And I think it's true not only in humans, but in many animals.
01:38:49 - And the first level of development is to have this system, this monitoring system.
01:38:56 - You monitor your behavior. So, I mean, the archaic system is let us say a system
01:39:01 - that just continuously adjusts like reference point learning to external contingency.
01:39:07 - Then the step, a bit more evolved than this very first step,
01:39:13 - is you monitor your behavior.
01:39:17 - And then the second step is you start to be able to monitor alternatives at the same time.
01:39:23 - So now to finish up our conversation, so we have really now this very deep understanding
01:39:35 - of decision-making and the frontal lobes.
01:39:38 - And this is based on a long track record of outstanding work,
01:39:41 - both experimental and theoretical, which I think makes it really so unique.
01:39:46 - So if we would like to follow in that tradition, what would be a chance law that we should follow?
01:39:52 - What's a chance law? To study the brain.
01:39:57 - Uh...
01:39:59 - My view is that,
01:40:03 - to do both experiments and modeling, to try to have in the same restricted team the two competencies.
01:40:17 - Because experiments are very important, because you can develop very nice model
01:40:22 - and very sophisticated, very beautiful model,
01:40:25 - mathematically beautiful model, and they are very satisfactory for our intellect,
01:40:34 - but actually they don't explain what humans do.
01:40:39 - So doing experiments forces you to move forward and not to stay in a comfortable
01:40:45 - way with very nice, beautiful models.
01:40:50 - And nature is not always perfect. Perfect.
01:40:55 - So this model, I mean, you need to develop models that are maybe less beautiful,
01:41:01 - but they are more efficient, more pragmatic.
01:41:05 - Okay. So the other thing is, so five years from now, I'm going to go visit you
01:41:09 - in Paris, and I'm going to confront you with a prediction you're going to make today.
01:41:15 - So I'm going to ask you, look, did this really work out?
01:41:18 - So what's this one prediction you're most passionate about today?
01:41:22 - Which is one prediction you would make? Like, okay, I would try to think about
01:41:29 - a prediction that is testable. Okay.
01:41:37 - I have one prediction that, a strong prediction from our data,
01:41:42 - is that humans cannot monitor more than three strategies at the same time,
01:41:49 - three, four strategies. Yes.
01:41:54 - Because I am interested about this prediction because this is what we found
01:42:00 - in our experiment, but maybe in other experiments it could be different.
01:42:05 - So I would like to know whether this capacity limit we found in our experiment
01:42:10 - is quite general, is it true or whether it's just an anecdotal finding,
01:42:18 - and I think yeah it's quite simple and I think it's testable Wonderful,
01:42:24 - so Etienne Costelan thank you very much for this conversation Thank you for
01:42:27 - your question, I really enjoyed them and enjoyed discussing with you.
01:42:32 - Music.
01:42:36 - The CSN podcast was produced by the Convergent Science Network of Biometrics
01:42:42 - and Biohybrid Systems, a project funded by the European Sevens Research Framework Programme.
01:42:50 - For more interviews, recorded lectures or upcoming conferences in the field
01:42:55 - of biometrics and biohybrid systems, go to csnnetwork.eu.
01:43:02 - And thank you for listening.
01:43:02 - Music.
01:43:05 - Thank you for watching!

CSN Podcasts

Etienne Koechlin on prefrontal cortex and cognitive control

Season 2013

Description

CSN Podcasts

Episodes

Eberhard Fetz on brain-computer interface and neurochip

Tim Pierce on olfaction and chemical sensing

Marc Toussaint on planning as inference and graphical models

Cyriel Pennartz on hippocampus and ventral striatum

Paul Verschure on consciousness and distributed adaptive control

Timestamp

Be the first to leave a comment

Leave a commentCancel

Legal

Follow us on

Etienne Koechlin on prefrontal cortex and cognitive control

Description

Episodes

Timestamp

Be the first to leave a comment

Leave a commentCancel

Tags

Legal

Follow us on

Login to enjoy full advantages

Please login or subscribe to continue.

Go Premium!

Enjoy the full advantage of the premium access.

Stop following

Cancel subscription