cover riccardo sanz

Riccardo Sanz on machine consciousness and control engineering

  • cover play_arrow

    PLAY EPISODE


cover riccardo sanz
Season 2010
Season 2010
Description arrow_drop_down

Description

What happens when engineered systems become too complex for humans to understand, let alone control? Riccardo Sanz argues that the path forward requires machines capable of controlling themselves , and that this leads, perhaps inevitably, toward machine self-awareness. Subscribe for more from the Convergent Science Network podcast series. Riccardo Sanz approaches consciousness not from philosophy or neuroscience, but from the hard edge of control engineering. In this interview, he explains why traditional control theory breaks down when the controller itself becomes so complex that it can fail in ways no human operator can diagnose. Modern countrywide electrical grids, flight control systems, and computing infrastructures already exceed human comprehension during failure states , leading to blackouts, crashes, and cascading breakdowns. Sanz’s provocative claim is that the only scalable solution is to give these systems the capacity to model and manage themselves. This is not, he insists, an attempt to mimic human consciousness. Instead, his research group arrived at concepts of self-awareness and self-modeling from purely technical requirements for robust, adaptive control. The convergence with consciousness research was discovered after the fact, when they found that the competences they needed, self-monitoring, self-repair, cognitive flexibility, overlapped with properties that consciousness researchers attribute to sentient systems. The distinction matters: Sanz argues that copying the human brain would reproduce its evolutionary limitations, whereas extracting the underlying principles of self-awareness could yield systems that far exceed human capabilities in speed and information integration. The conversation probes the risk of infinite regress , if a controller needs a meta-controller, what controls that? Sanz proposes that each successive layer of self-representation compresses complexity, collapsing into increasingly compact models until the system converges on a unified self-description. He draws parallels to industrial process control, where hierarchies of control loops ultimately reduce to a single variable like profitability, but notes that current systems lack the self-awareness to handle their own failures. On the question of existential risk from superintelligent machines, Sanz is sanguine. He believes that by the time engineering reaches the sophistication needed to create deeply self-aware systems, the technology for bounding their behavior will be equally mature. His core message is a call for rigor: the fragmentation of control engineering, neuroscience, and philosophy into separate communities with incompatible vocabularies is the real barrier to progress.

Tagged as:

About the author call_made

CSN Podcasts

Both the triumphs of humanity and its most evil deeds have resulted from collaboration. In a time where humanity is required to aspire to the former and minimize the latter, the question arises of how collaboration arises and why it fails. Surprisingly, this phenomenon, so central to who we are, is not well understood. Hence, a collaborative effort is required to understand collaboration in its full biological, psychological, sociological, cultural, and economic complexity and to translate this understanding into operational impact. This series of podcasts is one step toward achieving these complementary goals. The Collaboration Podcast presents interviews with people who are central orchestrators of collaboration in various domains including business, government, science, art, health, sustainability, and the military. The discussions were conducted by Prof. Dr. Paul F.M.J. Verschure and members of the Program Advisory Committee of the Ernst Strungmann Forum on Collaboration (https://www.esforum.de/forums/ESF32_Collaboration.html) during 2021 and had the goal to sketch a map of opportunities, challenges, and obstacles in human collaboration. The forum took place in May 2022, and now we would like to share this series of interviews with a broader audience. The full report of the Forum will be published in 2023 by MIT Press. The podcast was produced by the Convergent Science Network (https://www.convergentsciencenetwork.org/). Context: The stability of social systems depends critically on realizing sustainable methods of “collaboration,” yet how and by which means collaboration is achieved is not clearly understood; neither are the conditions or processes that lead to its breakdown or failure. Collaboration can be understood as cooperation between agents toward mutually constructed goals. Part of the reason for our lack of understanding is that the phenomenon of collaboration is, by nature, a highly multidisciplinary problem, and effective research into its complexities has been difficult to achieve across the broad range of scientific and technical disciplines involved. The need for a fundamental understanding of collaboration, however, has become increasingly important. Not only does humankind demand answers as it attempts to address critical challenges at multiple scales (e.g., climate change, migration, enhanced automation, social and economic inequality), but ever-increasing technological and economic means of interconnecting people and societies are disrupting long-established, familiar patterns of how we interact. Radical technological changes that are ongoing have the potential to reshape collaboration in ways that are currently hard to predict or influence (e.g., by altering configurations in interaction, information creation, and modes of communication). On one hand, such changes could disrupt hitherto stable forms of collaboration by affecting critical communication channels and traditional roles, as can be observed in the rapidly changing patterns in governance, commerce, and social interaction. Conversely, technology could lead to the emergence of novel, successful forms of collaboration that deviate from traditional “hierarchical” architectures. Evidence of this can be seen in areas as diverse as highly automated manufacturing plants, the open science movement, collaborative software repositories, user-centered services, and the sharing of economy-based modes of organization. Without a fundamental understanding of the mechanisms, processes, and boundary conditions of collaboration, it is not possible to evaluate or predict which of these possible scenarios are sustainable or even plausible. The Forum “How Collaboration Arises and Why it Fails” (May 8–13, 2022, Location: Frankfurt am Main, Germany) Chairs: Andreas Roepstorff and Paul Verschure Program Advisory Committee: Jenna Bednar, Julia R. Lupp, Bhavani R. Rao , Andreas Roepstorff, Ferdinand von Siemens, and Paul Verschure

More posts

Timestamp

  • fast_forward00:00:00 - Ricardo, one of the speakers at our summer school, and Ricardo,
  • fast_forward00:00:05 - you presented a view on machines, if you want, and robots,
  • fast_forward00:00:13 - control systems, in which you made the argument that actually they should naturally
  • fast_forward00:00:19 - include phenomena you could call consciousness.
  • fast_forward00:00:25 - So, what are your considerations there to come to this rather,
  • fast_forward00:00:29 - let's say, counterintuitive conclusion?
  • fast_forward00:00:31 - Well, I think it is not a counterintuitive conclusion.
  • fast_forward00:00:35 - I think it's just a need of how things are developing and all the needs of having
  • fast_forward00:00:40 - more robust systems today where we can rely our lives upon in this changing
  • fast_forward00:00:46 - environment where things are changing so fast.
  • fast_forward00:00:48 - So, the question for the new machines and the machines of the future is that
  • fast_forward00:00:53 - they must be much more adaptive in a sense.
  • fast_forward00:00:56 - And not just adaptive to changes in the environment, but adaptive to their own changes.
  • fast_forward00:01:01 - And that takes us to this question of consciousness.
  • fast_forward00:01:04 - It's a question of name, more than a question of fashion.
  • fast_forward00:01:08 - Thank you. Yeah, but in some sense, you make an argument based on the sense of complexity, right?
  • fast_forward00:01:16 - Where you would say, well, if we want to start to deal with the world,
  • fast_forward00:01:20 - the dynamical world, in a way that is acceptable, the controllers will get more complex.
  • fast_forward00:01:26 - And at some point, the controller will find it necessary to control itself.
  • fast_forward00:01:30 - And it's sort of somewhere at that point that you're saying consciousness comes
  • fast_forward00:01:34 - in. So how should I understand that? Where's that transition point exactly?
  • fast_forward00:01:38 - Well, the question is the moment where humans cannot really control or address
  • fast_forward00:01:44 - the complexity of the systems.
  • fast_forward00:01:45 - And the only possibility for that is to have a scalable approach.
  • fast_forward00:01:50 - And having a scalable approach, humans don't scale well. That's a problem because
  • fast_forward00:01:54 - humans typically think one by one.
  • fast_forward00:01:56 - And thinking by teams is not yet solved for humans.
  • fast_forward00:02:00 - But for machines, it could be. And the question is to transfer the competences
  • fast_forward00:02:05 - for managing systems, managing complex systems into systems because they can scale up.
  • fast_forward00:02:11 - And that's the question at the end. Because growing complexity can only be handled
  • fast_forward00:02:16 - by systems that can grow.
  • fast_forward00:02:18 - And humans cannot grow as they are today. And that's the problem.
  • fast_forward00:02:22 - When we are in the case of technical systems, we are touching the limits of understandability.
  • fast_forward00:02:28 - So in many cases failures cannot be understood by humans because of the complexity
  • fast_forward00:02:34 - and problems that are appearing for example in countrywide electrical networks
  • fast_forward00:02:39 - cannot be understood by humans and that's the problem,
  • fast_forward00:02:43 - we need to transfer into systems that scale more than humans and that are able
  • fast_forward00:02:48 - to handle these problems that seems counterintuitive, no?
  • fast_forward00:02:52 - Because in some sense you're saying we want to have scalable systems to come
  • fast_forward00:02:57 - to a scalable system they must be sentient they should have consciousness humans
  • fast_forward00:03:02 - are not scalable, they don't scale very well however.
  • fast_forward00:03:08 - You didn't say that, but that's what I was thinking. Humans are conscious. Yeah.
  • fast_forward00:03:12 - So that would suggest that consciousness is not a necessary ingredient of these
  • fast_forward00:03:16 - scalable systems. Or did I misunderstand the argument?
  • fast_forward00:03:19 - No, the question, we are real. The question of consciousness and self-awareness,
  • fast_forward00:03:24 - not trying to mimic humans at all.
  • fast_forward00:03:27 - We reached a point finding some basic design principles or structure principles.
  • fast_forward00:03:33 - And then we found this community dealing with self-awareness and consciousness
  • fast_forward00:03:38 - in discussing these topics.
  • fast_forward00:03:40 - But we came not from the side of mimicking humans, but from the side of finding
  • fast_forward00:03:45 - some structural competences into
  • fast_forward00:03:47 - the systems themselves, related to self-awareness and self-competence.
  • fast_forward00:03:52 - And then we found that the people doing research on machine consciousness,
  • fast_forward00:03:55 - in many of the cases, they are just trying to mimic human behavior.
  • fast_forward00:03:59 - Without this understanding of the basic principles regarding self-awareness
  • fast_forward00:04:05 - and self-management at the very end and deeply.
  • fast_forward00:04:09 - So we came from the purely technical side into the topics where those people are also discussing.
  • fast_forward00:04:16 - So in a sense, we are not really trying to build machines that mimic human consciousness at all.
  • fast_forward00:04:22 - We are trying to build machines that have the competences that are associated
  • fast_forward00:04:27 - to this self-awareness in humans, but perhaps not the same way.
  • fast_forward00:04:32 - Okay, but explain something to me that I can, I mean, in some sense,
  • fast_forward00:04:39 - you're coming from a control engineering perspective, right?
  • fast_forward00:04:41 - And you say, okay, in control engineering, we're hitting a boundary now.
  • fast_forward00:04:44 - But then what's this traditional control engineering view that we now have to change?
  • fast_forward00:04:51 - What's this traditional view and why does it break down exactly?
  • fast_forward00:04:54 - Well, the breakdown is that typically control engineers model,
  • fast_forward00:04:58 - have a model of the plant they are controlling.
  • fast_forward00:05:00 - But they don't have a model of the controller because
  • fast_forward00:05:03 - the complexity of the controller is so high that they really cannot have
  • fast_forward00:05:06 - a mathematical model the way they had in the past so you
  • fast_forward00:05:09 - can have a differential equation model of a car and then you build a controller
  • fast_forward00:05:12 - for it but then what happens when the controller is failing how can you control
  • fast_forward00:05:17 - the controller and this is missing and that's the problem where traditional
  • fast_forward00:05:21 - control engineering is not addressing at all because for example in other cars,
  • fast_forward00:05:26 - you have plenty of networking inside,
  • fast_forward00:05:27 - plenty of electronic control units and computing.
  • fast_forward00:05:30 - If something fails there, there is no control over that.
  • fast_forward00:05:33 - So if the failure is in the control system, current control technology is not
  • fast_forward00:05:40 - addressing it. And that's the problem.
  • fast_forward00:05:43 - Because they are only addressing faults at the level of the plant,
  • fast_forward00:05:46 - but not faults at the level of the controller.
  • fast_forward00:05:48 - So we need controllers that are able to control themselves to overcome these difficulties.
  • fast_forward00:05:54 - Because of the growing complexity of the control systems. In that case.
  • fast_forward00:06:00 - So that's clear to me. Following this argument, I could then say,
  • fast_forward00:06:05 - well, but how do I now prevent falling into an infinite regress of these controllers
  • fast_forward00:06:11 - and meta controllers and supervisory controllers and so on?
  • fast_forward00:06:14 - Yeah, the question is something that was mentioned during this course.
  • fast_forward00:06:17 - We bound the infinite regress if in every step the needs for representation are reduced.
  • fast_forward00:06:28 - So, in a sense, the system is collapsing into a single compact and simplified
  • fast_forward00:06:34 - representation of itself.
  • fast_forward00:06:36 - So, that's the question. The question is not just to have a whole representation
  • fast_forward00:06:39 - of the complete system, but a
  • fast_forward00:06:42 - compact, more synthetic and more simple representation at each meta level.
  • fast_forward00:06:48 - So at the end, the system will collapse into a single representation that is
  • fast_forward00:06:51 - addressing, so to say, in a pyramidal way, the understanding of the lower layers.
  • fast_forward00:06:58 - So is that a hope, a hypothesis, or is it a reality? No, it's just a hypothesis. Okay.
  • fast_forward00:07:05 - It does sound a little bit like the old intentional systems stance of Dan Dennett,
  • fast_forward00:07:11 - where he would say like, well, we can resolve the homunculus problem because
  • fast_forward00:07:15 - we can just split it up in simpler, simpler, simpler homunculi.
  • fast_forward00:07:18 - And in the end, it's just a binary decision that has to be made to its lowest level.
  • fast_forward00:07:22 - Yeah. Yeah, but this is, in some sense, reducing away the problem and not really solving it, right?
  • fast_forward00:07:30 - You explain it away because, actually, then you have to show that this is possible,
  • fast_forward00:07:34 - that you can actually, if you use the word simplicity for this,
  • fast_forward00:07:38 - that you can get to this kind of compression of complexity. Yeah.
  • fast_forward00:07:44 - Could you give an example of this kind of compression? How do you think about
  • fast_forward00:07:48 - this, even for a simple controller?
  • fast_forward00:07:50 - For a simple controller, you can find this kind of approach in very complex
  • fast_forward00:07:54 - plants where you can have thousands of control loops controlling basic properties
  • fast_forward00:08:03 - in plants. And then you have loops over loops.
  • fast_forward00:08:05 - And when you go up through the pyramid of control, at the very end,
  • fast_forward00:08:09 - what you have is just money.
  • fast_forward00:08:11 - And you have a single loop controlling the money, the money that the plant is producing.
  • fast_forward00:08:15 - And then beyond that, the safety and the maintainability and environmental impact.
  • fast_forward00:08:23 - And then you have a unit control and then you have, well, this is a scale down
  • fast_forward00:08:28 - and you can find this inside the plants.
  • fast_forward00:08:30 - But the problem is that in that system, this kind of hierarchy is not self-aware.
  • fast_forward00:08:34 - So it's not addressing the problems it has upon itself and humans are there
  • fast_forward00:08:39 - to handle And it's the same problems that you can find in information technology,
  • fast_forward00:08:42 - in networks and computing systems and infrastructures.
  • fast_forward00:08:46 - It's the same problem that the computers controlling things are not controlling
  • fast_forward00:08:51 - themselves. That's the big problem.
  • fast_forward00:08:53 - But this is an interesting example, definitely.
  • fast_forward00:08:57 - Also, as you say, right, it's always the humans enter the loop somewhere.
  • fast_forward00:09:01 - And maybe it's on the basis of that luxury that we have these general purpose
  • fast_forward00:09:06 - controllers running around, which are human beings.
  • fast_forward00:09:10 - That then these engineered systems can actually operate.
  • fast_forward00:09:13 - And this might give us then the false belief that the principles in which they
  • fast_forward00:09:16 - operate are valid principles.
  • fast_forward00:09:19 - Do you think that's a reasonable criticism? Yeah, it is.
  • fast_forward00:09:23 - The question about humans is an old question in automation.
  • fast_forward00:09:28 - Humans should be there or shouldn't be there. And the question is that today,
  • fast_forward00:09:32 - in many of the systems, humans cannot cope with the problem.
  • fast_forward00:09:36 - And that's the problem we are facing these days. Humans are not able to solve
  • fast_forward00:09:40 - problems at the level of electrical networks, countrywide electrical networks.
  • fast_forward00:09:44 - They are not able to solve problems at the level of flight control systems when
  • fast_forward00:09:48 - faults appear. Because of the requirements of speed, of information integration,
  • fast_forward00:09:53 - of focus of attention, humans cannot solve the problems.
  • fast_forward00:09:58 - And then what happens? When those faults appear, systems do crash.
  • fast_forward00:10:04 - And there is no solution today for this kind of problem. Then we have blackouts,
  • fast_forward00:10:08 - we have very big airplane accidents.
  • fast_forward00:10:12 - Because humans cannot cope, and we don't have solutions for handling those problems
  • fast_forward00:10:16 - at the level that we need.
  • fast_forward00:10:18 - Following your earlier argument, I could then say, well, humans cannot cope
  • fast_forward00:10:21 - because the engineered system has created a situation in which it cannot cope
  • fast_forward00:10:26 - and subsequently the human user cannot cope either.
  • fast_forward00:10:29 - Yeah. Is that fair? It's fair. Okay. And that counts because we are all the
  • fast_forward00:10:33 - time pushing forward the requirements for the technical systems.
  • fast_forward00:10:36 - And now we're in the frontier of pushing forward the requirements of technical
  • fast_forward00:10:41 - systems beyond human control competence.
  • fast_forward00:10:44 - And we are in such a frontier today. There are phenomena that are appearing
  • fast_forward00:10:49 - in technical systems that humans don't understand.
  • fast_forward00:10:52 - Take an example of that? An example, for example, there are waves of electrical
  • fast_forward00:10:56 - voltage crossing across Europe, north and south, and people don't understand what's going on.
  • fast_forward00:11:04 - Because there is no way of understanding the whole picture at all.
  • fast_forward00:11:08 - We need an information theory and a model of what's going on that goes beyond
  • fast_forward00:11:12 - the capabilities of humans.
  • fast_forward00:11:14 - That happens also with technical systems in information and infrastructures.
  • fast_forward00:11:21 - The understanding of how a complex server, a computing server that is composed
  • fast_forward00:11:27 - by thousands of computers at
  • fast_forward00:11:29 - the same time, how it behaves when something fails, when there is a virus.
  • fast_forward00:11:34 - Humans cannot cope with that. The only solution is to shut down the systems
  • fast_forward00:11:38 - and start from scratch because nobody is understanding what's going on and there
  • fast_forward00:11:42 - is no possibility of doing that.
  • fast_forward00:11:44 - But the only strategy that we can think of is making systems keep themselves working.
  • fast_forward00:11:51 - However, the way you phrase the problem now, which definitely is a serious problem,
  • fast_forward00:11:56 - seems to create almost a paradox because certainly where I'm coming from,
  • fast_forward00:12:01 - there's this belief like, okay, if we take a biomimetic approach,
  • fast_forward00:12:04 - an approach more based on understanding of the brain, we can solve these kinds of problems.
  • fast_forward00:12:09 - But in some sense, what you're saying for all practical purposes,
  • fast_forward00:12:12 - also the human brain is not able to solve these problems today.
  • fast_forward00:12:15 - So my belief is actually based on a false assumption.
  • fast_forward00:12:19 - Is that correct? Yeah, in a sense, it is correct. But the question is that we
  • fast_forward00:12:23 - are proposing is not bio-inspired things. What we are saying is that the things
  • fast_forward00:12:27 - that humans do are an instance of this kind of approach.
  • fast_forward00:12:30 - That is not enough. I mean, there is a kind of solutions that are,
  • fast_forward00:12:36 - so to say, the self-awareness solutions.
  • fast_forward00:12:38 - And humans are just an instance of that with their own limitations.
  • fast_forward00:12:41 - And making a copy of humans for solving those problems is not the solution.
  • fast_forward00:12:46 - The solution is going beyond particular human implementations and particular
  • fast_forward00:12:52 - human brains and understanding the basic principles that are there.
  • fast_forward00:12:55 - And that's the problem. We need to understand the very principles of self-awareness
  • fast_forward00:12:59 - to be able to create a new class of system that has much more competence regarding speed and.
  • fast_forward00:13:07 - Capability of integrating information that humans do have
  • fast_forward00:13:10 - but to do that we can get by inspiration
  • fast_forward00:13:13 - but in the sense of understanding how humans do and extracting
  • fast_forward00:13:16 - the basic principles of that not copying the brain but understanding how it
  • fast_forward00:13:19 - works and use that kind of architecture to scale it up that's very exciting
  • fast_forward00:13:24 - so you're saying actually the human brain is is like an imperfect approximation
  • fast_forward00:13:29 - of some ultimate brain i would say that we might that we might be able to figure
  • fast_forward00:13:34 - out i would say say that. That's a good word for it.
  • fast_forward00:13:36 - Okay, so how do I know that I found that perfect brain?
  • fast_forward00:13:43 - No, the question is, can we find those patterns in the brain?
  • fast_forward00:13:47 - Can we understand how humans think about themselves and get some basic principles about that?
  • fast_forward00:13:52 - Can we have a good theory of how humans think about humans?
  • fast_forward00:13:55 - And that's a big problem. Not just for extracting the basic principles,
  • fast_forward00:14:00 - that's a big problem also for interacting systems with humans.
  • fast_forward00:14:03 - The main problem, to my understanding about humans interacting with computers
  • fast_forward00:14:07 - is that computers don't have a good theory of the human.
  • fast_forward00:14:09 - It's not a problem of having good theories about computers.
  • fast_forward00:14:13 - It's the question of how humans are working and how is the perceptual systems of humans.
  • fast_forward00:14:18 - The question for me is, can we understand humans in a sense of understanding
  • fast_forward00:14:24 - how they work to make systems that are not human at all, but are exploiting
  • fast_forward00:14:29 - some basic competences that humans are able to do?
  • fast_forward00:14:34 - For example, a basic competence is the cognitive flexibility that humans do have.
  • fast_forward00:14:39 - Humans can learn how things are working. Can we translate that into a machine?
  • fast_forward00:14:44 - Not by building a human, but by building into the machine the capability of
  • fast_forward00:14:49 - understanding how things are working.
  • fast_forward00:14:51 - This is a basic competence that we need for technical systems.
  • fast_forward00:14:54 - And that does not mean making a human.
  • fast_forward00:14:56 - That's a big mistake. People try to think that to make intelligent machines,
  • fast_forward00:15:00 - we should copy humans. I don't think that's the problem.
  • fast_forward00:15:04 - Would you say then that the human brain is compromised because of,
  • fast_forward00:15:10 - let's say, the way it's embodied?
  • fast_forward00:15:14 - Of the evolutionary constraints imposed? Or why is our brain imperfect in that sense?
  • fast_forward00:15:19 - Well, the brains are imperfect because evolutionary change does not create optimal
  • fast_forward00:15:25 - solutions, just good enough solutions.
  • fast_forward00:15:27 - Solutions that are good enough for the ecological needs. They are working.
  • fast_forward00:15:30 - So humans' brains are good for controlling mammals in the context of 60,000
  • fast_forward00:15:37 - years ago or so. They are not optimal solutions.
  • fast_forward00:15:40 - They are scaled up to the level of solving the problems at that time.
  • fast_forward00:15:44 - And that means that if we want to translate that into a different body with
  • fast_forward00:15:49 - a different set of requirements, we cannot really copy it unless we are doing a humanoid robot.
  • fast_forward00:15:55 - Would have the same kind of problems and the same kind of interaction of humans.
  • fast_forward00:15:58 - If the context, if the ecological niche for the machine is very different,
  • fast_forward00:16:04 - copying the brain will not work.
  • fast_forward00:16:05 - We need something very different for that.
  • fast_forward00:16:08 - So there are plenty of constraints in the human brain that are related with
  • fast_forward00:16:11 - environments of the humans, with the body of the humans, with the evolutionary history of humans.
  • fast_forward00:16:17 - And we need to identify them to get rid of the constraints and the limitations
  • fast_forward00:16:21 - and go to the basic principles indeed. Right.
  • fast_forward00:16:25 - Now, there's some fear mongering going on certainly across the Atlantic Ocean
  • fast_forward00:16:30 - about what's called the singularity. In case we're going to build this perfect
  • fast_forward00:16:34 - machine, it might decide it doesn't need humans anymore.
  • fast_forward00:16:37 - We get in some sort of Skynet Terminator scenario.
  • fast_forward00:16:41 - Do you share these concerns? Not really. Not really for a single reason.
  • fast_forward00:16:46 - I don't think that we are able to create the level of self-awareness and selfiness,
  • fast_forward00:16:54 - I would say, into machines that humans do have.
  • fast_forward00:16:57 - We can build some basic properties of humans into the machines, but not create….
  • fast_forward00:17:04 - Today, these complex entities. I'm not afraid of that.
  • fast_forward00:17:09 - I'm pretty sure that when we reach the level of being able to create this kind of entity,
  • fast_forward00:17:15 - the technology will be so sophisticated that we can put any limitation that
  • fast_forward00:17:21 - we want in the sense of Asimov's laws of robotics.
  • fast_forward00:17:26 - We will be able to bound the behavior. That's a big problem because today the
  • fast_forward00:17:31 - problem against building complex systems is how to bound the wrong behaviors at the end.
  • fast_forward00:17:38 - And that's a very big problem, but I think that technologies today are very aware of that.
  • fast_forward00:17:43 - That before you press the start button of a machine, you need to really bound
  • fast_forward00:17:49 - it, bound the behavior, and know how they are bound. Right, absolutely.
  • fast_forward00:17:53 - So now, coming from this perspective of control engineering,
  • fast_forward00:17:59 - and also as we discussed in the summer school, there is often this idea like,
  • fast_forward00:18:03 - Like, well, the brain is a control system.
  • fast_forward00:18:05 - But now, as we discussed, the control systems that we know as engineers are incomplete.
  • fast_forward00:18:13 - So how far does it get us to just make that claim the brain is a control system?
  • fast_forward00:18:20 - I mean, is it a bit too easy to say?
  • fast_forward00:18:23 - At what point of definition does it actually give us insight?
  • fast_forward00:18:28 - The understanding of why I say that the brain is a control system is very simple.
  • fast_forward00:18:35 - Most of the inputs to the brain and the outputs of the brain are just feeding
  • fast_forward00:18:41 - the brain and getting some products, waste products on it.
  • fast_forward00:18:47 - And all the meaningful inputs are information.
  • fast_forward00:18:51 - And that means that what is happening in the brain is an informational process.
  • fast_forward00:18:56 - It's not a chemical process producing any kind of molecule that the body is using.
  • fast_forward00:19:03 - The molecules that appear there are molecules related to information.
  • fast_forward00:19:08 - So, what is happening in the brain is information coming in,
  • fast_forward00:19:11 - information coming out. That is
  • fast_forward00:19:12 - the function of the brain, and that is the function of the control system.
  • fast_forward00:19:15 - So when systems that do that are information processors, and when information
  • fast_forward00:19:21 - processors link sensors and actuators, they are controllers,
  • fast_forward00:19:25 - they are no other thing. So it's not a question of...
  • fast_forward00:19:28 - We can say that it's a question of observing the thing. So we get cables coming
  • fast_forward00:19:34 - from sensors into informational processor and then going out into actuators.
  • fast_forward00:19:39 - That's a control system as we build it today.
  • fast_forward00:19:43 - But that's in some sense a very broad framework in which we define control.
  • fast_forward00:19:48 - So can we be more specific? I mean, do you say like, well, we have engineered
  • fast_forward00:19:53 - specific kinds of control systems that we think are particularly relevant when
  • fast_forward00:19:59 - we look at the brain or when we look at a certain area in the brain?
  • fast_forward00:20:02 - Is that... No, in fact, this is the whole thing. This is all materials from the 50s and 60s.
  • fast_forward00:20:07 - This is the whole cybernetics movement in the... The problem that happened with
  • fast_forward00:20:11 - cybernetics is that it was split in different fields.
  • fast_forward00:20:15 - One was information technology. The other one was control technology that is
  • fast_forward00:20:19 - related to building technical systems at the end.
  • fast_forward00:20:22 - Some people kept it into the world of networks and brain understanding and biology.
  • fast_forward00:20:27 - But the field is the field of the 50s and 60s. So this is an old picture.
  • fast_forward00:20:33 - This is not something new. And in fact, if we read what people like McCulloch
  • fast_forward00:20:39 - and Wiener said, they were saying that controllers in technical systems and
  • fast_forward00:20:44 - brains are doing the same thing.
  • fast_forward00:20:45 - And the analysis they did in the past are the analysis that we need to foster today.
  • fast_forward00:20:50 - And now we have something that they didn't have, that these huge amounts of competing.
  • fast_forward00:20:56 - Fast enough as to control bodies of enormous complexity.
  • fast_forward00:21:00 - So we are now in a situation that we really can go back into this cybernetic
  • fast_forward00:21:05 - vision and really understand how brains work as controllers. I mean, this picture.
  • fast_forward00:21:12 - So in the domain of, let's say, the neuroscience of motor control,
  • fast_forward00:21:17 - there's quite some excitement nowadays on particular kinds of concepts coming
  • fast_forward00:21:22 - from control engineering and people believe, okay, this is now explaining what
  • fast_forward00:21:26 - the brain is doing, what structure X is doing in terms of forward modeling or inverse modeling, etc.
  • fast_forward00:21:33 - What do you think of these approaches?
  • fast_forward00:21:34 - Is this actually helpful or do you think it's sort of confusing?
  • fast_forward00:21:37 - No, it is very helpful. The problem that we have is that when we look at control
  • fast_forward00:21:42 - technology and control theory, the problem is that control theory, as we,
  • fast_forward00:21:47 - I would say, control engineers do, do is based on solid mathematical models
  • fast_forward00:21:54 - and sets of equations that we can solve.
  • fast_forward00:21:57 - And we can solve analytically, and that's the question.
  • fast_forward00:22:00 - When we have models that are so complex that we really cannot solve analytically,
  • fast_forward00:22:04 - then that's the field for other people trying to do simulations and trying to
  • fast_forward00:22:08 - do computer implementations and trying to do, well, environments for testing,
  • fast_forward00:22:13 - forward testing of models, in a sense.
  • fast_forward00:22:15 - And this is the field of people that are modeling brains, because the mathematical
  • fast_forward00:22:19 - description of a brain is so
  • fast_forward00:22:20 - complex that we really cannot solve the equations by analytical methods.
  • fast_forward00:22:25 - So the communities, in a sense, are split into two ways of doing things.
  • fast_forward00:22:29 - One way is the mathematical-analytical method, and the other way is the forward
  • fast_forward00:22:34 - model-based synthetic method of using computers for looking at what's going on.
  • fast_forward00:22:40 - Because really, we cannot invert the model, and that's the point.
  • fast_forward00:22:44 - When complexity grows, when non-linearity appears, we really cannot solve the
  • fast_forward00:22:49 - equations and invert the models to make the controllers.
  • fast_forward00:22:51 - Then we revert into this alternative way of exploring.
  • fast_forward00:22:57 - And by simulation what's going on and trying to understand that.
  • fast_forward00:23:00 - Because really we cannot analytically solve the problems.
  • fast_forward00:23:04 - But the community should get together. Again, there are problems and theoretical
  • fast_forward00:23:08 - concepts that are critical to be shared.
  • fast_forward00:23:11 - The concept of observability, for example, the concept of controllability.
  • fast_forward00:23:16 - Those are concepts that should be a common trait across all these technologies.
  • fast_forward00:23:23 - So how would you see a concept like controllability giving us leverage when we're look at the brain?
  • fast_forward00:23:28 - Well, the question of controllability is a question of how can we do the body
  • fast_forward00:23:33 - do whatever we want and what things we can do with this.
  • fast_forward00:23:37 - And the question is, can we solve analytically that problem?
  • fast_forward00:23:40 - Can we decide if a particular body, for example, of a humanoid robot is controllable?
  • fast_forward00:23:45 - And that is a theoretical problem
  • fast_forward00:23:47 - that can be solved. But many people in robotics is not trying to do that.
  • fast_forward00:23:51 - It's trying just to, for example, people that are trying to do flexible They
  • fast_forward00:23:55 - are trying to solve a problem by just experimentation and hacking,
  • fast_forward00:23:59 - I would say, instead of having the deep models that they need to address the problem.
  • fast_forward00:24:05 - But the problem here is that we have isolated communities with different ontologies,
  • fast_forward00:24:09 - some more rigorous and more mathematical, more physical, the other one more
  • fast_forward00:24:13 - related on empirical aspects and, so to say, philosophical even,
  • fast_forward00:24:17 - even metaphysical assumptions.
  • fast_forward00:24:19 - Assumptions, and we need to get rid of the separation and have a single community
  • fast_forward00:24:24 - across this, from the very simple controllers that we can find anywhere,
  • fast_forward00:24:29 - to the brain, and to the psychological,
  • fast_forward00:24:32 - more complex and more sophisticated psychological aspects, and have a single.
  • fast_forward00:24:37 - Ontology that can be shared across all this stuff.
  • fast_forward00:24:40 - Just take, for example, the concept of information.
  • fast_forward00:24:43 - Nobody agrees on that. And when we talk to different people,
  • fast_forward00:24:46 - they don't reach an agreement on that. And that's a problem.
  • fast_forward00:24:49 - The problem, I think, the big problem that we have is the community being split on those areas.
  • fast_forward00:24:57 - Now, to bring them together, if we, for instance, go back to this notion of
  • fast_forward00:25:02 - the brain as a controller, or more specifically, as was discussed in the school,
  • fast_forward00:25:06 - the brain is an adaptive filter.
  • fast_forward00:25:08 - And if you have this very broad definition of a controller, like,
  • fast_forward00:25:11 - okay, whenever I'm mapping some input states to output states, whatever
  • fast_forward00:25:14 - happens in between i consider a controller in some
  • fast_forward00:25:17 - sense then it's not very a very specific formulation right because in that sense
  • fast_forward00:25:21 - and indeed anything that is dealing with an input output mapping would fall
  • fast_forward00:25:25 - in that category and automatically also the brain so for adaptive filter it
  • fast_forward00:25:29 - would be sort of roughly the same thing because i have an input output mapping
  • fast_forward00:25:32 - that i'm just sort of adjusting to some to some criteria so is it.
  • fast_forward00:25:38 - So at what point does it actually give us leverage when we look at this?
  • fast_forward00:25:42 - When it really adds understanding and at what point is it sort of covering a
  • fast_forward00:25:47 - little bit actually the important details of the questions we should ask?
  • fast_forward00:25:51 - Yeah, this is related with what I was saying before about the ontology.
  • fast_forward00:25:55 - When people say an adaptive filter, when people say a filter,
  • fast_forward00:25:58 - when people say a controller, some people will say that they are different things.
  • fast_forward00:26:03 - But if you go at the basics of the thing, they are the same kind of thing.
  • fast_forward00:26:07 - You were saying input in, output out, and that's all.
  • fast_forward00:26:10 - So you have a system that has inputs and has outputs, and that's all.
  • fast_forward00:26:13 - If you connect that system a particular way, it will be a controller.
  • fast_forward00:26:17 - If you connect that system in another way, it will be a filter.
  • fast_forward00:26:20 - That will depend on how you use the inputs and outputs of the system itself.
  • fast_forward00:26:25 - And that's a very big problem that is related to something that has appeared
  • fast_forward00:26:28 - in the course, that is the idea of function.
  • fast_forward00:26:30 - What is the function of such a module? It is a controller or is a or is a filter.
  • fast_forward00:26:36 - And there are two interpretations of this function thing.
  • fast_forward00:26:40 - One is the relation input-output that should be described by the mathematics,
  • fast_forward00:26:44 - describing the thing. And the other one is what it is for.
  • fast_forward00:26:48 - What is the function? It is playing in the system and it can be a controller
  • fast_forward00:26:52 - or can be a filter, can be whatever.
  • fast_forward00:26:54 - The same kind of mathematical description input-output can be applied for different
  • fast_forward00:26:59 - functions in different systems.
  • fast_forward00:27:01 - And the clarification of the The terminology is one of the major tasks that we need to solve here.
  • fast_forward00:27:08 - Because we can have an endless discussion about is this a filter or is it a controller?
  • fast_forward00:27:15 - Or from a theoretical point of view, this discussion is no sense.
  • fast_forward00:27:19 - Because those two kinds of systems are the same kind of thing at the end.
  • fast_forward00:27:25 - The only difference is how you use them.
  • fast_forward00:27:27 - That will depend on the use, not on the thing.
  • fast_forward00:27:30 - The thing is not a filter, it's not an intruder, is a dynamical system. That's all.
  • fast_forward00:27:36 - And many of the discussions we can find in current designs are related to not
  • fast_forward00:27:41 - understanding those basic, so to say, theoretical underpinnings of the system.
  • fast_forward00:27:46 - But now you also have been in projects where you have been really trying to
  • fast_forward00:27:50 - link your understanding of control engineering to, let's say, the biology.
  • fast_forward00:27:54 - Yeah. So what are the lessons from that experience? How successful have these attempts been?
  • fast_forward00:28:00 - Well, I'm not very happy and I'm very happy. So how can that be? be.
  • fast_forward00:28:05 - I'm not very happy because it's difficult to break the frontiers between communities.
  • fast_forward00:28:12 - So having biologists talk with roboticists by providing a common language to them is not working.
  • fast_forward00:28:19 - It's not working because the languages of the communities are being kept as they were in the past.
  • fast_forward00:28:25 - It's not easy to move or to share concepts at all.
  • fast_forward00:28:29 - So it's a happy situation.
  • fast_forward00:28:32 - On the other side, it is good to to see that people is hearing and then trying
  • fast_forward00:28:38 - to collaborate and hearing the others.
  • fast_forward00:28:41 - So I think that the first thing to do is to get this common language at the
  • fast_forward00:28:45 - end, and then common ontology.
  • fast_forward00:28:46 - That is, to my understanding, the basic ontology of physics,
  • fast_forward00:28:50 - and then on that, going up.
  • fast_forward00:28:52 - And I think something will help is that, for example, in neuroscience,
  • fast_forward00:28:56 - most of people doing neuroscience today are physicists by training.
  • fast_forward00:29:01 - So they start to talk about the mathematical basic concepts or theoretical concepts that are there.
  • fast_forward00:29:10 - So I hope that things will change in the future. But I think that the main problem
  • fast_forward00:29:14 - is that of the communities.
  • fast_forward00:29:15 - In my experience, the problem is the breaking of the communities because the
  • fast_forward00:29:19 - problem is so big that nobody has the possibility of addressing the solution.
  • fast_forward00:29:25 - Because someone has the tools, the other one has the global understanding,
  • fast_forward00:29:29 - another one has the data, the ground truth, the experimental conditions,
  • fast_forward00:29:34 - and we need to put all there into a single picture, otherwise it will solve the problem.
  • fast_forward00:29:39 - To work our way towards the finish line.
  • fast_forward00:29:44 - You have been going around in these fields for quite a bit and you also have
  • fast_forward00:29:49 - this very unique experience of interfacing two of these other disciplines and
  • fast_forward00:29:54 - really to try to solve problems very concretely.
  • fast_forward00:29:56 - So if there is this one law, the one Ricardo Sanz law that we should all adhere
  • fast_forward00:30:01 - to and try to understand the brain and how to then build this ideal brain that
  • fast_forward00:30:06 - can be the ultimate controller, what's Ricardo Sanz law that we should adhere to?
  • fast_forward00:30:13 - The most important thing that we should do is to try to be rigorous and formal
  • fast_forward00:30:17 - about the terms that we are using.
  • fast_forward00:30:19 - If we are able to do that, then we will see that the concepts that we are using
  • fast_forward00:30:24 - in different fields are the same.
  • fast_forward00:30:26 - And that will help transition from a pre-scientific domain, that is what we
  • fast_forward00:30:33 - are now, into a newborn scientific community.
  • fast_forward00:30:38 - So I would say that the first step to do is to be rigorous about the terms we
  • fast_forward00:30:43 - are using that is the very first step very good five years from now I'm gonna go up to Madrid and.
  • fast_forward00:30:53 - I'm gonna ask you look Ricardo there was this prediction you gave me and now
  • fast_forward00:30:57 - I want to know whether it came out or not what's this one prediction you're
  • fast_forward00:31:00 - willing to stick your neck out for today well I hope that in some years we will have a,
  • fast_forward00:31:08 - real implementation of the fundamental concept.
  • fast_forward00:31:11 - Working in a wide scale of different systems, from simple machines and simple
  • fast_forward00:31:17 - robots into all networks and real large industrial plants.
  • fast_forward00:31:22 - Having a single set of concepts implemented in such a variety of systems will
  • fast_forward00:31:28 - give credit to the vision, in a sense.
  • fast_forward00:31:32 - So what's the most advanced concept I can hold you to five years from now?
  • fast_forward00:31:36 - The most advanced concept, I think, is a system that is able to control itself
  • fast_forward00:31:40 - based on the model that engineers used to build it.
  • fast_forward00:31:44 - Okay, cool. And that's our key cornerstone for the work.
  • fast_forward00:31:51 - Great. I'll be back and I'll ask you about it. Ricardo Sanz,
  • fast_forward00:31:54 - thank you very much for this conversation. Thank you.

Be the first to leave a comment

Leave a comment

Your email address will not be published. Required fields are marked *

Convergent PozitivLinia Logo

Exploring the convergence of neuroscience, robotics, and AI through conversations with leading researchers since 2010.

A project of the Convergent Science Network Foundation.

© CSN Podcasts. Developed by IMCreativeWEBC

0%

Login to enjoy full advantages

Please login or subscribe to continue.

Go Premium!

Enjoy the full advantage of the premium access.

Stop following

Unfollow Cancel

Cancel subscription

Are you sure you want to cancel your subscription? You will lose your Premium access and stored playlists.

Go back Confirm cancellation