cover ton coolen

Ton Coolen on immune networks and neural networks

  • cover play_arrow

    PLAY EPISODE


cover ton coolen
Season 2018
Season 2018
Description arrow_drop_down

Description

What if the mathematics behind neural networks could unlock the secrets of the immune system? Physicist Ton Coolen reveals how techniques from statistical mechanics, originally developed for obscure magnetic materials, now expose deep structural parallels between how brains store memories and how immune systems learn to fight disease. Subscribe for more from the Convergent Science Network podcast series. Ton Coolen joins Paul Verschure and Tony Prescott to explain how a collaboration with Italian researchers led him to apply finite connectivity analysis, a mathematical framework only available since around 2000, to models of immune network function. The resulting models map directly onto Hopfield-type attractor networks from neural network theory, with cytokine signaling playing the role of synaptic connections and B-cell receptor evolution functioning as a rewiring mechanism. The conversation traces how applied problems in biology have driven fundamental advances in theoretical physics, inverting the usual relationship between basic and applied science. The discussion explores what the immune system can teach neuroscience about memory and learning. Unlike neural networks where few patterns with many bits are stored, immune networks store many patterns with few bits each , a regime that demands entirely different mathematical treatment. Coolen argues that biological heterogeneity, shaped by evolution rather than randomness, represents a fundamental challenge that standard physical methods cannot handle, pointing toward a new class of problems at the boundary of physics and biology. Key topics include the mathematical parallels between Hopfield networks and immune models, why equilibrium statistical mechanics fails for living systems, how hypermutation and selection function as a learning algorithm, the unsolved problem of immune memory, and the tantalizing possibility that the nervous system anticipates and regulates immune responses through brain-immune coupling. Part of the Convergent Science Network podcast series from the BCBT Summer School.

Tagged as:

About the author call_made

CSN Podcasts

Both the triumphs of humanity and its most evil deeds have resulted from collaboration. In a time where humanity is required to aspire to the former and minimize the latter, the question arises of how collaboration arises and why it fails. Surprisingly, this phenomenon, so central to who we are, is not well understood. Hence, a collaborative effort is required to understand collaboration in its full biological, psychological, sociological, cultural, and economic complexity and to translate this understanding into operational impact. This series of podcasts is one step toward achieving these complementary goals. The Collaboration Podcast presents interviews with people who are central orchestrators of collaboration in various domains including business, government, science, art, health, sustainability, and the military. The discussions were conducted by Prof. Dr. Paul F.M.J. Verschure and members of the Program Advisory Committee of the Ernst Strungmann Forum on Collaboration (https://www.esforum.de/forums/ESF32_Collaboration.html) during 2021 and had the goal to sketch a map of opportunities, challenges, and obstacles in human collaboration. The forum took place in May 2022, and now we would like to share this series of interviews with a broader audience. The full report of the Forum will be published in 2023 by MIT Press. The podcast was produced by the Convergent Science Network (https://www.convergentsciencenetwork.org/). Context: The stability of social systems depends critically on realizing sustainable methods of “collaboration,” yet how and by which means collaboration is achieved is not clearly understood; neither are the conditions or processes that lead to its breakdown or failure. Collaboration can be understood as cooperation between agents toward mutually constructed goals. Part of the reason for our lack of understanding is that the phenomenon of collaboration is, by nature, a highly multidisciplinary problem, and effective research into its complexities has been difficult to achieve across the broad range of scientific and technical disciplines involved. The need for a fundamental understanding of collaboration, however, has become increasingly important. Not only does humankind demand answers as it attempts to address critical challenges at multiple scales (e.g., climate change, migration, enhanced automation, social and economic inequality), but ever-increasing technological and economic means of interconnecting people and societies are disrupting long-established, familiar patterns of how we interact. Radical technological changes that are ongoing have the potential to reshape collaboration in ways that are currently hard to predict or influence (e.g., by altering configurations in interaction, information creation, and modes of communication). On one hand, such changes could disrupt hitherto stable forms of collaboration by affecting critical communication channels and traditional roles, as can be observed in the rapidly changing patterns in governance, commerce, and social interaction. Conversely, technology could lead to the emergence of novel, successful forms of collaboration that deviate from traditional “hierarchical” architectures. Evidence of this can be seen in areas as diverse as highly automated manufacturing plants, the open science movement, collaborative software repositories, user-centered services, and the sharing of economy-based modes of organization. Without a fundamental understanding of the mechanisms, processes, and boundary conditions of collaboration, it is not possible to evaluate or predict which of these possible scenarios are sustainable or even plausible. The Forum “How Collaboration Arises and Why it Fails” (May 8–13, 2022, Location: Frankfurt am Main, Germany) Chairs: Andreas Roepstorff and Paul Verschure Program Advisory Committee: Jenna Bednar, Julia R. Lupp, Bhavani R. Rao , Andreas Roepstorff, Ferdinand von Siemens, and Paul Verschure

More posts

Timestamp

  • fast_forward00:00:03 - This is the Convergent Science Network podcast. Leading researchers in the domain
  • fast_forward00:00:10 - of neuroscience, brain theory and technology are interviewed by Paul Verschoor and Tony Prescott.
  • fast_forward00:00:21 - This is Paul Verschoor together with Tony Prescott for the Convergent Science
  • fast_forward00:00:27 - Network podcast. Here's the 10th edition of the Barcelona Coalition Brain and
  • fast_forward00:00:31 - Technology Summer School. And we're here with Ton Koolen.
  • fast_forward00:00:36 - And Ton, in your talk this morning, you were dealing with this,
  • fast_forward00:00:42 - the analogy or the similarities you might find between neural networks and immune networks. Yeah.
  • fast_forward00:00:51 - So how did you come to study or to raise that specific question?
  • fast_forward00:00:57 - What kind of mileage do you think we can get in addressing it?
  • fast_forward00:01:01 - Yeah, how did I come to it? It was really a case of being asked,
  • fast_forward00:01:06 - being lucky to be asked by a group of Italian researchers who had recently developed
  • fast_forward00:01:11 - some really nice new models.
  • fast_forward00:01:13 - And in the development of these models, they find that at some point,
  • fast_forward00:01:18 - solving these models required mathematical technology which had only been in
  • fast_forward00:01:24 - the public domain for a couple of years and they were not having the experience
  • fast_forward00:01:29 - to do this calculation and I happened to have been working on similar problems.
  • fast_forward00:01:35 - And in addition, we knew each other because one of these Italians had done a
  • fast_forward00:01:39 - PhD in London. So one thing led to another, and I was asked to join.
  • fast_forward00:01:43 - And I really enjoyed that. It was really very interesting. The model was interesting,
  • fast_forward00:01:47 - and it allowed me to do research in a field that was really very close mathematically
  • fast_forward00:01:55 - to the borderline of what was then the state of the art.
  • fast_forward00:01:59 - So this problem had everything you would wish for in a problem.
  • fast_forward00:02:02 - It has clear relevance, understanding the immune system has clear medical relevance.
  • fast_forward00:02:08 - And it also was very interesting scientifically as someone who likes to do a
  • fast_forward00:02:14 - calculation, to be involved in those calculations.
  • fast_forward00:02:18 - So what were the specific mathematical techniques that they needed?
  • fast_forward00:02:23 - In the analysis of systems with very large numbers of variables,
  • fast_forward00:02:27 - the traditional methods that were being used until around 2000.
  • fast_forward00:02:33 - Work only when the number of components that each element interacts with is
  • fast_forward00:02:39 - either very large or very small.
  • fast_forward00:02:41 - But in the intermediate regime where you have components that interact with
  • fast_forward00:02:44 - a finite number of other parts, there were no methods available.
  • fast_forward00:02:49 - And this is called finite connectivity analysis and there are different mathematical
  • fast_forward00:02:52 - medical tricks for dealing with it but they'd only been developed in this domain
  • fast_forward00:02:56 - around 2000 by various people including mainly Rémy Monasson in France Giorgio Parisi in Rome.
  • fast_forward00:03:06 - And this turned out to be needed to solve these particular immune models.
  • fast_forward00:03:10 - And these techniques were already developed looking at specific natural phenomena,
  • fast_forward00:03:15 - or these were being developed within theoretical physics or applied mathematics
  • fast_forward00:03:20 - just as a technique that people thought was important?
  • fast_forward00:03:24 - Well, these methods tend to come from a community of solid-state physicists,
  • fast_forward00:03:31 - theoretical physicists, is, but it is specifically the sub-part of that community
  • fast_forward00:03:35 - that look at interdisciplinary problems.
  • fast_forward00:03:37 - So I think the motivation for looking at finite connectivity problems might
  • fast_forward00:03:41 - have even come from optimization problems in computer science that were being
  • fast_forward00:03:45 - analyzed by methods from theoretical physics.
  • fast_forward00:03:49 - At least the names that come to mind at this moment are typical type of people
  • fast_forward00:03:54 - who've been working on optimization problems.
  • fast_forward00:03:56 - They convert these optimization problems into theoretical physics problems and
  • fast_forward00:04:00 - then use the technology.
  • fast_forward00:04:01 - What I find instructive about that is that what you see here is how applied
  • fast_forward00:04:06 - problems are driving the basic science. Oh, yes.
  • fast_forward00:04:09 - Right? Which is actually… That's very much so. It's counterintuitive.
  • fast_forward00:04:12 - Usually, there's this idea like, okay, the basic science happens because we
  • fast_forward00:04:15 - have this genius sitting somewhere in a dungeon somewhere, and then it comes
  • fast_forward00:04:20 - out and becomes applied. But here, you see it exactly the other way around.
  • fast_forward00:04:22 - Yeah. I mean, this has been going on for about 25, 30 years at least in that community.
  • fast_forward00:04:28 - Initially, physicists started looking at heterogeneous systems in the context
  • fast_forward00:04:33 - of some obscure magnetic materials that later turned out to have no use whatsoever
  • fast_forward00:04:38 - anywhere, and were called spin glasses.
  • fast_forward00:04:41 - Since then, the main application domains of the mathematical methods that came
  • fast_forward00:04:46 - from these obscure materials have been ranging from computer science to economics
  • fast_forward00:04:52 - to biology to medicine to statistics.
  • fast_forward00:04:56 - And now there is a very fruitful flowing back and forth of ideas and problems
  • fast_forward00:05:02 - between theoretical physicists who are only theoretical physicists by virtue
  • fast_forward00:05:07 - of the salary they draw from the physics department.
  • fast_forward00:05:10 - But they work largely on non-physical problems.
  • fast_forward00:05:14 - So this has gone on for quite some time. So one thing I found really remarkable
  • fast_forward00:05:19 - in the model you presented from these people in Rome on the immune system in
  • fast_forward00:05:23 - this case is that they actually stumbled, if you want,
  • fast_forward00:05:27 - into a description of this immune system in terms of an energy landscape,
  • fast_forward00:05:33 - if you want, or a dynamical landscape that you could call energy.
  • fast_forward00:05:36 - That was identical to what people like John Hopfield have proposed now 30 years
  • fast_forward00:05:43 - ago as effective macroscopic descriptions of the spin glass models initially
  • fast_forward00:05:49 - and then of neural networks.
  • fast_forward00:05:51 - So is this convergence? Was this happenstance, luck, bias because of the techniques
  • fast_forward00:05:57 - people had? How did that come together and why is it meaningful?
  • fast_forward00:06:01 - Well, I think it's a mixture of many things. First of all, you have to have
  • fast_forward00:06:05 - seen certain domains to recognize certain approaches or models or methods for what they are.
  • fast_forward00:06:12 - So it demands that you are someone who is willing and able to look over the
  • fast_forward00:06:18 - fence away from their own discipline.
  • fast_forward00:06:20 - And the people who did this in Rome had been involved in various projects related
  • fast_forward00:06:24 - from physical systems to neural networks and so on.
  • fast_forward00:06:28 - So while doing the calculations, while building the models, at some point you
  • fast_forward00:06:33 - recognize that models are developing into a certain form that resembles something else.
  • fast_forward00:06:38 - But you'll only recognize it if you've actually been reading papers or preferably
  • fast_forward00:06:43 - been carrying out research in these other disciplines.
  • fast_forward00:06:46 - So breadth helps you. And also the other thing you mentioned is mathematical technology.
  • fast_forward00:06:51 - Mathematical technology constrains what you will be writing down in the first
  • fast_forward00:06:55 - place because Because you know.
  • fast_forward00:06:58 - Only certain types of models will be solvable. So you steer away from constructions
  • fast_forward00:07:03 - that you know will lead you into the morass anyway.
  • fast_forward00:07:05 - So all these things work together.
  • fast_forward00:07:09 - From your talk, I understood that both the neural networks and the immune networks
  • fast_forward00:07:15 - have this beautiful abstract description in terms of statistical mechanics,
  • fast_forward00:07:21 - which you've explored in the talk.
  • fast_forward00:07:22 - But I'm interested in, I think you also pointed out in the talk to some other
  • fast_forward00:07:27 - sort of direct analogies between immune networks and neural networks that could
  • fast_forward00:07:33 - provide further levels of interaction between those two fields.
  • fast_forward00:07:37 - Can you say a bit more about what you think those are?
  • fast_forward00:07:41 - Yeah, so the similarities are at different levels.
  • fast_forward00:07:45 - You have the mathematical level, it was already mentioned just now,
  • fast_forward00:07:48 - but also even if you didn't buy these particular models, if you thought,
  • fast_forward00:07:52 - well, this is just a choice by a couple of researchers and we could have written
  • fast_forward00:07:55 - models in very different ways, although qualitatively perhaps similar.
  • fast_forward00:08:00 - But at a higher level you see that you are looking at two systems which share
  • fast_forward00:08:05 - various fundamental ingredients.
  • fast_forward00:08:07 - A, they have large numbers of variables. B, these variables interact with each
  • fast_forward00:08:12 - other in a heterogeneous manner.
  • fast_forward00:08:14 - And C, this system is not a static system.
  • fast_forward00:08:17 - The forces that all these objects exert onto each other are evolving over time
  • fast_forward00:08:23 - on the basis pieces of experience in response to the environment that they work in.
  • fast_forward00:08:29 - And these are ingredients that are exactly the same in neural networks and in the immune model.
  • fast_forward00:08:36 - There are also differences, that's very clear.
  • fast_forward00:08:38 - But the way you can exploit these similarities is first in terms of intuition.
  • fast_forward00:08:46 - If you have, as is the case here, one field that has already been studied for
  • fast_forward00:08:50 - decades, like neural networks,
  • fast_forward00:08:51 - you develop a pretty good intuition for what to expect, pretty good intuition
  • fast_forward00:08:57 - for what would happen if you added certain ingredients or took certain ingredients away.
  • fast_forward00:09:02 - And that intuition is probably the most powerful thing to carry over to other
  • fast_forward00:09:07 - fields, because the details may differ.
  • fast_forward00:09:09 - But statistical mechanics teaches us that as soon as you have a very large system,
  • fast_forward00:09:14 - microscopic details of how exactly the model is built or what exactly you choose
  • fast_forward00:09:20 - for the types of variables or the parameters become less and less important.
  • fast_forward00:09:25 - So when at some microscopic level of description what is built into the model
  • fast_forward00:09:30 - is similar, then you also expect the behavior to be really similar.
  • fast_forward00:09:34 - And that helps you. That helps you in designing models. And there is never a
  • fast_forward00:09:38 - perfect model. Every model has a typical shelf life.
  • fast_forward00:09:42 - But it helps you in being effective in selecting models, in selecting directions
  • fast_forward00:09:48 - in which to to push your models.
  • fast_forward00:09:51 - Because at the end of the day, certain models are enormously powerful and helpful,
  • fast_forward00:09:57 - and many others will not be so for various reasons.
  • fast_forward00:10:01 - And this guides you, this analogy guides you. And I think for some of the neuroscientists
  • fast_forward00:10:06 - in the audience, it was also surprising to,
  • fast_forward00:10:09 - hear you talking about immune networks and the way that we talk about neural
  • fast_forward00:10:14 - networks as structures that have learning and memory, perhaps in quite a different way.
  • fast_forward00:10:20 - Way, and in some senses the mechanisms may be similar and other ones might be different.
  • fast_forward00:10:24 - But I guess that's one of the paths by which this research in immune networks
  • fast_forward00:10:29 - may come back to influence the neuroscience.
  • fast_forward00:10:32 - Yes, that is something that is not guaranteed to happen, but it is something
  • fast_forward00:10:36 - that I certainly think is worth thinking about.
  • fast_forward00:10:40 - I think in science generally it's very rare for one to make an interesting discovery,
  • fast_forward00:10:47 - for that discovery not to have
  • fast_forward00:10:48 - implications outside of the initial domain where the discovery was made.
  • fast_forward00:10:53 - And the main short-term lesson that I think neuroscience can take from this
  • fast_forward00:11:00 - immunological way of working is the possibility of having attractor neural networks
  • fast_forward00:11:06 - in which one is not recalling one attractor at a time,
  • fast_forward00:11:11 - an attractor which contains a very large number
  • fast_forward00:11:14 - of bits of information, one image at a time,
  • fast_forward00:11:17 - but where you see that there are multiple attractors which can be recalled independently
  • fast_forward00:11:22 - at the same time, even though you are looking at a system that is fully interacting,
  • fast_forward00:11:27 - not directly, but there are no subsystems that are disconnected from each other.
  • fast_forward00:11:32 - But if you then change parameters, all of a sudden you can get a transition
  • fast_forward00:11:35 - where all these subsystems are no longer acting independently.
  • fast_forward00:11:38 - Now, many of the things that you think about in neuroscience are somewhat reminiscent
  • fast_forward00:11:46 - of that, especially in making recognition on the basis of recognizing subparts of images.
  • fast_forward00:11:53 - Whether or not you combine them, or whether you want to create bigger objects
  • fast_forward00:11:59 - that are composed of things that you've seen before, but never in that combination.
  • fast_forward00:12:05 - May be relevant, may not be relevant, but certainly worth exploring.
  • fast_forward00:12:09 - These models operate in a slightly different regime in terms of parameters and
  • fast_forward00:12:13 - scales and numbers of bits of patterns and things like that.
  • fast_forward00:12:17 - One thing I find interesting here is also how you described the development
  • fast_forward00:12:20 - of these techniques. We start with spin glasses.
  • fast_forward00:12:24 - Those techniques made certain assumptions about symmetry and homogeneity of
  • fast_forward00:12:29 - the system. So those techniques would not scale to complex biological phenomena
  • fast_forward00:12:34 - like the immune system or nervous systems. For that, you need new techniques.
  • fast_forward00:12:39 - Would that actually help us to also distinguish living from non-living systems?
  • fast_forward00:12:43 - Because apparently, mathematically, there's a unique set of tools you need to
  • fast_forward00:12:48 - describe living systems as opposed to a pure physical system like a spin glass.
  • fast_forward00:12:53 - Would you go that far? Yeah, well, one thing I've learned over the years working
  • fast_forward00:12:58 - on biological systems is that when it comes to modeling, biology is much harder than physics.
  • fast_forward00:13:03 - And there are some really very clear reasons for that.
  • fast_forward00:13:07 - One very simple reason is that very often in biology, you look at systems in
  • fast_forward00:13:13 - which the number of elements involved is neither really small nor really big, the mesoscopic area.
  • fast_forward00:13:20 - And the problem with statistical methods from physics is they only work when
  • fast_forward00:13:24 - you have really large numbers.
  • fast_forward00:13:25 - In biology, that's not always the case. That's the first obvious thing.
  • fast_forward00:13:29 - The second thing is that many physical methods rely on the systems that study going to equilibrium.
  • fast_forward00:13:35 - Well, that's never happening in biology, at least not until we are permanently horizontal.
  • fast_forward00:13:41 - So that rules out about three quarters of the methodology of statistical physics.
  • fast_forward00:13:45 - But I think the main one, and that is also the most interesting one for a modeler
  • fast_forward00:13:51 - like like me, is that in biology, we always have heterogeneity.
  • fast_forward00:13:56 - We have heterogeneity in interactions between elements, heterogeneity in molecular structures.
  • fast_forward00:14:04 - And physics started out, at least statistical mechanics started out,
  • fast_forward00:14:08 - looking at systems in which every particle was interchangeable with every other particle.
  • fast_forward00:14:14 - Two electrons are exactly the same. The forces between particles would always be the same.
  • fast_forward00:14:20 - Physicists tended to look at pure systems, and they saw heterogeneous systems
  • fast_forward00:14:23 - as, say, impurities of pure systems, perturbations of pure systems.
  • fast_forward00:14:29 - Biology is not like that. Now, in the 70s, there was a whole area opened up
  • fast_forward00:14:35 - in statistical mechanics to look at heterogeneous systems. This is where the spin classes came in.
  • fast_forward00:14:40 - But one fundamental difference has remained, and that is, in using the physical
  • fast_forward00:14:45 - methods, you always have to be able to represent the heterogeneity as randomness.
  • fast_forward00:14:52 - You write a probability distribution for the particular parameters or forces
  • fast_forward00:14:57 - that are heterogeneous.
  • fast_forward00:14:59 - And that's exactly what is different in biology. Biological systems are heterogeneous,
  • fast_forward00:15:04 - but the heterogeneity is never random.
  • fast_forward00:15:07 - It has always been a consequence, at least partly, of evolution and selection.
  • fast_forward00:15:12 - And that rules out immediately all these methods. And the best example to think
  • fast_forward00:15:17 - about is probably the modeling of proteins.
  • fast_forward00:15:21 - Proteins are these large molecules assembled of little parts that are called amino acids.
  • fast_forward00:15:27 - And nature uses proteins as the building blocks of cells, as messengers.
  • fast_forward00:15:32 - It's the basic hardware of cellular biology.
  • fast_forward00:15:36 - If you look at randomly selected macromolecules of amino acids,
  • fast_forward00:15:42 - they behave completely differently from proteins.
  • fast_forward00:15:44 - That means that it is essential that, in order to understand how these biological
  • fast_forward00:15:50 - systems operate, you must study the evolution that gave rise to these structures.
  • fast_forward00:15:58 - But that is a very complicated thing, because it is a two-time-scale process
  • fast_forward00:16:02 - in which the processes on the slowest timescale influence the fast ones,
  • fast_forward00:16:06 - and the fast ones feed back to evolution.
  • fast_forward00:16:10 - You will never be able to write a heterogeneity in terms of a probability distribution.
  • fast_forward00:16:15 - That is really interesting because it means that here is a type of problem that
  • fast_forward00:16:21 - we cannot handle the way we would handle a heterogeneous physical system.
  • fast_forward00:16:25 - There are ways, there are openings, but it is a completely different field.
  • fast_forward00:16:30 - You mentioned this in your talk that you were continually discussing
  • fast_forward00:16:34 - with the biologists just about which aspects of the system they consider to
  • fast_forward00:16:38 - be important and in your models you have a noise term which i guess at the moment
  • fast_forward00:16:44 - captures a lot of that detail yeah and there may be aspects of that detail that
  • fast_forward00:16:48 - they think has to be pulled out and put into the model,
  • fast_forward00:16:52 - is is what you're saying that the mask gets hairier and hairier as you pull
  • fast_forward00:16:57 - more of those details out and put into the model or is is there some sort of
  • fast_forward00:17:02 - possibility of actually going into
  • fast_forward00:17:04 - that space and it's not going to become increasingly difficult.
  • fast_forward00:17:08 - There are going to be ways of making models that are more biologically rich,
  • fast_forward00:17:14 - sort of things that you can work with. You can still show interesting results.
  • fast_forward00:17:19 - I think there's no golden and perfect answer to this.
  • fast_forward00:17:25 - It's always the case that in the development of models, you have these three
  • fast_forward00:17:28 - distinct stages as I see them.
  • fast_forward00:17:30 - So stage one is where you have a problem from the real world and you have really
  • fast_forward00:17:34 - no idea how to model it in such a way that you can make progress.
  • fast_forward00:17:37 - Now, I think we are beyond that point in immunology now.
  • fast_forward00:17:41 - And the second stage is that there are some models that seem to make sense,
  • fast_forward00:17:46 - but they are really very simplified, extremely abstract, and one has sacrificed
  • fast_forward00:17:51 - a lot of the realism in order to get something that works.
  • fast_forward00:17:55 - That's where we are now. And the third stage is, as you say,
  • fast_forward00:17:58 - where you make these models more realistic.
  • fast_forward00:18:01 - And that means having a good intuition for knowing in which order to bring these ingredients in.
  • fast_forward00:18:07 - And there will be some ingredients that have a serious impact on the methodology
  • fast_forward00:18:12 - that you will need to still solve the models.
  • fast_forward00:18:15 - And there are some ingredients that are relatively easily incorporated.
  • fast_forward00:18:19 - To give an example in these immune models, the initial models of Adriano and Elena,
  • fast_forward00:18:25 - had the unwelcome element that a T-cell, a single T-clone, could be simultaneously
  • fast_forward00:18:31 - excitatory to some B-cell and inhibitory to another.
  • fast_forward00:18:35 - It's just like, and it's a nice analogy, the same thing was assumed in neural
  • fast_forward00:18:39 - network theory in the very beginning by Hopfield.
  • fast_forward00:18:43 - We know that the hardware is different. The hardware doesn't allow for that.
  • fast_forward00:18:47 - And in neural networks, we could remove this and we could still solve models. The same happened here.
  • fast_forward00:18:52 - So that happened to be a relatively easy ingredient to bring in.
  • fast_forward00:18:57 - There are other things that are going to be much harder. We can see that coming.
  • fast_forward00:19:00 - And that is when you include the evolution of antigen.
  • fast_forward00:19:03 - Because then you have to study in great detail the dynamical processes of hypermutation.
  • fast_forward00:19:09 - Mutation, because it is a rat race between the virus or the bacterium that has
  • fast_forward00:19:16 - come in and that will multiply, and the immune system trying to catch up.
  • fast_forward00:19:21 - So when you start working on those models, you can forget about equilibrium
  • fast_forward00:19:24 - studies, you can forget about energy functions, because these systems are no
  • fast_forward00:19:28 - longer going to equilibrium.
  • fast_forward00:19:29 - So you're going to move away from statistical mechanics to another modelling approach?
  • fast_forward00:19:33 - Well, one would move away from equilibrium statistical mechanics, which
  • fast_forward00:19:36 - is the part that has been used so far on these particular models
  • fast_forward00:19:39 - then you would use non-equilibrium statistical mechanics and
  • fast_forward00:19:42 - that is a slightly more specialist area
  • fast_forward00:19:46 - of statistical mechanics it is a small subset of the broader area most of statistical
  • fast_forward00:19:54 - mechanics but this non-equilibrium system mechanics has been developing by looking
  • fast_forward00:19:59 - at biology or it came from other domains of physics oh it came also from from
  • fast_forward00:20:03 - standard domains of physics,
  • fast_forward00:20:05 - because even if you know what the equilibrium behavior is, you might still be
  • fast_forward00:20:08 - interested in studying how the system would get there.
  • fast_forward00:20:11 - So they were developed already within the physics community,
  • fast_forward00:20:16 - but you just find that in biology, that will turn out to be your only toolbox,
  • fast_forward00:20:20 - because most biological systems don't go to equilibrium.
  • fast_forward00:20:24 - Okay, the analogy between neural systems, biological neural systems,
  • fast_forward00:20:30 - and immune immune systems is that they can learn and have memory, right?
  • fast_forward00:20:37 - So, in the nervous system, and there's also many talks on this,
  • fast_forward00:20:40 - the dominant view on that is that this memory has something to do with the connectivity
  • fast_forward00:20:45 - in the network and how it's regulated.
  • fast_forward00:20:48 - It might not be literally the
  • fast_forward00:20:49 - synapse, but it is the regulation of the synapse that's key for memory.
  • fast_forward00:20:53 - How should we think about memory now in this immune system? What's the substrate
  • fast_forward00:20:57 - of memory in the immune system?
  • fast_forward00:20:59 - Yeah, that has been a very tricky discussion in the biology community in the
  • fast_forward00:21:04 - sense that there are various competing views.
  • fast_forward00:21:08 - Some are fashionable in certain eras and then get ditched a couple of years
  • fast_forward00:21:13 - later, and that has happened in this community.
  • fast_forward00:21:15 - For instance, Jern's model of idiotic networks envisaged a mechanism of memory
  • fast_forward00:21:22 - that was very different from what most biologists now assume is the case.
  • fast_forward00:21:26 - Most biologists now assume that bee clones, after they've been activated,
  • fast_forward00:21:30 - after they've been producing antibodies, then go into a state where they become
  • fast_forward00:21:36 - so-called memory cells.
  • fast_forward00:21:37 - So somehow you want to retain and memorize the particular shapes of antibodies
  • fast_forward00:21:44 - that were very successful in attacking the antigen that came in.
  • fast_forward00:21:48 - So the thinking is that those B cells that produce those antibodies,
  • fast_forward00:21:51 - they convert into a state where they can live for 25 years. I find that intuitively slightly.
  • fast_forward00:22:01 - Not maybe unrealistic, but slightly far-fetched. Is there any empirical support for that idea?
  • fast_forward00:22:08 - Well, I'm not an immune biologist, and I'm sure there is some support.
  • fast_forward00:22:13 - But I would be surprised if that was a clear-cut and completely closed case.
  • fast_forward00:22:22 - I think there is still room for alternative. But that would then predict that
  • fast_forward00:22:26 - there is some reservoir of these memory B-cells that reside somewhere in the
  • fast_forward00:22:30 - body. Yes, that's the problem.
  • fast_forward00:22:31 - That's expanding, right? That's called ditty expanding. Well,
  • fast_forward00:22:33 - the thinking is they don't expand.
  • fast_forward00:22:35 - The thinking is that you have this group of B cells that have been activated,
  • fast_forward00:22:38 - and they go and sit somewhere, and they live 30 years, just in case at some
  • fast_forward00:22:44 - point that particular virus reappears.
  • fast_forward00:22:47 - I don't buy that. But I cannot argue that there is evidence to reject this.
  • fast_forward00:22:52 - It just seems a bit far-fetched to me. Okay, but it means for you,
  • fast_forward00:22:54 - this means the memory problem for immune systems is still not resolved.
  • fast_forward00:22:59 - If we don't actually really know?
  • fast_forward00:23:01 - I think there will be an explanation which is a bit more natural and elegant than the memory cell.
  • fast_forward00:23:07 - So your modeling approach, would that lend itself to building models that contest
  • fast_forward00:23:14 - the alternative hypothesis?
  • fast_forward00:23:16 - I think so. Yes, I think so. I think so. But at this moment,
  • fast_forward00:23:20 - we're not there yet because the models are relatively primitive.
  • fast_forward00:23:23 - I mean, compared to, if you look at, that's why I find the history of neural
  • fast_forward00:23:26 - networks here is so interesting because we've gone through many of those things.
  • fast_forward00:23:30 - And you can look back and see, yeah, it was very similar then.
  • fast_forward00:23:33 - Hopfield models were primitive, really very primitive. Lutz was wrong with them.
  • fast_forward00:23:39 - And the interesting thing was, at that time even, Hopfield models were still
  • fast_forward00:23:43 - solved using equilibrium statistical mechanics, and in these models,
  • fast_forward00:23:46 - one sacrificed biological realism in order to be able to do that.
  • fast_forward00:23:51 - And when you talked to people in those days, it was very often suggested,
  • fast_forward00:23:55 - well, we will never be able to do the dynamics, so this is why we do it that way.
  • fast_forward00:23:59 - Turns out to be not true. People are sometimes too pessimistic about what is
  • fast_forward00:24:03 - possible, and that is very dangerous, because you miss opportunity.
  • fast_forward00:24:08 - Turned out to be nonsense.
  • fast_forward00:24:09 - Doing the dynamics was, in retrospect, relatively easy, which was very good
  • fast_forward00:24:14 - because then we could remove all these biologically unrealistic model ingredients.
  • fast_forward00:24:18 - I expect the same to happen here.
  • fast_forward00:24:21 - But the other thing is that for memory, it's mysterious.
  • fast_forward00:24:26 - In some sense, we could argue memory is equally mysterious for the brain,
  • fast_forward00:24:31 - but there is consensus on what that substrate would be, more or less.
  • fast_forward00:24:36 - But now let's look at the learning component, right?
  • fast_forward00:24:38 - So for the immune system, you would have hypermutation and then selection upon
  • fast_forward00:24:44 - this set of hypermutated B cells as your core learning mechanism, right?
  • fast_forward00:24:51 - And now in the brain, yes, people like Jerry Edelman have proposed that similar
  • fast_forward00:24:57 - mechanisms would work in the brain to select synapses and in that way have specificity
  • fast_forward00:25:02 - of connectivity and learning.
  • fast_forward00:25:05 - But it's not the only way in which we can think about learning in the nervous system.
  • fast_forward00:25:09 - It might also just be the differential strengthening of synapses that allow
  • fast_forward00:25:15 - you, as in a Hopfield network, right, to sort of bias the network towards certain
  • fast_forward00:25:20 - responses away from other responses without necessarily selecting across a repertoire.
  • fast_forward00:25:24 - You remove certain possibilities completely. Completely.
  • fast_forward00:25:28 - So do you believe that selection is then the only mode of learning in the immune network?
  • fast_forward00:25:36 - Or is there actually more subtle than that?
  • fast_forward00:25:40 - It's an interesting question. I think that over the years we find that a lot
  • fast_forward00:25:45 - more is going on than we presently think.
  • fast_forward00:25:48 - I would also, for instance, expect that at some point there is some interface
  • fast_forward00:25:52 - between the nervous system and the immune system. And even today I had some
  • fast_forward00:25:57 - very nice discussions with some of the people here where these ideas came up too.
  • fast_forward00:26:02 - If the nervous system, for instance, would be able to predict or anticipate
  • fast_forward00:26:08 - certain invasions of antigen, then you can prepare the immune system.
  • fast_forward00:26:15 - I think there's a lot going to be found that we don't know present.
  • fast_forward00:26:20 - Well, in that sense, for instance, if you look at sepsis as an inflammatory
  • fast_forward00:26:25 - problem, it's extremely interesting to look at.
  • fast_forward00:26:28 - If you look at these broad organ systems that are affected in sepsis by inflammation,
  • fast_forward00:26:33 - it's a rather complex dynamics in which, let's say, the inflammatory load to
  • fast_forward00:26:39 - organs can be very specifically changing or transiently changing during such a disease process.
  • fast_forward00:26:47 - And actually, also many patients who suffer from sepsis, some of them might
  • fast_forward00:26:52 - appear in all standard biomarkers healthy.
  • fast_forward00:26:56 - They're sent home, and a few months later, they still die, right?
  • fast_forward00:26:59 - So that would also suggest there's much more active regulation going on of the global response,
  • fast_forward00:27:05 - response yeah with the possible anticipatory component to
  • fast_forward00:27:08 - the immune response that might or
  • fast_forward00:27:11 - might not reside in the immune system itself so
  • fast_forward00:27:15 - how do you see that coupling exactly between the brain and
  • fast_forward00:27:18 - the immune system do you expect the brain to really model let's say the health
  • fast_forward00:27:21 - status of organ systems and then bias in some way the immune response and if
  • fast_forward00:27:28 - so what's the substrate of that you have to think really about these things
  • fast_forward00:27:32 - in in terms of the coding that the different system use.
  • fast_forward00:27:35 - Brains think in terms of electric potentials and the immune system thinks in terms of chemistry.
  • fast_forward00:27:41 - It communicates by docking and sending cytokines and things like that.
  • fast_forward00:27:47 - And there are certain constraints if you think about possible mechanisms and possible interfaces.
  • fast_forward00:27:53 - For instance, immune cells cannot pass the blood-brain barrier,
  • fast_forward00:27:57 - so they cannot directly enter the brain.
  • fast_forward00:28:00 - However, various molecules that they produce could.
  • fast_forward00:28:04 - We also know that in the brain, as I understand it, there is an interface between
  • fast_forward00:28:10 - communication by spikes and chemistry.
  • fast_forward00:28:14 - The release of all kinds of chemicals happens also in the brain.
  • fast_forward00:28:19 - So I don't think it's far-fetched. I don't think it's far-fetched.
  • fast_forward00:28:22 - There are all kinds of opportunities, but we will never be able to find out
  • fast_forward00:28:25 - until we have at least a satisfactory description of the immune system itself.
  • fast_forward00:28:32 - There's so much we still don't know about the immune system.
  • fast_forward00:28:35 - Going to an interface now is premature.
  • fast_forward00:28:39 - But if I were a betting person, I would put my money on there being one.
  • fast_forward00:28:43 - It just seems to be giving you an enormous evolutionary advantage.
  • fast_forward00:28:49 - And therefore I think it will be there.
  • fast_forward00:28:53 - So the algorithms that are involved in learning, presumably,
  • fast_forward00:28:56 - I mean, you didn't talk about them in any detail in the talk,
  • fast_forward00:28:59 - but there's something analogous happening there to natural selection,
  • fast_forward00:29:04 - and is it involving both the B-cell and the T-cell populations in some interesting
  • fast_forward00:29:10 - ways that alters the population dynamics so as to give you some form of optimal search?
  • fast_forward00:29:15 - Yeah, so it's interesting if you look at the literature on immunology that most
  • fast_forward00:29:21 - studies that have been carried out tend to have been carried out on B-cells
  • fast_forward00:29:26 - and on evolution of receptors of B-cells.
  • fast_forward00:29:29 - A lot is known about them, much less is known about T-cells.
  • fast_forward00:29:32 - And partially this is due to the fact that T-cells were only discovered later.
  • fast_forward00:29:37 - The regulatory T-cells were only discovered in the 1990s.
  • fast_forward00:29:41 - So there is a good reason why we know much more about B-cells than we know about T-cells.
  • fast_forward00:29:46 - When you think about learning, and this is This is again where I find these
  • fast_forward00:29:50 - models very helpful, the models
  • fast_forward00:29:53 - that create this mapping between neural network and the immune system.
  • fast_forward00:29:56 - If you look at those models, you see where is the learning residing,
  • fast_forward00:30:01 - what is the substrate for the learning? It is in the cytokine variable.
  • fast_forward00:30:06 - So if you think about what they do, the cytokine variables simply tell you exactly
  • fast_forward00:30:11 - which B clone talks to which T clone, and within these models,
  • fast_forward00:30:15 - it is also in the other direction.
  • fast_forward00:30:18 - And that is exactly what is being changed as a result of hypermutation and selection,
  • fast_forward00:30:23 - when we go through that process.
  • fast_forward00:30:26 - Because if we change the receptor properties of the Bs and the Ts,
  • fast_forward00:30:30 - then we also change who talks to whom.
  • fast_forward00:30:32 - So we're effectively changing the patterns that, in the language of neural networks,
  • fast_forward00:30:38 - would be the patterns stored in a Hopfian model. So we're rewiring the network?
  • fast_forward00:30:41 - We are rewiring the network. At the same time as we're changing the specificity of the cells? else.
  • fast_forward00:30:47 - So in a way, the network is a consequence of the information being stored.
  • fast_forward00:30:51 - And since we changed the information being stored, we're effectively rewiring the network.
  • fast_forward00:30:56 - So we could see the cytokine as a virtual synapse now that connects certain... No, not the synapse.
  • fast_forward00:31:01 - The synapse is expressed in terms of the cytokines.
  • fast_forward00:31:04 - So the cytokines would be... They're the connection.
  • fast_forward00:31:07 - They form the connection between the B's and the T's. They are the equivalent
  • fast_forward00:31:10 - thing of the bits in the patterns of the Hopfield model.
  • fast_forward00:31:12 - Exactly. And they build the synapse.
  • fast_forward00:31:15 - So what So what's now the capacity of then such a memory?
  • fast_forward00:31:21 - If the cytokines would then, in this, the analogy we're now pursuing,
  • fast_forward00:31:24 - the cytokines are in that your connectivity matrix, right?
  • fast_forward00:31:28 - So given the scale of the immune system and the number of elements,
  • fast_forward00:31:32 - what kind of capacity would we then look at in these cytokines?
  • fast_forward00:31:35 - And what kind of learning rates would you expect or the rate of change that you can support?
  • fast_forward00:31:40 - Yeah, this we can now answer as a result of the models. And the answer is, in fact, very trivial.
  • fast_forward00:31:45 - This is always when you have understood something you
  • fast_forward00:31:48 - know that you could have thought about it before but in standard
  • fast_forward00:31:52 - hopfield models what you know is that you can store if you
  • fast_forward00:31:55 - want to have perfect recall only relatively small number of patterns but each
  • fast_forward00:31:59 - of these patterns has an extensive number of bits here is the other way around
  • fast_forward00:32:02 - you you can store and recall an extensive number of patterns proportions to
  • fast_forward00:32:07 - a number of b clones but each of them will have only a finite that number of
  • fast_forward00:32:11 - bits of information in that.
  • fast_forward00:32:14 - So the capacity is in the order of n, and precise numbers you find by calculating
  • fast_forward00:32:20 - the phase diagrams, which has been possible now.
  • fast_forward00:32:23 - You can compute these things. Right. But then another kind of connectivity matrix
  • fast_forward00:32:29 - might be defined by the T helper cells, for instance, that would modulate the
  • fast_forward00:32:33 - interaction between the Ts and the Bs.
  • fast_forward00:32:36 - For the nervous.
  • fast_forward00:32:41 - Just focus the whole learning problem towards the synapses keeps things relatively
  • fast_forward00:32:46 - controllable, understandable.
  • fast_forward00:32:48 - But maybe in the immune system, there might be multiple levels at which this
  • fast_forward00:32:52 - plasticity now gets expressed or multiple substrates or not.
  • fast_forward00:32:55 - Are you happy with just focusing it fully for now at this cytokine modulation?
  • fast_forward00:33:02 - I think for now, this is what is being done simply because we are already very
  • fast_forward00:33:07 - happy with what we can achieve with this model.
  • fast_forward00:33:09 - But in the future we
  • fast_forward00:33:13 - will want to study in more detail the actual process
  • fast_forward00:33:16 - of what you would say learning in neural networks what we
  • fast_forward00:33:18 - would call hypermutation and thereby the changing
  • fast_forward00:33:21 - of all these cytokine variables in the immune system and
  • fast_forward00:33:25 - then we will undoubtedly find that things get more
  • fast_forward00:33:28 - complicated we will have to look at different types
  • fast_forward00:33:31 - of p cells different types of t cells and all those
  • fast_forward00:33:33 - things so you will inevitably get a
  • fast_forward00:33:36 - next layer of complexity simply because you move away
  • fast_forward00:33:39 - from what is currently one of the most primitive starting
  • fast_forward00:33:42 - points imaginable so that
  • fast_forward00:33:45 - will happen that will happen but um i
  • fast_forward00:33:49 - think on top of that we will find and that is going to be done by experimentalists
  • fast_forward00:33:55 - we will probably find new players that we haven't even been thinking of not
  • fast_forward00:33:59 - just b cells or t cells but other cells that do things we might find interactions
  • fast_forward00:34:04 - mechanisms Mechanisms that we don't know now.
  • fast_forward00:34:08 - But now think about the signaling. So in the nervous system,
  • fast_forward00:34:11 - in the end, it's electrical, as we discussed.
  • fast_forward00:34:14 - In the immune system, it could be through the metabolites, molecules that are
  • fast_forward00:34:19 - going through the circulatory systems.
  • fast_forward00:34:23 - What are the forms of signaling that now the immune system has available that
  • fast_forward00:34:28 - would make it different from the kind of signaling you would see in a nervous system?
  • fast_forward00:34:34 - Um let me think about that one so in the nervous system i think there's one,
  • fast_forward00:34:42 - complication that maybe is not so much a complication in the immune system and
  • fast_forward00:34:46 - that is in the nervous system we also have topology we have wiring and at least
  • fast_forward00:34:52 - in the immune system we don't have that because these cells just move around
  • fast_forward00:34:55 - in the blood the assumption is that
  • fast_forward00:34:57 - there are no barriers that single out certain B-cells as being having given
  • fast_forward00:35:02 - access to certain areas as opposed to others.
  • fast_forward00:35:05 - There are some aspects only which are that the hyposomatic mutation takes place
  • fast_forward00:35:11 - in certain parts of the body.
  • fast_forward00:35:13 - But apart from that, we have no topology and that makes your life a lot easier.
  • fast_forward00:35:17 - So in terms of the signaling, there is the extra constraint in neural network
  • fast_forward00:35:22 - systems that you have to look carefully at the wiring.
  • fast_forward00:35:26 - These things cannot float around, neurons cannot float around and establish
  • fast_forward00:35:30 - and interact with whoever they want.
  • fast_forward00:35:33 - And space is a tricky thing, and especially if you think about mathematical methods from physics.
  • fast_forward00:35:40 - In one area, and that is space, they are remarkably unsuccessful.
  • fast_forward00:35:45 - And that is when you look, for instance, at a very simple cubic lattice of molecules
  • fast_forward00:35:50 - that only talk to their neighbors, that is still an unsolved problem.
  • fast_forward00:35:55 - Even if these variables are just binary, they can be on or off,
  • fast_forward00:35:58 - and they talk to their neighbors in a three-dimensional arrangement,
  • fast_forward00:36:01 - it's an unsolved problem so space brings an
  • fast_forward00:36:04 - extra dimension that makes problems really
  • fast_forward00:36:07 - significantly harder and that is
  • fast_forward00:36:10 - absent in the immune system so that is a difference which actually works
  • fast_forward00:36:13 - to the advantage of modeling in the immune system and
  • fast_forward00:36:17 - in brains we can get away with it we have been able to
  • fast_forward00:36:20 - get away with it for a long time because at least neurons
  • fast_forward00:36:23 - talk to a great number of other neurons if neurons
  • fast_forward00:36:26 - had only been talking to neighboring neurons we would
  • fast_forward00:36:29 - be in real trouble and thank god we're
  • fast_forward00:36:32 - not right so then
  • fast_forward00:36:35 - but how can we now test these models because um also look at the case of neuroscience
  • fast_forward00:36:41 - right there was a lot of enthusiasm for the hopfield kind of networks because
  • fast_forward00:36:46 - people had this this hope like okay now it becomes mathematically tractable
  • fast_forward00:36:50 - neuroscience can become like physics we can have analytic solutions god knows what,
  • fast_forward00:36:56 - but that was 85 more or less, right? So we're talking about 30 years.
  • fast_forward00:37:01 - And has that impact been so overwhelming? Are we really now radically,
  • fast_forward00:37:05 - have we radically changed the way which we think about the brain?
  • fast_forward00:37:08 - Not so sure about that, right? So on the other hand, it has been very,
  • fast_forward00:37:13 - as a heuristic, it has helped people to pose certain questions.
  • fast_forward00:37:17 - So now in something you can argue for the immune system, you stand now at the
  • fast_forward00:37:20 - beginning of an equivalent process, right?
  • fast_forward00:37:23 - Where it is this fundamental insight using these techniques from theoretical
  • fast_forward00:37:28 - physics has helped you to get the handle on a system that you don't understand.
  • fast_forward00:37:33 - But what's the trajectory that you now envision? What are the next steps to
  • fast_forward00:37:37 - also make it empirically testable?
  • fast_forward00:37:39 - Yeah, so we are very aware of the danger of the same thing happening in immune
  • fast_forward00:37:46 - modeling and what happened in neural networks.
  • fast_forward00:37:48 - And we can come back to what I think are the reasons for that because some of
  • fast_forward00:37:52 - them are not scientific but psychological.
  • fast_forward00:37:54 - Now at the moment what we're trying to do is we're trying to,
  • fast_forward00:37:58 - in a particular multidisciplinary project that we've started in the UK involving
  • fast_forward00:38:03 - not just people like me who do models but also bioinformaticians and experimental immunologists,
  • fast_forward00:38:09 - we're trying to find indeed models in which we can test the predictions of the
  • fast_forward00:38:16 - model using experiments.
  • fast_forward00:38:18 - So we would do this in the form of having a model in which an antigen comes
  • fast_forward00:38:24 - in that has never been seen before, and then predicting the dynamical response, first and secondary.
  • fast_forward00:38:33 - And see to what extent this is similar to what we get in what is called unchallenged
  • fast_forward00:38:38 - individuals in an immunological response.
  • fast_forward00:38:43 - And then try to build the models from there, test every time that we can make a prediction.
  • fast_forward00:38:49 - At the end of the day, the only real test of whether you have extracted knowledge
  • fast_forward00:38:53 - in any area of modeling is whether you can predict unseen situations with the
  • fast_forward00:38:58 - model and and then test them. If you can't, you haven't learned anything.
  • fast_forward00:39:02 - If you can, you have. And what's the status of the measuring techniques that
  • fast_forward00:39:07 - the immunologists have to detect this sort of...
  • fast_forward00:39:11 - I mean, for your models, you're looking for quite a lot of detail in terms of
  • fast_forward00:39:15 - the temporal and spatial dynamics, or certainly temporal dynamics of what's unfolding.
  • fast_forward00:39:20 - So this is really difficult, in the sense that it is very hard,
  • fast_forward00:39:24 - and we do struggle with that, to find people with
  • fast_forward00:39:28 - the right type of data because you're looking for individuals
  • fast_forward00:39:31 - who are who've never
  • fast_forward00:39:35 - experienced challenged by a particular antigen before so
  • fast_forward00:39:38 - you have to go for really rare diseases you cannot go for
  • fast_forward00:39:41 - flus or things like that because there is almost no one who will have a primary
  • fast_forward00:39:45 - response to that so you have to go for very funny things like yellow fever and
  • fast_forward00:39:50 - And our collaborators are trying to get these people from populations of individuals
  • fast_forward00:39:56 - who are about to make a trip abroad to an area where they've never been,
  • fast_forward00:40:00 - and who are then being inoculated beforehand.
  • fast_forward00:40:03 - And these are the ones that we try to get the data from, because then they get
  • fast_forward00:40:06 - a response to this inoculation that should be very similar to primary response.
  • fast_forward00:40:12 - Response, because then at least we can do tests before having modeled the secondary
  • fast_forward00:40:18 - response, before having modeled the consequences of the hypermutation.
  • fast_forward00:40:22 - But why no animal models?
  • fast_forward00:40:25 - Well, why no animal models? It just happens to be that the collaborators we
  • fast_forward00:40:30 - have don't work with animals.
  • fast_forward00:40:32 - They work with people, because they come from a clinical environment where it's
  • fast_forward00:40:36 - human immunology they work on. And can you do some of this in a sort of petri
  • fast_forward00:40:42 - dish where you just put some antigen into a sample of blood?
  • fast_forward00:40:47 - I've been told that this is really hard. Right, okay.
  • fast_forward00:40:50 - That it is one of the tricky aspects of experimental immunology.
  • fast_forward00:40:56 - Look, here we have all these Brits, they're going on holidays to faraway exotic
  • fast_forward00:41:01 - places where they're going to run the risk to catch all sorts of strange diseases
  • fast_forward00:41:07 - and and they get inoculated, and imagine this whole thing now works,
  • fast_forward00:41:11 - what's the specific prediction you would like to see tested first?
  • fast_forward00:41:15 - We would like to see the evolution in time of the distribution of clone sizes.
  • fast_forward00:41:21 - So if immune system is not challenged, so let's say before these people get
  • fast_forward00:41:26 - the inoculation, the theory can do a prediction already on the distribution
  • fast_forward00:41:29 - of clone sizes in normal state.
  • fast_forward00:41:32 - Then comes a challenge. This challenge has a consequence, and you see this in
  • fast_forward00:41:37 - the distribution of clone sizes.
  • fast_forward00:41:40 - And the distribution of clone sizes is a typical thing that can easily be measured.
  • fast_forward00:41:44 - It but this is the first test it doesn't mean that this is the perfect test
  • fast_forward00:41:47 - just means this is a very clear transparent test and if it doesn't even pass
  • fast_forward00:41:53 - that test then you know you have missed something important in your model you
  • fast_forward00:41:56 - go back to the drawing board and you think what that could be,
  • fast_forward00:41:59 - but this is relatively accessible data and this is what we're focusing on so
  • fast_forward00:42:04 - it's a distribution of the sizes of the clones but it's like fine-tuning the
  • fast_forward00:42:08 - model is the model falsifiable
  • fast_forward00:42:10 - so what would be the observation in this experiment that
  • fast_forward00:42:13 - would completely kill the model that you know
  • fast_forward00:42:15 - proposed with your colleagues well i don't
  • fast_forward00:42:18 - know what the definition of completely kill is because you can always go back
  • fast_forward00:42:21 - to the drawing board well that's the question right yeah exactly so but it this
  • fast_forward00:42:25 - is a falsifiable model in the sense that you could find a clone size distribution
  • fast_forward00:42:29 - which is qualitatively very different from what the model predicts and it's
  • fast_forward00:42:34 - not something that you could fix with matching parameters and things like that because.
  • fast_forward00:42:39 - So it could work it could not work it is a falsifiable thing but but.
  • fast_forward00:42:45 - The tricky area is the area in the middle, where it works to some extent.
  • fast_forward00:42:50 - It is not complete nonsense that comes out. It is also not perfect.
  • fast_forward00:42:53 - This is the general situation we find ourselves in, in modeling biological systems.
  • fast_forward00:42:58 - So then you try to systematically think about the ingredients without just putting
  • fast_forward00:43:05 - your modeling hat on, but also in talking to your experimental colleagues,
  • fast_forward00:43:09 - because things that you do have to make sense, see what could be responsible
  • fast_forward00:43:13 - for the deviation between the immune experiment.
  • fast_forward00:43:16 - This is the tedious cycle that we have to go through, whether in physics,
  • fast_forward00:43:19 - biology, any discipline.
  • fast_forward00:43:21 - That's correct. But now, so here we have this model.
  • fast_forward00:43:27 - Right now, we're in a phase of the science that we actually try to understand
  • fast_forward00:43:31 - how this system works at all, right? This is what you were clearing in your talk.
  • fast_forward00:43:35 - There's a lot of aspects of this immune system and its responses that we just don't understand.
  • fast_forward00:43:41 - But the next step would be to control this immune system. So imagine we move
  • fast_forward00:43:45 - to this phase, now you have the model, we understand how it works,
  • fast_forward00:43:49 - understanding would imply we can control it.
  • fast_forward00:43:51 - So what kind of control would you envision we should pursue there?
  • fast_forward00:43:57 - Well, I have a very simple but clear vision on that.
  • fast_forward00:44:02 - I don't know whether it's realistic, but this is what I'm heading for.
  • fast_forward00:44:05 - And that is in the immune system there's
  • fast_forward00:44:08 - a very clear division of objects that
  • fast_forward00:44:11 - immune cells can sense being categorized either
  • fast_forward00:44:14 - as self or non-self as enemy or
  • fast_forward00:44:18 - friend and we know one thing and
  • fast_forward00:44:20 - that is the immune system is capable of changing
  • fast_forward00:44:24 - its mind from one day to the next and this happens certain diseases when all
  • fast_forward00:44:29 - of a sudden finds that the immune system starts attacking a particular type
  • fast_forward00:44:33 - of cell cell and this is not desirable but we know that the immune system can
  • fast_forward00:44:39 - do it what we do not know is what actually made that change how came that change about.
  • fast_forward00:44:45 - So we first need to understand this. We need to understand what is happening
  • fast_forward00:44:50 - when someone develops alopecia, where all of a sudden your hair fall out.
  • fast_forward00:44:54 - Because it's a mechanism that is possible, but it's a mechanism that we would
  • fast_forward00:44:58 - like to exploit and control.
  • fast_forward00:45:00 - Because if I can manipulate this, if I can tell the immune system which objects
  • fast_forward00:45:07 - are from today onwards to be seen as enemy,
  • fast_forward00:45:10 - which until now was seen as friend, then we can
  • fast_forward00:45:14 - redirect the immune system to attack anything we like
  • fast_forward00:45:16 - and this is what i see as as really the
  • fast_forward00:45:19 - future of medical applications of the immune system and
  • fast_forward00:45:23 - this can be um especially relevant in
  • fast_forward00:45:26 - cancer medicine where there
  • fast_forward00:45:30 - are different levels of ambition one can have so
  • fast_forward00:45:32 - one level of ambition i think is a bit too far for now and that is to
  • fast_forward00:45:35 - train the immune system to see the difference between a healthy
  • fast_forward00:45:38 - cell and a cancerous cell of a given organ
  • fast_forward00:45:41 - but we don't need to do that we can also lower
  • fast_forward00:45:44 - our ambitions and say look there are certain organs that a person can live without
  • fast_forward00:45:49 - thyroid is one example if we could at least instruct the immune system to regard
  • fast_forward00:45:55 - all thyroid cells as enemies then you could treat a patient first with surgery remove the thyroid,
  • fast_forward00:46:01 - and then you switch your immune system to attacking all remaining residual thyroid cells.
  • fast_forward00:46:09 - And the difference between that and the normal route, which is chemotherapy,
  • fast_forward00:46:13 - is that if you do it right, you will not have any off-target pathologies that
  • fast_forward00:46:21 - you create, any damage that you create.
  • fast_forward00:46:23 - And it is an active agent. It is not a passive chemical.
  • fast_forward00:46:28 - It is an active agent that will seek out anything that looks like a thyroid cell.
  • fast_forward00:46:32 - So I can see this as the future. But isn't it, there's some molecular stuff
  • fast_forward00:46:38 - going on there which is not captured in your model yet, presumably.
  • fast_forward00:46:41 - No, these models are far from that stage. But I'm now looking ahead.
  • fast_forward00:46:45 - This is one of the things that could become possible once we have an acceptable
  • fast_forward00:46:50 - level of description of the immune system.
  • fast_forward00:46:53 - Because I think we have to be extremely careful at the same time.
  • fast_forward00:46:56 - Simply because the immune system is so powerful, and we've seen that in some
  • fast_forward00:47:00 - trials that went dreadfully wrong. on, tinker with it the wrong way,
  • fast_forward00:47:03 - you can do tremendous damage too.
  • fast_forward00:47:05 - And it occurred to me that that's another use for your model because you can
  • fast_forward00:47:09 - test interventions in the modeling environment before you try them.
  • fast_forward00:47:13 - Because you explained how this particular disastrous trial started out with
  • fast_forward00:47:20 - the successful animal models.
  • fast_forward00:47:21 - So the animal models don't always predict what will happen in a human.
  • fast_forward00:47:24 - But maybe a computational model will show you a variety of possible results
  • fast_forward00:47:28 - and you you could then look out for bad outcomes that you might not have seen in the animal.
  • fast_forward00:47:35 - Yeah, a colleague of mine from Brazil, from Berlin, once described it in the
  • fast_forward00:47:39 - following way, which I think is a very good way of seeing it.
  • fast_forward00:47:41 - If we build a new car and we crash test it, we don't run Mercedes against walls.
  • fast_forward00:47:45 - We do it on a computer first. That's why we do the crash test.
  • fast_forward00:47:49 - In medicine, we're not often doing that, sometimes because we don't have the
  • fast_forward00:47:54 - computational ability or we don't have the knowledge to do it.
  • fast_forward00:47:57 - But I think that is the future.
  • fast_forward00:47:59 - Exactly as you say, you have a decent model and you have to be careful that
  • fast_forward00:48:02 - you test your model against experimental data, but once you get to the stage
  • fast_forward00:48:06 - where you trust your model to some extent, see if it exists.
  • fast_forward00:48:10 - Then you can crash test ideas. And in the model, you can make those changes
  • fast_forward00:48:15 - that you think on the basis of the theory would be the changes required,
  • fast_forward00:48:20 - and then test whether it actually works like that or does have perhaps unforeseen
  • fast_forward00:48:25 - consequences that are dangerous to the patient.
  • fast_forward00:48:28 - I think it's more than crash testing as well. I mean, the driverless car research
  • fast_forward00:48:32 - is now moving into simulation because they can't capture enough real-world data
  • fast_forward00:48:37 - of unlikely things that will happen when you have millions of driverless cars.
  • fast_forward00:48:42 - So you want to simulate those situations.
  • fast_forward00:48:45 - But there are already attempts underway, as you also mentioned,
  • fast_forward00:48:50 - also failed attempts, to try to really master and control the immune system
  • fast_forward00:48:56 - for more personalized healthcare. Yes.
  • fast_forward00:48:59 - So to what extent do you see these current steps as being consistent with the
  • fast_forward00:49:05 - concept of the immune system that you are advancing with this model um i think
  • fast_forward00:49:11 - it is going in the right direction in in early.
  • fast_forward00:49:15 - Immunological therapy ideas very often
  • fast_forward00:49:19 - what was done was non-cell specific interventions in the immune system which
  • fast_forward00:49:24 - you can sort of see as similar to just rebooting it giving it a big kick and
  • fast_forward00:49:29 - then hope that it settles into a new state which is doing what it was designed
  • fast_forward00:49:33 - to do and more recently differently,
  • fast_forward00:49:36 - especially with this example of these CAR T cells,
  • fast_forward00:49:39 - the research has moved away in the direction of making genetic modifications
  • fast_forward00:49:46 - that are not generic, but they are tailored to the specific needs of an individual patient.
  • fast_forward00:49:53 - And in the case of CAR T cells, they create these receptors to be specifically
  • fast_forward00:49:59 - tailored to recognizing and docking to the cancer cell.
  • fast_forward00:50:05 - Now that is moving already in the right direction. It's the only thing is it's
  • fast_forward00:50:08 - fairly limited because this only is done for cytotoxic T cells.
  • fast_forward00:50:12 - So this doesn't exploit at all the interaction, the network concept,
  • fast_forward00:50:17 - the interaction between the T clones and the B clones.
  • fast_forward00:50:21 - And in that sense, it is still very different. It's not this idea that I have
  • fast_forward00:50:26 - of controlling the self non-self boundary.
  • fast_forward00:50:28 - That's not what's happening, but at least it
  • fast_forward00:50:31 - is moving in the the right direction it is trying to make non-generic changes
  • fast_forward00:50:35 - but it's trying to make changes to components that are really tailored to the
  • fast_forward00:50:40 - specific needs of an individual and they are spectacular some of these results
  • fast_forward00:50:45 - it isn't doing everything we would want these things to do the CAR T cells don't
  • fast_forward00:50:49 - work for solid tumors at the moment.
  • fast_forward00:50:52 - There's a lot of research going on in seeing what is the
  • fast_forward00:50:55 - reason why it doesn't work there what can we change and there
  • fast_forward00:50:59 - have been already several generations compared to the first generation
  • fast_forward00:51:02 - of these methods they've come a long
  • fast_forward00:51:04 - way and that's why they've now been uh approved by the various authorities right
  • fast_forward00:51:09 - but why why would this work on by manipulating the t cells and not the b cells
  • fast_forward00:51:14 - because you could imagine that the b cells that was labeling um whatever object
  • fast_forward00:51:20 - you would like to classify as non-self.
  • fast_forward00:51:23 - So can you explain from the perspective of your model why it has been effective
  • fast_forward00:51:26 - on the T cells and not the B cells?
  • fast_forward00:51:28 - That's a very tricky question because I don't really know the answer to that.
  • fast_forward00:51:31 - And I can only speculate as to why that may be.
  • fast_forward00:51:35 - But given the model, you must have an intuition, no?
  • fast_forward00:51:38 - Well, let's approach it from the other side. Would it be a viable alternative to do that?
  • fast_forward00:51:44 - I think it would be. But I need to look very carefully at the differences,
  • fast_forward00:51:49 - is if any between the response of the immune system and the way it does its business in,
  • fast_forward00:51:56 - attacking foreign antigen compared to how it deals with deranged own cells and
  • fast_forward00:52:02 - i would not be surprised if let's say the players responsible for the immune
  • fast_forward00:52:07 - response would not be the same,
  • fast_forward00:52:09 - because b cells are being trained very specifically in the germinal center to
  • fast_forward00:52:14 - look away from the cell cells, to really look only at foreign antigen.
  • fast_forward00:52:21 - So it may well be that the way the immune system handles cancerous cells is
  • fast_forward00:52:26 - different, distinct from the way it is most effective in handling foreign antigen.
  • fast_forward00:52:33 - There must be studies looking at that, but I'm not aware of them at the moment.
  • fast_forward00:52:37 - But it is a very good point.
  • fast_forward00:52:43 - So you move from surgical physics to neural networks to the immune system.
  • fast_forward00:52:49 - Do you in any way miss the study of neural networks and the brain,
  • fast_forward00:52:54 - or not that much? Do you find this more challenging?
  • fast_forward00:53:01 - That's an interesting question again. and I've had
  • fast_forward00:53:05 - a really good time studying neural networks but I
  • fast_forward00:53:08 - also find very often that when I've worked
  • fast_forward00:53:11 - on something for 10 years or more I tend
  • fast_forward00:53:14 - to look for something different and it
  • fast_forward00:53:18 - can be that different in terms of the modeling
  • fast_forward00:53:22 - and the analysis can be found within
  • fast_forward00:53:24 - the same domain main or outside and
  • fast_forward00:53:28 - and what i've seen at the time is that i
  • fast_forward00:53:31 - found all kinds of new things to work on which happened not to be neural networks
  • fast_forward00:53:35 - but now just by accident i'm moving back and i'm touching that field again simply
  • fast_forward00:53:42 - because the technology leads me there um i've always enjoyed I thought it was
  • fast_forward00:53:49 - a fascinating area to work in,
  • fast_forward00:53:51 - but at some point I got the impression that in this community it became an inappropriate
  • fast_forward00:54:01 - norm that one would have to know everything about biology,
  • fast_forward00:54:05 - everything about the computer science aspects, everything about the theory,
  • fast_forward00:54:09 - in order to be effective. I believe that's a fallacy.
  • fast_forward00:54:14 - And I think it has done some damage to that field and it has sort of pushed communities apart.
  • fast_forward00:54:21 - I find these things very interesting because I've now, about 10 years ago,
  • fast_forward00:54:25 - moved into what I call quantitative medicine, did a lot of work on protein interaction
  • fast_forward00:54:30 - networks, gene regulation.
  • fast_forward00:54:33 - And I've asked myself, how can we avoid that in the future?
  • fast_forward00:54:39 - So what we've done is we've set up a mechanism where young people are given
  • fast_forward00:54:44 - a program, a training program, in which they are being taught by people from
  • fast_forward00:54:49 - different disciplines, at a level that they can understand.
  • fast_forward00:54:52 - And we try to educate them to work differently, to not believe that in working
  • fast_forward00:54:59 - in an interdisciplinary team, one has to know everything about all the disciplines
  • fast_forward00:55:02 - because one ends up being mediocre at everything.
  • fast_forward00:55:05 - But they have to identify what they are in terms of their primary discipline
  • fast_forward00:55:10 - and make sure that they are really on the ball in that.
  • fast_forward00:55:12 - But they need to speak the jargon of the people at the other side of the various
  • fast_forward00:55:18 - fences, experiences, and they need to know what keeps them awake at night.
  • fast_forward00:55:21 - And that's the only crucial thing. But they should not make the mistake to think
  • fast_forward00:55:24 - that they have to be experts at three different disciplines.
  • fast_forward00:55:27 - It's not realistic. It's counterproductive. I think at the time,
  • fast_forward00:55:32 - but maybe I'm wrong because you know that community better than I do,
  • fast_forward00:55:36 - there was a bit of that happening in between the 1990s and 2000s.
  • fast_forward00:55:42 - It's still like that. Yeah. Yeah, yeah. I mean, multidisciplinarity is clearly
  • fast_forward00:55:50 - still a challenge also in the field of neuroscience,
  • fast_forward00:55:52 - where there's from the side of the people doing the experimental work,
  • fast_forward00:55:58 - which is extremely intense.
  • fast_forward00:56:02 - Of course, they would feel like, well, these guys haven't made the miles to
  • fast_forward00:56:06 - really appreciate the complexities of what we're doing here.
  • fast_forward00:56:09 - They will not really appreciate the meaning of this kind of data.
  • fast_forward00:56:13 - And the modelers or the theoreticians would often have the feeling like,
  • fast_forward00:56:18 - well, we're not being listened to, right? So, there's this massive disconnect.
  • fast_forward00:56:21 - And that's also why I think in neuroscience, theory overall has made less of
  • fast_forward00:56:26 - an impact than you might have wished.
  • fast_forward00:56:30 - But now, so, okay, given this long trajectory you've made, what should be Ton's
  • fast_forward00:56:38 - law if we want to understand the immune system, the brain, living systems? So, what's Ton's law?
  • fast_forward00:56:45 - Well, I don't think I have one law. I've learned a couple of things,
  • fast_forward00:56:48 - and I can go through them if you want. The first is interdisciplinary interactions.
  • fast_forward00:56:54 - What i just explained to you and i think where we should move to is a is a way of working based on,
  • fast_forward00:57:00 - respect of expertise on the other side right if i work with an experimentalist
  • fast_forward00:57:04 - a bioinformatician i have to trust them to know what they do and i i leave them
  • fast_forward00:57:10 - to it what i do need to know is,
  • fast_forward00:57:13 - what is it they really find interesting but what i
  • fast_forward00:57:16 - should not try to do is to convince them what they should find interesting
  • fast_forward00:57:19 - is what i find interesting yeah so these
  • fast_forward00:57:21 - are the types of of psychological things i've learned that means empathy not
  • fast_forward00:57:25 - not that empathy i just need to know what what excites you
  • fast_forward00:57:28 - paul i need to know what what is it that makes you really
  • fast_forward00:57:31 - happy you need to know what makes me happy and we have to accept that these are
  • fast_forward00:57:33 - different things but they could all be found in the same project but they can
  • fast_forward00:57:38 - all be found in the same project and that's how we can work together best and
  • fast_forward00:57:41 - i don't and i should not try to do someone else's work or even attempt to know
  • fast_forward00:57:46 - what they know this is completely counterproductive so that's that's that's one of the first things.
  • fast_forward00:57:54 - Generally, I think we live in a bit of a strange time in terms of how science
  • fast_forward00:58:01 - is organized generally, and I think there will come a point where there will be major revisions.
  • fast_forward00:58:05 - One of the crucial lessons is never sacrifice quality. Everything else you can
  • fast_forward00:58:10 - sacrifice, but never sacrifice the quality of what you do.
  • fast_forward00:58:12 - Be your own worst referee yeah because fashions
  • fast_forward00:58:17 - will come and go reputations will come and go but you want your name to be a
  • fast_forward00:58:22 - sign of quality the only way to protect that is to never sacrifice quality and
  • fast_forward00:58:28 - if you then perhaps are six months later in submission of a paper and it doesn't
  • fast_forward00:58:33 - end up in nature who Who cares?
  • fast_forward00:58:37 - But maybe in 15 years from now, they will still read that paper.
  • fast_forward00:58:40 - And all the nature papers will have been seen and forgotten about in six months.
  • fast_forward00:58:45 - Quality is everything in science. That should never be sacrificed.
  • fast_forward00:58:49 - Okay. Another loss you want to add to that or two is enough for now?
  • fast_forward00:58:53 - No, I drink lots of coffee, Paul. Okay, good.
  • fast_forward00:58:57 - So to finish up, so we're going to send Tony from Sheffield to London five years from now.
  • fast_forward00:59:03 - Okay. Okay, given the current political situation, let's just assume it's London,
  • fast_forward00:59:09 - it might be somewhere else.
  • fast_forward00:59:10 - You never know, with Brexit hanging in the air.
  • fast_forward00:59:14 - But okay, there comes Tony, he visits you at King's College to check whether
  • fast_forward00:59:20 - a specific hypothesis was falsified or verified.
  • fast_forward00:59:24 - And that is the hypothesis that you're going to specify today.
  • fast_forward00:59:27 - So what's the one hypothesis that you really would like to see tested in this five-year frame?
  • fast_forward00:59:32 - That it is possible to instruct the immune system to change which object it
  • fast_forward00:59:40 - sees as self and it sees as non-self.
  • fast_forward00:59:42 - I think that would be a big, big step forward if we can achieve that and actually
  • fast_forward00:59:46 - have experimental validation for it.
  • fast_forward00:59:50 - Great. Tom Cullen, thank you very much for this conversation.
  • fast_forward00:59:53 - Thank you. Thank you very much.
  • fast_forward00:59:57 - The CSN Podcast was produced by the Convergent Science Network of Biometrics
  • fast_forward01:00:03 - and Biohybrid Systems, a project funded by the European Sevens Research Framework Programme.
  • fast_forward01:00:11 - For more interviews, recorded lectures or upcoming conferences in the field
  • fast_forward01:00:16 - of biometrics and biohybrid systems, go to csnnetwork.eu.
  • fast_forward01:00:22 - Music.

Be the first to leave a comment

Leave a comment

Your email address will not be published. Required fields are marked *

Convergent PozitivLinia Logo

Exploring the convergence of neuroscience, robotics, and AI through conversations with leading researchers since 2010.

A project of the Convergent Science Network Foundation.

© CSN Podcasts. Developed by IMCreativeWEBC

0%

Login to enjoy full advantages

Please login or subscribe to continue.

Go Premium!

Enjoy the full advantage of the premium access.

Stop following

Unfollow Cancel

Cancel subscription

Are you sure you want to cancel your subscription? You will lose your Premium access and stored playlists.

Go back Confirm cancellation