cover dmitri chklovskii

Dmitri Chklovskii on predictive coding and lattice filter

  • cover play_arrow

    PLAY EPISODE


cover dmitri chklovskii
Season 2012
Season 2012
Description arrow_drop_down

Description

Can the brain’s visual wiring be explained by the same engineering principles that optimize telephone networks? Dmitri Chklovskii shows how predictive coding theory and lattice filters map onto real neural circuits, from fly photoreceptors to the mammalian LGN.

Subscribe for more from the Convergent Science Network podcast series.

Chklovskii bridges theoretical physics and neuroscience by applying adaptive signal processing frameworks to sensory systems. Building on Barlow’s redundancy reduction principle and the predictive coding work of Srinivasan, Laughlin, and Dubs, his group derives normative predictions for neural filter shapes with no free parameters: once you specify the natural stimulus statistics and signal-to-noise ratio, the optimal filter is uniquely determined. The biphasic temporal response and center-surround spatial receptive fields of retinal and LGN neurons emerge naturally as mechanisms for subtracting predictions from incoming signals, compressing redundant information.

The key evidence supporting this framework over simple biophysical explanations like after-hyperpolarization comes from stimulus-dependent filter changes. At high contrast, neurons show sharp biphasic responses with strong negative components; at low contrast, the filter shifts toward broader low-pass characteristics with weakened negative phases. This adaptive behavior matches predictive coding predictions but would require different physiological implementations at each contrast level, suggesting the filter shape is functionally optimized rather than a fixed biophysical artifact.

Chklovskii introduces the lattice filter as a specific circuit implementation where decorrelation occurs in hierarchical stages, each operating at a different timescale. This architecture predicts that LGN temporal receptive fields should be longer than retinal ones, which matches electrophysiological observations. It also predicts two distinct LGN cell types corresponding to forward and backward prediction error pathways, identifiable with the known lagged and non-lagged cell classes. At Janelia Farm, his group has reconstructed the connectome of the fly visual system through the first two neuropils, mapping approximately 10,000 synaptic connections among 50 neurons per processing column. The L1 and L2 large monopolar cells show response properties consistent with the dual pathways of a lattice filter, and inter-column connections provide the substrate for motion detection.

Tagged as:

About the author call_made

CSN Podcasts

Both the triumphs of humanity and its most evil deeds have resulted from collaboration. In a time where humanity is required to aspire to the former and minimize the latter, the question arises of how collaboration arises and why it fails. Surprisingly, this phenomenon, so central to who we are, is not well understood. Hence, a collaborative effort is required to understand collaboration in its full biological, psychological, sociological, cultural, and economic complexity and to translate this understanding into operational impact. This series of podcasts is one step toward achieving these complementary goals. The Collaboration Podcast presents interviews with people who are central orchestrators of collaboration in various domains including business, government, science, art, health, sustainability, and the military. The discussions were conducted by Prof. Dr. Paul F.M.J. Verschure and members of the Program Advisory Committee of the Ernst Strungmann Forum on Collaboration (https://www.esforum.de/forums/ESF32_Collaboration.html) during 2021 and had the goal to sketch a map of opportunities, challenges, and obstacles in human collaboration. The forum took place in May 2022, and now we would like to share this series of interviews with a broader audience. The full report of the Forum will be published in 2023 by MIT Press. The podcast was produced by the Convergent Science Network (https://www.convergentsciencenetwork.org/). Context: The stability of social systems depends critically on realizing sustainable methods of “collaboration,” yet how and by which means collaboration is achieved is not clearly understood; neither are the conditions or processes that lead to its breakdown or failure. Collaboration can be understood as cooperation between agents toward mutually constructed goals. Part of the reason for our lack of understanding is that the phenomenon of collaboration is, by nature, a highly multidisciplinary problem, and effective research into its complexities has been difficult to achieve across the broad range of scientific and technical disciplines involved. The need for a fundamental understanding of collaboration, however, has become increasingly important. Not only does humankind demand answers as it attempts to address critical challenges at multiple scales (e.g., climate change, migration, enhanced automation, social and economic inequality), but ever-increasing technological and economic means of interconnecting people and societies are disrupting long-established, familiar patterns of how we interact. Radical technological changes that are ongoing have the potential to reshape collaboration in ways that are currently hard to predict or influence (e.g., by altering configurations in interaction, information creation, and modes of communication). On one hand, such changes could disrupt hitherto stable forms of collaboration by affecting critical communication channels and traditional roles, as can be observed in the rapidly changing patterns in governance, commerce, and social interaction. Conversely, technology could lead to the emergence of novel, successful forms of collaboration that deviate from traditional “hierarchical” architectures. Evidence of this can be seen in areas as diverse as highly automated manufacturing plants, the open science movement, collaborative software repositories, user-centered services, and the sharing of economy-based modes of organization. Without a fundamental understanding of the mechanisms, processes, and boundary conditions of collaboration, it is not possible to evaluate or predict which of these possible scenarios are sustainable or even plausible. The Forum “How Collaboration Arises and Why it Fails” (May 8–13, 2022, Location: Frankfurt am Main, Germany) Chairs: Andreas Roepstorff and Paul Verschure Program Advisory Committee: Jenna Bednar, Julia R. Lupp, Bhavani R. Rao , Andreas Roepstorff, Ferdinand von Siemens, and Paul Verschure

More posts

Timestamp

  • fast_forward00:00:03 - This is the Convergent Science Network podcast. Leading researchers in the domain
  • fast_forward00:00:10 - of neuroscience, brain theory and technology are interviewed by Paul Verschoor and Tony Prescott.
  • fast_forward00:00:18 - So this is Paul Verschoor with the Convergent Science Network podcast.
  • fast_forward00:00:24 - And in this episode that we recorded as part
  • fast_forward00:00:27 - of the CSN Barcelona Cognition Brain and
  • fast_forward00:00:30 - Technology Summer School series I'm talking
  • fast_forward00:00:33 - to Dimitri Sklosky and Dimitri
  • fast_forward00:00:36 - you come out of engineering now of theoretical physics and from theoretical
  • fast_forward00:00:41 - physics you went into neuroscience and also in your in your presentation this
  • fast_forward00:00:46 - morning you also show how let's say this interest in theory is translating towards
  • fast_forward00:00:51 - how you understand brains.
  • fast_forward00:00:54 - So how do you see that link exactly between theory and the practical sides of neuroscience?
  • fast_forward00:01:05 - So historically, neuroscience has been rich on a number of experimental observations
  • fast_forward00:01:14 - and facts that have been assembled through a large number of techniques
  • fast_forward00:01:19 - on a variety of different levels and animals.
  • fast_forward00:01:24 - But what has been lacking is a theoretical framework that would allow to put
  • fast_forward00:01:29 - these facts into a unified perspective and to make future predictions and eventually
  • fast_forward00:01:35 - to understand how the brain works.
  • fast_forward00:01:41 - Neuroscientists always are interested in finding an appropriate framework to put the facts in.
  • fast_forward00:01:51 - In our case, we had a lot of success with doing it by borrowing the ideas from
  • fast_forward00:01:59 - electrical engineering, specifically from the field of adaptive signal processing. Right.
  • fast_forward00:02:05 - So what you emphasized a lot there was how very specific ideas about signal
  • fast_forward00:02:13 - processing, as developed within engineering for a long time by now,
  • fast_forward00:02:18 - might help us to get some leverage in how we can understand the brain.
  • fast_forward00:02:21 - And so what do you see as some promising starting points there when you talk
  • fast_forward00:02:28 - about adaptive filtering?
  • fast_forward00:02:29 - Because in some sense, there are quite a number of people who go around saying,
  • fast_forward00:02:33 - well, the brain is an adaptive filter.
  • fast_forward00:02:36 - Common filters have been very successful in engineering, so the brain also must
  • fast_forward00:02:40 - operate like one, et cetera.
  • fast_forward00:02:41 - But to make that now more specific, where do you really see leverage in these
  • fast_forward00:02:46 - kinds of more normative approaches towards the brain? Yeah.
  • fast_forward00:02:50 - Well, I think to make such ideas successful and of practical value is to make
  • fast_forward00:02:59 - a connection between theory and experiment on a very specific level,
  • fast_forward00:03:03 - so that we could predict, for example,
  • fast_forward00:03:05 - response properties of individual neurons given a certain stimulus presentation,
  • fast_forward00:03:12 - which could be compared with electrophysiological recordings, for example.
  • fast_forward00:03:15 - And I think one of the places where this could be done easiest are sensory systems,
  • fast_forward00:03:23 - like visual, for example,
  • fast_forward00:03:26 - where you have complete control over the stimulus and you also have access to
  • fast_forward00:03:32 - the neurons by recording in the retina or further down in the vertebrate pathway in the LGN.
  • fast_forward00:03:38 - And the reason I think that the engineering principles should apply there is
  • fast_forward00:03:44 - because both systems have to deal with the same kind of limitations that are
  • fast_forward00:03:50 - presented by the physical world,
  • fast_forward00:03:52 - such as limitations on the dynamic range and the bandwidth of the communication
  • fast_forward00:04:00 - from the retina, say, to the LGN and from the LGN to the cortex.
  • fast_forward00:04:05 - But now, can you give me an example? Could you talk me through an example of
  • fast_forward00:04:09 - how such a filter would help us to understand what happens in the retina and LGN?
  • fast_forward00:04:15 - So one example which goes
  • fast_forward00:04:18 - back in time is the receptive fields of the retinal or LGN neurons that have
  • fast_forward00:04:28 - been known to be biphasic in time and center-surround in space.
  • fast_forward00:04:36 - Both of those observations can be explained based on the predictive coding framework
  • fast_forward00:04:43 - that originates in engineering,
  • fast_forward00:04:46 - where you say that the system is trying to compress the incoming signal by subtracting
  • fast_forward00:04:54 - a prediction from the actual signal value.
  • fast_forward00:04:57 - So, in terms of biphasic temporal response, for example, you would use the previous
  • fast_forward00:05:02 - values of the signal to predict the current one, and that's how you get a biphasic shape.
  • fast_forward00:05:08 - In the case of the spatial response shape, the center-surround shape,
  • fast_forward00:05:13 - you use the surrounding values of the signal, like the surrounding values of
  • fast_forward00:05:19 - image pixels, to predict the value of the pixel, a central pixel in the image,
  • fast_forward00:05:25 - and subtracting that prediction.
  • fast_forward00:05:27 - And that's how you can explain both the biphasic and the center-surround shape of the response.
  • fast_forward00:05:33 - But now, as a start, the intuition to look at encoding by neurons,
  • fast_forward00:05:40 - certainly in sensory systems, from this perspective, goes back quite a long way.
  • fast_forward00:05:45 - I mean, you mentioned yourself, Barlow, for instance.
  • fast_forward00:05:48 - That's right. When he tapped into this. So what does progress mean if we compare
  • fast_forward00:05:54 - it to the intuitions of Barlow and where we are today?
  • fast_forward00:06:02 - So Barlow and Atnav particularly borrowed the ideas from engineering.
  • fast_forward00:06:10 - And I think Barlow's point of view is usually summarized by the maximum redundancy reduction,
  • fast_forward00:06:17 - where the idea is that you just transmit the part of the signal that is non-redundant
  • fast_forward00:06:24 - or new or surprising or couldn't have predicted.
  • fast_forward00:06:27 - And then it has been followed by the introduction of predictive coding framework,
  • fast_forward00:06:33 - which is a concrete quantitative framework that allows to want to generate the
  • fast_forward00:06:39 - predictions for the receptive fields like biphasic and center-surround responses that I talked about.
  • fast_forward00:06:43 - And that was done by Srinivasan Laughlin and Dubs in 1982.
  • fast_forward00:06:49 - And then this line of work was continued by several people, most notably Attick and Van Hatteren.
  • fast_forward00:06:57 - And what we did recently is just try to take this work to its logical conclusion
  • fast_forward00:07:04 - and derive a normative theory in the most direct and transparent way.
  • fast_forward00:07:11 - And in that way, we could compare the predictions to the actual measurements
  • fast_forward00:07:18 - of spatio-temporal receptive fields that were done in catangion and insect visual system.
  • fast_forward00:07:25 - Okay, so what would now be the key parameters of this model?
  • fast_forward00:07:30 - So if I would like to take the model as a filter to look again at the brain
  • fast_forward00:07:35 - itself, what are the key parameters I should be sensitive to?
  • fast_forward00:07:39 - Right, so the model actually has no free parameters in a sense that once you
  • fast_forward00:07:49 - define a natural stimulus ensemble,
  • fast_forward00:07:51 - um so that's a statistics of the
  • fast_forward00:07:54 - input and um you specify
  • fast_forward00:07:57 - the signal to noise ratio then there
  • fast_forward00:08:00 - is a unique prediction for the filter shape and that
  • fast_forward00:08:03 - can be compared with the um electrophysiologically measured um receptive fields
  • fast_forward00:08:10 - by say a reverse correlation method but now um in some sense if you take something
  • fast_forward00:08:16 - like the signal to noise ratio of these neurons this might not be a constant necessarily, right?
  • fast_forward00:08:20 - This could vary depending on, let's say, the presence of neuromodulators or not.
  • fast_forward00:08:25 - So how well would it generalize beyond this fixing of these kinds of parameters?
  • fast_forward00:08:31 - Right. I was actually referring to the signal-to-noise ratio in the input that
  • fast_forward00:08:37 - would have to do with the absolute intensity level.
  • fast_forward00:08:44 - But there is also internal noise, of course,
  • fast_forward00:08:47 - and the impact of that noise depends on the specific circuit implementation of the filter,
  • fast_forward00:08:57 - which we have one proposal for that I discussed earlier this morning,
  • fast_forward00:09:04 - which is based on the lattice filter idea.
  • fast_forward00:09:07 - Before we go to the lattice filter, let's look at the simpler case first.
  • fast_forward00:09:12 - Because In some sense, the key signature that you see as confirming the physiology
  • fast_forward00:09:18 - is biphasic response, which essentially means I would have, let's say,
  • fast_forward00:09:23 - some onset-driven response,
  • fast_forward00:09:25 - and then I would have, let's say, an orthogonal or an opposite response in turn,
  • fast_forward00:09:31 - like I might have, let's say, a depolarizing response to something,
  • fast_forward00:09:35 - and then I'm hyperpolarizing, right?
  • fast_forward00:09:37 - This would be a quick characterization of, let's say, the simplest form of such a filter.
  • fast_forward00:09:43 - And so so what i'm curious about is how
  • fast_forward00:09:46 - specific can you make these responses right so
  • fast_forward00:09:49 - also what i post you the question post in the morning one way
  • fast_forward00:09:53 - to to look at these these the negative side of
  • fast_forward00:09:56 - the response the the negative tail could be to say like well after hyperpolarization
  • fast_forward00:10:01 - is a standard feature of all neurons whether they're encoding anything or not
  • fast_forward00:10:05 - so this is just a non-specific component and now you're saying no no it's a
  • fast_forward00:10:10 - specific component because that's exactly what I need for my predictive filter.
  • fast_forward00:10:14 - So how could we make this experimentally testable? Or do you think the data
  • fast_forward00:10:18 - is already out there to allow us to make a decision on this?
  • fast_forward00:10:23 - I think the answer is partially yes. And the reason, I mean,
  • fast_forward00:10:29 - of course, there is a physiological mechanism like after hyperpolarization that
  • fast_forward00:10:34 - has to underlie this response.
  • fast_forward00:10:36 - But I think the evidence to say that this response is there to implement productive
  • fast_forward00:10:44 - filtering comes from the change in the filter shape in response to the change
  • fast_forward00:10:50 - in the stimulus statistics.
  • fast_forward00:10:51 - So, for example, at high contrast, at high signal-to-noise ratio,
  • fast_forward00:10:56 - you get very strong biphasic response with a strong and sharp negative component.
  • fast_forward00:11:02 - When you go to low contrast, to low signal-to-noise ratio,
  • fast_forward00:11:07 - the filter changes and it carries most of the weight in the first peak,
  • fast_forward00:11:14 - which gets wider, and the negative rebound actually gets much weaker.
  • fast_forward00:11:22 - And that would be consistent with the filter doing more of low-pass filtering
  • fast_forward00:11:27 - rather than high-pass filtering.
  • fast_forward00:11:31 - Such change would be expected from the predictive coding framework,
  • fast_forward00:11:36 - but would require a different physiological implementation.
  • fast_forward00:11:41 - And so the fact that the neuron response follows this prediction suggests that
  • fast_forward00:11:48 - the predictive coding framework has value.
  • fast_forward00:11:52 - Yeah, but that would also mean that for the predictive coding framework,
  • fast_forward00:11:56 - you should see a stimulus-specific modulation of both the positive and the negative
  • fast_forward00:12:01 - phase, right? That's correct.
  • fast_forward00:12:04 - And is there sufficient evidence for that? If we go to just,
  • fast_forward00:12:08 - let's say, standard physiology of the visual system, and we start to look at,
  • fast_forward00:12:13 - let's say, the encoding of more or less complex scenes as an example,
  • fast_forward00:12:16 - do you see examples of this?
  • fast_forward00:12:21 - Only very few. As I mentioned, there is a change in contrast that people looked
  • fast_forward00:12:27 - at, and lately they've been adding more and more noise to the images and looking at the response.
  • fast_forward00:12:34 - But I think that this line of work actually should become a bigger project right now,
  • fast_forward00:12:44 - and we're trying to make connection with experimentalists to actually test this
  • fast_forward00:12:50 - idea more exhaustively by playing with the different statistical ensembles of
  • fast_forward00:12:57 - natural scenes and seeing if the filter would change accordingly.
  • fast_forward00:13:00 - So it is in some sense work in progress. Right, okay, understood.
  • fast_forward00:13:04 - Now my other question is also, so before we go to the complex version of the
  • fast_forward00:13:12 - model, which is this lattice filter system.
  • fast_forward00:13:17 - Something funny happens in the argument about adaptive filters, right?
  • fast_forward00:13:20 - Because, as we also discussed earlier, originally the intuition is like, well,
  • fast_forward00:13:25 - adaptive filters were actually a near optimal way or an optimal way,
  • fast_forward00:13:30 - if you want, to show how you can transduce information through some channel, okay?
  • fast_forward00:13:36 - And this is also developed as a technique, given the limitations and the possibilities
  • fast_forward00:13:42 - of the engineering that we do. And there, indeed, bandwidth is always an issue.
  • fast_forward00:13:47 - But now for brains themselves, for the brain, maybe bandwidth is not an issue in the same way.
  • fast_forward00:13:55 - Like, for instance, you could argue that actually the whole principle of the
  • fast_forward00:13:59 - brain is to do massive IO, but all these connections that you have available
  • fast_forward00:14:04 - to you with actually minimal local computation,
  • fast_forward00:14:08 - because biophysically that's more complex.
  • fast_forward00:14:10 - So the whole design might actually be exactly orthogonal to what the engineers
  • fast_forward00:14:14 - were thinking of when they designed these adaptive filters.
  • fast_forward00:14:17 - So maybe then it's the wrong metaphor to apply to this system.
  • fast_forward00:14:22 - Yeah, it's actually a very astute observation, Paul, because when we started
  • fast_forward00:14:28 - this work, our main motivation was a so-called communication bottleneck,
  • fast_forward00:14:34 - as I think Barlow referred to it, where he said, well,
  • fast_forward00:14:38 - you know, there are many more photoreceptors in the vertebrate retina than there are ganglion cells.
  • fast_forward00:14:45 - And so there is a bottleneck for transmitting information to the rest of the
  • fast_forward00:14:50 - brain, and therefore there must be compression, which could be done by redundancy reduction.
  • fast_forward00:14:55 - And that's the philosophy that we started with.
  • fast_forward00:14:59 - But I think as the work progressed, especially after looking at other systems, such as, for.
  • fast_forward00:15:10 - As explicit as it is in the vertebrate pathway, we started questioning this
  • fast_forward00:15:18 - assumption of the need for compression.
  • fast_forward00:15:22 - And at the same time, what started to become clear that there could be computational
  • fast_forward00:15:27 - advantages to implementing this predictive coding framework and decorrelating the incoming signal,
  • fast_forward00:15:36 - especially when it's done in a stage-wise fashion like it's done in the lattice filter.
  • fast_forward00:15:42 - Because when I looked up in signal processing textbooks,
  • fast_forward00:15:47 - it turns out that lattice filters, in addition to performing the correlation
  • fast_forward00:15:53 - of the original signal stream, They also are used for feature construction,
  • fast_forward00:16:01 - because if you output signal from each stage of the lattice filter,
  • fast_forward00:16:09 - you get a set of orthogonal features,
  • fast_forward00:16:11 - which are very convenient to be used as a set for predicting or training or
  • fast_forward00:16:21 - learning a correct response to another input.
  • fast_forward00:16:24 - Input, which presumably is what the brain is trying to achieve in associative
  • fast_forward00:16:29 - learning or something like that.
  • fast_forward00:16:31 - So now we move to this lattice filter. What makes the lattice filter interesting
  • fast_forward00:16:36 - and how is it different from the filter we just talked about?
  • fast_forward00:16:40 - Right, so lattice filter is a specific circuit implementation of a predictive coding filter,
  • fast_forward00:16:49 - and the defining characteristics is that decorrelation is done in stages, and,
  • fast_forward00:16:58 - where each consecutive stage decorrelates the signal on a different time scale.
  • fast_forward00:17:06 - Which seem to correspond to electrophysiological
  • fast_forward00:17:10 - observations of receptive fields in the retina and LGN.
  • fast_forward00:17:15 - Specifically, it has been measured that the temporal receptive fields in LGN
  • fast_forward00:17:20 - are longer than the ones in the retina.
  • fast_forward00:17:23 - And that's why it
  • fast_forward00:17:26 - seems that the lattice filter may be a good
  • fast_forward00:17:29 - model for the system so the key
  • fast_forward00:17:32 - thing is a lattice filter is hierarchical at every
  • fast_forward00:17:35 - state it performs the same operation roughly which essentially is to just decorrelate
  • fast_forward00:17:39 - the image is that the reasonable way to interpret it that's right on a different
  • fast_forward00:17:43 - time scale okay so that means you you decorrelate at varying time scales as
  • fast_forward00:17:48 - you go through the filter as you go through this cascade of filters essentially
  • fast_forward00:17:51 - that's right but With the local operations, it would be the same.
  • fast_forward00:17:54 - Similar. In the simplest model, they're exactly the same, but the lattice filter
  • fast_forward00:17:59 - can be modified to have slightly different operations.
  • fast_forward00:18:04 - So what does the lattice filter now solve that your previous linear filter did not solve?
  • fast_forward00:18:12 - So I think a better way to put it is that the general linear filter is a mathematical
  • fast_forward00:18:20 - concept that performs optimal prediction and therefore widens the incoming stimulus.
  • fast_forward00:18:29 - Leitz's filter is a specific circuit implementation of that filter.
  • fast_forward00:18:34 - And in particular, it's the one which allows you to do decorrelation in stages
  • fast_forward00:18:39 - by using biologically realistic elements such as neurons that have relatively short time constants.
  • fast_forward00:18:48 - Distance, but giving you the ability to de-correlate the signal over a longer
  • fast_forward00:18:55 - time scale due to the cascade hierarchical structure that you mentioned. Okay.
  • fast_forward00:18:59 - But now, so if it's getting close to implementation, it also means that you
  • fast_forward00:19:05 - must be able to make more specific predictions about how a biological system
  • fast_forward00:19:08 - could implement such a filter.
  • fast_forward00:19:11 - So what would be these specific predictions that would come out of that?
  • fast_forward00:19:14 - That's exactly right. Right, that's the reason to go for this specific circuit implementation,
  • fast_forward00:19:20 - and I think the strength of the lattice filter model is that you can map the
  • fast_forward00:19:26 - specific units in the circuit to the specific neurons in the brain.
  • fast_forward00:19:33 - And in particular, then, we could predict the responses of the retinal neurons
  • fast_forward00:19:39 - versus the LGN neurons, according to what the lattice filter tells you, but also we.
  • fast_forward00:19:46 - Can predict that there should be at least two different types of responses in LGN, for example,
  • fast_forward00:19:54 - which correspond to the so-called forward and backward prediction error filters in the lattice filter.
  • fast_forward00:20:02 - And those responses actually have been discovered electrophysiologically,
  • fast_forward00:20:07 - and they could be identified with the classes of cells that are called lagged
  • fast_forward00:20:13 - and non-lagged cells in the LGN.
  • fast_forward00:20:16 - And the non-lagged were discovered by Mastro Nardi more than 20 years ago,
  • fast_forward00:20:22 - and they have a distinct property that, although they also have a biphasic response,
  • fast_forward00:20:29 - but the second phase is greater in amplitude than the first,
  • fast_forward00:20:34 - which is rather unusual.
  • fast_forward00:20:36 - Is that still a negative phase, or that can be a positive phase, or it doesn't matter?
  • fast_forward00:20:40 - Right. So the important thing is that the two phases have the different signs,
  • fast_forward00:20:44 - It doesn't matter whether the first one is plus and the second is minus,
  • fast_forward00:20:48 - or the first is minus and the second is plus.
  • fast_forward00:20:52 - In this way, the classification into lagged and non-lagged cells is separate
  • fast_forward00:20:57 - from the classifications of cells into on and off.
  • fast_forward00:21:01 - So both lagged and non-lagged cells can come in on and off varieties.
  • fast_forward00:21:08 - What's the duration of this lagged response?
  • fast_forward00:21:15 - So what lagged cells do is in response to a step stimulus, they respond with
  • fast_forward00:21:23 - an initial delay which could be a few tens of milliseconds and that's why they're
  • fast_forward00:21:28 - called lagged. Right, okay.
  • fast_forward00:21:29 - But then how would your filter cascade account for that kind of lag given that
  • fast_forward00:21:36 - you would only have, let's say, three or four synaptic steps to get there, right?
  • fast_forward00:21:41 - If we go from the photoreceptor to our ganglion cells and then to your lag cells,
  • fast_forward00:21:46 - actually would, well, let's say two to three synaptic steps.
  • fast_forward00:21:50 - So how would you now relate that to this cascade of filters,
  • fast_forward00:21:55 - which are positive and your negative prediction components?
  • fast_forward00:22:00 - So, it is a little bit hard to explain without showing any diagrams,
  • fast_forward00:22:04 - but I can say... But for that, we recommend people to look at the video lecture. Exactly.
  • fast_forward00:22:10 - But the basic idea is that the photoreceptors at the very front of the cascade
  • fast_forward00:22:18 - perform low-pass filtering,
  • fast_forward00:22:20 - thus introducing the delay of various frequency components.
  • fast_forward00:22:26 - And then each stage of the pathway invokes a so-called all-pass filter,
  • fast_forward00:22:34 - which transmits all the frequency components equally, but introduces differential
  • fast_forward00:22:42 - phase delays depending on the frequency, which could be thought of as delays,
  • fast_forward00:22:47 - as just pure delays.
  • fast_forward00:22:49 - And as those delays accumulate from stage to stage, they lead to this electrophysiologically.
  • fast_forward00:22:59 - Notable lagged responses.
  • fast_forward00:23:01 - But it would mean in your case, the prediction would be that somewhere in this
  • fast_forward00:23:05 - network of ganglion cells or so, this buildup of delays would then happen.
  • fast_forward00:23:10 - Is that the logical consequence? That's right.
  • fast_forward00:23:14 - I think those interactions could happen already on the bipolar cell level.
  • fast_forward00:23:23 - For example, it is known that although off-bipolar cells have fast response
  • fast_forward00:23:32 - and they use ionotropic ion channels.
  • fast_forward00:23:41 - The on-bipolar cells have a delayed response because they use a metabotropic ion channels.
  • fast_forward00:23:50 - And that delay response I think is about 20 ms or so.
  • fast_forward00:23:55 - So that could be the original source of the delay.
  • fast_forward00:24:00 - But then there is further processing, of course, in the bipolar to ganglion
  • fast_forward00:24:07 - cell synapses and in the amacrine cell network interacting with bipolar cells.
  • fast_forward00:24:13 - So what I liked also in this model that you presented is that you actually purposefully
  • fast_forward00:24:20 - want to apply it both to, let's say, vertebrates and invertebrate systems, right?
  • fast_forward00:24:23 - Because you do believe, or wish at least, that the same principles will hold.
  • fast_forward00:24:30 - Right. This is correct. Correct.
  • fast_forward00:24:32 - So as an example, you talked about a specific system in the fly brain, right?
  • fast_forward00:24:39 - So how well did the model map onto that system? How well did that work out?
  • fast_forward00:24:44 - Yeah, I think what you said is really important that, you know, really good.
  • fast_forward00:24:51 - Powerful and correct theory has to apply across species, has to be general enough.
  • fast_forward00:24:57 - And so we're particularly pleased that this theory works for invertebrates as well, like in flies.
  • fast_forward00:25:03 - And in particular, in flies, the photoreceptors synapse on the cells called,
  • fast_forward00:25:14 - large monopolar cells, which have two biggest classes that are called L1 and L2,
  • fast_forward00:25:26 - and they're very similar in their initial response and the part of their anatomical
  • fast_forward00:25:35 - features, which led us to think about the dual pathway communication,
  • fast_forward00:25:40 - which is a hallmark of the lattice filter itself.
  • fast_forward00:25:44 - And that's how we got to the idea of the lattice filter, thinking about L1 and
  • fast_forward00:25:50 - L2 as being those two pathways, the forward and the backward prediction error filter.
  • fast_forward00:25:56 - So what evidence did you find that they could indeed exchange information in
  • fast_forward00:26:01 - a way that would be consistent with that model?
  • fast_forward00:26:04 - So that evidence is still somewhat sketchy because it is very hard to do electrophysiology in flies.
  • fast_forward00:26:15 - And what electrophysiology has been done was based mostly on recording from
  • fast_forward00:26:20 - cell bodies, although there was some in the axons.
  • fast_forward00:26:26 - But those measurements initially did not show difference in response properties between L1 and L2.
  • fast_forward00:26:32 - However, more recent measurements of the calcium dynamics
  • fast_forward00:26:37 - in the axon terminals of L1, L2, which is the output of those cells,
  • fast_forward00:26:44 - showed a different response between L1 and L2.
  • fast_forward00:26:48 - And in fact, the kind of response that has been reported by the clandinin lab
  • fast_forward00:26:53 - shows features indicative of the forward and backward pathways of the lightest Right.
  • fast_forward00:27:06 - So, now you're in a very unique position, right?
  • fast_forward00:27:09 - Because you work at Janelia Farms, and you also have been very much involved
  • fast_forward00:27:15 - in a very detailed reconstruction of the brain of these flies, right?
  • fast_forward00:27:21 - Now, the data set that you have for playing there, and I hope that you can explain
  • fast_forward00:27:25 - to me a little bit what you guys have been doing there, this is also giving
  • fast_forward00:27:29 - you now a grounding again to look at this more theoretical model, right?
  • fast_forward00:27:33 - So what's the data you have in your hands now on that fly brain at the anatomical
  • fast_forward00:27:38 - level that would help you understand this kind of filter model?
  • fast_forward00:27:43 - So what you're referring to is another direction in my group,
  • fast_forward00:27:49 - which is a high-throughput reconstruction of the connectome or the wiring diagram
  • fast_forward00:27:57 - of the brain on the synapse level.
  • fast_forward00:27:59 - And we've been doing that in the fly visual system, and this project is now bearing fruits.
  • fast_forward00:28:08 - And in particular, we were able to assemble the visual pathway in fly through
  • fast_forward00:28:16 - the first two neuropills, lamina and medulla.
  • fast_forward00:28:19 - And medulla was done for the first time. And what we have now is a kind of idealized processing column,
  • fast_forward00:28:31 - which repeats itself in the visual system in parallel.
  • fast_forward00:28:38 - And that column contains about 50 neurons, and we have attempted to map out
  • fast_forward00:28:45 - all the synaptic connections between those 50 neurons.
  • fast_forward00:28:49 - How many synaptic connections do they have?
  • fast_forward00:28:51 - So it's of order of 10,000. Okay.
  • fast_forward00:28:56 - That's including the gap junction, so that's only the synaptic connections?
  • fast_forward00:28:59 - The current imaging techniques that we're using based on electron microscopy
  • fast_forward00:29:05 - doesn't allow us to see gap junctions clearly in our data set.
  • fast_forward00:29:11 - So what we're reporting is just the chemical synapse. Right, okay.
  • fast_forward00:29:15 - So now the wiring diagram that you now have extracted from that column How does
  • fast_forward00:29:23 - it map onto this model of an adaptive filter?
  • fast_forward00:29:27 - So parts of that wiring diagram are consistent with the lattice filter,
  • fast_forward00:29:31 - but what we are also seeing is that the circuit is more complex,
  • fast_forward00:29:37 - and in particular, it seems that it isn't just focused on the decorrelation,
  • fast_forward00:29:42 - as one would expect from the compression and redundancy reduction point of view,
  • fast_forward00:29:47 - but also on the feature extraction that we mentioned previously,
  • fast_forward00:29:52 - using, of course, the correlation, but on feature extraction that can be used
  • fast_forward00:29:57 - to build other features for more specific purposes, such as,
  • fast_forward00:30:05 - for example, motion detection.
  • fast_forward00:30:07 - But how would I see that at an anatomical level? So I have my photoreceptor
  • fast_forward00:30:11 - projections, and I have my lobula, and I have my lamina and medulla.
  • fast_forward00:30:18 - So what are the specific wiring templates, if you want, that you can now extract from this?
  • fast_forward00:30:23 - Okay, this wiring template is more the predictive filter, and that wiring template
  • fast_forward00:30:28 - is, let's say, feature extraction.
  • fast_forward00:30:31 - Yeah, so, of course, just from anatomy, it might be hard to know that conclusively.
  • fast_forward00:30:42 - So what we
  • fast_forward00:30:45 - know already is that you know l1 and
  • fast_forward00:30:48 - l2 both are postsynaptic
  • fast_forward00:30:51 - to photoreceptors which is what you would
  • fast_forward00:30:54 - expect in the first stage of the lattice filter we know that they interact by
  • fast_forward00:30:58 - means of gap junctions which have been reported by the bores lab so they could
  • fast_forward00:31:02 - potentially be the two pathways forward and backward pathways of the lattice
  • fast_forward00:31:06 - filter and their output as measured by calcium imaging seems to support that interpretation.
  • fast_forward00:31:13 - What happens afterwards in the medulla is.
  • fast_forward00:31:19 - Are rather somewhat unclear and still work in progress, but it is already known
  • fast_forward00:31:26 - that L1 pathway and L2 pathway are involved in motion detection.
  • fast_forward00:31:31 - And so what we are doing,
  • fast_forward00:31:35 - we're tracing through the cells postsynaptic through L1 and L2 to see if they
  • fast_forward00:31:42 - would be consistent with the interpretation of feature construction or decorrelation.
  • fast_forward00:31:51 - Okay, but now if you have 50 neurons a column you have about 10,000 connections
  • fast_forward00:31:58 - so I could argue that in that setup
  • fast_forward00:32:01 - you can find basically any type of connection pattern you would like.
  • fast_forward00:32:05 - So in some sense the question is more about which obvious connection patterns are absent.
  • fast_forward00:32:11 - So which pattern of connectivity is absent that would support your hypothesis?
  • fast_forward00:32:18 - So, let me say it this way.
  • fast_forward00:32:22 - So, it is true that there is a zoo of connections, and we have to orient ourselves in it.
  • fast_forward00:32:28 - But there are a few things that help us. The first one is that each connection
  • fast_forward00:32:35 - has a multiple number of synapses in parallel.
  • fast_forward00:32:40 - So if neurons A and B have synaptic connection, there are usually tens or sometimes
  • fast_forward00:32:46 - more than a hundred synaptic contacts in parallel.
  • fast_forward00:32:49 - And so we can order connections in terms of their strength by using the number
  • fast_forward00:32:55 - of contacts as a proxy for connection weight.
  • fast_forward00:32:58 - And of course, initially we focus on the strongest connection.
  • fast_forward00:33:01 - So in some sense, we first look at the scaffolding of that network.
  • fast_forward00:33:07 - That's the first thing that we use.
  • fast_forward00:33:10 - The second is how that circuit is divided in between columns.
  • fast_forward00:33:19 - So the fly visual system, if you think about looking at a fly eye,
  • fast_forward00:33:25 - starts with about 800 so-called amatidia, which correspond to basically pixels
  • fast_forward00:33:32 - of the image that the fly sees.
  • fast_forward00:33:38 - And then each of those pixels is initially processed independently from the other.
  • fast_forward00:33:44 - And that forms the basis of the so-called column that has about 50 neurons.
  • fast_forward00:33:50 - That's the one I described.
  • fast_forward00:33:51 - And the processing structure is rather periodic.
  • fast_forward00:33:55 - So it's almost a crystalline structure of 800 units. Okay.
  • fast_forward00:34:00 - And we focused initially on the connection within the unit.
  • fast_forward00:34:06 - But if the structure were to perform motion detection, it has to correlate signals
  • fast_forward00:34:13 - from adjacent pixels, at least.
  • fast_forward00:34:17 - And therefore, there must be connections between the columns.
  • fast_forward00:34:20 - And so we know then that the connections that are necessary for motion detection
  • fast_forward00:34:26 - should span multiple columns.
  • fast_forward00:34:28 - And that's how we can determine which connections would have to be involved in motion detection.
  • fast_forward00:34:37 - So if we did not see any connections between columns at this stage,
  • fast_forward00:34:41 - we would be very surprised because then the system couldn't do motion detection.
  • fast_forward00:34:46 - Fortunately, we found such connections, and they are a natural substrate for motion detection.
  • fast_forward00:34:52 - Did you find any evidence for the famous Reichardt detector that sort of tries
  • fast_forward00:34:57 - to extract motion by correlating different input signals?
  • fast_forward00:35:01 - So I think this is a million-dollar question, of course,
  • fast_forward00:35:05 - and I think philosophically the Reichardt detection is correct in the sense
  • fast_forward00:35:10 - that the system is comparing a signal from one pixel with a delayed signal from an adjacent pixel.
  • fast_forward00:35:18 - But we seem to favor a slightly different form of such comparison,
  • fast_forward00:35:27 - which actually does not involve multiplication,
  • fast_forward00:35:31 - but contains another non-linearity that is necessary for forming a motion signal.
  • fast_forward00:35:38 - Which is what, the threshold? Erectification. Okay.
  • fast_forward00:35:43 - So this is pretty amazing, right? So sometimes you guys have it all now because
  • fast_forward00:35:48 - you have access to an exquisite data set of this brain. You have a theory.
  • fast_forward00:35:55 - Now you're trying to match. But in some sense, I could say, but maybe you're
  • fast_forward00:35:59 - barking up the wrong tree, right?
  • fast_forward00:36:01 - Because maybe the fly brain or any brain did not evolve to optimize signal transduction.
  • fast_forward00:36:07 - It just got optimized to generate behavior.
  • fast_forward00:36:09 - And all you're telling me now is how in this complex set of connections in the
  • fast_forward00:36:15 - fly brain, I can optimize a signal.
  • fast_forward00:36:18 - But at some point, this fly just has to go left or right or up and down the
  • fast_forward00:36:21 - land or whatever, or, you know, pursue the sugar.
  • fast_forward00:36:26 - So where does this mapping take place? How do I get behavior out of this and
  • fast_forward00:36:32 - also functionally relevant responses?
  • fast_forward00:36:36 - Right. So this is, of course, the hologram of neuroscience. How do you get from
  • fast_forward00:36:41 - sensory inputs to behavior?
  • fast_forward00:36:44 - And we think that we're moving in the right direction.
  • fast_forward00:36:48 - And there are two arguments that I can make.
  • fast_forward00:36:53 - Well, the first one is the idea behind those redundancy reduction and predictive
  • fast_forward00:36:58 - coding approaches is to come up with some theoretical framework which is not
  • fast_forward00:37:04 - based on the specific task that the animal has to perform, right?
  • fast_forward00:37:11 - So you say, well, I want to communicate information to the rest of the brain
  • fast_forward00:37:15 - domain as fully and as quickly as possible.
  • fast_forward00:37:19 - And then whatever task is needed to do will be possible to do if we preserve all the information.
  • fast_forward00:37:27 - So initially, of course, this requirement was viewed as the strength of a theory,
  • fast_forward00:37:31 - because you can come up with the predictions that were task-independent,
  • fast_forward00:37:36 - which I think is completely appropriate for the front end of the visual system.
  • fast_forward00:37:40 - As we are moving further into the visual system, of course, this is not good
  • fast_forward00:37:45 - anymore and we have to come up with task-specific computations.
  • fast_forward00:37:49 - And I think motion detection is the first step in that direction.
  • fast_forward00:37:53 - The second part of my answer is, you know, I would put it in the fallen way,
  • fast_forward00:38:05 - which I learned from talking with Sydney Brenner, who spearheaded the reconstruction of the C.
  • fast_forward00:38:14 - Elegans conictome about 30 years ago.
  • fast_forward00:38:18 - The explanation is the following. Whenever you come up with a model, there is no way to prove.
  • fast_forward00:38:26 - Fully and rigorously that the model is correct. You can only disprove models
  • fast_forward00:38:31 - by saying that they don't fit experimental facts, and if the model is not disproven,
  • fast_forward00:38:37 - it's still in the running.
  • fast_forward00:38:40 - But you always get questions, you know, how do you know that your model is the right one?
  • fast_forward00:38:47 - How come there couldn't be some other wire that sends information directly from
  • fast_forward00:38:54 - the photoreceptors to the avoidance response neurons in flies.
  • fast_forward00:39:00 - And that's exactly why we are doing a complete conic-tome reconstruction.
  • fast_forward00:39:04 - Because once we reconstruct all the connections in the fly brain,
  • fast_forward00:39:08 - we can answer that there is no other wire.
  • fast_forward00:39:13 - And so I think that if we combine the theoretical approaches with electrophysiology
  • fast_forward00:39:19 - and with behavioral tests with the conic-tome,
  • fast_forward00:39:22 - that's when we can say well no we
  • fast_forward00:39:26 - are not barking up the wrong tree um this is
  • fast_forward00:39:30 - a necessary condition the model would have to
  • fast_forward00:39:32 - use this particular pathway but now
  • fast_forward00:39:35 - that but there's one aspect that i'm missing because you could
  • fast_forward00:39:39 - also are you look certainly if you look at the insect case we know
  • fast_forward00:39:42 - that that after the medulla we start
  • fast_forward00:39:46 - to hit these whitefield neurons that we know physiologically show
  • fast_forward00:39:49 - responses that are highly adapted to the behavior of
  • fast_forward00:39:52 - the animal like you might have specific optic flow patterns you
  • fast_forward00:39:56 - might have approaching obstacles like time to contact type responses so in that
  • fast_forward00:40:02 - sense you just want in your in your model and in your reconstruction just one
  • fast_forward00:40:06 - synapse away from that response so isn't another test of your model to show
  • fast_forward00:40:12 - that which is one single synaptic step,
  • fast_forward00:40:16 - you can then generate these kinds of established and also behaviorally relevant
  • fast_forward00:40:21 - physiological responses of your wide-field neurons.
  • fast_forward00:40:24 - So how would that work in your model?
  • fast_forward00:40:28 - Yeah, so I think that I cannot give you a complete answer now because we haven't
  • fast_forward00:40:36 - done that part of the work,
  • fast_forward00:40:37 - but this is something that people have thought about in the motion detection
  • fast_forward00:40:41 - context because once you have elementary motion detection that detect motion
  • fast_forward00:40:46 - in a local part of the visual field,
  • fast_forward00:40:50 - then the inputs of those local detectors can be combined to produce a global
  • fast_forward00:40:58 - motion response in a given neuron.
  • fast_forward00:41:04 - And one example of those neurons are, for example, the neurons that are supposed
  • fast_forward00:41:10 - to reflect rotations around different axes,
  • fast_forward00:41:16 - in the flight of a fly, which require a very particular map of the motion response direction,
  • fast_forward00:41:26 - and they can be, of course, built by using motion detectors in different parts
  • fast_forward00:41:35 - of the visual field with different directional selectivity.
  • fast_forward00:41:39 - Right. But then, in your case, to extract those features, you would have to
  • fast_forward00:41:46 - tap into the filter cascade at multiple levels.
  • fast_forward00:41:50 - You cannot just read it out at a single level, if I understood it right.
  • fast_forward00:41:54 - It depends on how complicated a temporal sequence you need to predict the response.
  • fast_forward00:42:06 - I think for the motion detection, you need to just compare two points in time,
  • fast_forward00:42:13 - at least if I take a correlation-based framework of Reichert and Hassenstein seriously.
  • fast_forward00:42:20 - So I think that you should be able to do that relatively simply.
  • fast_forward00:42:24 - But of course, for more complex predictions, then you would have to tap into
  • fast_forward00:42:29 - the lattice filter on many stages.
  • fast_forward00:42:31 - Right. Okay. So that might be not a testable prediction that would come from this framework.
  • fast_forward00:42:37 - That's correct. Okay. So that means if in your reconstruction of this fly brain,
  • fast_forward00:42:42 - you do not find a very wide multi-scale readout from these wide-field neurons
  • fast_forward00:42:48 - in this whole column or this set of columns,
  • fast_forward00:42:53 - then the cascade filter might not be the way the problem is solved.
  • fast_forward00:42:56 - That is true, of course. But I have to say that our observation is that,
  • fast_forward00:43:03 - unlike engineering application of lattice filters, for example,
  • fast_forward00:43:06 - in speech processing, it's not uncommon to have a hundred or more stages of the lattice filter.
  • fast_forward00:43:13 - We think in the brain, there aren't as many stages.
  • fast_forward00:43:17 - And just having a few stages can accomplish a lot because the processing in
  • fast_forward00:43:25 - the brain is applied not on a single channel level,
  • fast_forward00:43:29 - because those columns actually interact with each other the further you go into the system.
  • fast_forward00:43:40 - Then the filter can accomplish a lot, actually, with just a few stages.
  • fast_forward00:43:45 - Okay, but that's the kind of compression of such a filter that the engineers
  • fast_forward00:43:48 - haven't really tried yet.
  • fast_forward00:43:50 - Well, I cannot say that they haven't tried, but it's certainly not just a textbook version of a filter.
  • fast_forward00:43:58 - I don't know, it may exist in the literature on some level.
  • fast_forward00:44:02 - I think one thing that it seems that the brain is using all the time that is
  • fast_forward00:44:11 - rare in engineering is the use of nonlinearities.
  • fast_forward00:44:16 - And I think that's one thing we can learn from brains. But I guess engineers
  • fast_forward00:44:22 - have good reasons not to use them.
  • fast_forward00:44:25 - Because it complicates their lives. Exactly. It's difficult to analyze and understand. Exactly. Right.
  • fast_forward00:44:31 - And nature doesn't have these kinds of scruples.
  • fast_forward00:44:34 - So we also touched upon this whole issue of, let's say, optimal encoding frameworks
  • fast_forward00:44:41 - versus finite capacity models, right?
  • fast_forward00:44:46 - So the adaptive filter is more finite capacity, because you start to squeeze
  • fast_forward00:44:49 - as much information as you can through a channel that's this bottleneck.
  • fast_forward00:44:54 - But in optimal coding frameworks you start to sort of you're not too worried
  • fast_forward00:44:58 - about that problem right you just worry about how do I sort of compress my information
  • fast_forward00:45:03 - in an optimal way in sort of information theoretical terms,
  • fast_forward00:45:07 - so do you see this as contradictory approaches or do you see this as complementary
  • fast_forward00:45:12 - do you see this column of 50 cells in the thigh brain maybe doing both or is
  • fast_forward00:45:17 - it really sort of more an exclusive choice that we have to make here.
  • fast_forward00:45:22 - I'm not sure I got the exact... Well, in the engineering literature,
  • fast_forward00:45:29 - these would be seen as different approaches, right?
  • fast_forward00:45:32 - It's possibly also contradictory, whether you deal with optimal coding or with
  • fast_forward00:45:36 - finite capacity channels.
  • fast_forward00:45:39 - Oh, yeah. Yeah, so I think that at this point, the experimental evidence is
  • fast_forward00:45:48 - just maybe barely sufficient to make such a fine distinction.
  • fast_forward00:45:55 - But that's currently, you know, we're investigating that currently and see if
  • fast_forward00:46:00 - we need additional experiments to make that distinction clearly. Right.
  • fast_forward00:46:05 - So the other thing is, if we now go back to the vertebrate case,
  • fast_forward00:46:08 - we have the retina, now you have the LGN, you give us a model of how we can think about this.
  • fast_forward00:46:12 - There's of, let's say, optimally decorrelating these inputs.
  • fast_forward00:46:17 - And now we hit the cortex. And then you could argue, well, now the cortex has
  • fast_forward00:46:21 - this perfectly massaged signal.
  • fast_forward00:46:23 - So that means from a signal processing perspective, the game for cortex should
  • fast_forward00:46:27 - become a different game, right?
  • fast_forward00:46:29 - Because now it's It's optimally decorrelated. It's a perfect signal.
  • fast_forward00:46:33 - So from the perspective of an adaptive filter, your cortex, so this higher level
  • fast_forward00:46:37 - processing story, will be playing a different game.
  • fast_forward00:46:40 - What would that game be if you would have to guess today?
  • fast_forward00:46:44 - Well, I guess I would have to speculate at this point.
  • fast_forward00:46:49 - But I think the way I would put it is that if the goal of predictive coding
  • fast_forward00:46:59 - is to de-correlate the signal as much as possible,
  • fast_forward00:47:05 - then the stages in the retina and the LGN take you as far as possible with linear filters.
  • fast_forward00:47:14 - Yes, there are non-linearities in neurons along the way, but there is also evidence
  • fast_forward00:47:17 - they can conspire in a way to generate a linear response.
  • fast_forward00:47:24 - Now, when you get to the cortex, the responses of the cells are decidedly non-linear,
  • fast_forward00:47:31 - as for example exhibited by the complex cells in V1.
  • fast_forward00:47:35 - And so what I think is happening is that the cortex may be continuing the job
  • fast_forward00:47:43 - of the decorrelating of the input stimulus and the feature construction,
  • fast_forward00:47:49 - which the lattice filter stages were doing before,
  • fast_forward00:47:53 - but invoking additional non-linearities to make even better predictions and
  • fast_forward00:48:01 - make the outgoing signals even more independent than is possible with linear filters.
  • fast_forward00:48:07 - Okay. Or possibly actually start to correlate again, to group features together in meaningful ways.
  • fast_forward00:48:14 - To predict function. Yeah. Predict behavioral output, yes. Yeah.
  • fast_forward00:48:22 - Okay, that's great. So we made progress here in understanding sensory systems.
  • fast_forward00:48:28 - So, but now, Dimitri, so to finish up, we always have two questions,
  • fast_forward00:48:34 - right? So you come from theoretical physics.
  • fast_forward00:48:36 - You have been working very hard on, let's say, the anatomy of these brains,
  • fast_forward00:48:42 - so you know really how hard that is.
  • fast_forward00:48:44 - And then you try to combine that now with also theoretical work,
  • fast_forward00:48:48 - which I think is actually the only way forward, right?
  • fast_forward00:48:49 - Look at all this detail within anatomy.
  • fast_forward00:48:53 - So on the basis of experience, what would be Dimitri's law that we should all
  • fast_forward00:48:57 - follow in our aims to understand the brain and behavior?
  • fast_forward00:49:09 - That's a tough question.
  • fast_forward00:49:18 - So, I come from a tradition in theoretical physics which favored starting with a simple,
  • fast_forward00:49:31 - and intuitive model of even the most complex phenomena that one could be studying.
  • fast_forward00:49:37 - And the reason for doing this is, I think, because our brains are better suited
  • fast_forward00:49:46 - at analyzing the simple models.
  • fast_forward00:49:50 - And by simple, I mean models involving very few relevant parameters.
  • fast_forward00:49:56 - And if you build such a model, it kind of prevents you from overfeeding experimental
  • fast_forward00:50:03 - data. And also, it makes all the assumptions very transparent.
  • fast_forward00:50:08 - And I think that my.
  • fast_forward00:50:15 - My style of work in theoretical neuroscience is strongly based on that tradition
  • fast_forward00:50:22 - in theoretical physics,
  • fast_forward00:50:23 - where the simplicity and the clarity of the model is the main driving force.
  • fast_forward00:50:33 - And so, even when studying such complex things like the brain and how it computes,
  • fast_forward00:50:42 - I would like to start with the situations that can be broken down on the very simple level involving,
  • fast_forward00:50:51 - very few variables and few or no adjustable parameters.
  • fast_forward00:50:57 - And that's why we focused on the sensory system.
  • fast_forward00:51:02 - And only once we get a foothold there, I think then we can move on further.
  • fast_forward00:51:09 - But again, chipping away a small and digestible part of the problem that we
  • fast_forward00:51:17 - can solve in a clear and intuitive way, and then move on.
  • fast_forward00:51:22 - Okay, so Dmitri's Law is keep it simple. Pretty much.
  • fast_forward00:51:25 - Okay, very good. Good. And the second one, so if I'm going to get you back here
  • fast_forward00:51:29 - five years from now, since I'm such an unpleasant person, I would like to remind
  • fast_forward00:51:33 - you of the predictions you have been making.
  • fast_forward00:51:36 - So what's the one prediction that you feel most strongly about today?
  • fast_forward00:51:41 - I should remind you of five years from now to ask you, does that really come out?
  • fast_forward00:51:47 - What's that one prediction I should remind you of five years from now?
  • fast_forward00:51:55 - So I think that our strongest predictions are the shapes of receptive fields of the LGN neurons,
  • fast_forward00:52:05 - and although some of them have been seen already, I think that there are more
  • fast_forward00:52:16 - details there that the lattice filter model contains,
  • fast_forward00:52:20 - but haven't been completely verified experimentally, and in particular on the
  • fast_forward00:52:26 - circuit level, I think we have a model for computation,
  • fast_forward00:52:33 - but how it is implemented in terms of individual neuron synaptic properties.
  • fast_forward00:52:40 - For example, in LGN, there is a structure called triadic synapse and so on.
  • fast_forward00:52:44 - On how that structure builds up the receptive field, the lattice filter predicts, is not clear.
  • fast_forward00:52:53 - But the prediction would be
  • fast_forward00:52:55 - that it is exactly the computation that is proposed by the lattice filter,
  • fast_forward00:53:02 - which is an all-pass filter, which has a frequency-depending phase delay.
  • fast_forward00:53:10 - Right. Very good. Good. So, Dmitry Slotsky, thank you very much for this conversation.
  • fast_forward00:53:14 - Thank you very much, Paul, for having me here. Great.
  • fast_forward00:53:20 - The CSN podcast was produced by the Convergent Science Network of Biometrics
  • fast_forward00:53:26 - and Biohybrid Systems, a project funded by the European 7th Research Framework Program.
  • fast_forward00:53:32 - Music.

Be the first to leave a comment

Leave a comment

Your email address will not be published. Required fields are marked *

Convergent PozitivLinia Logo

Exploring the convergence of neuroscience, robotics, and AI through conversations with leading researchers since 2010.

A project of the Convergent Science Network Foundation.

© CSN Podcasts. Developed by IMCreativeWEBC

0%

Login to enjoy full advantages

Please login or subscribe to continue.

Go Premium!

Enjoy the full advantage of the premium access.

Stop following

Unfollow Cancel

Cancel subscription

Are you sure you want to cancel your subscription? You will lose your Premium access and stored playlists.

Go back Confirm cancellation