Chapter 18 Chapter Sixteen Speculations

"I'd rather make an ongoing mistake whenever it's filled with the seeds of continual self-correction. And you go with your dead truth!" —Vilfredo Pareto① At a given moment, the firing of certain neurons is associated with certain properties of visual perception.The experiments outlined so far will help us identify these neurons.A region of the visual cortex on one side of the monkey's brain has about half a billion neurons.Are there some clues that could lead us to the neuron we're looking for? There is a possibility that although only a fraction of these neurons will become awareness neurons at any given moment, all of them have the potential to fulfill this role.This seems unlikely given the behavior of neurons during binocular rivalry (see discussion in the previous chapter).But it's also possible that some nonaware neurons do this in certain situations.It is more probable that there are several forms of visual perception, perhaps very brief for simple features, and longer lasting for vivid visual perception; perhaps there is a deeper form which It is indeed related to vision, but does not correspond to visual "images" that seem to appear in the brain.I have touched on this issue when I outlined David Marr's views (see Chapter 6) and Jackendorf's views (see Chapter 14).To simplify matters, let us focus on vivid visual perception (Jackendorf's point of view is roughly equivalent to Marr's 2.5-dimensional map at this time).

A remarkable feature of the internal picture of our visual world is that it is fairly well organized, and psychologists will be happy to show us that it is not as regular as we often imagine—that is, our perception of relative size and Judgments of distance are not always as precise as an engineer's drawing.But in general, we rarely confuse them when we observe our surroundings.The fact that the real external world is always there is a fact that the brain can use to test any tentative judgments it might make.But even so, when our brains generate a symbolic representation of the visual world in front of us, the representation is still very well organized spatially.

It would not be very surprising if neurons at all levels of the visual hierarchy were sensitive to the precise location in the visual field of the features they responded to.But we have seen that this is not true.Some neurons responded particularly well to complex objects, such as a face, regardless of whether the face was directly in the center of the animal's gaze, or slightly to one side, or even further than its normal straight-up position. The responses were almost as good.This is reasonable.For all high-level features, it is almost impossible to have a separate neuron corresponding to each possible location.It is impossible to have enough neurons for this task.

Neurons in V1, on the other hand, are indeed sensitive to the precise location in the field of view of features of interest (such as orientation, motion, color, parallax, etc.).They can do this because those features are relatively simple and fixed.At the same time, this is also due to the fact that V1 is particularly rich in neurons that process features that appear near the center of gaze. In 1974, psychologist Peter Milner published an insightful article(1).In the article he argues that, for the above reasons, the primary visual cortex (such as area vi) is also as tightly involved in visual perception as the higher visual cortex.The mechanism by which this is achieved, he speculates, may involve extensive feedback from neurons higher up in the visual hierarchy to lower layers.The exact function of these feedbacks is unclear.Since they are connections between the cortex, they all come from neurons that transmit excitability.The key question is how strong they are.Opinions vary.One possibility is that, while these feedbacks are sufficient to regulate any firing elicited by other inputs, they are often not strong enough alone to make cells fire rapidly enough.This may mean that its effect is too weak to have an impact on the subsequent stages. If area C is back-projected to area B, and B is also back-projected to area A, people may suspect that unless from C has a direct feedback to A, otherwise the events that occur in C can have a sufficient impact on A through B indirectly.We diagram this as:

A<————————B<———————————C (Only the return path is shown). Can C affect A?Perhaps we need an additional pathway (shown above the other two pathways) to do this? ↙………………<………………↖ ↓↑ A<———————————B<—————————————C So we ask, which cortical areas in the monkey brain directly back-project to V1? Referring to the connectivity schematic in Figure 52, we can see that almost all visual areas no higher than the V4 and MT levels do have direct connections back to v1, while most levels higher in the hierarchy do not.Does this mean that only the neurons in the lower part of Figure 52 are directly related to vivid visual perception?

Since cortical V2 is also large and has a complete retinal counterpart, perhaps we should consider, as an alternative, only those areas that project back to V1 or V2.This would involve more cortical areas, but not the infratemporal areas (those whose names begin with IT). I believe these views contain some truth, but the arguments are flimsy and cannot serve as a basis for our exploration.It hints, but it's not convincing.In addition, more recent work has shown that more cortical areas project back to V1 than originally thought.At this point it is best to keep this in mind for a while while exploring the problem, but not to believe in it too much.At this stage, it is of utmost importance to learn more about the anatomy and behavior of these numerous cortical feedbacks.

Another possible strategy is to investigate whether awareness in some sense requires the participation of the brain in communicating with itself.In neurological terms, this might mean that, as Gerald Edelman once proposed, reentry pathways to return to the starting point after one or more steps are essential.The problem, however, is that it is difficult to find a pathway that is not re-entrant.From this judgment rule, the hippocampus is the exact location of consciousness (reentrant since most of its input comes from the entorhinal cortex, and most of its output goes back there), but it is not.This negative result suggests that we must use the reentry rule with care.

In its simplest form, re-entry pathways may arise between just two cortical areas.For example, region A projects to region B, and B projects to A; but usually this always happens, and it doesn't help us much.Can we make the idea of ​​reentry more precise and more useful? Recall that for many cortical regions, if region A projects to layer 4 of B, then B does not project to layer 4 of A.Backcasting avoids that layer.We can represent this notationally as: <——————— AB ———> Among them, the solid arrow indicates "entering the 4th layer".This suggests that we only need to look for two cortical regions that project to each other into layer 4 in far fewer cases, using the convention above as

<————— AB —————> In the hierarchy of Figure 52, this is the case between cortical regions on the same hierarchy, but not always. MT, V4 and V4t are obvious examples. This point of view appeals to me.It is easy to get some theoretical arguments that give it some academic status.Unfortunately, the fine neural interconnection details of this so-called layer 4 have not been carefully studied.This view is indeed worthy of attention. Let's try a rather different approach.So far we have mainly been talking about cortical areas.Can we go a step further and try to guess which layers of the cortex might be involved in representation awareness?Or even further, what types of neurons might be involved in these layers?Now we do have the few scattered pieces of evidence.

One class of cortical neurons that stands out is some pyramidal cells in layer 5.They are the only neurons that project outside of the cortical system (by cortical system I mean the cerebral cortex and areas closely related to it, such as the thalamus, claustrum, etc.).It might be argued that what passes from one part of the brain to the other should be the result of neural computations.I have already said that visual perception may correspond to a subset of these results.This made people wonder about these special pyramidal cells, do they have other unusual properties? (After all, what scientists call "proof" is the eventual agreement on many apparently disparate aspects of an object or concept.) In fact, some of these neurological incompetences fire in a peculiar fashion.Many neuroscientists have found that such neurons ① tend to fire in "clusters".They injected electrical current into many different individual neurons on slices of the cortex and found three types of firing patterns.The first corresponds to inhibitory neurons, the second most pyramidal cells, but the third neurons appear to be mostly the larger pyramidal cells of layer 5, which tend to cluster in this environment issued.The apical dendrites of these cells extend to the top layer of the cortex (layer 1), where they may receive input from the aforementioned backprojections.

All this evidence is rather crude, and still raises the question of whether these layer 5 pyramidal cells are closely related to awareness.Even if the pyramidal cells of layer 5 do express the "results" of cortical computations, it does not follow that some form of awareness occurs when all such neurons fire in various cortical areas.Other mechanisms may be required to develop awareness, for example, some special form of short-term memory, such as the reverberant loop discussed later in this chapter. While these ideas are speculative, it does outline the importance that when a neuroscientist reports the results of certain experiments, he should know the layer and, if possible, the neurons he is recording from. What type of neuron is it.This is often technically difficult when studying alert animals, although new, more elaborate methods can make it easier. A more general view is that we should pay closer attention to the various layers of the cortex.Although a neuron's dendrites and axons often extend into several layers, which layer the cell body is located in may be genetically determined during normal embryonic development. (In contrast, the details of a neuron's connections are largely influenced by its experiences.) If there are indeed some special types of cortical neurons whose firings correlate with what we see, we might expect that the cell bodies of these neurons Located in only one or a few cortical layers or sublayers. The brain tries to make sense of the information coming into the eyes and to represent it in a compact, well-organized way, the result being visual perception.But unless it's really useful to the organism, there's no need to do it.It may be required in several different areas.Where is this information sent in the brain?Two prominent sites are the hippocampal system (including the temporary storage or encoding of event memories) and the motor system (especially its higher planning levels).Can we trace back connections from these two destinations to localize visual perception on the cortex? Unfortunately, currently this approach creates more difficulties than it solves.Visual perception is likely to be combined at some stage with information from other senses, such as hearing and touch.When you drink a cup of coffee, you can feel the appearance and feel of the cup, as well as the smell and taste of coffee.Higher visual areas do project to multisensory cortical areas.It is unclear which of the vivid surface visual perceptions of 2.5D sketches, and the less visual information of 3D models, is more closely related to the type of visual perception sent to the hippocampus and motor system.Perhaps both are needed. The anatomical connections between cortical visual areas, multisensory areas, and hippocampal structures are well understood (see Figure 52).They clearly show that visual areas such as V4 and MT, as well as the infratemporal cortex, do not project directly to the hippocampus.Visual information has to pass through other cortical areas to get there.Unfortunately, our current understanding of the behavior of neurons in these regions is rather superficial and further research is needed. The pathways to the motor cortex have been studied somewhat, but much remains to be done.In addition, there are other routes that reach the motor cortex more indirectly.There are numerous pathways from the cortex to the striatum, and interestingly these connections also come from some of the pyramidal cells in layer 5.From there, the information travels to part of the thalamus and on to multiple motor and premotor areas of the cortex.There is also a pathway from the cortex to the cerebellum, then back to the thalamus, and then to the cortex.Some of these pathways may be involved in "unconscious", rather mechanical activities.Much more experimental study of these parts of the brain is needed if we hope to understand various forms of vision and other sensory perceptions. The firing of awareness neurons may often be the result of relevant neural network decisions, which is one of its characteristics.Making a fair compromise is a linear process, whereas making a sharp decision is highly nonlinear.For example, electing the president of the United States is a non-linear process, while proportional representation is more linear, at least after everyone has cast their vote.The behavior of neurons and, by extension, neural networks is highly nonlinear, and there is no difficulty in principle. For neurons, this mechanism is likely to be a winner-take-all process like a presidential election-that is, many neurons compete with each other, but only one (or very few) can win, which means that its firing is more important. to fire more intensely, or in some particular fashion, while all other neurons are forced to fire more slowly, or not at all. This is easily achieved in artificial neural networks by making each neuron have an excitatory output while suppressing all other competitors.The most active neurons can hopefully overwhelm all opponents (like in elections!) But it is not so simple for real neurons, since most of the time the output of a single neuron can only be excitatory or Inhibitory, but not both.There are many strategies that might circumvent this difficulty. For example, if all excitatory neurons stimulate an inhibitory neuron, which in turn inhibits all excitatory neurons, then the neuron with the largest average inhibitory advantage has Likely to be a winner.Designing a neural network that satisfactorily executes the winner-take-all operation is tricky, but it can be done, especially if more than one winner is allowed. There seems to be no reason to think that nature has not evolved such a mechanism.The problem is how to discover the exact location in the brain where this operation is taking place.So far we don't know enough about the highly complex local circuits in and around the cortex to be of great help.Of course this will change as our knowledge increases.One might find that the neural interactions within the cortex are so complex that no simple mechanisms are involved.But it is also possible that this critical process employs some special neural strategy, and all we can do is keep an eye out for promising signs.The problem is complicated by the fact that awareness does not always require a decision between two or more options (like looking at a Necker cube).In other cases, it may be more effective to reach a compromise between information from different sources, such as using different depth cues to judge the distance of an object in the field of view.Conversely, decision-making is essential in determining whether an object is in front of another object and partially occludes it. So far we have had quite a few clues on which to rely on perceptual neurons, although it points in some promising directions.Do we have more avenues to follow?Could studying the neural mechanisms of short-term memory teach us something useful about visual perception?In fact it seems certain that without short-term memory we would not be conscious, but how short should it be and what are its neural mechanisms? Recall that there are two main types of memory.When you actively recall something, somewhere in your brain neurons must be firing to express the memory.However.You can remember many things, such as the Statue of Liberty, or your birthday, but at a certain moment you are not recalling them. Normally, this underlying memory does not require the relevant neural firing.In storing memories, the strengths (among other parameters) of many synaptic connections are altered so that, given appropriate cues, the desired neural activity can be regenerated.This way the memory is stored in the brain. Active recall or latent memory, which (of these two forms of memory) is involved in the very short-term memory we are interested in?More likely is an active form of memory, that is, your immediate memory of an object or an event is likely to be based on active firing of nerves.How did this happen?I think there are at least two possible ways. Due to certain intrinsic properties of a neuron, such as its many ion channels, once it is fired, it may continue to fire.The firing persists for a while and then fades, or the neuron fires until it receives some external signal that stops it firing.The second mechanism is quite different, involving not only the neurons themselves, but also how other neurons are connected.There may be some "reverberating loops," a closed loop of neurons in which each neuron fires the next neuron, and keeps that activity looping.Both mechanisms are possible and they are not mutually exclusive. Also, is it possible to have some underlying form of short-term memory?This would mean that the participating neurons would start firing when stimulated, and then stop firing: but if there was a strong enough cue to awaken the underlying memory to become active, those neurons would quickly start firing again.But how could this happen unless the first round of distribution left some traces in the system?Perhaps, instantaneous changes in the relative synaptic strength (or other neuronal parameters) could embody this short-lived underlying memory over a short period of time?In fact, is there experimental evidence for such a momentary change in synapses?Incidentally, this variation was proposed by Cristoph vonder Malsburg in a rather obscure theoretical paper mentioned earlier. Unbeknownst to Christophe, there had been some experimental evidence of transient synaptic changes.They were first discovered in the 50s and are located where nerves and muscles join (that is, where the nerve that fires the muscle meets that muscle), far from the brain.Shortly afterwards, similar transient synaptic changes were found in the hippocampus (for review, see ref. 6).When an axonal pulse reaches a synapse, it alters the synapse almost simultaneously so that the synapse increases in strength.A rapid pulse train produces a larger increase.This increase in synaptic strength then decays in a complex fashion, some as fast as about 50 milliseconds and slower for a fraction of a second to a minute or so.This is exactly the time involved in short-term memory.There is also some evidence that this also occurs at synapses in the neocortex.It appears that this is primarily caused by changes in the input side of the synapse (the presynaptic side), and may involve nearby calcium ions, as well as the movement of synaptic vesicles near the synaptic junction. ① Whatever the cause, it almost certainly exists.Its size is perceptible. Unfortunately, very little work has been done on these transient changes, mainly because long-term changes in synaptic strength (a currently hot topic) are easier to study.Most theoretical work on neural networks does not consider this case either.We are thus in a strange situation: a phenomenon that may be of great importance to consciousness (especially visual perception) has been neglected by both experimentalists and theorists. Perhaps this momentary change in synaptic weight is also important for briefly maintaining the echo loop.This increase in synaptic strength helps the circuit maintain its reverberant firing. How to prevent this continuous release from overspreading and affecting other circuits is a more difficult problem.There are so many complex circuits in the brain that pinpointing the exact location of a reverb circuit, if it exists at all, is nearly impossible.Is it possible that this type of reverberation (associated with active short-term memory) occurs in only one or a few specific locations?Is there any indication that such circuits are constructed somewhat isolated from nearby circuits of the same form, so that memories do not spread in an uncontrolled fashion? One circuit is thought to be possibly involved in very short-term memory.It projects from the thalamus to a type of pyramidal cell in layer 6 of the cortex, which in turn signals back to the same part of the thalamus.Both thalamic and cortical neurons have very few laterally extending axonal collaterals, so they likely have very few interactions with their immediate neighbours.This gives them the partially isolating properties just mentioned. Research on the pathway has focused on cortical V1 area and its connection to the lateral geniculate body.The anterior pathway from the lateral geniculate body to the pyramidal cells in layer 6 appears to be weak.The return pathway from layer 6 to the lateral geniculate body has an extremely large number of axons, perhaps 5 to 10 times as many as the main forward connection from the lateral geniculate body to layer 4.That's surprising in itself, especially since it's so hard to discover what functions they have.However, most experiments on this pathway have been performed with animals anesthetized; very short-term memory may be weak or non-existent, and the animals are therefore unconscious.In the article mentioned a few pages earlier, Livingstone and Huber noted that they found that the activity of neurons in the lateral geniculate body decreased during slow-wave sleep.This can have an impact.Although signals could travel from the lateral geniculate body to cortical V1 (as they found), these signals were not large enough to sustain any reverberant activity.It is now known that there are pathways from the brainstem that alter the activity of the lateral geniculate body (and, by extension, the activity of other parts of the thalamus) during slow-wave sleep. It could then be hypothesized that these layer 6 neurons are closely related to a key element of consciousness—maintaining echo circuits that embody very short-lived memories.This is consistent with the earlier general notion that activity in predominantly lower cortical layers is generally associated with consciousness, and visual perception in particular. Might there be such reverberating loops associated with all cortical areas?In other words, do all cortical areas have pyramidal cells in layer 6 that project to some part of the thalamus and from there project back to those same layer 6 pyramidal cells?Unfortunately, we don't fully understand this yet.Perhaps only the lower and intermediate levels of sensory processing (which have perceptible layer 4) have the layer 6 echo loops required for this form of short-term memory.This is what Jackendorf calls conscious awareness.Perhaps a stronger input to layer 4 could energize the reverberating loop in layer 6 even more.If all of this turns out to be true, this meaningfully connects brain structure and Jackendoff's hypothesis.The possibilities are exciting. Let's put those speculations aside for now.Is there evidence that the sustained firing of neurons is associated with some forms of short-term memory?Building on previous work, Patricia Goldman-Rakic ​​of Naru University and her colleagues conducted such an experiment.They trained a monkey to gaze at a spot in the center of a TV screen while a target stimulus was randomly presented elsewhere on the screen.When the target was no longer presented, after a delay, the monkey was asked to move its eyes to where the target had been.The experimenters studied the responses of visual neurons in the prefrontal area of ​​the animals' brains.Usually when an object is presented in a particular place on the screen, one specific neuron will respond to it, while other neurons will respond to objects in different places on the screen.Strikingly, the neurons typically kept firing many seconds after the stimulus was withdrawn, until the monkeys responded.Moreover, if the activity was not maintained (which happens occasionally), the monkeys were likely to make mistakes.In short, it appears that these neurons are part of a working memory system that corresponds to visually specific spatial locations. ① There may be such systems elsewhere in the brain that correspond to other types of working memory.Thus we have at least one instance in which persistent firing of neurons is involved in short-term memory,2 although the evidence in other cases is doubtful. Note that this is a single task, so the monkey may be repeating the task in the brain with a delay. It is not known what the neuron activity will be if the monkey has to perform two different tasks.Nor do we understand the neural mechanisms that maintain this sustained firing.Like the study of attention, we can say that the study of the neural mechanisms of short-term memory has begun, but a lot of experimental work is still needed to unravel its mysteries. ①Vifredo Pareto (1848-1923), Italian economist and sociologist, he applied mathematics to economics after Wallace, and his elite theory of society had a great influence on the later Mussory Ni's fascist party has a great influence. ——Translator's Note ①This article was written while he was on vacation, and is not well known.Neither Christopher Koch nor I have heard of it.Fortunately, in 1991 we attended a conference with Peter in Arizona and he spoke to us about this almost forgotten article.In this article he also presents a related distribution point of view to solve the bundling problem.Over the years Stephen Grosberg, Antonio Damasio, Simon Ullman, and others have advanced similar ideas about the function of these return pathways. ①The axonal pulses produced by these neurons are not completely regular, but the time intervals are not random; instead, they tend to produce short clusters of several pulses at a time, with longer intervals between different clusters, And very little to no pulse. ① If they are only presynaptic—they do not depend on everything happening on the postsynaptic side—they cannot be Hebbian, as von der Marsberg claims.Whether there is a Hebbian-type transient change is still under study.Non-Hebbian transient changes are still under study.Non-Hebbel instantaneous changes have been ignored by theorists for a long time. ① They also used the 2-deoxyglucose technique to show that areas connected to the prefrontal cortex, such as the hippocampus, posterior parietal cortex, and the dorsal mid-thalamus nucleus, were more active during such tasks. ②Unfortunately, the way these neurons fire does not prove the existence of an echo loop.