Chapter 19 Chapter 17 Oscillator and Processing Unit

"Prediction is a difficult thing, especially if it involves the future." So far I've said little about possible solutions to bundling problems.Different features of an object (or event) correspond to different neuron firings in the brain.The bundling problem is how to tie these neurons together.This problem is especially pronounced if more than one object is perceived at a perceptual moment.The importance of binding is that it may be necessary for at least some types of awareness.It was mentioned in Chapter 14 that bundling may be achieved by relative firing of related neurons.A very simple form of correlated firing is one in which all involved neurons fire simultaneously in a rhythmic pattern (although rhythm is not essential to correlation).Figure 57 is an idealized example showing a neuron bursting every 100 milliseconds with a frequency of about 10 Hz.Rhythms with frequencies around this are called "alpha rhythms".This rhythm, as well as others, can be detected in brain waves recorded from the scalp (ie, an electroencephalogram, or EEG), a rather disjointed signal.Is there experimental evidence for correlated firing in populations of neurons?

It has been known for some time that correlated firings in the form of oscillations occur in the olfactory system, but until recently such oscillations were clearly observed in the visual cortex.The most exciting results come from two research groups in Germany.Wolf Singer, Charles Gray and colleagues in Frankfurt observed oscillations in the visual cortex of cats.These oscillations are in the 35 to 75 Hz range and are often referred to as "gamma oscillations" or, less precisely, "40 Hz oscillations".This oscillation was independently observed by Reinhard Eckhom in Marburg and his colleagues.They were able to observe this phenomenon particularly clearly, using an electrode designed to detect "field potentials."Roughly speaking, field potentials show the continuously varying average activity of a group of neurons near an electrode, much like the chatter you hear in a crowd at a cocktail party.

These experiments are relatively new, and newer experimental results are still emerging. Here, I only give a very simple description. As mentioned above, when appropriate stimuli appear in the visual field, some neurons in the visual cortex will become active and fire in a certain rhythmic form.The average local electrical activity (field potential) in their vicinity often appears as oscillations in the 40 Hz range.The pulses emitted by such neurons do not appear randomly, but are "in tune" with local oscillations (see Figure 60).A neuron fires in short bursts of two or three spikes in time, or sometimes it may not fire at all; but when it does fire, it is often approximately synchronous with some of its neuronal "companions."These oscillations are not very regular.Their waveform is more like a roughly drawn wave than a very regular mathematical wave with a constant frequency.

Singer and colleagues have often found that when recording from two electrodes that are not too far apart, if neurons near one electrode fire, they tend to synchronize with the firing of neurons near the other electrode, even when The field potential may also have in-phase oscillations with two electrodes separated by up to 7 mm.But this happens more often when the motor stimulus that excites them belongs to the same object rather than two objects.It's just that the experimental evidence to support the last statement is quite scant.In addition, experiments have shown that the movement of light rods can cause the rhythmic firing of the same phase in the corresponding positions of the first visual area and the second visual area, which just shows that synchronization can occur between neurons in different cortical areas.In addition, experiments have shown that synchronization can occur between the cortex of the two hemispheres of the brain.

Both research groups in Germany believe that these 40-hertz oscillations may be the brain's answer to the problem of bundling.They propose that neurons that mark all the different properties of the same object (shape, color, motion, etc.) tie these properties together by firing in sync.Koch and I took this idea a step further, arguing that this synchronous firing in time with (or around) the Y-oscillations (in the 35 to 75 Hz range) might be the neural correlate of visual perception.Such behavior would be a special case of correlated issuance proposed by other theorists. We also suggest that the main function of the attention mechanism may be to select an object to be attended to and then help to synchronously combine all neurons corresponding to the brain's best interpretation of this part of the visual input.We suspect that the thalamus is the "organ of attention," and that parts of it control the "searchlights" of attention that jump from one salient object to another in the visual field.

These pioneering experiments were performed while cats were lightly anesthetized, no oscillations were observed in cats very deeply anesthetized (using barbiturates), but neuronal activity was extremely reduced anyway , so the result itself is not very informative.Recent experiments have been carried out on conscious cats (Charles Gray mentioned this in a private correspondence with me).Oscillations at 40 Hz are also present here, so the oscillations are not an anesthesia-induced artifact.Some new experiments using lightly anesthetized monkeys also found oscillations in cortical area VI.Experiments in the cortical MT area of ​​awake monkeys showed that oscillations were observed when a motion stick was used as visual input, but not when presented with a pattern of pseudorandomly moving dots.There is currently no explanation for this difference in behavior.It is more likely that oscillations are involved in figure/background discrimination than in visual perception.Eberhard Fetz and colleagues also clearly observed oscillations in the motor/somatosensory cortex of awake monkeys, especially when performing a complex operant task requiring attention. when.

The observed oscillations are usually rather brief.Their duration often depends on the length of presentation of the visual signal used.Correlated oscillations between groups of neurons at different locations last only a few hundred milliseconds, as some theories predict.In general, it is difficult to convince people that the vivid picture of the outside world in our minds depends entirely on such messy and difficult to observe neural activity. You may feel confused now, like a police officer in the early stages of a difficult murder case.There are many clues here, but none convincingly point to a possible solution to the mystery.This is the kind of police work that the public least appreciates—the systematic, laborious pursuit of many rather weak leads.The same is true for the scientific inquiry into visual perception, we all want to know the answer, but we cannot find it without carefully examining the different "traces".There may be many clues that turn out to be misleading or even downright wrong.

From all these considerations we know that there may be several forms of visual perception; and, by extension, that there may be even more forms of consciousness in general.Can we find some way to relate these different forms of visual perception to the structure and behavior of the primate visual system? Recall that I have described three possible phases of visual processing: a very brief phase, roughly corresponding to Marr's elemental map; levels; there is also a three-dimensional object-centric process that does not correspond to what we actually see, but rather certain conjectures about the objects we see.I vividly saw the outline and visible surface of a particular object, which indicated that it was a teacup, with the inferred 3D shape.Usually see this word includes such two usages.If I say "Do you see that cup over there?" I use the word look in two senses.I may simply be referring to the visible surface of the cup presented to me, but I may also be referring to the inferred three-dimensional shape of the entire cup.Note that 2.5-dimensional graphs and 3-dimensional models are two inferences of a class of problems, ie they both have interpretations of this visual input, and both can be wrong.Our everyday use of words may not accurately describe the real behavior of the brain.

There is an idea that there is some thalamic region corresponding to each level of visual processing, (1) which I call the processing hypothesis.What do cortical areas that receive input from the same thalamic nucleus have in common?This crucial issue is rarely mentioned. We all know that the lateral geniculate body (part of the thalamus) is primarily associated with V1 in the primate visual system.The primate thalamus has a large section called the "postthalamic tubercle," where the other visual areas of the thalamus are located (see Chapter 15).It has a large number of different subregions, some of which may consist of several smaller subregions.Is each region associated with a certain stage of visual processing?There are two possibilities.These subregions (three of which are major, i.e., anterior, lateral, and central posterior thalamic tubercle) may each be related to a stage in David Marr's theory (i.e., feature map, 2.5D map, and 3D model) or to a certain There are strong associations with similar things.It is also possible that smaller and more numerous subregions are each strongly associated with one level of van Essen's visual scale (Fig. 52).There is, of course, an element of truth to both of these possibilities.

What do I mean by "strong correlation"?The connections from the thalamus to the cortex come in two forms: one goes to layer 4 (or layer 3); the other avoids these intermediate layers and usually has many projections to layer 1.The first type of connection may be driving, while the second is more like regulating an event that has already occurred.By strong correlations I mean those driving connections to the middle layer.In this brief consideration I temporarily suspend another type. At its simplest, he said, the processing hypothesis is that any one cortical area is only closely related to a certain part of the thalamus.This view is not entirely implausible.The cortical Vl area is only closely related to the lateral geniculate body, but not to other parts of the thalamus.It was found that the features that form Marr's element map (or something similar) do appear in the V1 region.The information identified there corresponds to fairly simple local features, such as the orientation of a small portion of the image in the field of view.Koch and I hypothesized that area Vl might be the seat of very transient forms of visual perception.We don't think this requires an attention mechanism.Experiments have shown that the monkey's attention does not affect the firing of neurons in the Vl area, which can be considered as a support for this formulation.

We do not yet know enough detail about the thalamic connections in other parts to tell whether the processing hypothesis is correct.Does each cortical area except area V1 have strong connections to only one part of the PST?If not, how are they connected?More experiments are needed to answer this question.It is also possible that some thalamic regions happen to have strong connections with cortical regions involved in visual perception. And what about the hypothetical 3D model stage?We hardly know where to start in this situation, and the psychologist Irving Biedrman thinks that the representation will be based on some primitive three-dimensional shapes that he calls "geometrics(1)".Some theorists (such as Tommaso Poggio) argue that what we have in our minds is a series of two-dimensional views of an object, and the ability to interpolate between them.Both views are likely to be true.Exactly where in the monkeys' brains all of these occur, if indeed they exist, remains to be determined.In the absence of such knowledge, it is difficult to evaluate processing hypotheses, and many hypotheses that at first glance seem promising are often stalled due to experimental uncertainties. Nevertheless, there is something attractive about dealing with assumptions.It shows that we may use the terms conscious and unconscious for many different activities.They should probably be replaced by some phrase like "processing unit" or in some cases "awareness unit".Each perceptual unit has its own semi-global representation, usually covering several cortical areas.They may have their own characteristic processing time, each corresponding to the characteristic time of very short-term memory (for example, V1 area is very short, while higher cortical areas are longer), and, more importantly, its own special representation Form: simple features of the V1 area, 2.5-dimensional objects expressed by the next higher cortex, etc., the characteristics of each form of processing unit will depend on the content and organization of that particular representation, and it is possible that each particular Each thalamic region uses its own form of attention, allowing neurons in groups of its cortical areas to send messages to thalamic neurons, which in turn feed back information, and so coordinate their firing in some way .There is also a speculative idea here (described in Chapter 16) that thalamo-cortical-thalamic circuits may be echo circuits closely related to very short-term memory. Of course, there are complex, direct connections that do not go through the thalamus between many different cortical areas, as shown in Figure 52.The processing assumption does not imply that neuronal activity flows in only one way, from lower processing units to higher ones.There is almost certainly information flow in multiple directions. This does not mean that the thalamus itself can generate all the different forms of awareness.Awareness requires electrical activity in various cortical areas in addition to the thalamus, much like conducting an orchestra that needs to play music. ① (Thus) the thalamus cannot be ignored if you are interested in visual perception or other aspects of consciousness, to say the least.Some people may despise the "insignificant" lateral knee body, saying that it is just a relay station.But students who study the visual system will interject, "Why does the posterior tubercle have to be there?" It's not an insignificant area of ​​the brain; Big.It may have some important function, but so what?Although vague on the details, the processing hypothesis does raise a possibility. The thalamus is a key player in the process of consciousness.This is not a new idea.This point was made long ago by Wilder Penfield.In a newly published article, James Newman and Bemard Baars expand on the latter view (which is briefly discussed in Chapter 2), arguing that the thalamic area Certain nuclei known as "layer kernels" propagate information to what they consider a global workspace.One of these nuclei, called the central nucleus, is closely associated with the visual system.They project primarily to an important part of the brain, the striatum, and to a lesser extent to many cortical regions.The striatum has strong connections to the motor system, but parts of it may also be involved in issues of a more cognitive nature.It is one of the parts of the brain that Parkinson's disease attacks. What kind of specific information the core of each layer sends out remains to be explored (2).Newman and Bales also emphasize the role of the reticular nucleus of the thalamus (described in Chapter 10).As I have considered, they believe that the reticular nucleus may be involved in the control of attention.It is unclear whether the reticular nucleus can perform the required degree of selectivity in the thalamus.It may have only one function, which is to fully control the activity of the thalamus and cortex during states such as sleep and wakefulness.If the thalamus is indeed key to consciousness, the reticular nucleus is likely involved in some control of consciousness. There is another brain region that must be briefly mentioned here, the claustrum.It is a thin sheet of neurons near the lower cortical area near the "insula" (part of the cortex).Its input is mainly from the cortex, and most of the output is also returned to the cortex, so it is like a satellite in the cortex.It receives input from many cortical areas and may send connections back to them all.Some (but not all) visual areas of the cortex project to a part of it, where (in the cat brain) a corresponding map of individual retinal areas is formed.These visual inputs may overlap with inputs from other claustrums.There seems to be little work on the claustrum in monkeys in recent years, so there may be some inaccuracies in what has been said above. (For example, there might be two visual projections in there.) The function of the claustrum is not well understood.Why is all this information aggregated into one sheet?One might suspect that the claustrum has some form of global function, but no one knows what that is.Even though it's a fairly small area of ​​the brain, it shouldn't be completely ignored. There is likely to be a hierarchical system of processing units, in the sense that some may exercise some type of global control over other parts.There are also populations of neurons (such as the claustrum and the lamellar nucleus of the thalamus) with broad projections to the cortex that may play this role. A review of the previous two chapters shows that there is no shortage of plausible ideas and experiments.Disappointingly, none of the ideas currently appear to be organized in a convincing way to form an exhaustive, plausibly correct neural hypothesis.If you feel like I'm groping my way through the jungle, you're absolutely right.This is often the case when researching frontiers.But I do feel now that I have a much deeper understanding of what the key issues are than I did ten years ago.I often even say to myself that I can glimpse some answers.But this is a common illusion that people have when they study a problem for a long time.We've broken through to higher levels, so even though the road is long and hard, we've seen the best directions to explore. Despite all these uncertainties, after deliberating all these very scattered facts and speculations, is it possible to paint some big picture, even tentatively, to roughly guide us through the jungle before us? ?Let me set aside those cautions and sketch a possible model.The reality may be much more complicated than that, and it is unlikely to be simpler. Consciousness is associated with certain kinds of neural activity.A plausible model is that these activities occur in the lower layers of the cortex, such as layers 5 and 6.This activity expresses the local (temporary) results of much of the "computation" that occurs primarily at other layers of the cortex. Not all cortical neurons in the lower layers express consciousness.The most dominant species are the large "clumps" of cone cells located in layer 5, such as those that project out of the cortical system. Unless these particular lower-level activities are maintained by some form of very short-term memory, it cannot reach consciousness.It stands to reason that this may require an effective reverberating loop from cortical layer 6 to the thalamus and back to cortical layers 4 and 6.If this loop is lacking, or if layer 4 is too small, it will be impossible to sustain these reverberations.Thus only some cortical areas are capable of expressing consciousness. A processing unit (of which only some are relevant to consciousness) is a collection of cortical areas (1) that are on the same level of the visual hierarchy and project to each other towards layer 4.Each such collection of cortical areas is strongly connected to only a small area of ​​the thalamus.Such an area coordinates the electrical activity of its associated cortical area by firing synchronously. The thalamus is closely related to the attention mechanism.The special bundles required for object identification operations (particularly figure/background separation) usually have a tuned emission pattern whose rhythm is usually in the 40 Hz range. Areas involved in consciousness may affect (not necessarily directly) part of the voluntary motor system. (There may be some unconscious operations in between, such as thinking.) To repeat, consciousness mainly depends on the connection between the thalamus and the cortex, and it is only possible if certain cortical areas have echo loops (including cortical layers 4 and 6) and have projections strong enough to produce significant reverberations. So much for this plausible model.I hope no one will call it the Crick (or Crick-Koch) theory of consciousness.When I wrote this model, I was quite hesitant about the choice of materials.If it was proposed by someone else, I would not hesitate to accuse it of being a paper house that collapses at the first touch.That's because it's patchwork, and there isn't enough critical experimental evidence to support its various parts.Its only value lies in the potential to push scientists and philosophers to think about these questions from a neural perspective, thereby speeding up experimental progress on consciousness. What about more philosophical questions?I am sure that when we fully understand the neural mechanisms of consciousness, this knowledge will answer two important questions: What is the general nature of consciousness?This in turn allows us to speak meaningfully about the nature of consciousness in other animals, as well as in man-made machines such as computers.What benefits does consciousness bring to an organism so that we can discover why there is consciousness.It may eventually turn out that visual perception arises because its detailed information needs to be sent to several different areas of the brain.It may be more effective to make these messages explicit than to implicitly pass them down different channels.Having a single distinct representation also prevents one part of the brain from using one interpretation of a visual scene while another part uses a quite different interpretation.When information only needs to be sent to one place, it gets there by experience and without awareness. What really proves difficult or impossible to establish are the details of the subjective nature of consciousness, since this would depend on the precise system of symbols used by each conscious organism.Unless we can link the two brains together in a sufficiently precise and detailed way, we cannot directly transfer the symbology from one brain to the other.Even if we can do this, it may have its own problems.But without an understanding of the neural correlates of consciousness, I don't believe that any of these questions can have answers that thinking people can accept. I would like to say a few words in particular to the many scientists who are currently quite actively working in the field of brain (especially vision) research.It is their rather conservative attitude that hinders the smooth progress of experimental research. They give too much weight to many complex issues that I vaguely pass over.They should not use these mistakes and omissions as an excuse for not confronting the vast message of this book.What happens in our brains as we watch?It is not feasible to ignore this overall problem and only study some special problems of vision.A layman would think this attitude is too narrow, and it is, as I have tried to show that the present problem of visual perception is open to exploration both experimentally and theoretically.Furthermore, if we actively confront this difficulty, we start to think about the problem from a completely new perspective, looking for information that previously seemed irrelevant or of little interest (such as dynamic parameters or short-term memory).I hope that before long every laboratory that studies the human and other vertebrate visual systems will have a prominent sign on the wall that reads: consciousness right now 1 This view was recently developed by the Harvard mathematician David Mumford.An unpublished article sent to me by Wu Quanfeng (transliteration) also deals with this. (2) The central nucleus is thought to be involved in the control of gaze. ① Some sets may have only one member, such as the Vl region.