Chapter 16 Chapter 14 Visual Perception

"The universe is like a great book unfolding before our eyes. Philosophy is written in it. But we cannot understand them without first learning and mastering the language and symbols in which they are written." — Galileo Let's now take an overview of the areas we've covered so far.The theme of this book is the "surprising hypothesis" -- that the behavior of each of us is nothing more than the activity of a population of interacting neurons.Christof Koch and I believe that the best way to explore the question of consciousness is to study visual perception. This includes studying humans and their close relatives. However, it is not a straightforward thing for people to see things. It is a constructive one. , complex processing.Psychological research shows that it has a high degree of parallelism and is processed in a certain order, and the "attention" mechanism is at the top of these parallel processes.Psychologists have proposed several theories that attempt to explain the general laws of visual processing, but none that involve more of the behavior of neurons in the brain.

The brain itself is composed of neurons and a large number of supporting cells.Considered molecularly, each neuron is a complex object, often with irregular and unusual shapes.Neurons are electrical signaling devices.They respond rapidly to incoming electrical and chemical signals and send their high-speed electrochemical pulses down the axon, often over distances many times greater than the diameter of the cell body.There are huge numbers of these neurons in the brain, and they come in many different types.These neurons have complex connections with each other. Unlike most modern computers, the brain is not a general-purpose machine.When fully developed, each part of the brain performs certain different specialized tasks.On the other hand, in almost all reactions, there are many parts interacting.This general idea is supported by studies of the human brain, including studies of brain-injured individuals and studies of the human brain from outside the skull using modern scanning methods.

The number of distinct cortical areas in the visual system is much greater than one might expect.They are connected in an approximate hierarchical manner.In lower cortical areas, neurons with the shortest connections to the eyes are primarily sensitive to relatively simple features in a small area of ​​the visual field, but nonetheless these neurons are not affected by the visual environment in which that area is located.Neurons in higher cortical areas respond to much more complex visual objects, such as faces or hands, and are insensitive to the object's position in the field of view. There doesn't seem to be a single cortical area (so far) that corresponds to the full repertoire of visual perception.

To understand how the brain works, we must develop theoretical models that describe how groups of neurons interact.These current models oversimplify neurons.Although modern computers are much faster than their predecessors, only a small number of these simplified neurons and their interactions can be simulated.Still, these different types of simplified models, while primitive: often exhibit surprising behavior.These behaviors have similarities with certain behaviors of the brain.They offer new avenues for studying the ways the brain might work. The above is background knowledge.On this basis, we set out to address the problem of visual perception: how to interpret what we see in terms of neuronal activity.In other words, what are the "neural correlates" of visual perception?Where exactly are these "awareness neurons" located?Are they concentrated in a small area or scattered throughout the brain?Is there anything special about their behavior?

To begin, let us review the various perspectives outlined in Chapter 2.What kind of psychological processing does visual perception involve?If we could figure out exactly where in the brain these different processes are located, it might help locate the sensory neurons we're looking for. Philip Johnson-Laird believed that the brain, like modern computers, has an operating system.The behavior of this operating system corresponds to consciousness.He puts this idea in a broader context in his book Mental Medels.He argues that the difference between conscious and unconscious processes is that the latter are the result of highly parallel processing in the brain.As I've described in the visual system, this parallel processing is the ability of large numbers of neurons to work simultaneously, rather than sequentially processing information one after the other.This makes it possible for organisms to evolve specialized, fast-moving sensory, cognitive, and motor systems.A more sequential operating system has global control over all these activities, allowing for quick and flexible decisions.Roughly speaking, it is as if the conductor of an orchestra (equivalent to an operating system) controls all members of the orchestra to play at the same time.

Johnson-Laird posited that while this operating system could monitor the output of the nervous systems it controlled, all it could use were the results they passed to it, not the details of how they worked.Through introspection we can only perceive a very small fraction of what is going on in our heads.We cannot intervene in the many operations of the operating system that generates information and transmits it to the brain.Because he views operating systems as primarily sequential, he argues, "When introspecting, we tend to force concepts that would otherwise be parallel into the narrow confines of sequential." This is where using introspection goes wrong s reason.

Johnson-Laird makes his point clearly and convincingly.But the location and nature of this operating system must also be identified if we hope to understand the brain neurally.It doesn't necessarily correspond to many properties of modern computers.The brain's operating system may not be clearly located in a particular location.It's more like distributed in two senses: it may involve several separate parts of the brain interacting, with information about the activity of one part being spread across many neurons.Johnson-Laird's description of the brain's operating system is somewhat reminiscent of the thalamus, but the thalamus has too few neurons to express the full content of visual perception (although this is verifiable).It seems more likely that some but not all neurons in the neocortex express visual perception under the influence of the thalamus.

Where would the neural correlates of awareness we seek be at on the functional scale of the brain?Johnson-Laird believed that the operating system was at the highest level of the processing hierarchy, while Ray Jackendorf believed that awareness was more connected to the middle levels.Which point of view is more reasonable? Jackendorf's ideas about visual perception1 are based on David Marr's idea of ​​a 2.5-dimensional diagram rather than a three-dimensional model (roughly the observer-centered representation of visible surfaces described in Chapter 6). ).This is because what people directly perceive is only the side of the object in the field of vision; there is an invisible part behind the object is only speculation.On the other hand, he believed that understanding of visual input (i.e., what we perceive) is determined by three-dimensional models and "conceptual structure" (conceptual structure, another grandiose way of saying the mind).The above is his middle level theory of consciousness.

The following example will help to understand the theory.If you see a person whose back is turned to you, you can only see the back of his head, not his face.However, your brain deduces that he has a face.We'll reason this way because you'd be quite surprised if he turned around and showed that the front of his head didn't have a face.The observer-centered representation corresponds to what you see of the back of his head.This is what you really feel.The inference your brain makes about its face is derived from some sort of three-dimensional model representation.Jackendorf argues that you are not directly aware of this three-dimensional model (nor are you, for that matter, your own mind).As an ancient poem says: If you haven't heard what I said, how can you know what I think?

Since it was not easy to understand Jackendoff's language when he first read his work, I put the penultimate version of his theory in a footnote. ②If my understanding of his theory is correct, his idea applied to vision is that "morphological differences" (including the position, shape, color, movement, etc. of a visual object) are related to a kind of short-term memory (or caused/supported/projected by it) as a result of a "winner takes all" mechanism (a selection mechanism), enriched by the role of the attention mechanism. The value of Jackendorf's point is that it reminds us not to assume that the highest levels of the brain are necessarily the only levels involved in visual perception.The vivid mental representation of the scene before us may involve many intermediate layers, and other layers may not be vivid enough, or, as he speculates, we may not be able to perceive them at all.

This does not mean that information flows only from surface representations to three-dimensional representations; bidirectional flow almost certainly exists.In the example above, when you imagine the front of a face, what you perceive is the perceptible surface representation produced by the imperceptible three-dimensional model.The distinction between the two representations may need to be clarified as the subject develops, but it gives an initial, rough view of what we are trying to explain. The exact location of these layers in the cortex is not yet known.As far as vision is concerned, they are more likely to correspond to the middle of the brain (the lower temporal lobe and some parietal areas) than to the frontal areas of the brain, but which of the visual hierarchy (see Figure 52) is Jackendorff referring to? part, which remains to be explored (discussed in more detail in Chapter 16). After looking at some psychologists' perspectives on the problem, we now look at this conundrum from the perspective of neuroscientists who understand neurons, their connections, and how they fire.What are the general characteristics of the behavior of neurons associated with consciousness (or not)?In other words, what are the "neural correlates" of consciousness?It seems plausible that, in some sense, the activity of neurons is essential to consciousness.Consciousness may be linked to a particular type of activity in certain neurons in the cortex.No doubt it takes different forms, depending on which parts of the cortex are involved.Koch and I hypothesize that there is only one (or a few) underlying mechanisms.We argue that consciousness at any moment in time will correspond to a particular type of activity in a momentary collection of neurons.These neurons are just a part of the pool of candidates with considerable potential.So, at the neural level, the question is: • Where are these neurons located in the brain? · Do they belong to some special neuron type? • If their connection is specific, what is it? · If there is some special way of their distribution, what is it? How to find those neurons related to visual perception?Are there clues to the pattern of neuronal firing associated with this awareness? As we have seen, psychological theory has several hints for us.It is likely that some form of attention is involved in the perception process, so we should study the mechanism of the brain's selective attention to visual objects.Perceptual processes are likely to include some form of very short-term memory, so we should also explore the behavior of neurons when storing and using this memory.Finally, the fact that we seem to be able to attend to multiple objects at once poses problems for some neural theories of awareness, so we begin by addressing this issue. What exactly happens in our brain when we see an object?The number of possible, different objects we would see is almost infinite.It is impossible for every object to have a corresponding responding cell (such cells are often called "grandmother cells").Expressing so many objects with different depths, movements, colors, orientations, and positions in space, the possible combinations are staggeringly large.However, this does not rule out the possibility that there may be specialized populations of neurons that respond to fairly specific, ecologically important goals, such as the appearance of a face. It seems likely that, at any instant in time, each particular object in the field of view is represented by the firing of a population of neurons. ① Since each object has different characteristics, such as shape, color, motion, etc., which are processed by several different visual areas, it is reasonable to assume that seeing each object often involves neurons from many different visual areas.How do these neurons temporarily fire as a whole at the same time?This problem is often referred to as the "binding problem".Since visual processes are often accompanied by auditory, olfactory or tactile sensations, this binding must also occur between different sensory modules. ② We all have the experience of having a global perception of an object.This leads us to think that all neurons respond positively to different features of a seen object, and that the brain somehow coordinates them together.In other words, if you are focusing on a friend with whom you are discussing a point, some neurons in your brain respond to the movement of his face, some respond to the color of the face, neurons in the auditory cortex The neuron responds to what he said, and probably has those stored memory traces of who the face belongs to, and all of these neurons will be bundled together so that they carry the same marker to indicate that they collectively generated a response to that face. Awareness of a particular face. (Sometimes the brain can also be tricked into making false bindings, such as mistaking the heard ventriloquist's voice for that of the mimic.) Bundling comes in several forms.A neuron that responds to a short line can be thought of as tying together the points that make up the line.The input and behavior of such neurons may have been originally determined by genes (and developmental processes) that evolved as a result of experience in our distant ancestors.Another form of binding, such as recognition of familiar objects, or letters of the alphabet, may be acquired through frequent, repetitive experience, that is, through repeated learning.This might mean that a large number of neurons involved in a process end up with tight connections with each other. ①The fairly permanent binding of these two forms can produce populations of neurons that, as a whole, respond to many objects such as letters, numbers, and other familiar symbols.But it is impossible to have enough neurons in the brain to encode an almost infinite number of perceivable objects.The same is true for language.Every language has a large but finite number of words and an almost infinite number of correctly formed sentences. We are most concerned with the third form of binding.It is neither determined by early development nor acquired by repeated learning.It works especially well for objects that are novel to us, such as a new animal we saw at the zoo.In most cases, there are not necessarily strong connections between neurons that are actively participating in the process.This bundling must be able to be achieved quickly.It is therefore primarily ephemeral, and must be able to tie visual features together into an almost infinite number of possible combinations, though perhaps at a given moment it can form only a few of them.If a particular stimulus is present frequently, this third form of momentary binding will eventually establish the second form of binding that is learned over and over again. Unfortunately, we do not understand how the brain expresses this third form of binding.It is particularly unclear whether, during focused awareness, we perceive only one object at a time, or whether we can perceive several objects at once.On the surface, we can never perceive more than one object at a time, but could this be an illusion?Can the brain really process information about multiple objects one after the other so quickly that they appear to be in our mind at the same time?Maybe we can only pay attention to one object at a time, but after paying attention, we can roughly remember a few of them.Because we don't know for sure, we have to consider all of these possibilities.Let us first assume that the brain can only process one object at a time. What type of neural activity might be involved in the bundling?Of course, the neural correlates of consciousness may consist only of a particular type of neuron, say, a type of pyramidal cell in a particular cortex.One of the simplest views is that awareness arises when certain members of this particular group of neurons fire at a fairly high frequency (say, about 4oo or 500Hz), or for a reasonably long period of time.Thus, bundles correspond to only a relatively small fraction of cortical neurons that fire simultaneously at high frequency (or all fire for long periods of time) in several different areas of the cortex.This appears to have two consequences: the rapid or sustained firing will intensify the effect of this excited group of neurons on the projecting neurons corresponding to the brain's perceived The "meaning" of known objects.At the same time, this rapid (or sustained) firing will activate some form of very short-term memory. However, if the brain can accurately perceive more than one object at the same time, then this view cannot be established.Even if the brain processes one object at a time, it must distinguish objects from background.To understand this, imagine that in a field of vision near the center of vision, there happens to be a red circle and a blue square.Then, some neurons corresponding to awareness will fire rapidly (or keep firing for a while), some will be marked with red, some with blue, others with circles, and of course some with squares.How does the brain know which colors go with which shapes?In other words, if awareness corresponds only to rapid (or sustained) bursts, the brain will probably mix up the properties of different objects. There are many ways to get around this difficulty.Perhaps vivid awareness of an object is formed only when the brain pays attention to it.Perhaps the attention mechanism boosts the activity of neurons that respond to the object being attended to, while dampening the activity of neurons that respond to other objects.If so, the brain would just jump from one object to another with the attention mechanism, processing them one after the other, which is what happens when we move our eyes, after all.We focus first on one part of our field of vision, then on to another, and so on.Since we can see multiple objects at the same time without moving our eyes, the attention mechanism must be faster than this and work between eye turns. A second alternative explanation is that the attention mechanism somehow causes different neurons to fire in somewhat different ways, where the key is correlated firing. ①It is based on the idea that it is not only the average firing rate of neurons that matters, but the precise timing of each neuron firing. For simplicity, let us only consider two objects. The neurons corresponding to the characteristic response of the first object all fire in a certain pattern at the same time, and the neurons corresponding to the second object all fire at the same time, but at a different time than the first group of neurons. An idealized example can illustrate this problem more clearly.Assume that the neurons in the upper group fire quickly.Perhaps they will be sent again, say after 100 milliseconds.Similarly, after the second burst is fired, it is fired again 100 milliseconds, and so on.Suppose the second group of neurons also fires bursts of high-speed spikes every about 100 milliseconds, but only when the first group of neurons is at rest.That way, other parts of the brain don't mix up the firings of the two groups of neurons, because they never fire at the same time. The basic idea here is that many spikes arriving at a neuron at the same time will have a greater effect than the same number of spikes arriving at different times. ②The theoretical requirement is that the firing of the same group of neurons has a strong correlation, and at the same time, the correlation between neurons of different groups is weak, or even has no correlation at all. ③ Let's get back to our main problem.This is what locates the "perception" neurons and reveals what is the mechanism that makes their firing symbolic of what we see.It's like trying to solve a murder mystery.We know a few clues about the victim (the nature of awareness) and many messy facts that may be relevant to the crime.Where does progress look most promising?So what's the next step? The most direct lead will be to catch the suspect at the scene.Can we find those neurons whose behavior is consistently associated with visual perception?One possibility is to set up an environment (viewing the Necker cube described in Chapter 3) so that the visual information entering the eye remains constant, but perception changes.Which neurons change their firing, or how they fire, when perception changes, and which do not?If the firing of a particular neuron does not change with perception, this provides an "it's not present" evidence.On the other hand, if its emission is indeed related to perception, we still need to determine whether it is the "culprit" or "accomplice". Let's change our strategy.Can we limit the location of the crime to a particular town, a district, or a unit in a building?This would allow us to search more effectively in our question, namely, can we say roughly where visual perception neurons might be located in the brain?Obviously, we speculate it's in the neocortex.Although we cannot completely ignore the close neighbors of the neocortex, such as the thalamus and claustrum, and the evolutionarily older visual system (older visua1 system) and superior colliculus, let alone the striatum and cerebellum.Visual perception is less likely to reside in areas such as the auditory cortex, so we can focus primarily on the many visual cortical areas shown in Figure 48.Perhaps we can find evidence that certain regions are more closely involved in visual perception than others. That's not enough to find the killer, but it might point us in the right direction.Criminals can be a certain type of person.For example, a strong man, a disturbed teenager, or a gang of gangsters.What types of neurons might be involved here?Is it an excitatory neuron?Or inhibitory neurons?Astrocytes or pyramidal cells?If they are on the cortex, in which layer or layers of the cortex are they found? Another tactic would be to look for some form of communication between them that would give them away.If this was done by a gang, did they use a mobile phone in the car?In neurological terms, does awareness depend on some particular form of neural circuitry that occurs only in certain locations in the brain? Perhaps someone will look for a motive for the crime.Can the murderer get some benefit from committing a crime?Can he get financial benefits?If so, where did the stolen money go?If we can find it there, we might be able to track down the killer.In neural terms, where in the brain is the visual information sent?How are these sites connected to cortical visual areas? Also, one asks if there is some particular behavior that leads us to the suspect.It might be a correlated firing between groups of neurons, or it might be a rhythmic or patterned firing of one form or another.If we suspect a gang of bandits, who is most likely to be the leader?Who decides the actions of the bandits?We believe that the process of awareness often involves the brain making judgments about which explanation is most plausible.This could be a "winner wins" mechanism involving certain groups of neurons, and if we can discover this, the neural nature of the winner might point us to the perception neurons.Were any special weapons used in the crime?As stated earlier, it is a safe guess that very short-term memory is an essential feature of awareness.Also some form of attentional mechanism may assist in producing vivid awareness, so anything we know about these workings at the neural level will lead us in the right direction. In short, he said that a large number of experimental methods can lead us conceptually to the neurons we are looking for and their behavior.At this stage, because of the difficulty of the problem we have to solve, we cannot ignore any clue that even appears to have little promise, let us now examine more closely the nature of these different approaches. ① visual awareness (visua1a Wareness).In this book, both consciousness and awareness mean consciousness, but the former is a wider and more written word, while the latter is more used for the sensory system (especially the visual system) and is a more colloquial word (See the footnotes of Chapter 1). In the first and second parts of this book, they are both translated as "consciousness", which does not cause ambiguity.But in the third part, the author takes visual awareness as the breakthrough point of consciousness research, and needs to distinguish these two words.Therefore, in the third part (Chapter 14 to 18), awareness is specially translated as awareness in psychology. ——Translator's Note ①It is not easy to generalize Jackendoff's views without misinterpreting his meaning.If readers want to understand further, they can refer to his book.I do not intend to describe his arguments on phonology, syntax, semantics, etc., and his insights on musical cognition.Instead, I will try to refine his basic ideas, especially their visual application. ①Readers wishing to understand Jackendoff's words precisely are referred to his writings. (The final version of his theory, Theory Eight, also talks about emotion.) ② His original words are: "The morphological differences expressed by each form of awareness are caused/supported/projected by the structure of the intermediate level corresponding to the form. This structure is part of the matching set of short-term memory representations, And this representation is assigned by the selection mechanism and enriched by attentional processing. In particular, language perception is caused/supported/projected by the phonological structure, music perception corresponds to the musical surface, and visual perception comes from 2. 5D map." ① If the neurons in a group are not spatially close together (meaning they may have some kind of interconnection), receive somewhat similar inputs, and project to more or less similar regions, then this does not cause any particular difficulty .In this case they are like neurons in a single neural network.Unfortunately, usually such simple neural networks can only process one object at a time. ②It is still not completely sure that the binding problem really exists as I said, or whether the brain has bypassed it through some unknown technique. ① Recall that most cortical neurons have thousands of connections, many of which are weak to begin with, which means that learning is possible easily and correctly only when the brain is already roughly structured in the right way. ① This view is Christoph Feng.proposed by Christoph von der Malsburg in a rather cryptic 1981 article.It has been described before by Peter Milner and others. ① Of course, the pulses of axons within a group do not have to be precisely synchronized with each other.As electrical potential changes travel down the dendrites of the receiving neuron to the cell body, their effects are diffused in time.In addition, the time delays when pulses propagated along many different axons also varied.In this way, the firing times of a group of neurons need only be simultaneous on the order of milliseconds. ②A slightly more detailed theory introduces this inevitable time delay in the process of axonal transmission, so that synapses farther from the cell body receive input slightly earlier than those closer to the cell body, so that due to the temporal delay of dendrites For small differences, the maximum effect of both signals will reach the cell body at the same time.A more detailed theory also considers the modulation of inhibitory effects produced by local inhibitory nerves.All such qualitative considerations should be quantifiable through careful simulation, simulating in silico, of how individual neurons would behave in this environment, introducing factors such as time delays. ③ This kind of distribution is likely to be as regular as shown in Figure 57.