Chapter 12 Chapter 8 Avoiding Einstein Syndrome—The Importance of Convergent Evidence
"Biology experiment unlocks life's mysteries!", "New breakthrough in mind control!", "California scientists discover way to delay death!" -- as you can see, trying to parody a headline that fills tabloid and electronic media headlines "Breakthrough" news on the headlines is easy.Given that some irresponsible media outlets routinely produce such "headlines," it's no wonder that most scientists advise the public to treat such news with skepticism.However, the purpose of this chapter is not merely to counter exaggeration and misrepresentation, nor merely to remind people that when evaluating reports of scientific progress, they must examine their sources carefully. It is a comprehensive and comprehensive view of scientific progress.To this end, we will elaborate on the systematic positivism and public knowledge introduced in Chapter 1.
Such so-called "breakthrough" headlines in the media have largely misled the public about psychology and other sciences.A particularly typical misunderstanding is that they lead the public to think that all the problems in a certain field of scientific research can be solved by a key experiment, or that some important inspiration has made theoretical progress and completely reversed many previous problems. The totality of knowledge accumulated by researchers.This view of scientific progress is very much in line with the appetite of the news media for hype. In the way the media operates, the tracing of history presents fragmented and incoherent small events.It's also a rather convenient model for the Hollywood entertainment industry, where events must have a beginning and a happy ending, where ambiguities are cleared up.However, this is only a misrepresentation of scientific progress, which, if taken to be true, can lead to false perceptions about scientific progress and impair one's ability to assess scientific knowledge on a given issue.In this chapter, we will discuss two principles of science—the principle of connectedness and the principle of convergent evidence—that describe scientific development more accurately than the “leap model.”
While denying the validity of all "leaps" of scientific progress or key experimental patterns, we are not saying that this key pattern of experimental and theoretical development never occurred, on the contrary, some famous cases in the history of science show that this pattern did occur .Einstein's "Theory of Relativity" is by far the most famous example. So far, a series of extraordinary theoretical inspirations have redefined basic concepts such as time, space and matter.
However, Einstein's achievement stood like a monument, allowing this model of scientific development to rule the public mind.This dominance is enduring because it fits closely with the implicit "script" that the media uses to cover most news events.There are not many theories in human history that have suffered as much nonsense and false inferences as the theory of relativity (no, Einstein did not prove that everything is relative" - see Holton, 1996; Randall, 2005) Of course, our purpose is not to refute these fallacies, but to pave the way for the later discussion and evaluation of theories in psychology.
In Einstein's theory, the redefined concepts of the physical world are so fundamental that popular literature often equates them with conceptual shifts in the arts (a second-rate poet, reevaluated, becomes Genius; a school of art declared extinct).This approach ignores the most fundamental difference in the art and science of conceptual change.
Conceptual change in science obeys a principle of relevance that does not exist, or at least is extremely rare, in art (see Bronowski, 1956, 1977; Dobzhansky, 1973).That is to say, a new scientific theory must be related to previously established empirical facts.A new scientific theory should not only explain new facts, but also be compatible with old facts, so that it can be considered a real theoretical progress.The new theory can explain the old evidence in a very different way, but it has to make sense.These requirements ensure that science continues to advance on its original basis.No real progress will take place unless the range of theoretical explanatory power is broadened.If a new theory explains some new phenomena but fails to explain most of the old facts, it will not be considered as a comprehensive transcendence of the old theory, so it will not immediately replace those old theories, and the new theory and Old theories will coexist in competing guises until a new theory emerges that integrates the two.
No matter how shocking the new concepts in Einstein's theory are (clocks slow down, mass increases with speed, etc.), they all obey the principle of associativity.While declaring the hysteresis of Newtonian mechanics, Einstein's theory did not deny the facts of motion based on Newton's views, or make them meaningless.Instead, at lower velocities, the two theories made essentially the same predictions.The brilliance of Einstein's theory is that it can explain a wider range of new (sometimes surprising) phenomena that Newtonian mechanics cannot.Thus, even Einstein's theory, the most astonishing and fundamental conceptual reconstruction in the history of science, still obeys the principle of relevance.
The "leap and bound" mode of scientific development—what we might call Einstein's syndrome—leads us astray into thinking that new discoveries necessarily violate the principle of associativity.This concept is dangerous, because if the principle of relevance is abandoned, the biggest beneficiaries will be those who sell pseudoscience and pseudotheory.The reason why these theories get favor and attention is that they are always said to be "new". "After all, relativity was new for its time, right?" is often used as a way to justify something new.Of course, the accumulated factual data may seem like a huge stumbling block in this field of pseudoscientists.In fact, however, this stumbling block can not stop these pseudo-scientists, because they have two powerful tactics to resolve this trouble.One trick we discussed earlier (see Chapter 2) is to render the theory unfalsifiable before explaining the data, thus rendering the previous data useless.
The second trick is to declare that previous data are irrelevant to their subject and therefore not considered.To achieve "out of consideration" results, they usually emphasize that the new theory presents an "unprecedented" novelty.Phrases like "new conception of reality" and "unprecedented" were used frequently.But actually, the real trick is yet to come. The "new theory" was destined to be so groundbreaking that experimental evidence derived from tests of other theories was declared irrelevant.Only data are considered that are compatible with the framework of the new theory, that is, the principle of associativity is completely violated.Apparently, the theory is so new that they can confidently say that empirical evidence for it does not yet exist.In this way, you have a good soil for the development of pseudoscience: old, "irrelevant" data is wiped out, and new relevant data does not yet exist.This trick works easily because Einstein syndrome blinds the principle of associativity.Ironically, the importance of the principle of relevance is demonstrated by Einstein's theory itself.
California paleontologist Kevin Padian gives another example of how the nature of science can be misunderstood when people fail to appreciate the importance of the principle of relatedness.Referring to the Kansas school board's decision to remove evolution from students' required courses, Padian noted: "We're talking about a misunderstanding about 'how science is integrated. To have such a central theory as the theory of evolution that can tie the whole of biology together—and to think that it has no other connections—is ludicrous" (Carpenter, 1999, p. 117).The biological philosopher Michael Ruse (1999) pointed out that the theory of evolution has shown relevance to many independent scientific fields, such as paleontology, embryology, morphology, biogeography, neuroscience and so on.Likewise, Shermer (1997) states, "If the universe and the earth were only 10,000 years old, then cosmology, astronomy, physics, chemistry, geology, paleontology, paleoanthropology, and the sciences of early human history would be wrong” (p. 143).Renowned science writer and archaeobiologist Stephen J. Gould echoed this sentiment, “Teaching biology without teaching evolution is like teaching English without teaching English grammar” (Wright, 1999, p.56).
Ruth (1999) recounts an example of how Darwin used the principle of relevance and discarded a new theory that lacked the necessary relevance to other disciplines.At that time, Darwin wanted to explore a genetic mechanism that could match his theory of natural selection, so he tried to establish a so-called "pangenesis" theory. "Small germs are produced in various parts of the body, which circulate through the body and collect at the sex organs, where they are passed on to the next generation" (p. 64).One problem is that this theory doesn't agree with the cell theory.The second problem is that Darwin did not explain how these germs were transported, because blood transfusion experiments have shown that germs cannot be transmitted through blood.For these two and other reasons, pangenesis was thrown out of the scientific camp "because it was incompatible with the rest of biology" (p. 64).
The same situation applies to psychology. The philosopher Mario Bimge (1983) has pointed out that if cognitive psychology denies classical conditioning and operant conditioning from the beginning, then it will not be able to be used in psychology. neutral because it is incompatible with other knowledge in the behavioral sciences.Recalling the discussion of "assisted communication therapy" in Chapter 6, the reason why it cannot "cure" language impairment in autism is that it breaks the principle of associativity—if the treatment is effective, it will require us to rebuild neurological, Knowledge within the fields of genetics and cognitive psychology.This hypothetical therapy has no connection to other knowledge in science.The same goes for creationism's objection to evolution, which scientifically does not obey any principle of relatedness.On the contrary, the theory of evolution is inextricably linked with other sciences.As the biologist Sean Carroll (2005) puts it, "Evolution is not just a biological subject, it is the foundation of the discipline. Biology without evolution is like physics without gravity" . (p.52)
There is such an example from psychology.Suppose that two therapies are developed to help alleviate the problems of a child with severe dyslexia.Neither therapy has been empirically tested.The first, Treatment A, is a training program designed to promote children's recognition of fragments of language at the phonemic level.The second, treatment B, trains the receptivity of the vestibular organ by letting the child walk the balance beam blindfolded.Treatment A and Treatment B agree in one respect—their effects have not been directly tested empirically, and neither has received a good response.One of these treatments, however, has the upper hand in terms of the principle of associativity.Treatment A is consistent with broad consensus in the research literature that children with severe dyslexia are hindered because the child has not yet developed an adequate awareness of the segmental structure of language (Snowling & Hulme , 2005; Vellutino et al., 2004).Treatment B was not associated with any corresponding academic consensus.This difference in association suggests that treatment A is a better choice, even though neither has been directly tested.
This tendency to make Einsteinian innovations typical of science tricks us into thinking that all scientific progress consists of giant leaps.The problem is that people tend to generalize these examples into a notion that scientific progress is supposed to happen this way.In fact, progress in many fields of science does not rely on a sudden breakthrough, but consists of a series of pauses and iterations between advances that do not constitute a major impact.
The uncertainty of scientific work is largely unknown to the public.Scientific experiments rarely completely determine a question, or support a theory to the exclusion of others.New theories rarely comprehensively transcend all preexisting competing conceptual systems.The determination of many problems is not determined by a key experiment as depicted in science movies, but until the scientific community gradually begins to agree that the evidence for a certain theory is stronger than the evidence for any other theory much.Instead of evaluating data from a single, perfectly designed experiment, scientists evaluate evidence from dozens of experimental papers, each with its flaws, but all of which provide part of the answer.This incremental mode of scientific development is hampered precisely because Einstein Syndrome has created a mind-set in the public that all sciences are the same as physics, because for physics the leap mode of scientific progress Perhaps the most applicable.
Consider the leaps and bounds in genetics and molecular biology over the past century.These advances did not come about because a giant Einstein came along at a critical moment and fixed everything.Instead, dozens of inspirations and insights from hundreds of flawed experiments have led to the integration of modern biology.The occurrence of these advances does not rely on the revolutionary reconstruction of some major concepts, but the long-term and repeated confrontation and confrontation between several tenable different explanations.After more than ten years of inconclusive experiments, countless theoretical ideas, debates and criticisms, scientists finally figured out whether genes are composed of proteins or nucleic acids.They reach a new consensus, but not through a leap of faith.Ernst Rutherford, the discoverer of the atomic nucleus, emphasized the importance of the principle of relevance, "A scientist cannot rely on the opinion of only one, but on the wisdom of thousands" (Holton & Roller, 1958, p.166 )
Rutherford's point highlights another way of distinguishing science from pseudoscience.Science, always guided by the principle of connectedness, is characterized by the participation of many individuals whose contribution is judged by the extent to which it advances our understanding of the natural world.No single individual can dominate the discussion by virtue of its special status.Of course, in Chapter 1, we have already discussed this public nature of science. In contrast, pseudoscience often believes that specific authorities and researchers have "special" opportunities to approach the truth.
We have suggested two ideas that provide a useful context for understanding the rules of psychology.First of all, no experiment in science is perfectly designed, and there are uncertainties in the interpretation of any experimental data.When evaluating a theory, scientists often do not wait for a perfect or critical experiment to appear, but evaluate the overall trend of a large number of partially flawed experiments.Second, much science has progressed even without Einstein.These advances have been staggering and meandering rather than stepping forward through the great "Einsteinian integration."Like psychology, many other sciences are the accumulation and splicing of knowledge that originally lacked a common theme.
The previous discussion led to a principle of evidence evaluation that is of central importance in psychology.It is often referred to as the principle of aggregate evidence (or the principle of operational aggregation).Scientists and those who use scientific knowledge often have to make judgments about what the body of evidence actually says.In such cases, the principle of aggregated evidence becomes a very important tool.The principle of convergent evidence is also a useful tool for lay users of scientific information, especially when they are evaluating psychological claims.Although an exhaustive technical discussion of the concept of aggregated evidence will quickly turn our heads, in fact, the practical application of this concept is easy to understand.We will explore two ways of formulating this principle, one in terms of the logic of "flawed experiments" and the other in terms of theoretical testing.
Taken to the extreme, there are countless ways for an experiment to go wrong (or, to use the term, to get confused).In most cases, however, there tend not to be so many interfering key factors.Scientists with extensive experience in a particular field often have a good idea of what the most critical factors are.As a result, scientists can always spot critical flaws in an experiment when reviewing a study's results.Next, the principle of aggregated evidence prompts us to examine patterns of flaws in the relevant research literature that either support or disprove our desired conclusions.
Assume that results from a large number of different experiments consistently support a particular conclusion.If the experiments themselves are not perfect, we should continue to assess the nature and extent of these research flaws.If all experiments were flawed in the same way, these circumstances would reduce our confidence in the conclusions of the experiments, since the consistency of the conclusions might be due to only one particular flaw, common to all experiments; On the other hand, if all the experiments showed different flaws, our confidence in the conclusion would be greatly increased, because the consistency of the results does not appear to be due to a single confounding factor that muddles the results of all the experiments.As Anderson (1996) put it, “Different methods are likely to involve different hypotheses. When a hypothesis can pass many falsification tests based on different hypotheses, we can say that we have obtained a strong conclusion” (p. 742).
Each experiment helps to correct the design errors of the other experiments, which in turn examines its flaws and makes them supportive.Although each has different shortcomings and experimental techniques have their own advantages and disadvantages, as long as a large number of experiments can obtain similar results, then we can say that our experimental evidence has achieved aggregation.Even though none of the experiments were perfectly designed, we got a fairly convincing result.Thus, the principle of convergent evidence allows us to base conclusions on a large number of somewhat different experimental sources.The reason why this principle allows us to obtain convincing results is that the consistency of the results obtained by this method is unlikely to be caused by the specificity of an experimental procedure.
Convergent evidence principles can also be formulated in the form of theoretical tests.Research is highly convergent when a series of experiments consistently supports a putative theory while collectively ruling out the most dominant competing theories.Although no single experiment can rule out all possible explanations, a series of experiments with some diagnostic power (if all the data show a certain trend) can lead to a very convincing conclusion.
For example, suppose five different theories (call them A, B, C, D, and E) exist simultaneously for a certain phenomenon, all of which have been tested by a series of experiments.Suppose some experiments test theories A, B, and C with strong force, and the resulting data rejects A and B and supports C.Now imagine that other experiments test theories C, D, and E with equal force, and that the data disprove D and E, and support C.In this case, we have strong convergent evidence for theory C.We not only have data in favor of Theory C, but also against other competing explanations.To emphasize, no single experiment can test all theories, but taken together, a series of experiments can make strong inferences.
Conversely, if all known studies only robustly test B, C, and E, and the data support C and disprove B and E, then theory C is not as strong as in the previous example.The reason is that, despite the generation of data supporting theory C, there is still no strong evidence to rule out the other possible theories (A and D).Research is thus highly convergent when a series of experiments consistently supports a putative theory while collectively ruling out competing theories of great importance.Although no single experiment can rule out other possible explanations, a more convincing conclusion can be drawn if a series of partially diagnostic studies are aggregated in the manner shown in the above example.
Finally, the principle of aggregated evidence enables us to dispel a myth that arose from the oversimplification of our discussion of falsification in Chapter 2.The discussion at the time seemed to make people think that when the first evidence contradicting one's theory appeared, the theory was considered falsified.However, this is not the case (Pigliucci, 2002).Just as a theory is supported by aggregated evidence, it is disproved by aggregated research findings.
The reason for emphasizing the importance of aggregation is that psychological conclusions are often based on principles of aggregated evidence.This fact is certainly not unique or uncommon (and in many other sciences conclusions are not based on a single, conclusive experimental evidence, but on numerous experiments with unclear results).But this is particularly the case in psychology, where experiments tend to be less diagnostic.That is, the data in support of a theory can often only rule out a small subset of possible explanations, leaving behind many "alternate" theories that might replace it.As a result, convincing conclusions can only be drawn after collecting and comparing data from a large number of studies.
Psychology experiments have a high element of ambiguity, not surprisingly, since the questions they study involve complex human behavior.If psychologists were open to acknowledging this fact, and then patiently explaining the consequences of that fact, the public would be better able to understand the science.Psychologists should acknowledge that, although psychological science exists and is advancing, progress has been slow, and many conclusions have often come from excruciatingly long periods of synthesis and debate.We should always be skeptical of so-called breakthroughs that the media often proclaims, but the skepticism that psychological claims are subjected to is real.
In psychology, we have to walk a tightrope with caution.For example, we must resist the temptation to treat a hypothesis as a proven theory when the evidence is not convincing.This skepticism is emphasized repeatedly in successive chapters of this book.Be careful not to infer causation from correlation, rejecting witness narrative evidence.At the same time, we shouldn't overreact to things like incomplete knowledge and unexplored conclusions, and start to doubt that psychology can produce convincing conclusions at all.Nor should we be seduced by the irrational claim that psychology cannot be a science.From this position, the principle of aggregated evidence can be used to balance overinterpretation of hypothetical knowledge.Although all psychological research has flaws of one kind or another, aggregation allows us to draw convincing conclusions.
The best way to demonstrate the principle of convergent evidence is to examine some of the still controversial areas of psychology.Let's look at the importance of the principle of convergent evidence through an example.The question is whether exposure to violent television programs increases aggressive behavior in children.On this issue, the current scientific consensus is that watching violent TV programs (or movies) can indeed increase aggressive behavior in children.This effect is not very large, but it is real.Scientists' confidence in this conclusion does not come from a single, authoritative study, but from the aggregation of many research results (eg, Anderson, Berkowitz, Donnerstein, Huesmann, Johnson, Linz, Malamuth, & Wartella, 2003; Anderson & Dill, 1999; Anderson & Huesmann, 2005; Anderson, Huston, Schmitt Linebarger, Linebarger, & Wright, 2001; Bushman & Anderson, 2002; Paik & Comstock, 1994).This research conclusion applies to video games, television, and movies (Anderson & Bushman, 2001).The studies used varied widely in study design, subject size, and specific techniques, but it is now clear that these differences are strengths rather than weaknesses of the various studies in this field.
Despite the overwhelming evidence that television is an industry that can negatively impact children, the bosses of the television networks and video game industries naturally resist the evidence.They launched a campaign to mislead the public, taking advantage of the public's "failure to appreciate that findings are based on the aggregation of many studies rather than a single, conclusive proof."The network companies continue to cherry-pick individual cases and suggest that all it takes to prove that each study is flawed is to disprove the overall conclusion.Although social science researchers may respond to criticism of a particular study, it does not follow that researchers always readily admit that a particular study is flawed.The key difference is that the researchers rejected the suggestion that acknowledging that a particular study was flawed would negate the general scientific consensus that film and television violence has an impact on aggressive behavior.The reason for this is that general conclusions derive from aggregation.Even studies that do not contain such flaws have results that point in the same direction.This study has its own problems, of course, but other studies have corrected for this and come to similar conclusions.
On this issue, for example, early research has revealed a correlation between the amount of viewing of violent programming and aggressive behavior in children.Such correlative evidence cannot be considered causal, and it is very right to point out that.Perhaps a third variable is responsible for this association, perhaps more aggressive children choose to watch more violent programming (a question of directionality).
But the scientific community's conclusions are not based solely on relevant evidence.Researchers not only perform simple measurements of the association between two variables, but also use more complex correlation techniques that allow researchers to draw some tentative conclusions of causal nature (one such as partial correlation, in mentioned in Chapter 5).One such technique uses a longitudinal design in which the same two variables—in this case, television violence and aggression—are measured at different points in time.The correlation patterns obtained by this design can tell us whether the two are causally linked.Studies of this kind have already been done, and the results show that watching violent TV programs may indeed increase people's later aggressive behavior.
Likewise, it is not unreasonable to criticize vertical correlation technology as controversial, because it is.The point is that the conclusion that there is a causal link between TV violence and aggressive behavior does not rest on simple or complex correlative evidence, since researchers have also conducted countless laboratory studies in which TV The numbers of violence are directly manipulated, not just assessed.In Chapter 6, we discussed the manipulation of variables, and the joint use of control methods such as manipulation and random allocation can avoid the shortcomings of related research in explaining problems.If there are two groups of children who, after all other variables have been experimentally balanced, still exhibit different levels of aggressive behavior; if the only difference between the two groups of children is that one group watches violent programs and the other does not , then we can make the correct inference: the manipulated variable (TV violence—independent variable) caused the change in the outcome variable (aggressive behavior—dependent variable).This result has appeared in most experimental studies.
These studies have fueled some of the "this isn't real life" objections that have been discussed in previous chapters, along with baseless accusations.In any case, the effects of TV violence are not specific to a particular group of children, as these results have been confirmed in different parts of the United States and in different countries around the world.Various studies using different experimental situations and different TV programs as experimental stimuli have obtained highly consistent results.
Importantly, the same conclusions were also drawn from field experiments rather than laboratory experiments.A design called a field experiment has also been used to study television violence/aggressive behavior.The existence of this type of research design reminds us not to assume that there is a necessary connection between the experimental situation and the experimental design.It is sometimes believed that we can only manipulate variables in the laboratory and that we can only conduct related research in non-laboratory settings.This idea is incorrect.Related research is often done in the laboratory, and variables can often be manipulated in non-laboratory settings.Although considerable creativity is sometimes required to manipulate variables in non-laboratory settings to conduct field experiments, this approach is increasingly used in the field of psychology.
Of course, field experiments themselves have flaws that are often the strong point of other studies.Overall, the evidence used to link viewing of TV violence to increased aggressive behavior in children does not rely solely on one study or even one type of study.
This situation is similar to the relationship between smoking and lung cancer.Tobacco bosses often try to mislead the public by implying that smoking causes lung cancer is based on individual studies, and then criticize individual studies.On the contrary, there is a lot of converging evidence in support of this conclusion.The aggregation of data from different studies is strong, and the aggregation of these data will not be completely changed by criticism of a certain study.
In fact, it is necessary to discuss a scientific issue similar to the cause of lung cancer here.Many decisions in medical diagnosis and treatment are based on whether the results of different studies can be converged into a single conclusion.For example, when different types of epidemiological investigations (so to speak, a field study involving humans seeking to link a disease to environmental and geographic factors), carefully controlled animal experiments, and clinical trials with human subjects When the research results tend to converge on one conclusion, the medical profession will have greater confidence in this conclusion and believe that this conclusion is reliable, and doctors will be willing to implement treatment plans on the basis of these evidences.
However, all three types of research have their own drawbacks.Epidemiological studies are often correlational, and the likelihood of spurious correlations between variables is high.Laboratory research can be highly controlled, but the subjects are often animals rather than humans.Clinical trials in hospital settings use human subjects in a true therapeutic setting, but there are still many control issues due to the placebo effect and the expectation effect of the medical team treating the patient.As in the smoking and lung cancer examples, medical researchers can draw convincing conclusions when data from different methods are pooled strongly, although each study has its own problems.This is in the same way that psychologists can use principles of convergent evidence to help them conclude that television violence has an effect on aggressive behavior.
The problem of assessing the impact of television violence is a classic example of how, in psychology, data ends up being used to solve problems.Especially in areas of urgent societal concern, it is important to remember that the answers to these questions will only emerge slowly through the convergence of results from a large number of different studies.It is impossible to solve these problems through a single breakthrough research.To sum it up with a simple principle: when evaluating empirical evidence in psychology, what is in mind is "scientific consensus", not "major breakthrough"; it is "gradual integration", not "big leap".
The principle of "consensus, not breakthrough" can be illustrated by the controversy over "compensatory early childhood education programs."In the late 1960s and early 1970s, when the debate was raging over whether President Lyndon B. Johnson's "Quality Society Program" was actually working, the public would often see it in newspapers Headlines like: "Early Intervention Can Raise IQ by 30 Points" and "Mind Opening is a flop" and so on.How should a layman face such contradictory information?In this example, the principle of “scientific consensus rather than major breakthroughs” can certainly help, as it reminds us that both newspaper headlines may be premature.In fact, it took another decade for researchers to reach a scientific consensus on this important social issue.
这一共识的产生并非源自于某个单独的重大研究成果，而是当康奈尔大学的一组研究人员（Lazar, Darlington, Murray, Royce, & Sniper, 1982）在20世纪60年代和70年代早期将来自11个不同早期教育项目上的几百个被试的数据汇总起来分析时，这种共识才得以建立。尽管单一项目的研究结果有时候很难去解释，但当它们汇聚在一起时，整体的研究结果就非常清晰了。早期教育干预的短期项目没有顺理成章地让IQ增加30分。另一方面，心智开启计划以及一些类似项目也并没有绝对失败。早期教育干预项目的确能对参与此项目的儿童后继的教育历程产生具体的影响。这些儿童更少会留级，更少被安排到特殊教育班，而且对学校和学业成绩有更为积极的态度，并持续表现出学业成绩提高（也可见Lee, Brooks-Gunn, Schnur, ScLiaw, 1990; Ramey, 1999）。
加拿大心理学家提莫西·摩尔（Timothy Moore, 1996）认为，如果人们能更加普遍地意识到聚合性原则，那么在法庭上将会更好地利用专家证词。他特别讨论了依赖专家证词的问题。证词属于个人意见，难以代表该领域专家的共识。摩尔引述了在一案中的专家证词。这起案件涉及两个青少年的自杀，他们的父母控告摇滚乐队“犹大圣徒”在歌曲中传达的潜意识信息诱发了他们孩子的自杀。尽管专家证词指出，当时的科学共识是：没有任何证据显示那些潜意识信息能产生这种效果（即使是现在，这一共识仍然成立），然而，在一个不能反映实证性共识的学者进行了一番言之凿凿的心理动力学解释之后，这个案子的法官多少还是受了些影响。摩尔总结说，这个学者误导了法庭，“他的观点虽然极富想象力和逻辑性，但与当时对于此问题的主流科学理解相悖。长长的履历和尊贵的职位并不足以保证其观点是科学有效的，单个专家的证词是独特的、个人化的，并且未经更广泛的科学团体的认定，这样的专家不足以引导整个法庭”（p.38）。
聚合原则同样也意味着，我们应当乐于看到多种不同方法应用于各个心理学研究领域之中。因为不同的研究技术各有其优势和不足，用于获得特定结论的各种方法之间呈现一种相对的平衡是比较理想的。心理学长期以来都因过于依赖基于实验室的实验技术而受到诟病。这种批评在心理学家之间也是个争议性的话题。然而，一种确定无疑的趋势是，近年来，心理学各个领域都已经开始使用不同的研究方法了。例如，由于过度依赖实验室技术，社会心理学家遭受的批评可能是最多的，但社会心理学家已经开始转向了更富想象力的现场设计，以寻求聚合性的证据来支持他们的理论（Kunda，1999）。
心理学者比伯·拉坦（Bibp Latane）和约翰·达利（John Darley）的研究为此提供了一个很好的例子。这些研究者们因对“旁观者现象”的研究而广为人知。旁观者现象是指，一些人在看到他人处于危难之中时并不施以援手。拉坦和达利（1970）指出了这样一个事实：在很多危急时刻，当有其他旁观者在场时，某个旁观者伸出援手的可能性会更低。
然而，这两位研究者清楚地知道，这些仅凭被试在实验室里的反应而做出的结论太过单薄了。因为在实验室中，被试都是在自愿报名到实验室来参加实验之后才目睹紧急事件的。为此，拉坦和达利设计了另外一个有趣的实验，希望在另一个情境中重现这一现象。他们找到一个愿意合作的卖酒的商店，该商店同意假装店里发生了盗窃事件。当收银员在店铺的后面为一个“顾客”拿啤酒时，该“顾客”（实际上是研究者的同伴）拿起一箱啤酒走出店门。这一幕总发生在收银台前一个或两个真正的顾客的眼皮底下。收银员回来后问这一个或两个顾客，“嗨，刚才在这里的那个人到哪儿去了？你看见他离开了吗？”这样，就给了顾客一个机会向收银员报告刚才发生的盗窃事件。与实验室实验的结果吻合：当旁观者在场的时候，向收银员报告盗窃案的行为受到了抑制。
社会心理学家并不是唯一试图在不同的情境中重复其研究结果的人。认知心理学家们也开始探索如何推广他们的许多实验结果。例如，吉格伦泽（Gigerenzer, 1984）研究了“频率-效力效应”的普遍性。这个效应是指，一个陌生但看似有理的论断，不管是真是假，只要经过不断地重复，就会增加人们对它的相信程度。这个效应成功地得到了重复验证，但是这些研究都是在实验情境中，以大学生为被试（并且绝大多数在美国）。于是吉格伦泽做了一个非实验室情境、以非大学生为被试的研究。他在德国慕尼黑测试了许多非大学生的成年人，测试在这些人家里进行，也发现了“频率-效力效应”，而且其程度与实验室中美国大学生被试所得的几乎相同。
在第10章中，我们将讨论许多带有概率性质的决策原则，这些决策原则最早都产生于实验室，但都经过了现场式的检验。例如，研究者检验了理疗师、股票经纪人、陪审员、经济学家及赌徒在各自所属情境下，是以何种方式做出概率推理的（Belsky & Gilovich, 1999; Gilovich, Griffin, & Kahneman, 2002; Hilton, 2003; Kahneman & Tversky, 2000）。行为决定理论的原则业已用于许多应用性的领域，例如，决定丹佛市警局使用哪种型号的子弹最为理想，以及决定是否在亚利桑那州中部建立大坝（Hammond, Harvey, & Hastie, 1992）。
实验与非实验结果的聚合性也成为教育心理学领域的突出特点。例如，针对不同课程安排所做的实验研究和现场研究都表明，早期语音教学有助于阅读技巧的习得（Ehri, Nunes, Stahl, & Willows, 2001; Pressley, 2005; Snowling & Hulme, 2005; Stanovich, 2000; Vellutino et al., 2004）。
总的来说，当前的心理学研究采用多种类型的实验技术和情境。尽管对于很多问题的研究有时候过于集中在使用某些特定的技术，但在心理学中，研究方法的多样性比过去几年丰富多了。
对于某个特定问题的研究，通常是从相对较弱的方法过渡到可以做出较强结论的方法。例如，研究者对某个特定假设的兴趣，常常源于某个异常感兴趣的特殊个案。正如我们在第4章中讨论的，这就是个案研究的真正作用：为更有效力的进一步研究提供一些假设，同时激发科学家们用更为严格的方法去研究这些假设。个案研究之后，研究者多采用相关研究来确认变量之间是否存在真正的关联，而不仅是存在于几个个案中的巧合现象。如果相关研究证实了变量之间的关联，研究者就开始尝试采用实验法来对相关变量进行操纵，借以找到变量之间可能存在的因果关系。这个递进的顺序就是：从个案研究到相关研究，再到操纵变量。尽管并非每个研究领域都遵循这个渐进式途径（有时不同类型的研究同时进行），但这一向更有效方法迈进的进程的确是普遍发生的。
在讨论“向更有效的研究方法迈进”之前，我们必须纠正读者的一个错误概念，这个错误概念源于第5章的讨论，那就是“相关研究在科学中没有什么用处”。的确，当一个因果关系的假说需要验证时，操纵变量的研究方法更受青睐。然而，这并不意味着相关研究对于知识的获得毫无帮助。首先，许多科学假设是以相关或者不相关的形式来表述的，因此这类研究是在直接验证这些假设。第二，尽管相关并不意味着因果关系，但因果关系一定包含相关。也就是说，如果一个相关研究不能肯定地证实因果关系的假设，那它可以起到排除这一因果假设的作用。第三，相关研究或许比它们看上去更有用，因为最近新发展的复杂相关设计可以让研究者做出有限的因果推论。我们在第5章讨论了偏相关这种复杂的相关技术，这一技术有可能检验出变量间的关联是否能够被第三变量所解释。
然而，最重要的原因可能在于，有时出于道德或伦理的考虑，我们无法对一些变量进行操纵（例如，营养不良或肢体残障）。而另外一些变量，诸如出生顺序、性别、年龄等，则因其无法被操纵而具有天然的相关性，涉及它们的科学知识也因此必须建立在相关证据基础上。当然，这一情况并不是心理学领域所独有。天文学家们显然无法操纵所有影响其研究对象的变量，然而他们依然能够做出结论。
在健康心理学中，有一个研究方法演进的例子，它涉及A型行为模式和心脏病之间的关系（Curtis & O' Keefe, 2002; Matthews, 2005; Smith, 2003; SulsScBunde, 2005）。最初，A型行为模式这一概念源于两位心脏病专家的观察，这二位医生从他们一些病人的行为中发现了一种稳定的模式，这种行为模式包括时间紧迫感、飘忽不定的敌意，以及对成就的极度渴求。于是，一些医生通过对少数个案的观察，提出了“A型人格”这一想法。这些个案研究提出了这个概念，但并不能作为有力证据来证明这种特定的行为模式是导致心脏病的原因之一。要证明这一点，需要的不仅是少数几个个案研究，它还需要由心脏病专家、生物化学家和心理学家团队数十年的努力。
很快，这个研究从永远也不可能证实假设的纯粹个案研究，转向了更有效力的研究方法。研究者发展和检验了A型行为模式的操作性定义。大范围的流行病学研究证实了A型行为和心脏病之间的相关性。然后这种相关研究工作就变得很复杂了。研究者使用复杂的相关技术来搜寻潜在的第三变量。由于行为模式与其他传统心脏病风险因素中的一种（例如吸烟、肥胖和血液中胆固醇水平）存在相关，因此A型行为和心脏病之间有可能存在虚假相关。当其他的变量在统计上被排除后，A型行为模式和心脏病之间仍然具有关联。
最后，研究者釆用了实验研究对变量进行操纵，以期证实二者间是否具有因果关系。一些研究试图去验证是否某些生理机制影响了两者之间的关系，并以动物作为被试——某些人所谓的“不是真实的生活”的研究方法。另外一些研究则以犯过心脏病的人为被试。这些被试被随机分配到两个组中的一组。一个组接受咨询，帮助他们避免传统的风险行为，例如吸烟或者吃高脂肪食物；另一组在接受同样的咨询的同时，还接受了一个以减少他们的A型行为为目的的训练项目。三年之后，在接受A型行为辅导的病人中，心脏病复发的情况要明显少很多。
简而言之，证据汇聚起来支持了“A型行为模式是导致心脏病的重要原因”这一假设。对这个问题的研究提供了一个很好的范例，从中我们能清楚看到，研究是怎样从一个感兴趣的个案研究走向相关技术，最后到可以操纵变量的实验研究的。
我们能从这个例子中得到的最后一点经验就是，科学概念总是在不断地演进。这个论点是在第3章讨论操作性定义时首次提出的。最近的研究似乎表明，将A型行为与心脏病之间的关系说成是整体性的显得过于简单化了。原因在于，只有该概念中的特定成分（特别是对抗性敌意）才与心脏病有关联（Curtis & O' Keefe, 2002; Matthews, 2005; SulsScBunde, 2005）。因此，这是个很好的例证，从中可以看出，随着科学的进步，它是如何不断地揭示特定的关联，以及理论概念是如何被细化的。
聚合性原则的最后一个启示是，当一个问题的最初的研究结果看上去有些矛盾时，我们不应当对此感到绝望。在科学中，证据融合的过程就像投影仪慢慢将一张未知的幻灯片的焦点调清晰。起初，屏幕上的模糊影像可能代表任何东西。接着，随着一点点地调整焦距，虽然这个图像仍不能被清楚地识别出来，但许多其他的可能假设也许会被排除。最后，当焦距调准，就可以非常有信心地做出最终的判断。证据融合过程就好比一个调焦过程。幻灯片的模糊影像就如同互相矛盾的数据，或者是那些支持多重假设的证据。
因此，研究早期所获得的矛盾数据不应该让我们对发现真相感到绝望。类似的情况不光发生在心理学领域，同样也发生在一些相对成熟的科学中。的确，公众经常意识不到科学中经常会得到一些矛盾的数据。这些矛盾只不过是因为我们对问题理解得还不够充分，这些矛盾还可能仅仅是偶然事件（我们将会在第11章中对此展开讨论），或者源于不同实验在方法上的细微差异。
在达成共识之前，其他许多科学也都经历了令人困扰的不确定时期（Ioannidis, 2004; Simonton，2004）。格兰德威尔（Gladwell, 1996）讲述了近来关于脑创伤患者紧急救治认识的演进过程。一名纽约患者非常幸运地得到了世界顶级专家之一简姆·加哲医生（Drjam Ghajar）的治疗。这位医生始终致力于改变该领域中一个临床上的错误看法。格兰德威尔说，若干年前，当加哲和其他五位研究者在对一些创伤治疗中心进行调研时发现，尽管类固醇已经被反复地证明无助于减少颅内压力（而且会带来潜在危害），然而仍有75%的昏迷状态的病人是用类固醇来治疗的。He wrote:
当谈到几年前他的同行的观点时，加哲说，并不是说神经外科医生太过懒惰，而是这儿的信息太过庞杂，让人感到困惑（p.39）。
简而言之，和心理学的众多领域一样，该领域也充斥着许多尚未聚焦的研究，并且未能以一种有利于找到聚合性的方式加以概念化。因此在1994年，加哲和他的几个同事参加了一系列学术会议，在这些会议上，他们试图对所有的证据进行综合，以期发现某种聚合性。这些会议是由大脑创伤基金会发起的，研究者们审阅了涉及大脑创伤处理的14个领域、超过4000份的科学论文。大脑创伤基金会的执行主席描述了神经外科专家是怎样工作的：“他们所做的工作是对科学文献的证据进行论证，一旦有人说'我以往的经验就是如此'，所有人都会说，'哦，不，那不算，我们要看到证据'”（Gladwell, 1996，p.40）。最后的结果被证明是富有成效的：
从这个例子可以明显看出，并不仅是心理学的研究领域中遍布各种发现，在其他科学中也不乏这种由于数据模式的模糊性而难以达成一致结论的例子。在一篇名为“图片问题”的文章中，格兰德威尔（Gladwell, 2004）讨论了人们为何很难理解医生对于乳腺X光片的作用还存在着分歧。这是因为乳腺X光透视在大多数人看来是如此地“精确有力”，以至于他们认为仅凭它就能做出确诊。其实这些人不理解，医生的诊断虽必不可少，但乳腺X光片评估和疾病预测从本质上来说是具有概率性的。格兰德威尔说，“图片保证确定性，但它不能兑现这种承诺。经过40年的研究之后，对于女性在50岁至69岁期间接受乳腺X光透视的益处，仍然存在着不小的分歧。进一步的争议则在于，是否有足够的证据能够证明，50岁以下和70岁以上的女性定期需要接受乳腺X光透视检查”（p.81）。然而格兰德威尔继续谈到，和心理学领域一样，在医学领域里，知识即使不确定也依然有用：“答案是乳腺X光透视不需要完全准确无误才能拯救生命……它没有我们想的那么好。但总归比没有它要强”（p.81）。
在心理学和其他科学里，将来自不同研究的证据整合起来形成一个结论，已经能够通过一种更为正式的方法来实现，这就是一种叫做元分析的统计技术（Cooper & Hedges, 1994; Hunter & Schmidt, 1990; Rosenthal, 1995）。在医学语境中，元分析就是：
使用元分析来确定心理学结论的有效性，和医学的情形是类似的。两个实验组的比较得出的效应，可以纳入一个常规的统计矩阵中，这个矩阵能进行研究之间的比较。接着，这些结果以一种标准化的方式加以统计整合（Cooper & Hedges, 1994; Hunter & Schmidt, 1990）。如果整合过程达到了一定的统计学标准，就能形成一个关于这些效应的结论。当然，在某些情况下，有可能无法确定地得出一个结论，这时元分析的结果就是非结论性的。
越来越多的评论者开始呼吁，应更加重视元分析，并将之视为一种方法，来消除科学领域内相互对立研究之间的不断争议。这种方法有助于终止这种“公说公有理，婆说婆有理”的争论。对元分析的强调也揭示了一种观点：专业杂志上常见的观点对立可能只是表面现象，实际上我们拥有更多可靠和有用的发现。
国家阅读评审小组（NRP, 2000; Ehri, Nunes, Stahl, & Willows, 2001）对一些关于阅读教育的研究所做的元分析就证明了这一点。例如，他们得出结论，对38个不同的研究结果的元分析“有力地支持了这一观点，即相比其他课程提供的非系统或非语音教学，系统的语音教学在孩子的成长中发挥了更大的作用”（p.2-84）。在报告的另一部分，NPR报告说，对于52个语音意识训练研究的元分析说明，“教孩子掌握在语言中运用声音，能帮助他们学会阅读，在不同的教学、测验及参与者的个性条件下，其效应量都远远大于随机水平，并且，虽然这些效应有大有小，但大部分都处于中等水平”（p.2-5）。
美国心理学会的一支工作团队在心理学期刊上所做的关于统计方法的一番阐述，为本节内容提供了一个恰当的总结（Wilkinson，1999）。这个工作团队说：“研究者不应仅针对单个研究的结果做出解释，就好像其他文献所报告的结果与之毫无关系似的”（p.602）。不同研究结果之间达成聚合效应，才有利于推动科学进步。一个研究的结果也只有通过针对特定问题的诸多研究获得聚合性解释，才是有意义的。
在这一章中我们看到，为何“跃进”模式对于心理学来说是一种糟糕的模式，以及为什么“渐进整合”模式提供了一个更好的框架，凭借这个框架，我们就能够理解心理学中的结论是如何形成的。聚合性证据原则描述了心理学上研究结果是如何被整合的：没有一个实验是可以一捶定音的，但是每一个实验至少都能帮助我们排除一些可能的解释，并让我们在接近真理的道路上向前迈进。多种不同方法的使用让心理学家更为确信，他们的研究结果是建立在稳固的实证基础上的。最后，当概念上的变化发生时，它必须遵循关联性原则：新的理论不仅要能解释新的科学数据，还必须能解释已有的数据。
Notes:
Notes: