Sample from Chapter 7 of Becoming a Critical Thinker

....The testing of causal hypotheses in science is based, in part, on the same principle as polling. Each is based on the principle that we need only study a part of a class to gain knowledge about that class in general. A scientist does not have to study every virus of a certain type in order to draw justifiable conclusions about that type of virus. Since the number of individual viruses of any given type might be extremely large, it is possible that in some cases the percentage of individuals of a class actually observed might be close to zero. The same would be true for an astronomer studying any particular type of star. No matter how many stars astronomers actually observe, that number will be an infinitesimally small fraction of all stars. The same would be true for a chemist studying human DNA. And the same is true for a pollster studying the opinions or behavior patterns (beliefs and actions) of human populations. For example, there are about 30,000,000 Californians. The poll which concluded that 78% of all Californians are "extremely concerned" about crime was based on a sample of 1,003 adults, or 0.0033 percent of the total population were in the sample. That is, each person polled represented about 30,000 others. The sample may seem like a small percentage of the population, but it is immensely larger than any percentage of the "population" a chemist, biologist or physicist is likely to study in a lifetime of scientific investigation of molecules, cells, or atomic particles. However, the samples studied by the chemist, the biologist and the physicist are generally homogenous. Once the type of item to be studied is identified, selecting which items to study is not a major problem since "one water molecule is pretty much like any other water molecule," etc. What matters is not what percentage of the total target population is in the sample. What matters is how likely it is that the sample is a good cross section of the target population. What matters is how likely it is that the items in the sample are typical of the target population.

More problematic than the typical size of polling samples is that opinions are treated as if they were observable, measurable and fixed qualities. The "technique of polling promotes the assumption that an opinion is a thing inside people that can be exactly located and extracted by the pollster's questions." Furthermore, it promotes the assumption that we ought to have an opinion on the issue being polled. Maybe what we ought to have is information that would help us make a reasonable judgment about the issue. Maybe questions ought to stimulate thought and discussion about an issue rather than end it by gathering and announcing some statistic. In other words, we probably ought to be more concerned with the encouragement of thoughtless opinionizing than with whether or not a sample of 1,000 is sufficient to warrant valid generalizations. Nevertheless, let's return to the issue of valid generalization.

The basic justification for empirical generalizations, whether by a pollster or by a scientist, is that the sample (the items observed and studied) be representative of the target population (all of the items of the class being considered). Insofar as we are justified in believing that a sample of a population is representative, we are justified in believing that we have knowledge about that population in general. Such knowledge can be used to do what the prophets of old allegedly did: predict the future and guide us in our actions.

Justified empirical generalizations can provide us with a useful means of facing the future because they are predictive. They tell us not only about the items we've observed and measured; they tell us about items we haven't observed or measured. Therein lies their beauty. Of course, all depends on their being justified.