On the basis of the research on SSR one would expect AR to give positive results. Students are, after all, provided with more access to books and more reading time. But the results presented thus far, with the exception of the Idaho study, are not impressive. Sometimes claimed gains diminish when we consider published data from the school districts and the state and/or examine the reported data carefully. In fact, only the Idaho study provides truly strong data.

In addition, the Renaissance Independent Research Reports, according to Renaissance, are "sent in by independent researchers." They are suggestive and interesting, but are not scientific studies. It is possible that there is a selection bias: It is possible that only the apparently successful cases are submitted to Renaissance, and it is possible that only the apparently successful ones are posted on the website. I have of course no evidence that this is so. But in scientific studies, results are published regardless of whether they favor one hypothesis or another.

Even if AR did well in these studies, they would not demonstrate its efficacy: it is possible that the only aspect of AR that has a positive effect is the increased access to books and increased time to read them. The only way to settle this is a controlled study.

AR Students Compared to Traditionally Taught Comparison Groups

In the following studies, a real control group is used. In each case, however, the control group followed traditional language arts instruction.

Peak and Dewalt (1994) investigated the impact of AR on 50 randomly selected ninth graders in North Carolina who had used the system for five years; these students made better gains in reading than comparison students. Over five years, experimental students went from a mean of 716 (scale score) on the California Achievement Test (CAT) at grade 3 to a mean of 788 at grade 8, an average gain of 14.4 points per year, while comparisons went from 724 to 766 (8.4 points per year).

In Mathis (1996) the AR group consisted of 30 sixth graders. Comparisons were the same children one year earlier. Thus Mathis measured the effect of AR on a group of children during grade six (one year duration) and compared gains to gains made by the same children in grade five. Using AR, the children dropped about 1/2 of a point (.53) on the Stanford Achievement Test of reading. The year before, they fell 1/4 of a point (.27). The decline with AR was slightly greater, but these are very small declines. Both groups gained approximately one year in one year's time and AR made no difference.

Pavonetti, Brimmer, and Cipielewski (2003) administered the Title Recognition Test to seventh graders in three districts. The Title Recognition Test is a checklist that correlates highly with various measures of reading achievement (Cipielewski and Stanovich, 1992; Allen, Cipielewski, and Stanovich, 1992). Some of the titles included were: A Wrinkle in Time, The Outsiders, and My Side of the Mountain. For all districts combined, Pavonetti et. al. reported no difference between those children who had had AR in elementary school and those who had not.

Table 8: Studies with Comparison Groups doing Traditional Instruction

Table 8 summarizes studies with comparison groups doing traditional instruction. One would expect AR and related programs to do well in these studies, as experimental students clearly read more than comparisons. Only Peak and Dewalt (1994), however, provide positive results. The other two, one with a long duration, showed no difference between AR and comparisons.

8