Major distortions in the summary of the National Reading Panel report on phonics*

 

 


From Reading Process and Practice, 3rd ed. (2002)

by Constance Weaver, Western Michigan University

*This clarification deals with phonemic awareness as well as phonics in the concluding summary section.

______________________________________________________________________________

Terms relating to phonics

Phonological awareness--Awareness of the sound units in a language. In English, the largest sound unit is the syllable and the smallest significant sound unit is the phoneme.

Phoneme--Phonemes are what we have learned to hear as the smallest separate, or separable, sounds in a language: the smallest unit of sound that is capable of signaling a difference in meaning, as the sound / p / in pit and the sound / b / in bit differentiate the two words.

Phonemic awareness--Awareness that there are such separable sounds in words; the ability to hear separate phonemes in words. As currently used by skills researchers (such as most of the panelists on the National Reading Panel), phonemic awareness usually includes the ability to segment a word into phonemes and to manipulate phonemes in different ways.

Phonics--Letter-sound relationships, and the related skills used in analyzing words into phonemes or larger units and blending them to form recognizable words. Some researchers define phonics as teaching letter-sound relationships, or teaching relationships between the spelling and sound systems in a language.

______________________________________________________________________________

The official summary of the National Reading Panel report includes inaccurate claims about the teaching of phonics, when compared with the detailed analysis of the subgroup on phonics. Within a stretch of two paragraphs, the writers of the summary on phonics in the widely distributed 33-page booklet (NRP, 2000c) make several blanket statements that vastly overstate what the subgroup on phonics wrote in the detailed analysis. (References to both documents and to the important Minority View, by Joanne Yatvin, are included at the end of this discussion.)

Let us take each of three misleading statements in turn and contrast them with statements from the detailed report.

__________________________________________________________________

What is "reading," as discussed in the National Reading Panel report?

"Reading was defined to include several behaviors such as the following: reading real words in isolation or in context, reading pseudowords that can be pronounced but have no meaning, reading text aloud or silently, and comprehending text that is read silently or orally" (NRP, 2000c, p. 5).

__________________________________________________________________

Distortion from the Summary booklet

"The meta-analysis revealed that systematic phonics instruction produces significant benefits for students in kindergarten through 6th grade and for children having difficulty learning to read" (NRP, 2000c, p. 9). This implies that children in grades K-6 generally benefited from phonics instruction, which is not true.

But from the phonics subgroup report . . .

__________________________________________________________________

Phonics and comprehension

"Systematic phonics instruction is focused on teaching children the alphabetic system and explicitly how to apply it to read and spell words. Phonics skills would be expected to show effects on text comprehension to the extent that phonics skills help children read words in texts. This is one reason why phonics instruction may have exerted less impact on text comprehension outcomes than on word reading outcomes, because the impact is indirect. In addition, although phonics programs do give children practice reading connected text, the purpose of this practice is centered on word recognition rather than on comprehending and thinking about the meaning of what is being read. This may be another reason why effect sizes on text comprehension were smaller than effect sizes on word reading" (NRP, 2000b, p. 2-115).

Furthermore, there are decades of evidence—from miscue research as well as other observational/descriptive research—that focusing too much on word identification often keeps readers from focusing on meaning (e.g. Brown, Goodman, & Marek, 1996).

__________________________________________________________________

Distortion from the Summary booklet

"The ability to read and spell words was enhanced in kindergartners who received systematic beginning phonics instruction. First graders who were taught phonics systematically were better able to decode and spell words and to read text orally, and they showed significant improvement in their ability to comprehend text" (NRP, 2000c, p. 9).

But from the phonics subgroup report . . .

Distortion from the Summary booklet

"Older children receiving phonics instruction were better able to decode and spell words and to read text orally, but their comprehension of text was not significantly improved" (NRP, 2000c, p. 9).

But from the phonics subgroup report . . .

__________________________________________________________________

What about these errors in the influential summary of the report?

In a response to Elaine Garan’s first published critique of the NRP report (2001), Linnea Ehri and Steven Stahl had this to say:

Garan expresses many concerns about the clarity and accuracy of the writing in the reports, particularly with regard to differences between the Summary and the Reports of the Subgroups. We admit that some of the writing is not as clear as it should be. And Garan rightly notes that conclusions resulting from the distillation process in the Summary sometimes lack the details, qualifiers, and cautions that appear in the more extensive Reports of the Subgroups. However, the number of mistakes is small and the robustness of the findings is overwhelming" (Ehri & Stahl, 2001, pp. 18-19).

Ehri's unwillingness to admit that there are major distortions in the Summary booklet perhaps reflects the fact that she was the primary architect of that highly misleading section (NRP, 2000c). Unfortunately, few policymakers or teachers have discovered just how substantial and misleading these Summary claims are.

__________________________________________________________________

Interpreting NRP Claims About Reading Growth

In the phonics section of the National Readping Panel Report, the term "reading" usually refers to phonics and word identification skills and does not usually include comprehension. Only 11 of 75, or 15%, of the comparisons involving kindergartners and first graders and only 11 of 83, or 13%, of the comparisons involving grades 2-6 students involved comprehending "text" (NRP, 2000b, p. 2-151). True, a different list on the same page indicates that 35 cases involved comprehending "text," but that amounted to 16% of the comparisons overall. Furthermore, "comprehending text" usually turned out to be reading and "comprehending" a single sentence, for the younger kindergarten and first grade readers, the only age group for which systematic teaching of phonics showed a greater effect size overall than teaching phonics unsystematically or not at all.

Therefore, when the subgroup that analyzed phonics research mentions "growth in reading" or "reading growth," they are usually referring mostly or exclusively to growth on phonics-related skills, including word identification.

Negative or no conclusions from the NRP subgroup report on phonics

Nor is it true, as many people mistakenly assume, that systematic phonics instruction means a "direct instruction" program. There are specific direct instruction programs with a script that tells the teacher exactly what to say, but there are also many other ways of teaching phonics directly and systematically. The term "direct instruction" is something like the word "kleenex," which is the brand name for a particular kind of facial tissue, but which has also become a generic term for any kind of facial tissue. That is, direct instruction originally referred to a specific scripted program, DISTAR/Reading Mastery, but "direct instruction" is now also used to describe any kind of direct teaching.

__________________________________________________________________ Summary of the Actual phonics-related NRP Findings

Here are key statements and data from the National Reading Panel report about phonemic awareness and phonics (NRP, 2000b):

1. In comparison with "disabled" and normally progressing readers, at-risk students showed the largest effects from phonemic awareness training, especially with regard to "reading outcomes" (NRP, 2000b, p. 2-23).

2. Overall, teaching phonemic awareness to young children who were progressing normally as readers, to young children "at risk" of reading failure, and to older "disabled" readers (mostly in grades 2-6) produced, in comparison with control groups, a large "effect size" of 0.86 on posttests of phonemic awareness but only a relatively small effect size of 0.34 on reading comprehension tests (NRP, 2000b, p. 2-3; Ehri, Nunes, Willows, Schuster, Valeska, Yaghoub-Zadeh, & Shanahan, 2001, p. 261). It is important to note that "reading comprehension tests at the beginning levels consist of short sentences and conceptually simple ideas. The primary determiner of success is whether the words can be read" (Ehri et al., 2001, p. 277).

3. When phonemic awareness is taught with letters—that is, when sounds are matched with letters—it is actually phonics instruction. "Teaching PA with letters helps students acquire PA more effectively than teaching PA without letters" (NRP, 2002b, p. 2-41). In fact, the difference is substantial. Thus phonemic awareness can be taught most effectively and most efficiently as part of phonics instruction in letter-sound relationships. (In the NRP report, none of the studies was included in both analyses.)

4. "Systematic phonics instruction was most effective in improving children’s ability to decode regularly spelled words and pseudowords," with effect sizes of 0.67 and 0.60 respectively (p. 2-125). These are moderate to large effect sizes.

5. "Phonics instruction produced substantial reading growth among younger children at risk of developing future reading problems. Effect sizes were d = 0.58 for kindergartners at risk and d = 0.74 for 1st graders at risk" (p. 2-125).

6. "Phonics instruction also improved the reading performance of disabled readers (i. e. children with average IQs but poor reading) for whom the effect size was d = .32," a small effect size (p. 2-125).

7. "However, phonics instruction failed to exert a significant impact on the reading performance of low-achieving readers in 2nd through 6th grades (i.e. children with reading difficulties and possibly other cognitive difficulties explaining their low achievement). The effect size was d = 0.15, which was not statistically greater than chance" (p. 2-125).

8. "Phonics instruction appears to contribute only weakly, if at all, in helping poor readers apply these [word reading] skills to read text and to spell words. There were insufficient data to draw any conclusions about the effects of phonics instruction with normally developing readers above 1st grade (p. 2-108).

9. "The overall effect size of systematic phonics instruction in 1st grade was d = 0.54," a moderate effect size. The mean effect size for kindergarten was similar: d = 0.56. "The conclusion drawn is that systematic phonics instruction produces the biggest impact on growth in reading when it begins in kindergarten or 1st grade before children have learned to read independently" (p. 2-125).

10. Similarly, the effects of phonemic awareness training were by far the greatest for preschoolers and next greatest for kindergartners (d = 0.95), most of whom had probably not yet learned to read independently. The effect size for kindergartners was much greater than that for first graders, which was d = 0.48. It should be noted, however, that although preschoolers show the greatest effects when taught phonemic awareness, there is no evidence that phonemic awareness needs to be taught before kindergarten. It’s simply that preschoolers are less aware of the separate phonemes in words and therefore learn more from phonemic awareness training.

______________________________________________________________________________


Instruction in phonemic awareness and phonics will have the greatest effect upon children who cannot yet read independently.

______________________________________________________________________________

It may also be noted that while the subgroup report on phonics suggests that intensive, systematic phonics be continued through grade 2, the report itself does not provide separate data for grade 2. The data for grade 2 is lumped together with the data for grades 3-6, suggesting that the data for grade 2 alone may be insufficient to warrant any conclusions.

We must remember, too, that when the National Reading Panel makes claims about children’s growth in reading, the word "reading" refers mostly to word reading skills.

 

What, then, can reasonably be concluded with regard to the values of teaching phonics---that is, letter-sound relationships and various skills---intensively and systematically to kindergartners and first graders?

Considerations and cautions

The National Reading Panel report itself includes various considerations and cautions with regard to the teaching of phonemic awareness and phonics. These are combined with other concerns, reminders, and cautions from earlier in Chapter 13 of Reading Process and Practice (Weaver, 2002):

1. Phonological skills, including phonemic awareness, do not need to be taught before a child begins to learn to read whole, interesting, and familiar texts—obviously with support. The same is true of phonics.

2. Whether taught apart from or during reading and writing, phonemic awareness develops best when sounds are taught in conjunction with letters—that is, when phonemic awareness is actually part of phonics instruction (NRP, 2000b, pp. 2-6, 2-33). This is important to remember as we read the following comments about teaching phonemic awareness.

3. "In addition to teaching PA [phonemic awareness] skills with letters, it is important for teachers to help children make the connection between the PA skills taught and their application to reading and writing tasks" (p. 2-33).

4. "One instructional activity that is maximally effective for teaching PA in a way that builds a bridge to reading and spelling is that of teaching children to invent phonemically more complete spellings of words. . . . The effect size [in a study by Ehri and Wilce, 1987] was large, d = 0.97. These findings indicate that teaching children to segment and spell helps them learn to read as well as spell words" (p. 2-39).

5. "In the NRP analysis, studies that spent between 5 and 18 hours teaching PA yielded very large effects on the acquisition of phonemic awareness. Studies that spent longer or less time than this also yielded significant effect sizes, but effects were moderate and only half as large" (p. 2-42).

6. "In the NRP data base, the average length of sessions was 25 minutes." The National Reading Panel concluded "that sessions should probably not exceed 30 minutes in length" (p. 2-42).

7. The NRP analysis found that children learned phonemic awareness better when taught in small groups rather than in large groups or individually (p. 2-42). For teaching phonics in general, the panelists found that individual, small group, and whole class teaching all seemed to be effective (p. 2-124).

8. "In kindergarten, most children will be nonreaders and will have little phonemic awareness; so, PA instruction should benefit everyone. In 1st grade, some children will be reading and spelling already while others may know only a few letters and have no reading skill" (p. 2-42). In first and second grade, teachers should determine individual children’s need for help in developing letter-sound knowledge, which includes phonemic awareness.

9. There are many ways that phonics can be taught intensively and systematically. These include synthetic and analytic phonics; analogy phonics, which involves teaching onsets and rimes and decoding new words by analogy; teaching the use of letter-sound knowledge along with context to identify unfamiliar words that readers encounter in texts ("phonics in context"), and teaching phonics through invented spelling ("phonics-through-spelling") (p. 2-81). (The NRP summary booklet describes phonics in context as "embedded phonics"; 2000c, p. 8). The National Reading Panel clustered the various ways of teaching phonics into three categories: "(1) synthetic phonics programs which emphasized teaching students to convert letters (graphemes) into sounds (phonemes) and then to blend the sounds to form recognizable words; (2) larger-unit phonics programs which emphasized the analysis and blending of larger subparts of words (i.e. onsets, rimes, phonograms, spelling patterns) as well as phonemes; (3) miscellaneous phonics programs that taught phonics systematically but did this in other ways" (NRP, 2000b, p. 2-85). These three ways of teaching phonics systematically did "not appear to differ significantly in their effectiveness," although there were slight differences in effect size (p. 2-85).

10. "It is important to note that acquiring phonemic awareness is a means rather than an end" (p. 2-33). More generally, "Phonics teaching is a means to an end" (p. 2-88).

And finally, the National Reading Panel cautions that "Phonics should not become the dominant component in a reading program, neither in the amount of time devoted to it nor in the significance attached" (p. 2-89).

______________________________________________________________________________

**In trying to generalize about the effects of systematic phonics instruction, the members of the National Reading Panel used a statistical concept called effect size. Effect size was calculated by subtracting the mean of the control group from the mean of the experimental group, with the difference expressed in standard deviation units: d = 0.20, for example. The panelists categorized effect sizes according to Cohen's rule of thumb (1988): an effect size between zero and 0.20 is considered small; an effect size of 0.50 is considered moderate; and an effect size of 0.80 is considered large. The panelists also noted that "A treatment can be very effective but yield only a small effect size if instruction in the control group is also effective" and, conversely, that "if the control group's instruction is particularly ineffective . . . then the effect size is inflated" (NRP, 2000b, p. 2-95).  

References

Brown, J., Goodman, K. S., & Marek, A. M. (1996). Studies in miscue analysis: An annotated bibliography. Newark, DE: International Reading Association.

Ehri, L. C., Nunes, S. R., Willows, D. M., Schuster, B. V.; Yaghoub-Zadeh, Z.; & Shanahan, T. Phonemic awareness instruction helps children learn to read: Evidence from the National Reading Panel's meta-analysis. Reading Research Quarterly, 36 (3), 250-287.

Ehri, L. C., & Stahl, S. A. (2001). Beyond the smoke and mirrors: Putting out the fire. Phi Delta Kappan, 82 (1): 17-20.

Garan, E. M. (2001). Beyond the smoke and mirrors: A critique of the National Reading Panel report on phonics. Phi Delta Kappan, 82(7), 550-506.

National Reading Panel [NRP]. (2000a). Report of the National Reading Panel. Minority View, by Joanne Yatvin. Washington, DC: National Institute of Child Health and Human Development. <http://www.nichd.nih.gov/publications/nrp/minorityView.pdf>

National Reading Panel [NRP]. (2000b). Report of the National Reading Panel: An Evidence-Based Assessment of the Scientific Research Literature on Reading and Its Implications for Reading Instruction, Reports of the Subgroups. Washington, DC: National Institute of Child Health and Human Development. <http://www.nichd.nih.gov/publications/nrp/report.htm>

National Reading Panel [NRP]. (2000c). Report of the National Reading Panel: An Evidence-Based Assessment of the Scientific Research Literature on Reading and Its Implications for Reading Instruction. [Summary.] Washington, DC: National Institute of Child Health and Human Development. 33 pp. <http://www.nichd.nih.gov/publications/nrp/smallbook.htm>

Weaver, C. (2002). Reading Process and Practice (3rd ed). Portsmouth, NH: Heinemann.

 

Critiques of the National Reading Panel report

 

This is not necessarily a complete list of publications critiquing the NRP report; that list continues to grow.

Coles G. (2002). Reading unmentionables: Damaging reading education while seeming to fix it. Portsmouth, NH: Heinemann.

Coles, G. (2001). Reading taught to the tune of the "scientific" hickory stick. Phi Delta Kappan, 83 (3): 205-213.

Cunningham, J. W. (2001). The National Reading Panel report. Reading Research Quarterly, 36, 326-335.

Garan, E. (2002). Resisting reading mandates: How to triumph with the truth. Portsmouth, NH: Heinemann.

Garan, E. M. (2001a). Beyond the smoke and mirrors: A critique of the National Reading Panel report on phonics. Phi Delta Kappan, 82 (7), 550-506.

Garan, E. M. (2001b). More smoking guns: A response to Linnea Ehri and Steven Stahl. Phi Delta Kappan, 83 (1): 21-27.

Garan, E. M. (2001c). What does the report of the National Reading Panel Really
Tell Us About Teaching Phonics?" Language Arts, 77 (5), 59-68.

Krashen, S. D. (2001). More smoke and mirrors: A critique of the National Reading Panel Report on "Fluency." Phi Delta Kappan, 83(2), 118-121.

Yatvin, J. (2000). Report of the National Reading Panel. Minority View. (This is listed in the references as NRP, 2000a. It is also included within NRP 2000b, the book version of the National Reading Panel Report.) <http://www.nichd.nih.gov/publications/nrp/minorityView.pdf>

Yatvin, J. (2002). "Babes in the Woods: The Wanderings of the National Reading Panel." Phi Delta Kappan, 83(5): 364-369.