Critiquing a Study

Hale, A.D., Hawkins, R.O., Sheeley, W, Reynolds, J.R. Jenkins, S., Schmitt, S.J. & Martin, D.A. (2011). An investigation of silent versus aloud reading comprehension of elementary students using maze assessment procedures. Psychology in the Schools, (48)1, 4-13.

The purpose of the above study was to examine "the possible differences between silent and aloud reading comprehension scores on Maze assessment probes."  The participants were 89 first and second graders from general education classes at a private school in the southeast. The researchers used grade-level appropriate Aimsweb Maze passages to assess "aloud and silent Maze comprehension." (The Maze test consists of a complete first sentence, after which every seventh word is deleted. Students have to choose from three words given in parentheses and circle the correct answer.) To compare results, the authors also used grade-level appropriate DIBELS (Dynamic Indicators of Basic Early Literacy Skills) reading passages and norm-referenced passages from the Woodcock-Johnson Tests of Achievement. The assessments were conducted over a three-month period in the spring. The tests were given individually in a private room attached to the classroom and administered by "one primary investigator and five graduate students." The "primary investigator ensured that each graduate student could administer each assessment appropriately and score each assessment reliably." Each child read three passages silently and three orally within a period of five consecutive school days "to limit the possible effects from increased skill development." 
Results indicated that reading silently or orally did not affect the students' understanding of the tested material.
The authors mention that the "sample size was relatively small and came from a private school district." Therefore, "the generalizability of the results of this study are limited." They also state that most of the children were above grade-level readers.
I'm satisfied wtih the way this study was conducted. In my opinion, the authors come across as upfront and truthful. There was triangulation of sorts, in that there were several administrators, all of whom followed the same testing and scoring procedures. The tests were timed and needed right-or-wrong responses, so there was no room for subjectivity. There was "rich data," in that different assessments were used, and the oral reading was tape-recorded.  Finally, although Hale et al. place a limit on the overall  generalizability of the results, the study does seem to have internal generalizability, which Maxwell defines as "the generalizability of a conclusion within the setting or group studied" (p. 115).