The focus of this study was the effectiveness of GO Math! © 2015 a mathematics program for elementary grade students published by Houghton Mifflin Harcourt®. The study included students from 29 different schools in 9 different states. The overall demographics of the study sample are representative of the demographics of students enrolled in public schools in the United States in terms of students eligible for free/reduced lunch programs and below national averages for non-White students.
The study was conducted with over 2,788 students enrolled in Grades 2, 4, and 6. Only those students who took both a pretest and posttest were included in the data analysis. Teachers used the program for their math instruction five days per week and more than 25 minutes per day. The program was being used by the teachers for the first time. All of the teachers had at least five years of teaching experience and most had 10 to 15 years of teaching experience.
Instruction included the first three units of the program. Pretests and posttests were written by math specialists based on the instructional units taught at each grade level. In addition to analyzing the gain scores for the total group of students at each grade, analyses were conducted separately for higher and lower scoring mathematics students. Higher and lower scoring students were identified by the students’ pretest scores. Those scoring highest on the pretests were designated as the high scoring mathematics students and those scoring lowest on the pretests were designated as the lower scoring math students.
The average gain scores for the total group of students at each grade were statistically significant. The effect sizes for all students at Grades 2 and 4 were large and at Grade 6 the effect size was medium.
In addition the average gain scores for the low and high scoring groups at each grade level were also statistically significant. The effect sizes for the high and low scoring groups were large at Grades 2 and 4. At Grade 6 the effect size was large for the low scoring students and medium for the high scoring students. All of the effect sizes at every grade exceeded by a large margin the effect sizes needed to determine a substantively important level.
Houghton Mifflin Harcourt, School Publishers contracted with Educational Research Institute of America (ERIA) to conduct a one-semester study to evaluate the effectiveness of the GO Math! Program for elementary school students. The study compared assessments administered to students at the end of August 2014 to assessments administered in the middle of December 2014.
The following research questions guided the design of the study and the data analyses:
- Does the implementation of GO Math! Elementary Program in Grades 2–6 program lead to improved student mathematics achievement?
- Does the implementation of GO Math! Elementary Program in Grades 2–6 lead to differential effects on student achievement as a function of student ability level?
The design of the program called for the implementation of the GO Math! program for Grade 2, 4, and 6 students during the first semester of the 2014–2015 academic year. The schools had not used the program prior to this time.
A total of 118 teachers in 9 different states participated in the study. The number of teachers at each grade included:
- Grade 2: 22 schools; 8 states; 60 teachers.
- Grade 4: 20 schools; 9 states; 51 teachers.
- Grade 6: 5 schools; 3 states; 7 teachers.
Teachers reported using the program 5 days a week with an average usage time of more than 25 minutes. All of the teachers had at least five years of teaching experience and most had 10 to 15 years of teaching experience.
Houghton Mifflin Harcourt GO Math! 2015© is a K–6 program written specifically to support the Common Core State Standards for Mathematics with an emphasis on developing 21st-century skills. The Standards for Mathematical Practice are integrated into the content, along with an equal emphasis on conceptual fluency. The program provides rigor, depth of understanding through interactive lessons, research-based instructional approaches, best practices, English learner support, and differentiated math instructional resources to ensure success for all students. The comprehensive digital resources promote college and career readiness and support students, teachers, administrators, and parents.
Incorporates Mathematical Practices in every lesson so students develop the mathematical thinking they need.
Features exploration-driven lessons that begin with problem-based situations and build to more abstract problems.
Elevates depth of understanding so students are ready for the rigor of CCSS and the Smarter Balanced Assessment.
The pretest and posttest used in the study were developed by ERIA mathematics curriculum experts. Tests were developed to match the content of the GO Math! chapters used in the study as well as to emphasize the National Council of Teachers of Mathematics (NCTM®) Standards.
The tests were developed to respond to the following emphases:
Innovative items that call for actual performance on the part of students that encourage divergent thinking and problem solving, emphasize on thinking skills, and align with the NCTM Standards and the State Common Core Standards.
The tests included both multiple choice and written responses. The make-up of items types on each of the three assessments were:
Table 1 provides the basic test statistics. The table shows that the reliabilities of the tests are high and provide adequate stability to assess mathematics achievement. Of particular importance is the fact that the test reliabilities are higher for the post-tests than for the pretests. This is almost certainly the result of instruction which would result in less random guessing on the post-tests than on the pretests.
Table 2 provides the demographic characteristics of the schools included in the study. It is important to note that the school data does not provide a description of the make-up of the classes that participated in the study. However, the data does provide a general description of the schools and, thereby, an estimate of the make-up of the classes included in the study.
The percentage of students enrolled in free/reduced lunch programs ranged from 1% to 99% and averaged 48% across the sample of schools. By comparison, the reported national average for students enrolled in free/reduced lunch programs in public schools was reported as 48.1%.
The percentage of students classified as minority students (non-White) ranged from 0 to 99% with an average of 40%. By comparison, 49.8% of the students enrolled in U.S. public schools were classified as non-White.
Standard scores were used for all data analyses. Raw scores were converted to standard scores with a mean of 300 and a standard deviation of 50. Data analyses and descriptive statistics were computed for the students’ standard scores.
For most of the comparisons, paired comparison t-tests were used to determine if differences in pretest and post test scores were significantly different. The comparisons were conducted for differences between the GO Math! August 2014 (pretest) and the GO Math! December 2014 (post test). The ≤.05 level of significance was used as the level at which differences would be considered statistically significant.
In addition, effect size (Cohen’s d) was computed for each of the comparisons. This statistic provides an indication of the strength of the effect of the treatment regardless of the statistical significance. Interpretations of effect sizes in this report include the following guidelines:
.20 to .49 = small
.50 to .79 = medium
.80+ = large
Table 3 shows that the average scores of the 1,047 Grade 2 students participating in the study increased their average test scores at a statistical significant level. The effect size was substantively important and is classified as large.
The total group of 1,047 Grade 2 students was divided into two approximately equal sized groups based on their pretest scores. The 523 students scoring lowest on the pretest were considered to be lower achieving mathematics students while the 524 scoring highest on the pretest scores were considered to be higher achieving mathematics students.
Table 4 shows that both groups made statistically significant gains. The effect sizes for both groups were substantively important and are classified as large.
Figure 1 provides a graphic representation of the gains achieved by the Grade 2 students. In one semester, the Grade 2 students increased their average scores by 28%. The low achieving mathematics students increased their scores 35% while the high achieving mathematics students increased their scores 21%.
FIGURE 1. Grade 2 Pretest Posttest Gain Comparison All Students, Low Pretest Students, High Pretest Students
Table 5 shows that the average scores of the 1,000 Grade 4 students participating in the study increased their average test scores at a statistical significant level. The effect size was substantively important and is classified as large.
Based on their pretest scores, the total group of 1,000 Grade 4 students was divided into two equal sized groups of 500 students. The students scoring lowest on the pretest were considered to be lower achieving mathematics students while the students scoring highest on the pretest scores were considered to be higher achieving mathematics students.
Table 6 shows that both groups made statistically significant gains. The effect sizes for both groups were substantively important and are classified as large.
Figure 2 provides a graphic representation of the gains achieved by the Grade 4 students. In one semester, the Grade 4 students increased their average scores by 25%. The low achieving mathematics students increased their scores 16% while the high achieving mathematics students increased their scores 14%.
FIGURE 2. Grade 4 Pretest Posttest Gain Comparison All Students, Low Pretest Students, High Pretest Students
Table 7 shows that the average scores of the 341 Grade 6 students participating in the study increased their average test scores at a statistical significant level. The effect size was substantively important and is classified as medium.
The total group of 341 Grade 6 students was divided into two equal sized groups based on their pretest scores. The 170 students scoring lowest on the pretest were considered to be lower achieving mathematics students while the 171 scoring highest on the pretest scores were considered to be higher achieving mathematics students.
Table 8 shows that both groups made statistically significant gains. The effect sizes for both groups were substantively important and are classified as large for the low scoring group and medium for the high scoring group.
Figure 3 provides a graphic representation of the gains achieved by the Grade 6 students. In one semester, the Grade 6 students increased their average scores by 8%. The low achieving mathematics students increased their scores 11% while the high achieving mathematics students increased their scores 6%.
FIGURE 3. Grade 6 Pretest Posttest Gain Comparison All Students, Low Pretest Students, High Pretest Students
This study sought to determine the effectiveness of the GO Math! K–6 mathematics program by comparing growth on reliable and valid pretests and posttests. The study took place during the first semester of the 2014–2015 academic year and was carried out in 9 states and included 29 different schools and 118 teachers. The student population included the same percentage of students eligible as the national average. The percentage of non-White students was about 10% lower than the national average.
Two research questions guided the study and the conclusions for each are reported below.
Does the implementation of GO Math! Elementary Program in Grades 2–6 program lead to improved student mathematics achievement?
Across all three grades mathematics student growth was statistically significant. The effect sizes at all three grades were above a substantively important level and were large at Grades 2 and 4 and medium at Grade 6.
Does the implementation of GO Math! Elementary Program in Grades 2–6 lead to differential effects on student achievement as a function of student ability level?
Across all three grades mathematics student growth for the high achieving and low achieving students was statistically significant. The effect sizes at Grades 2 and 4 were for both the high and low group students were above a substantively important level and were large at both grade levels. At Grade 6 the effect sizes were high for the low pretest scoring group and medium for the high pretest scoring group.
On the basis of this study, both research questions can be answered positively:
With the GO Math! program for students in Grades 2, 4, and 6 showed significant growth and the effect sizes were very large.
The GO Math! program for elementary grade students showed significant growth for both higher ability and lower ability students in Grades 2, 4, and 6. The effect sizes for students at Grades 2 and 4 were large. At Grade 6 the effect size was large for low scoring students and medium for high scoring students.