Math 180: Cobb County School District

Houghton Mifflin Harcourt’s MATH 180 program targets students struggling with early mathematics achievement. Developed for students in Grade 5 and above who are two or more years behind in math achievement, and in line with recommendations for a successful math intervention from the American Institutes for Research and the What Works Clearinghouse (WWC), MATH 180 provides explicit and systematic instructional methods, motivational strategies, and data-based decision making for teachers, among other innovative components. Using a combination of whole-class instruction, computer adaptive instruction and practice, and teacher-led small-group instruction, MATH 180 aims to create a learning community where students learn to persevere through mistakes, experience success, and develop a growth mindset to gain confidence and overcome previously experienced struggles with math. Teachers using MATH 180 can become more effective instructors with the use of real-time data, differentiated instruction, and embedded discussion questions that identify student thinking and misconceptions, and through continuously available support, professional learning, and coaching resources.

Prior research suggests that MATH 180 may have a positive impact on student math achievement. For example, an early outcomes study showed greater growth in math achievement for students using MATH 180 compared to other interventions, and a large majority of teachers reported that MATH 180 raised achievement in the classroom and improved their teaching (Houghton Mifflin Harcourt, 2014). However, this study did not involve random assignment, making causal conclusions problematic, and on average, students completed less than 50% of the program over the school year. Additionally, a recently produced WWC guide on how best to help students struggling with mathematics reports that, in general, little evidence is available on the impact of interventions for low-performing students (Gersten et al., 2009). Thus, a more rigorous efficacy study is needed to further evaluate the impact of this program on student math achievement.

In order to expand on the research base behind the MATH 180 program, JEM & R conducted a quasi-experimental study in Cobb County, GA.

The overarching purpose of the study was to evaluate the effectiveness of MATH 180 in helping middle school intervention students attain critical mathematics skills and closing the achievement gap. Specifically, a one-year study was conducted and designed to address the following questions:

• What are the effects of MATH 180 on student mathematics achievement? Specifically, how do changes in mathematics test scores achieved by MATH 180 students compare to changes achieved by similar students in the matched comparison group?
• How does MATH 180 differentially affect subgroups of students? Specifically, how do changes in mathematics test scores achieved by specific subgroups of MATH 180 students (male and female students; Black, Hispanic, and White students; English learners [ELs]; and students with disabilities) compare to changes achieved by subgroups of comparison group students?

In order to address these research questions, a quasi-experimental study (QES) was conducted during the 2017–2018 school year in Cobb County, Georgia. The study included five MATH 180 teachers and their 128 students in Grades 6–8.

The MATH 180 study commenced in October 2017 and concluded in May 2018. Two middle schools were selected as study schools within Cobb County Public Schools in Georgia. Students within these schools who were identified as needing additional math support and used MATH 180 over the school year participated in the study (n = 128). In total, seven math support classes and five teachers served as treatment (i.e., MATH 180) classrooms. During the study, students used MATH 180 as part of a second math class period. As a result, MATH 180 was used in addition to a core math course.

In order to allow for rigorous comparisons to be made between groups, each of the participating schools was matched to a comparison school that was not implementing MATH 180 during the 2017–2018 school year but possessed similar schoolwide demographic and prior performance statistics. While this helped to promote baseline equivalence, school level data with such a small sample cannot ensure baseline equivalence, which is a necessary precondition for making strong causal conclusions. Therefore, to ensure that there were no significant differences between students on important variables, math intervention students who met specific criteria (e.g., completing at least 10 sessions of MATH 180) were included in the final sample, and each student was matched to a student from a non-MATH 180 school.

The comparison sample was determined via propensity scoring methodology. Basically, a logistic regression model was used to model the propensity to be a MATH 180 student. The predicted probability from the logistic regression served as a measure of the propensity of being a MATH 180 student and was also used as a distance measure to implement the matching procedure (nearest neighbor algorithm). This predicted probability served to reduce the multidimensional student-level characteristics into a single number that was used to match MATH 180 and non–MATH 180 students. Control students were selected based on the distance measure so that a comparison sample of the most closely matched students was created. The final propensity score was based on the following variables:

• Grade
• Ethnicity
• Gender
• EL Status
• Special Education Status
• Percent Present 2016­–2017 School Year
• Math Inventory Data from Fall 2017
• Georgia State Assessment Data from Spring 2016

Trainings were designed to provide teachers with the necessary background and practical experiences to begin implementing the program with fidelity during Fall 2017. The focus of these trainings was on the instructional model of the MATH 180 program, the use of the materials and implementation of the key components, and how the program could best be used to effectively help students build strong math foundational skills.

A Houghton Mifflin Harcourt^® (HMH^®) professional trainer provided three days of training as well as coaching visits every 4–6 weeks. In addition to initially providing the details of the program pedagogy and components (when and how to use), follow-up trainings tended to be customized according to the needs of each site and teacher, and teachers could ask for help with specific components they wanted to become more proficient at incorporating. In addition, together with the teacher, goals were set following coaching visits to help teachers keep on track with their implementation of the program. Throughout the course of the study, participants had email access to the professional trainer so they could communicate on a real-time basis any questions or issues regarding training or implementation that arose.

Math Assessment

The Math Inventory was utilized as an outcome measure in this study. Key features of this assessment include:

• Computer adaptive assessment that measures math abilities and longitudinal progress from kindergarten through Algebra II
• Measures of the National Council of Teachers of Mathematics (NCTM^®) five content standards: Number and Operations Geometry, Algebra/Patterns and Functions, Data Analysis and Probability, and Measurement
• Mathematics growth is measured on the Quantile^® Framework for Mathematics—a scientific taxonomy of over 500 math concepts and skills—placing student readiness and difficulty of math tasks on the same scale.
• 40-minute group administration, which can be offered 3–5 times per year

The Math Inventory was administered in Fall 2017 and Spring 2018 by Cobb County. Data from the prior school year (2016–2017) was also provided in order to 1) check for baseline equivalence, and 2) measure growth from last year. Tests were scored by each respective system, and individual level results were shared with researchers.

School Characteristics

The study sites consisted of six schools located in the suburbs of Atlanta and within Cobb County Public Schools. All study sample students received math intervention in addition to their core math instruction.

Demographic and prior performance information is presented in Table 1 for each of the study schools. As shown, the population of students at MATH 180 School A and their matched comparison school (E) also tended have similar demographics. MATH 180 School B and comparison schools C, D, and F tended to have similar demographics, including higher percentages of minority students, students receiving a free/reduced-price lunch, and EL students, and a lower percentage of historical math proficiency, especially compared to a statewide average of 42%.

Student Characteristics

In total, the study sample consisted of 307 sixth- through eighth-grade students at six middle schools. Specifically, the full analytical sample consisted of 128 intervention students who used the MATH 180 program during the 2017–18 school year and 179 comparison students not using the program.

Demographic characteristics for both types of students in the study sample are shown in Table 2. The sample consisted of an ethnically diverse population, with over 80% minority students. As shown, the propensity matching procedure resulted in MATH 180 and control groups that were fairly comparable across all demographic categories. Indeed, there were no significant differences between the two groups with regard to any measured demographic factors, ps > .05.

Importantly, groups did not differ with regard to the average number of days students were enrolled or the percentage of days students were absent (see Table 3). There were also no differences in baseline math proficiency across groups as measured by math scale scores on the previous year’s end-of-grade test, and Math Inventory Quantile^® measures from the beginning of the school year, ps > .05. Still, in order to ensure baseline equivalence and enhance the sensitivity of analyses, baseline math performance was taken into account as a covariate in analyses comparing MATH 180 and non–MATH 180 students.

MATH 180 Program Description

MATH 180 focuses on reasoning and using visual models to make sense of math.

Utilizing a blended approach that incorporates both digital and print materials, the instructional model for MATH 180 is designed to help students master skills and advance at an accelerated pace. Instruction begins with a whole-class “Do Now” to help students warm up. Taking approximately 5–10 minutes, these classroom management routines encourage thinking, inspire mathematical habits of mind, and make connections to prior topics. Following this whole-class time, students break into two groups and rotate between group instruction and the personalized MATH 180 software. While one group of students utilizes the customized software component of the program for support and practice, the teacher works with the other half on building conceptual understanding, developing reasoning and communication skills, and interpreting student thinking. Following 20–25 minutes, students rotate again so that they experience both groups before the class period ends.

MATH 180 consists of two courses. Course I is designed for students who need to build numerical understanding and reasoning skills—the foundational concepts that students need to think algebraically. Concepts covered include multiplication, division, decimals, and fractions. Course II transitions students to pre-algebra with an emphasis on building proportional reasoning with rates, ratios and linear relationships, and functions.

For teachers, the Teaching Guide provides daily step-by-step instruction to develop understanding through discussion and problem solving. In addition to the ongoing formative assessments, the mSkills tests assess students for mastery of key, standards-aligned skills and concepts taught during whole-class and group instruction. 

A consumable student book, mSpace, is designed to promote students’ active participation by providing a place to record and share their work and understandings. The companion software’s game-like environment engages students with games, choices, and an opportunity to earn badges as rewards. In the Learn Zone, students gradually progress from guided to independent practice with interactive visual models, a metacognitive coach, and corrective feedback. In the Success Zone, students apply what they’ve learned to more difficult tasks.

What are the effects of MATH 180 on student mathematics achievement? How do changes in mathematics test scores achieved by MATH 180 students compare to changes achieved by similar students in the matched comparison group?  

In order to examine the effects of MATH 180 on math performance, students who used MATH 180 were compared to an equivalent sample of non–MATH 180 users via ANCOVAs. As previously noted, groups were matched based on important demographic variables. Nevertheless, to account for any baseline differences between the two groups, analyses included pretest scores as a covariate. Therefore, a significant effect for the group would indicate that after controlling for baseline performance, there were significant differences in the posttest scores of MATH 180 and comparison students.

Results showed that in the overall sample, there was a significant group difference in posttest performance on the Math Inventory, p < .05. As shown in Figure 1, when adjusting for baseline (Fall 2017) Math Inventory performance, the gains in Math Inventory Quantile measures were greater for MATH 180 students than comparison students (49 Quantile difference), p < .05.

Findings demonstrating greater learning gains among MATH 180 students were further supported by the effect sizes obtained in comparative analyses. Effect size is a commonly used measure of the importance of an observed difference. The effect size of the observed difference between groups on the Spring 2018 Math Inventory (d = 0.29 for differences in posttest Math Inventory Quantile measures, d = 0.33 for Math Inventory growth during the school year) can be classified as small-moderate and educationally meaningful.

Growth among Student Subpopulations

Analyses were also conducted to assess whether there were differences in math performance between MATH 180 and comparison students belonging to several subpopulations of interest. Specifically, sufficient data was available to conduct analyses by gender, ethnicity, EL status, and disability status (see Figures 2–5). As with the overall analyses, an ANCOVA was run to determine whether there was a significant interaction between subgroup categories (e.g., males and females) and intervention group (i.e., MATH 180 vs. comparison students). A significant interaction would suggest that the effect of MATH 180 differs for different categories within the subgroup. Because group differences may also exist within an individual subgroup category, separate analyses were also conducted to investigate potential group differences in specific subgroup categories. Analyses included baseline performance as a covariate in order to minimize the potential effect of any pre-existing group differences.

Results for analyses of Math Inventory growth revealed one significant interaction between subgroup categories and intervention group. When controlling for Fall 2017 scores, EL students in the MATH 180 program outperformed controls on the Spring 2018 test, p < .05. For all other subpopulations, interactions between subgroup category and intervention group were not statistically significant, ps > .05. However, analyses of individual subgroup categories yielded several group differences. In addition to differences among EL students, there were significant group differences among females, Hispanic students, and students without disabilities, ps < .05. There was also a marginally significant group difference in Math Inventory growth among Black students, p < .10. In all cases, learning gains were greater among students in MATH 180 than those observed in their counterparts in the comparison sample. 

Results on the outcome measures indicate that MATH 180 is associated with positive math learning gains among struggling math students. In comparison to students who did not use the MATH 180 program, significant differences in learning gains were observed. Overall, MATH 180 students outperformed comparison students on the Math Inventory. MATH 180 students demonstrated higher math performance as compared to non–MATH 180 students on the Math Inventory, which suggests that MATH 180 is designed to improve students’ foundational understanding of math concepts.

When assessing performance in different subpopulations, several significant differences emerged regarding improvements in Math Inventory performance among MATH 180 students. Specifically, there were significant group differences in the post-test scores among females, Black and Hispanic students, EL students, and students without disabilities. In all cases, differences reflected greater math learning gains among students using MATH 180 compared to the matched control sample. Such findings are particularly noteworthy given the small sample sizes involved.

In summary, the MATH 180 program was associated with positive learning gains among struggling math learners. Moreover, comparisons made with non–MATH 180 students suggest that this program is effective in improving math skills. However, more rigorous research with a larger generalizable sample and randomized groups is warranted in order to determine the efficacy of this program.

Cheung, A. C. & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K–12 classrooms: A meta-analysis. Educational Research Review, 9, 88–113.

Gersten, R., Beckmann, S., Clarke, B., Foegen, A., Marsh, L., Star, J. R., & Witzel, B. (2009). Assisting students struggling with mathematics: Response to Intervention (RtI) for elementary and middle schools (NCEE 2009-4060). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies. ed.gov/ncee/wwc/publications/practiceguides/

Houghton Mifflin Harcourt. (2014). Early outcome effects of a blended learning model for math intervention instruction with special population students. Retrieved from /research/math-180-early-outcome-effects-of-a-blended-learning-model-for-math-intervention-instruction-with-special-population-students

National Council of Teachers of Mathematics (NCTM) (2007). Effective strategies for teaching students with difficulties in mathematics and what are the characteristics of students with learning difficulties in mathematics? Reston, VA: NCTM.