STEM Education and Tutoring in the Capital City: Part 3 – Achieving Impact

Note: This post is part of a series of three on tutoring for science and mathematics among disadvantaged students. Part 1 looks at needs. Part 2 and part 3 give examples of successful programs.

by Quentin Wodon

The United States and especially the District of Columbia are lagging behind in STEM education, as discussed in the first blog post of this series. When Don and his team designed the small Rotary-led tutoring program described in the second post of the series, they did not start with a review of the evidence from the literature on what works. But through the experience of the teachers and principal at the school, as well as their own experience, they had a pretty good idea of what could be useful. As a result, the design of the program actually corresponds to what the literature recommends.

Don, a teacher, and a few of the tutored students
Don, a teacher, and a few of the tutored students

Lessons from the Literature

The literature on tutoring and out-of-school-time programs (see for example the review by Heinrich and Burch) suggests that in order to achieve impact, it is often useful to: (1) provide consistent and sustained instructional time, for a total of at least 40-45 hours; (2) provide tutoring to small groups of students, preferably less than ten at a time; (3) follow a curriculum that is rich in content and takes into account the specific needs of students while being also closely related to what students learn during the regular school day; (4) ensure that tutoring sessions are active and varied (for example by combining structured and unstructured instruction, as well as individual and collective work time) and focused on targeting the development of specific skills; (5) foster positive relationships between tutors and students; and finally (6) foster collaboration between teachers and tutors with support of administrators, including for constructive evaluation. All of these features are at work in Don’s program.

There is substantial interest in tutoring today in the US. As mentioned in the first post in this series, under the ‘No Child Left Behind’ Act adopted a dozen years ago, public schools not making enough progress in learning assessments for two consecutive years must provide tutoring services to children. Tutoring initiatives are being implemented throughout the country. Earlier this year Mayor Emanuel announced the expansion (with private funding) of a mathematics tutoring program in Chicago that University of Chicago researchers found helpful for at-risk students in public schools (see the review of the study in the New York Times).

Examples of Great Programs

Another example of intensive tutoring program having impact is Higher Achievement. The NGO operates in Washington, DC, Baltimore, Richmond, and Pittsburgh. Students in the program meet three days a week during the school year. They first complete homework with support from teachers and volunteers. They then have dinner and work on a specific subject in small groups of two or three with a trained volunteer mentor. This is a rigorous program – overall, students spend a total of 650 hours a year in the program between 5th and 8th grade.

Data from Higher Achievement suggest that three fourth of the enrolled students improve their grade point average (GPA) by at least one letter grade, and 96% graduate from high school – two times the rate of their peers. Three fourths of the students also go on to graduate from college – four times the rate of their peers. The program has been evaluated rigorously by MDRC, a nonprofit, nonpartisan research organization. Researchers from the University of Texas at Austin compared Higher Achievement students (“scholars”) with a control group of students who applied to the program, met the admissions criteria, but were not selected to participate through a randomized lottery.

According to the evaluation of Higher Achievement published last year, the program had a statistically significant positive impact after one year in the program on mathematics proficiency and reading comprehension, as measured by standardized tests. The mathematics impacts lasted four years after enrollment in the program. The program also increased the probability that the students would enroll in high performing private high schools. These findings suggest that intensive OST (out-of-school-time) programs like Higher Achievement can be beneficial.

Another program that also operates in Washington, DC, and that has been rigorously evaluated by MDRC is Reading Partners. The program serves more than 7,000 students in over 130 schools in California, Colorado, New York, Oklahoma, Maryland, South Carolina, Texas, and Washington, DC. As was the case with the small Rotary-funded program in Washington, DC, and the larger program operated by Higher Achievement, Reading Partners works in (large) part with volunteers, which helps in keeping costs down. The evaluation of Reading Partners was conducted in 2012-13 in a subset of the schools where the program operates. Results suggest gains in reading proficiency. While this evaluation was not about STEM, it suggests again that tutoring programs can make a difference.

Policy and What You Can Do

From a policy point of view, there are legitimate questions about the cost effectiveness of some tutoring programs. This cost effectiveness issue must be looked at carefully on a case by case basis. But when the programs are staffed in part or fully by volunteers, they are more likely to be cost effective. Tutoring may also in some cases – especially when it is profit-motivated, act as a substitute for good quality teaching. This may be a serious problem in some developing countries (as an example, see this paper on Nepal), but probably much less so in developed countries. In most situations, tutoring is likely to lead to positive changes.

For those who care about helping disadvantaged students better succeed in schools, the good news is that there are many ways to contribute. If you have or can take the necessary time to do so, you can get personally involved like Don and his fellow Rotarians are doing, going every week to a school and working with a few students. But if you do not have the time, you can still help by contributing funding to organizations that are doing a great job on the ground.

STEM Education and Tutoring in the Capital City: Part 2 – Measuring Gravity

Note: This post is part of a series of three on tutoring for science and mathematics among disadvantaged students. Part 1 looks at needs. Part 2 and part 3 give examples of successful programs.

by Quentin Wodon

Innovative Tutoring

Imagine a group of elementary school students gathering in a school gymnasium as part of a tutoring session. The students are trying to estimate the gravitational acceleration force on an object at sea level, where Washington, DC, is located. The students throw a golf ball in the air in the gymnasium. They record the time it takes for the ball to fall from apogee to the floor using a simple stop watch. They repeat the exercise 25 times. They also estimate the distance from apogee to the top of the ceiling, which is done by first measuring the distance from floor to ceiling and next by guessing by how much the ball misses the ceiling. The students’ estimate of ‘g’, the gravitational acceleration due to the force exerted by the earth on the golf ball, turns out to be within three percent of the accepted value for Washington, DC, even though each of the 25 individual computations per throw varied widely. This showed to the students how approximate values, when averaged, may converge on true values with reasonable accuracy.

Two students perform the gravity experiment
Two students perform the gravity experiment

Another experiment used a hygrometer, an instrument for measuring humidity or moisture content as well as temperatures. This was coupled with water and iced water in cans. Students had to figure out the temperature at which beads of water formed on the outside of the tin cans, which was followed by a discussion of what fog is, how temperature affects relative humidity, why clouds form and sometimes rain or snow is produced.

Two students work with a hygrometer
Two students work with a hygrometer

Program Characteristics

These scenes are not from a movie, but from a volunteer-based tutoring program run in a public school located in Anacostia, the poorest area of the city. Until recently, few children at the school passed standardized mathematics and reading tests, but things have improved. The tutoring program has now been in existence for six years. It is run by Dr. Don Messer a member of the Rotary Club of Washington, DC in District 7620. Together with teachers, school administrators, and a half dozen other tutors from his Rotary club Don designed the program in an innovative way.

The program focuses on mathematics and reading, and on the types of questions asked in standardized tests. This is not to “teach to the test”, but to ensure that children understand potential test questions well. Tutors work with students in small groups of three or four to generate interactions and more learning. The groups meet once or twice a week for the entire school year. The goal is not only to help the students learn, but also to help them understand that there is a future for them that often they didn’t know existed.

Tutoring can work to improve learning – this is why so many parents who have the means to do so invest in tutoring (there is a rather large literature on private tutoring – as just one recent example see this paper on Vietnam). But children from disadvantaged backgrounds do not have such opportunities, which is why volunteer-run programs are so important for those children.

To work well, tutoring sessions should be active, varied, and even fun. Sessions should combine structured and unstructured instruction, as well as individual and collective work, and they should focus on specific skills. In Don’s small but effective volunteer program the first part of each tutoring session focuses on prior test problems from DC standardized tests. These tests are augmented by problems that tutors or teachers prepare to emphasize special themes. In mathematics for example, a package would contain around 80 problems, ranging from routine arithmetic operations to data analysis (histograms, bar charts, tables), basic geometry, and problems that require reading to make sense of what is to be done. The problem set is paced by student progress, not by a time schedule. Tutors make sure that if a problem is difficult to understand for one or more of the students, all students understand what the problem is driving at before they start to work on the problem. Students work on the problem until all have finished, but if the tutor sees that at least one student remains confused, a group discussion is launched to help the students get the correct solution. The tutors also try to interject simple science illustrations within the problems to be solved, as illustrated earlier with the gravity constant and hygrometer experiments.

Impact and Recognition

How successful has Don’s program been? No impact evaluation is available to say for sure, but success rates at standardized tests have been systematically higher for tutored than non-tutored students year after year. The results, albeit not based on a randomized study, are encouraging. In part thanks to this program, the Rotary club of Washington, DC, was recognized two years ago as Volunteer Group of the Year by Chancellor Henderson of the District of Columbia Public Schools. For the Rotarian tutors, the experience has been highly rewarding. And in Don’s case, there was no better reward than having a fifth-grader tell him: “You know Dr. Messer, you’re my grandpa.”

In the third and last post in this series, I will discuss results from several programs that operate in Washington, DC, and have been rigorously evaluated, including Higher Achievement and Reading Partners.

Note: Part of this blog post is adapted from a section in a book published by the author entitled Membership in Service Clubs: Rotary’s Experience (Palgrave Macmillan, 2014).