The sixth free ebook in the Rotarian Economist Short Books series has been released. The book tells the story of an initiative by a Rotary club to improve its public image by writing articles in the local media about volunteering opportunities for residents to make a difference in their community. The articles feature great local nonprofits, some of which the club is partnering with in order to implement service projects. The initiative appears to have been a success. To download your free copy, please go here.
The first ebook in the Rotarian Economist Short Books Series has just been published. It provides 10 simple lessons for Rotary clubs to grow. The book is based on the success of the Rotary Club of Capitol Hill in doubling its membership in six months. The book is free and available here in multiple formats. Please share this link widely for others to benefit from this resource. And if you like the book, please consider writing a quick review!
Can internship and mentorship programs help students graduate from high school and prepare them for colleges and careers? What type of support is needed for the most disadvantaged youth, including those who suffer from homelessness? Do tutoring programs help students learn? How can we improve the research, writing, and presentation skills of middle and high school students?
These are some of the questions considered in a new series of briefs on innovations in education in the greater Washington DC, area of the United States. The series is launched jointly by the World Bank Group’s (WBG) education team, the WBG Community Connections Program (the WBG’s outreach program to the local community), and the Rotary Club of Capitol Hill. We hope to contribute to better education outcomes not only in the DC area but also elsewhere by showcasing innovative programs that make a positive difference in the life of students and how well they learn.
Why focus on DC? The WBG’s mission is to end extreme poverty and boost shared prosperity. While living standards in DC are better than in the developing work, poverty rates remain high and only about two thirds of students complete high school according to the Office of the State Superintendent of Education. While improvements have been achieved in recent years, innovations are needed to improve education and employment outcomes for disadvantaged children and youth.
The briefs, in a small but meaningful way, will hopefully contribute to debates on how to improve education by featuring successful programs and policies. The programs and policies featured in the briefs will likely be relevant for other regions in the United States and developing countries with similar challenges.
The first set of briefs in the series feature non-profits that benefited from monitoring and evaluation grants from the WBG’s Community Connections Program or pro bono evaluation support provided by World Bank staff. The Latin America Youth Center (LAYC), One World Education, and the Urban Alliance are among the nonprofits featured in the briefs released on the occasion of the WBG’s Volunteer Awareness Day on May 10.
Below is a brief description of the work of the three organizations together with links to the briefs on these organizations.
LAYC: Some 17,400 young adults aged 18 – 24 who are from the Washington Metropolitan Area are disconnected from work and school. These youth are often from low-income families, are not in school, and are out of work. They typically face multiple challenges, including homelessness, issues with the courts, or substance abuse. These challenges prevent them from successfully transitioning into adulthood. LAYC uses an innovative approach called the ‘Promotor Pathway’ to target high-risk youth. It’s a long-term, intensive, holistic case management and mentorship intervention which has led to positive changes in school enrollment, birth rates, and homelessness.
One World Education: During the 2015-16 school year, One World Education worked with District of Columbia Public Schools as a partner for the ‘Cornerstone’ initiative. Cornerstones are high-quality, in-depth core curricular experiences such as argumentative writing programs. Cornerstones aim to provide rigorous content to students, improved professional development for teachers, and continuity and consistency across grades and subjects. Some 5,200 students in 15 DC public schools in grades 10 and 12 have already participated in the One World Education programs.
Urban Alliance: This non-profit organization, which operates in DC, Baltimore, Chicago, and Northern Virginia, facilitates the transition of disadvantage youth from high school to college to employment. It does so by running a comprehensive early employment program that provides access and exposure to professional networks for youth enrolled in the program. Urban Alliance staff train and mentor the students through their first professional employment opportunities, which help propel them to future success. Over 90 percent of Urban Alliance alumni go on to college. The WBG has participated in the Urban Alliance program since 1997 and supported over 300 students through internships. Results from an external evaluation suggest that the Urban Alliance program improves high school graduation rates and the likelihood that students will go to college after graduation.
The objective of the series of briefs is to document these and other successful programs, so that they can inform education policy and practice not only in DC, but also elsewhere. If you are living in the Greater Washington, DC, area and if you have an idea for a potential brief in the series, please let me know or post your idea in the comment section below.
This post is reproduced with minor changes from a post published by the author on May 9, 2016 on the World Bank Education for Global Development blog.
Readers of this blog know that I have emphasized for some time the need to strengthen a culture of evaluation in Rotary. Evaluations should be undertaken not only for our service projects, but also to assess how our clubs meet, work, and grow – or wither away. This post is about a recent evaluation of an education project supported by my club, and how the evaluation is proving to be useful not only for the local nonprofit we worked with, but also for our club and more generally for practitioners and policy makers working in the field of education.
For several years my club has supported One World Education (OWEd), a great nonprofit based in Washington, DC. OWEd runs the largest argumentative writing program in public and charter schools in the city. The nonprofit reached 5,800 middle and high school students this past school year. The aim of the program, which runs for 4-5 weeks in the schools, is to improve the research, writing, and presentation skills of the students, many of whom are from disadvantaged backgrounds and do not do very well in school.
In previous years, our support to OWEd consisted in providing a bit of funding and volunteering at some of their events. This year, we provided college scholarships for some of the high school students (seniors) who participated in the program and worked especially hard. But we also did more. Together with a team at American University, we designed an evaluation of the program to better measure its impact. For more than 550 students, teachers collected essays written in class before and after the program. The essays were graded by professors and instructors in the Department of Literature at American University. This enabled us to assess whether the program made a difference in the writing skills of middle and high school students.
The evaluation demonstrated that the program has a positive impact. The program generates statistically significant gains in writing quality, especially for students who performed worst on the initial pre-program assignment. The positive impact of the program was confirmed through data on the perceptions of teachers and students about the program. Two summary briefs about those evaluation results have been written and are now available for public schools and for charter schools separately.
It is clear that this type of evaluation is beneficial for the nonprofits whose programs are evaluated, as the evaluations enable the nonprofits to measure their impact, and take corrective action when needed. The evaluations are also beneficial for our club in reassuring members that we are investing in worthwhile initiatives.
But there is more. Many others are interested in such evaluations and may learn from them, possibly generating larger impacts beyond the specific programs being evaluated. And these evaluations provide for great stories to be featured in local newspapers or magazines as well as social media, giving more visibility not only to the nonprofits and programs being evaluated, but also to the Rotary clubs that supported those evaluations.
This is what we are focusing on now – making sure that the positive results obtained by OWEd through its program are better known in Washington, DC, and beyond. We are writing short articles that document those results, and some of the stories of the students who benefited from the program. We have secured already two placements for stories in the local media and we hope to write additional articles for national publications about the results of the evaluation. In addition, we will also prepare technical papers for academic journals. It remains to be seen whether we will be successful, but we now have a stronger story to tell thanks to the evaluation.
Finally, as mentioned, the evaluation has been summarized in two easy-to-read briefs. The two briefs, together with briefs about the work of other nonprofits operating in the field of education and skills for youth in the city, will be included in a small brief series on innovations in education in Washington, DC to be published by the World Bank. We hope that this simple brief series will help attract attention to the nonprofits doing great work in the city, while also helping practitioners and policy makers learn from the experience of successful programs.
In summary, evaluation is essential not only to help improve service projects, whether implemented by Rotary clubs or nonprofits, but also to tell stronger stories about ways to improve the lives of the less fortunate. Investing more in evaluation seems to be a win-win for nonprofits as well as service clubs. And for Rotary as a whole, as I mentioned it in a previous series of posts on this blog, focusing more on partnerships, innovation, and evaluation seems key to achieve larger impacts.
by Quentin Wodon
Imagine a group of elementary school students gathering in a school gymnasium as part of a tutoring session. The students are trying to estimate the gravitational acceleration force on an object at sea level, where Washington, DC, is located. The students throw a golf ball in the air in the gymnasium. They record the time it takes for the ball to fall from apogee to the floor using a simple stop watch. They repeat the exercise 25 times. They also estimate the distance from apogee to the top of the ceiling, which is done by first measuring the distance from floor to ceiling and next by guessing by how much the ball misses the ceiling. The students’ estimate of ‘g’, the gravitational acceleration due to the force exerted by the earth on the golf ball, turns out to be within three percent of the accepted value for Washington, DC, even though each of the 25 individual computations per throw varied widely. This showed to the students how approximate values, when averaged, may converge on true values with reasonable accuracy.
Another experiment used a hygrometer, an instrument for measuring humidity or moisture content as well as temperatures. This was coupled with water and iced water in cans. Students had to figure out the temperature at which beads of water formed on the outside of the tin cans, which was followed by a discussion of what fog is, how temperature affects relative humidity, why clouds form and sometimes rain or snow is produced.
These scenes are not from a movie, but from a volunteer-based tutoring program run in a public school located in Anacostia, the poorest area of the city. Until recently, few children at the school passed standardized mathematics and reading tests, but things have improved. The tutoring program has now been in existence for six years. It is run by Dr. Don Messer a member of the Rotary Club of Washington, DC in District 7620. Together with teachers, school administrators, and a half dozen other tutors from his Rotary club Don designed the program in an innovative way.
The program focuses on mathematics and reading, and on the types of questions asked in standardized tests. This is not to “teach to the test”, but to ensure that children understand potential test questions well. Tutors work with students in small groups of three or four to generate interactions and more learning. The groups meet once or twice a week for the entire school year. The goal is not only to help the students learn, but also to help them understand that there is a future for them that often they didn’t know existed.
Tutoring can work to improve learning – this is why so many parents who have the means to do so invest in tutoring (there is a rather large literature on private tutoring – as just one recent example see this paper on Vietnam). But children from disadvantaged backgrounds do not have such opportunities, which is why volunteer-run programs are so important for those children.
To work well, tutoring sessions should be active, varied, and even fun. Sessions should combine structured and unstructured instruction, as well as individual and collective work, and they should focus on specific skills. In Don’s small but effective volunteer program the first part of each tutoring session focuses on prior test problems from DC standardized tests. These tests are augmented by problems that tutors or teachers prepare to emphasize special themes. In mathematics for example, a package would contain around 80 problems, ranging from routine arithmetic operations to data analysis (histograms, bar charts, tables), basic geometry, and problems that require reading to make sense of what is to be done. The problem set is paced by student progress, not by a time schedule. Tutors make sure that if a problem is difficult to understand for one or more of the students, all students understand what the problem is driving at before they start to work on the problem. Students work on the problem until all have finished, but if the tutor sees that at least one student remains confused, a group discussion is launched to help the students get the correct solution. The tutors also try to interject simple science illustrations within the problems to be solved, as illustrated earlier with the gravity constant and hygrometer experiments.
Impact and Recognition
How successful has Don’s program been? No impact evaluation is available to say for sure, but success rates at standardized tests have been systematically higher for tutored than non-tutored students year after year. The results, albeit not based on a randomized study, are encouraging. In part thanks to this program, the Rotary club of Washington, DC, was recognized two years ago as Volunteer Group of the Year by Chancellor Henderson of the District of Columbia Public Schools. For the Rotarian tutors, the experience has been highly rewarding. And in Don’s case, there was no better reward than having a fifth-grader tell him: “You know Dr. Messer, you’re my grandpa.”
In the third and last post in this series, I will discuss results from several programs that operate in Washington, DC, and have been rigorously evaluated, including Higher Achievement and Reading Partners.
Note: Part of this blog post is adapted from a section in a book published by the author entitled Membership in Service Clubs: Rotary’s Experience (Palgrave Macmillan, 2014).
by Quentin Wodon
Brandon was a quiet student enrolled in a primary school located in one of the poorest areas of Washington, DC, the capital city of the United States. Students in that area tend to have very low scores on standardized tests. Upon the recommendation of his teachers Brandon started to participate in the school’s tutoring program. He said little, but it was clear that he was absorbing the material being taught like a sponge. When the results from the District of Columbia’s comprehensive assessment system (DC-CAS) tests were announced, Brandon achieved proficiency in both mathematics and English. For his efforts and success, Brandon received a well-deserved award during the fifth grade graduation ceremony!
Tutoring and other supplemental education programs have received renewed attention in the United States. Under the much debated ‘No Child Left Behind’ Act adopted a dozen years ago, public schools that have not made enough progress in learning assessments for two consecutive years are in principle required to provide tutoring services to children. This makes sense given that there is scientific evidence that tutoring programs can make a difference in learning achievement if they are well implemented.
Series of Three Posts
This series of three posts on STEM (Science, Technology, Engineering, and Mathematics) education and tutoring in the capital city is written in recognition of World Science Day for Peace and Development celebrated each year on November 10. The day raises awareness of the importance of science and aims to bridge the gap between science and societies. The focus of World Science Day celebrations this year is about quality science education.
Improving science education is needed not only in developing countries, but also in developed countries, and especially so in the capital city of Washington, DC. This first post in the series documents the state of science education in the United States and in the District of Columbia. The second post will show how as individuals we can make a difference. That post will tell the story of Rotarians who have been actively involved in mathematics and science tutoring in one of the city’s schools for several years. The third post will argue that tutoring can be brought to scale and be part of the solution. That post will report on the impact of a tutoring program implemented in Washington, DC, and a few other cities by Higher Achievement.
Performance of the US
When Brandon received his award, he was enrolled in one of the worst performing public schools in Washington, DC (the schools has since made substantial progress under new management). The District of Columbia itself is one of the worst performing areas in the United States according to national assessment data. And the performance of the United States is one of the lowest among OECD and other developed countries according to international assessment data. Before talking about the potential promise of tutoring programs, providing a few statistics and basic facts about the performance of the United States, the District of Columbia, and schools within the District may be useful to underscore the magnitude of the problem we face.
Consider first the performance of the US as a nation. International comparable data on the performance of school systems in science, mathematics, and reading are available from PISA (Program for International Student Assessment). PISA measures skills for reading, mathematics and science literacy among 15 year olds. The test has been conducted every three years among a sample of students in each participating country since 2000. The latest round of data collection took place in 2012 with 65 countries participating. Results were released in December 2013.
Among 34 OECD countries, the US ranked 27th in mathematics, 17th in reading, and 20th in science, with no statistically significant improvement over time. This is despite the fact that the U.S. spends more per student than most other countries (only Austria, Luxembourg, Norway, and Switzerland spend more, but these countries do much better). More than one in four US students did not show basic mathematics proficiency on the test. The US also had a below-average share of top performers, and (not surprisingly) students from disadvantaged backgrounds performed worse on average.
Performance of the District of Columbia
Consider next the performance of the District of Columbia within the US. Comparable data on state-level performance are available from the 2013 National Assessment of Educational Progress (NAEP). Data on performance in mathematics are available in 4th and 8th grades.
Nationally, the average score for fourth-graders in mathematics was 242 in 2013. For the District of Columbia, the average was 229, the lowest score in the nation. Nationally, 83 percent of students performed at or above basic level. In the District, that share was 66 percent, again the lowest in the nation. Some 42 percent of students showed proficiency nationally, but in the District the proportion was only 28 percent. Only two states (Louisiana and Mississippi) performed worse. Gaps between the District and the nation are also large in eighth grade.
Whether those gaps are due to poor teaching or the fact that many children come from disadvantaged background is beyond the scope of this blog post (for an analysis of teacher value added in the district, see this recent paper). But whatever the reasons, the fact remains that many students in the District do not perform well. Furthermore, within the District of Columbia, inequalities in student performance also tend to be high between the well-to-do and the less fortunate.
Mentioning this inequality in performance between groups is just another way to emphasize how beyond broad averages, for the poor the likelihood to perform well on standardized tests in the District is really low. One way to show this inequality at work is to share a little known fact about the NMSQT/PSAT test administered each year in 11th grade by the College Board. For the high school class of 2015, the District (together with New Jersey) had the highest required qualifying scores for students to become National Merit Semifinalists. Students in the District had to obtain a score of 224 out of a maximum of 240 to qualify, a much higher threshold than in many other states. This is because while many students do poorly in the Districts, a few do very well, and the threshold to become a National Merit Semifinalist is state-specific and percentage based.
To sum up, the District of Columbia tends to be at the bottom in terms of average performance in mathematics (as well as science and reading) within the United States, with the United States also faring poorly internationally. That’s the problem. In the next two posts, I will discuss part of the solution – whether tutoring could help make a difference.