Infrastructure and Poverty: Part 3 – What Can Be Done?

by Quentin Wodon

Infrastructure matters for poverty and development (first post in this series), but the needs of the poor are not being met (second post). In this third post, the discussion shifts to what can be done to improve infrastructure services for the poor, considering first reforms and subsidies, and next projects including those by service organizations.

Water fountain (Photo: C. Tsimpo)
Water fountain (Photo: C. Tsimpo)

Reforms and Subsidies

PPIs (private participation in infrastructure) and IRAs (independent regulatory agencies) have been among the most important reforms implemented in the past two decades. The empirical evidence reviewed in my   book with Antonio Estache suggests that these reforms have improved investment and service quality (as well as reduced corruption in the case of IRAs), but there are differences in these impacts between sectors and the effects have been relatively modest. In addition, while these reforms have helped some households, they probably have helped mostly households among better off groups since the poor simply have no access to basic infrastructure countries in low income countries.

Another area of concern is that of subsidies. Many countries have (often large) consumption subsidies for basic infrastructure services such as electricity and piped water. These subsidies prevent cost recovery by the utilities and this reduces the incentives for them to expand the networks. But in addition, while the subsidies are justified in theory by the aim to make services affordable to the poor, they are in practice very poorly targeted to the poor. In most countries the average subsidy received by a poor person is only a third (or less) of the average subsidy received by a person randomly chosen in the population as a whole.

Most existing subsidies are consumption subsidies implemented through the inverted block tariff (IBTs) structures. With IBTs the unit cost per kWh or cubic meter of water is lower for the first few blocks of consumption for all those connected to the network, and often below cost recovery levels. In low income countries, these subsidies are poorly targeted to the poor simply because most of the poor are not connected to the networks.

The targeting performance of those subsidies could be improved by reducing the lower bands of the IBTs so that only those who consume very low amounts of water or electricity benefit from the largest subsidies. Another option is to shift to Volume Differentiated Tariffs, whereby only those consuming in a tariff band receive the subsidy for that bans (under IBTs, all clients receive the subsidies in the lower bands for the part of their consumption in those bands). Both approaches tend to have a limited positive impact on targeting performance, but they help in reducing the cost of the subsidies.

Another alternative is to shift from consumption to connection subsidies. Instead of subsidizing the consumption of those already connected to the water network or the electricity grid, the idea is to subsidize the connection of new and typically poorer households to the network. Still another alternative is to target subsidies to those in need more purposefully (this can be done among others through geographic targeting or proxy means- testing). In those cases, targeting performance to the poor can under some conditions be improved significantly. These are all high priorities that governments and utilities should implement, as argued for example here.

Projects

What about the role of the nonprofit community, including NGOs and service clubs? Precisely because so many among the poor do not have access to basic infrastructure services, the role of nonprofit organizations is important to fill part of the gap in coverage of basic infrastructure services. It is important however for nonprofits, including service clubs, to operate in a highly professional and sustainable manner.

Providing access to electricity, water, sanitation, or other basic infrastructure services in poor areas is hard. Many projects are implemented in an unsustainable way, so that they ultimately fail. Research I am doing with Clarence Tsimpo on Uganda suggests that many small water projects fail in large part because of lack of infrastructure functionality (facilities stop to work properly, even shortly after being installed) or lack of local responsibility (poor local leadership or lack of proper community arrangements hinder maintenance, thereby yielding a slower but often irreversible damage in the infrastructure).

Sometimes, expensive technologies are put in place that communities have simply no way to keep up because of the high cost of parts for repairs. Training for proper maintenance may not be provided at all, or provided in a haphazard way, to beneficiary communities. These and other factors lead to the failure of many projects despite the best of intentions. So what are NGOs and especially service clubs to do? They need to get professional advice. In the Rotary family, the good news is that advice may be available from Rotarian Action Groups (RAGs). In the case of water and sanitation WASRAG is ready to help.

My Rotary club was recently considering a promising water harvesting and sanitation project in India. We got detailed specifications ready and they looked good.  But we asked for a professional review by a district Rotarian expert in the field. He raised concerns and suggested we contact WASRAG for advice. We did, and in the end we decided to subsume our own small individual project and funds into a much larger project run by WASRAG. This gave us piece of mind that the project would benefit from the professional expertise it needed.

In Rotary, on average (there are of course exceptions), larger projects are likely to be better designed than small ones. They also tend to be better managed because the stakes are higher. This means that they probably  have (again on average) more impact, and are more likely to be sustainable. Larger projects also require less administrative work than multiple smaller ones. Not all Rotary service projects need to be large projects where many clubs and districts pool resources together with professional advise from RAGs. But in some cases, when the expertise of RAGs is available, it is a good idea to work with them and pool resources.

 

Infrastructure and Poverty: Part 2 – Are Household Needs Being Met?

by Quentin Wodon

The first post in this series asked whether infrastructure matters for poverty reduction and other development outcomes. It does. This second post asks whether the infrastructure needs of households are being met. They are not. Part of the analysis relies on my new  book with Antonio Estache on infrastructure and poverty in Africa.

Solar panels in Karamoja, Uganda (Photo: C. Tsimpo)
Solar panels in Karamoja, Uganda (Photo: C. Tsimpo)

Challenges in Low Income Countries

Infrastructure does matter for growth and poverty reduction, but there is probably a difference in that relationship between low and middle or high income countries. In low income countries, there is no guarantee that investments in infrastructure will benefit the poor in a straightforward way, unless the investments are designed from the start to do so.

Consider it this way. The priorities of private investors and households in poverty are likely to differ in low income countries. In the African context especially, the number of the poor is rather large, with many living in rural areas and having no access to basic infrastructure services. Only the better off tend to have access to those services, and even at the margin, new investments in infrastructure may not necessarily benefit the poor, simply because they live too far away from the electricity grid or the piped water network. Incentives for private utilities to reach the poor are limited.

What about the links between infrastructure and employment? There is no doubt that lack of infrastructure is an obstacle for firms to operate. In enterprise surveys, close to half of firms declare that lack of electricity is a constraint for them, and one fourth cites lack of telecom and transportation services. This compares to 40 percent of firms citing corruption as a major obstacle to doing business. These rates are high, suggesting that lack of infrastructure is indeed a major constraint to investments and growth. But at the same time, in low income countries only a small minority of workers is engaged in the formal sector where these firms operate. Even if the firms would do better, this still may not have a direct immediate impact on the poor, apart from trickle down effects.

Another challenge relates to the cost and quality of service provision – such as the generation, transmission, and distribution of electricity. In part because the networks are small in many countries, operating costs tend to be higher in Africa than in other regions of the world. In some cases high costs result from over-engineering of projects. As for quality, in part due to capacity constraints, service is often provided only intermittently. These and other challenges make it difficult to serve the poor, especially in Africa.

Gains in Coverage?

 Organizations such as NGOs and service clubs are not engaged in large infrastructure projects. But they can play an important role in meeting household demand for basic infrastructure – including for off-grid electricity, water, and sanitation. In the case of Rotary for example, the Water and Sanitation Rotarian Action Group (WASRAG) is actively involved in providing access to water and sanitation in local communities. The question of whether household demand for basic services is being met is thus important not only for governments and utilities, but also for nonprofit organizations.

Progress in meeting household demand has unfortunately been very slow, especially in Africa. In telecoms there has been dramatic progress thanks to the mobile cell phone revolution. But in other sectors, with the exception of gains in rural electrification in some countries, coverage rates have not improved much (see this paper). For piped water, coverage rates have remained below 20 percent on average across countries with no clear gain over time. For electricity, there has been an increase in coverage rates from a fourth of households to about a third thanks as just mentioned to gains in rural areas. For flush toilets as for piped water, access has also remained flat with only about one in ten household being served. Even when gains in coverage are being achieved, these tend to benefit mostly better off households.

Cost, Affordability, and Supply

In urban and peri-urban areas small-scale providers are filling some of the gaps left by national or regional utilities, but they often have high costs and substantial margins, and are thus expensive for households. In Niger, a study suggests that the cost of water per liter from street vendors could be up to five times higher than the cost from the piped network.

Is the lack of coverage of infrastructure in the population a demand or supply issue? Lack of affordability of modern infrastructure services is an issue for the poor, and it may reduce the demand for those services. Yet the main constraint is lack of supply, not lack of demand (see this paper). The fact is that it is often more expensive for households to meet infrastructure needs through small scale providers or alternative sources than through the networks. For electricity and lighting, the cost of batteries, candles, or kerosene lamps is often higher, at least per unit of efficient energy, than the cost of an electricity bill.

The problem is not that households do not want to connect to networks. It is that even though households would benefit from a connection to existing networks, the opportunity to do so is often not available. This may be because households live too far from the networks. But it may also be because connection costs requested by utilities are often high, especially for the poor and when the costs have to be paid in a single installment.

This quick diagnostic suggests that a lot of work remains to be done to provide basic infrastructure services to the poor in Africa and many other developing countries. In the third post of this series, the record of the reforms and policies of the past two decades will be discussed, together with their implications for projects by organizations such as Rotary.

Infrastructure and Poverty: Part 1 – Does Infrastructure Matter for Poverty Reduction?

by Quentin Wodon

World Toilet Day will be celebrated in a few days on November 19. Some 2.5 billion people do not have access to proper sanitation, including toilets and latrines. Lack of sanitation has dramatic consequences for health. Several million people, many of them children, die from diarrheal diseases every year. Many of these deaths are attributed to unsafe water, poor sanitation including lack of toilets, and poor hygiene. Access to basic infrastructure services – not only for sanitation, but also electricity, piped water, and transport – remains low in many countries.

Women in Uganda carry water (photo: C. Tsimpo)
Women in Uganda carry water (photo: C. Tsimpo)

This 3-part post series discusses the relationship between infrastructure and poverty. The focus is on Africa (the region discussed in my book with Antonio Estache published this week), but the lessons apply more broadly. I will ask three questions: (1) Does infrastructure matter and is funding sufficient? (2) Are household infrastructure needs being met?; and (3) Have reforms succeeded, and what does it mean for us?

Infrastructure Matters

Infrastructure has long been recognized as essential for growth, and growth in turn is empirically proven to be the best way to reduce poverty in the long run (reducing inequality also helps, but has a much lower impact, especially in very poor countries where there is not much to redistribute). Estimates suggest that the elasticity of GDP to infrastructure is in the 0.4 to 1.5 range. This is large – better infrastructure has a major impact on growth.

Infrastructure also matters for other development outcomes at the individual and household level. As mentioned above, many child deaths could be adverted with access to improved water sources and better sanitation. Infrastructure also helps households shift time from domestic chores to productive work.

This has gender implications. In the developing world women work on average longer hours than men. They are involved, as men are, in farm and labor market work, but in addition they have the responsibility to fetch firewood and water. This responsibility can be time consuming. As a villager from Uganda explains: “They are few public taps available here and there is a lot of congestion, making it hard to access water without waiting for a period of one to two hours”.

In work I am doing with Clarence Tsimpo on Uganda, regression analysis with the latest household survey suggests that in areas where the electricity grid or the piped water network is available, a connection to the grid or piped water network for those households not yet connected could enable women to decrease their domestic working time, and correspondingly increase their market working time by about two hours per week.

The additional earnings that could be generated through this shift could reduce the share of the population in poverty by about one percentage point for each of the two basic infrastructure services. While this would not by itself eradicate poverty in the country, it would help beneficiary households, and especially rural women, fairly substantially.

Infrastructure as a New Priority

Throughout much of the last two decades, funding allocated to infrastructure by governments fell in proportion of available budgets. It has also been said that transport was one of the forgotten MDGs (Millennium Development Goals). Today, the situation has changed and the crucial role of infrastructure is widely recognized. Yet funding remains a challenge.

According to the World Bank, private infrastructure investment in emerging markets and developing economies dropped from US$186 billion in 2012 to $150 billion last year. At its annual meetings last month, the World Bank announced the launch of a new global infrastructure facility. While developing countries invest US$1 trillion per year on infrastructure, this would need to be doubled to maintain current growth rates and meet future demand for infrastructure from firms, households, and regions.

The private sector will play a key role in future infrastructure investments, but governments will need to invest more as well. For this, they will need to rely on both their own tax revenues and the availability of foreign aid. For low income countries, concessional financing (grants or very low interest loans) will remain crucial.

When increasing funding for infrastructure, governments and donors will need to be careful to assess fiscal and institutional capacity – not all countries have the same absorptive and implementation capacity. The worst that could happen would be to have large investments in sub-optimal infrastructure projects. The risk of an increase in debt to unsustainable levels must also be managed. But many countries do have the capacity to absorb more funding for infrastructure.

This is the big picture about the relationship between infrastructure and poverty in Africa and many other parts of the world. The next post in this series will discuss whether household needs are being met, and if not, why not.

Preparing and Evaluating Great Conferences: Part 3 – Lessons Learned

by Quentin Wodon

In the first post of this series, a simple argument was made for the importance of evaluating annual conferences – whether for Rotary districts or other organizations. Major investments are made in those conferences in terms of time and money. They are highlights of the life of their organizations, and essential to build friendships and teamwork among members. In the second post, summary results from the evaluation of the latest annual conference of Rotary district 7620 were provided to show how simple evaluations can provide valuable insights. In this last post,  more information is shared on how the evaluations for the last three conferences of the district were designed, and what some of the recommendations of participants were for future conferences.

Rotarians pack meals for the homeless at a district conference session
Rotarians pack meals for the homeless at a district conference session

Design of the evaluations

The questionnaire of the surveys implemented among conference participants were administered through the web (Survey Monkey) a few days after each conference. Using web surveys reduces the time needed to tabulate data, and ensures that there is no waste of information – for example from qualitative feedback – due to legibility issues often encountered with printed surveys.

In 2014, a total of 100 Rotarians responded to the survey, generating a response rate of about 40 percent, which is fairly good for a web survey and is likely to provide a good level of representativeness. Response rates for the two previous surveys for 2012 and 2013 were good as well. However, it may be that Rotarians who respond are those who tend to be more involved in the activities of their clubs and districts.

The conference evaluation surveys have been implemented for three years. Very similar questionnaires were fielded in the three years to maximize comparability. In 2013 and 2014 however, additional questions were added versus 2012 to better capture preferences from participants for future conferences.

The 2014 survey had a total of 24 questions, some with multiple sub-questions. The questionnaires were designed to take about 15’ to complete, so that substantial information could be captured without taking too much time for respondents. Two emails (one initial email and one reminder email) were sent to participants to ask them to fill the survey – this was enough to generate good response rates.

In terms of the structure of the questionnaire, a first set of questions were asked to respondents about their profile (age, gender, Rotary status, length of membership, club affiliation, past attendance at district conferences, attendance rate at club meetings, positions of leadership in the organization, etc.). A second set of questions asked participants to evaluate all of the conference sessions to which they participated one by one, as well as their general appreciation of the conference along a number of characteristics and some of their preferences for future sessions. Finally, a last set of questions were open-ended to elicit qualitative feedback on the conferences. The questionnaire of the 2014 evaluation is available in the report Evaluating Rotary District Conferences: Lessons from District 7620).

Suggestions from Respondents

Key results from the evaluation of the 2014 survey were already provided in the second post in this series. But it may be useful to summarize some of the feedback received for future conferences. As mentioned earlier, while the results are strictly speaking valid only for Rotary district 7620, they probably have broader relevance for other districts and service organizations.

When asked what types of sessions they would like to see more of in future conferences, participants suggested having more sessions on successful projects and debates/discussions on Rotary and its future. In terms of the types of speakers, participants would like more motivational and entertaining speakers, as well as more speakers from the business world versus nonprofits. Participants would also like less time spent on award ceremonies. Having at least one session devoted to a service project – like packing meals for people who are homeless in the picture above, is highly appreciated.

Participants would like the conferences to be shorter (at two and a half days, the 2014 conference was shorter than the 2012 and 2013 conferences, but even shorter conferences would be better). Shorter conferences would also help reduce the cost of attending the conference. This in turn may make it easier to attract more Rotarians to these events, including some of the younger Rotarians for whom cost may be a more serious issue.

As to whether it is better to have one or more districts present at a conference, the feedback was split between the two options – some participants prefer to have only their own district, while others like the opportunity to meet members from other districts. Virtually all participants like opportunities for discussions with Interactors (high school members of Interact clubs) and Rotaractors (young professionals in Rotaract clubs).

Conclusion

Evaluating district conferences in a serious way is feasible at virtually no cost, as illustrated in the case of Rotary district 7620 in this series of three posts. The results suggest that most participants are highly satisfied with the events. The hotels are often great, as is the organization. Yet areas for improvement include the need to hold the cost of the conferences down and to organize the conferences in such a way that more learning on the future of Rotary and successful service projects can take place. Many of these recommendations have been observed for three years in a row in the evaluations of the conferences implemented by district 7620. The good news is that by learning from these evaluations, the district has been able to further increase satisfaction rates with the conferences.

Next year’s district conference promises to be a bit different from the past three – with more of an emphasis on being financially friendly to new members. The goal, as in previous years, will be to have as many new members in the district attend as possible. But the conference committee is exploring – among other ideas – the possibility of relying on the hospitality of Frederick Rotarians to open their homes for an overnight stay for attendees. With about 400 Rotarians in four clubs living in the Frederick area where the conference will take place, this could be very successful.

 

Preparing and Evaluating Great Conferences: Part 2 – A Case Study

by Quentin Wodon

For the past three years, Rotary district 7620 has conducted evaluations of its annual district conferences using web surveys. As mentioned in the first post in this series, conducting such evaluations is important. Millions of hours and tens of millions of dollars are invested every year by Rotarians in attending district conferences, yet these conferences are rarely evaluated thoroughly.

This post shows how such evaluations can be useful by summarizing results for the (highly successful) 2014 conference (the report for all three conferences combined is entitled Evaluating Rotary District Conferences: Lessons from District 7620). The third post will provide lessons learned on what Rotarians would like to see in future conferences. While the analysis is specific to district 7620, it probably has broader relevance as well.

The "See Something Say Something" theater performance was the highest rated session at the conference
The “See Something Say Something” theater performance was the highest rated session at the conference

Success of the Conference

The evaluations of the previous two conferences of the district (in 2012 and 2013) suggested that while participants enjoyed these conferences, they could have been shorter and less expensive with more engaging speakers and more learning opportunities. Conferences should also be fun.

Did the district succeed in organizing a great conference in 2014? To a large extent, the answer is yes. The conference was shorter, and had lots of fun, but its cost for participants remained relatively high. The conference was better attended than in previous years. Most participants were seasoned Rotarians, but many Interactors and Rotaractors participated as well, thanks in part to an Interact Leadership Conference organized as a smaller sub-conference within the main conference.

Almost 60 percent of participants rated the conference as better than previous conferences, which is a major achievement (in the previous two years most respondents rated the conference as on par with previous conferences). A total of 100 Interactors, Rotaractors, and Rotarians responded to the web-based evaluation survey for the conference, which makes its results reliable.

Evaluation by Category and Session

Data on satisfaction rates with the facilities and various aspects of the conference were obtained and are shown in the Figure below. Most of the ratings look great with large shares of respondents rating various aspects as very good or good. The hotel rooms as well as the conference and hotel facilities and the convenience of the location ranked at the top. The organization of the conference and the opportunities for fellowship were also well rated. Even the category on learning about Rotary was well rated, but as in previous years only one in five participants said that they had learned a lot of new information that is likely to be useful to them as Rotarians. Apart from the issue with the quality of the food served by the hotel, the cost of the conference was the category with the lowest ratings. This is in fact an issue that has been identified for three years in a row.

Selected Results from the 2014 D-7620 Conference Evaluation
Selected Results from the 2014 D-7620 Conference Evaluation

The evaluation provided feedback on all conference sessions. For 26 sessions the sample size was large enough to tabulate responses (a minimum of 10 respondents per session was required to assess a session individually). The conference was focused on youth (on Friday and Saturday) and Wounded Warriors (for the Sunday brunch). Nine of the 26 sessions got 60 or more “very good” ratings.

By and large these were the sessions on youth, including the traditional Four Way Test competition for high school students, a See Something Say Something theater performance on bullying by Interactors from North Carolina, a speech by Jack Andraka – the winner of the prestigious Intel high school competition, and a speech by Teresa Scanlan – a recent Miss America and founder of an orphanage in Haiti. Also highly rated were the Interact Leadership Conference and the Mother’s Day brunch with wounded warriors on Sunday. Hospitality suites as always also fared very well.

What Worked, What Could be Improved

For those who did not attend this specific conference, all this may seem a bit abstract. But these data and results are provided to make the point that by evaluating conferences through simple web surveys, you can know exactly which sessions were great, and which ones not so great. This obviously can be useful for preparing future conferences. In addition, for the 2014 conference, it was clear that the focus on youth and wounded warriors was a hit, something to keep in mind.

The evaluation – including feedback from open-ended qualitative questions included in the web survey, also suggested areas for improvement, together with data on preferences regarding the type of speakers to invite, the length and cost of these conferences, and some of their other features. All of this will be discussed in the third and last post on this topic.

Note: Part of the analysis in this post is updated from a section in a book published by the author entitled Membership in Service Clubs: Rotary’s Experience (Palgrave Macmillan, 2014).

Preparing and Evaluating Great Conferences: Part 1 – The Need

Note: This post is the first in a series of three on preparing and evaluating conferences. Part 1 is about the need to evaluate, part 2 about a case study doing so, and part 3 on some of the lessons learned.

by Quentin Wodon

Many organizations, including firms, NGOs, professional associations, as well as service clubs, organize annual conferences. These conferences are essential not only for running the business of the organization, but also for building team work among the members of the organization. It is not clear however whether these annual conferences are evaluated properly.

Past RI President Bill Boyd at the D-7620 2014 conference
Past RI President Bill Boyd at the D-7620 2014 conference

In the case of Rotary, a global service club organization founded in 1905 which has today 1.2 million members, a global annual convention is organized typically in June of each year. But what may matter more are the conferences organized by each of the 530 or so districts in the world. These district conferences are annual events to which Rotarians from the district are invited to participate. They represent the main event where Rotarians from different clubs belonging to a common geographic area can meet each other and exchange their experiences.

It is important for an organization such as Rotary International to pay attention to district conferences not only because they are essential events in the life of clubs and districts, but also because the resources invested in the conferences are substantial, in terms of both time and money. Right now is the time when many Rotary districts are actively preparing their next annual conference, since these events are typically held in April or May before the end of the Rotary year.

This first post in a series of three on preparing and evaluating great conferences provides quick and dirty “back of the envelope” calculations of the potential investments made in district conferences every year. Because this investment is rather large, the conferences should be evaluated seriously.

Investments in Time

Consider first the allocation of time. To provide a very rough estimate of the time that may be invested every year in preparing and attending district conferences, data from the last three conferences organized by District 7620 can be used. The first conference was organized by district 7620 jointly with district 7630 in 2012. Together the two districts cover the states of Maryland and Delaware, as well as the District of Columbia (that is, the capital city Washington, DC). For the second conference, district 7610 in Northern Virginia joined in as well. The third conference was organized solely by district 7620.

The total attendance at the three conferences combined was estimated at about 1,050 Rotarians and guests. If one considers that activities at the conference go on for up to ten hours per day, and if we assume that participants on average spend 2.5 days attending the conferences (including travel time), with 1050 participants this would represent an investment in time of 26,250 hours. Together, the two districts that organized the joint conference in 2012 have about 4,000 Rotarians, and when the three districts are included for the 2013 conference, membership is above 6,000. For district 7620 in 2014, the membership was at about 2,350. Over the three years, this represents a potential attendance at the conference of close to 12,400 Rotarians. Therefore the amount of time allocated on average for the two conferences was then approximately 2.12 hours per Rotarian per year.

Multiplying this estimate by the number of Rotarians in the world yields an allocation of about 2.54 million hours per year for Rotary worldwide just for Rotarians to attend district conferences. This estimate may well be on the low side given that the attendance rate at the two conferences evaluated in this report, at about 8.5 percent on average for the three years using the approach just outlined, may be lower than is typical in other districts.

In addition, one should include the time spent by district leadership teams and conference committee members as well as district executive secretaries in preparing the conferences. It is not unrealistic to suggest that for any given district at least 30 Rotarians will volunteer time to help prepare various aspects of the conferences, including preparing their own presentations. Some of these Rotarians will spend a very large amount of time on the conferences, especially if they are part of the core organizing committee, while others will spend less time.

Just for the sake of the argument, consider an average of 25 hours spent by each of the 30 Rotarians for preparing various aspects of the conference (this is probably a rather low estimate). With some 530 Rotary districts worldwide, this would generate another 400,000 volunteer hours for preparation. Thus, possibly three million hours are allocated to district conferences every year by Rotarians, and this may well be a conservative estimate. The bottom line is that clearly a large amount of time is allocated to prepare and attend these conferences, so that making these conferences a success does matter for all those involved.

Financial Investments

Consider next the question of cost. Most of the costs of district conferences are borne by participants who pay their hotel bill as well as a fee for attending the conference to cover meals and other costs. A typical fee in the United States to attend a district conference will run at a few hundred dollars including meals and hotel rooms.

District conferences tend to be organized in major hotels or resorts, at least in the United States, so that even when special rates are obtained, room fees will typically be in the $100 to $150 per night. Some Rotarians stay for three or more nights, others for only one or two nights, and some just come for a day. Guests are likely to have additional expenses both at the site of the conference and to get to the conference site.

Assume, again for the sake of the argument, that on average the total cost of the conference per attendee is of the order $600 – it will often be  higher for those staying three nights, but it will be lower for those coming just for one day without staying any night in the hotel or resort. Even based on the somewhat low attendance rate of about 8.5 percent mentioned above for the three conferences considered here, with 1.2 million Rotarians worldwide, this would generate a cost of $61 million per year.

This might be too high because the cost of district conferences may be lower in other countries than it is in the United States. But on the other hand, this does not include special costs for the attendance of high level Rotary officials (whether those are paid for by districts or Rotary International) and other invited speakers. This estimate does not include the cost at Rotary headquarters to oversee and monitor district conferences and it also does not include the cost of the annual Rotary convention that typically welcomes more than 20,000 Rotarians from all over the world each year and could in a way be considered as a super district conference.

This cost estimate also does not factor in the opportunity cost of the time allocated by Rotarians to prepare and attend the conferences (that opportunity cost could be very high if estimated as is normal practice at the wage rate of those involved). Overall, the costs of district conferences are likely to be substantial, and could be higher than those indicated here depending on conference participation rates.

Implication

Major investments in time and money are made every year by Rotarians to prepare and attend district conferences. These conferences are the highlight of the Rotary year in each district. They should be evaluated well. The second post in this series will show how this can be done professionally at very low cost with a case study, and the third post will discuss lessons from the evaluation of the last three conferences of district 7620.

Note: Part of this blog post is adapted from a section in a book published by the author entitled Membership in Service Clubs: Rotary’s Experience (Palgrave Macmillan, 2014).

 

STEM Education and Tutoring in the Capital City: Part 3 – Achieving Impact

Note: This post is part of a series of three on tutoring for science and mathematics among disadvantaged students. Part 1 looks at needs. Part 2 and part 3 give examples of successful programs.

by Quentin Wodon

The United States and especially the District of Columbia are lagging behind in STEM education, as discussed in the first blog post of this series. When Don and his team designed the small Rotary-led tutoring program described in the second post of the series, they did not start with a review of the evidence from the literature on what works. But through the experience of the teachers and principal at the school, as well as their own experience, they had a pretty good idea of what could be useful. As a result, the design of the program actually corresponds to what the literature recommends.

Don, a teacher, and a few of the tutored students
Don, a teacher, and a few of the tutored students

Lessons from the Literature

The literature on tutoring and out-of-school-time programs (see for example the review by Heinrich and Burch) suggests that in order to achieve impact, it is often useful to: (1) provide consistent and sustained instructional time, for a total of at least 40-45 hours; (2) provide tutoring to small groups of students, preferably less than ten at a time; (3) follow a curriculum that is rich in content and takes into account the specific needs of students while being also closely related to what students learn during the regular school day; (4) ensure that tutoring sessions are active and varied (for example by combining structured and unstructured instruction, as well as individual and collective work time) and focused on targeting the development of specific skills; (5) foster positive relationships between tutors and students; and finally (6) foster collaboration between teachers and tutors with support of administrators, including for constructive evaluation. All of these features are at work in Don’s program.

There is substantial interest in tutoring today in the US. As mentioned in the first post in this series, under the ‘No Child Left Behind’ Act adopted a dozen years ago, public schools not making enough progress in learning assessments for two consecutive years must provide tutoring services to children. Tutoring initiatives are being implemented throughout the country. Earlier this year Mayor Emanuel announced the expansion (with private funding) of a mathematics tutoring program in Chicago that University of Chicago researchers found helpful for at-risk students in public schools (see the review of the study in the New York Times).

Examples of Great Programs

Another example of intensive tutoring program having impact is Higher Achievement. The NGO operates in Washington, DC, Baltimore, Richmond, and Pittsburgh. Students in the program meet three days a week during the school year. They first complete homework with support from teachers and volunteers. They then have dinner and work on a specific subject in small groups of two or three with a trained volunteer mentor. This is a rigorous program – overall, students spend a total of 650 hours a year in the program between 5th and 8th grade.

Data from Higher Achievement suggest that three fourth of the enrolled students improve their grade point average (GPA) by at least one letter grade, and 96% graduate from high school – two times the rate of their peers. Three fourths of the students also go on to graduate from college – four times the rate of their peers. The program has been evaluated rigorously by MDRC, a nonprofit, nonpartisan research organization. Researchers from the University of Texas at Austin compared Higher Achievement students (“scholars”) with a control group of students who applied to the program, met the admissions criteria, but were not selected to participate through a randomized lottery.

According to the evaluation of Higher Achievement published last year, the program had a statistically significant positive impact after one year in the program on mathematics proficiency and reading comprehension, as measured by standardized tests. The mathematics impacts lasted four years after enrollment in the program. The program also increased the probability that the students would enroll in high performing private high schools. These findings suggest that intensive OST (out-of-school-time) programs like Higher Achievement can be beneficial.

Another program that also operates in Washington, DC, and that has been rigorously evaluated by MDRC is Reading Partners. The program serves more than 7,000 students in over 130 schools in California, Colorado, New York, Oklahoma, Maryland, South Carolina, Texas, and Washington, DC. As was the case with the small Rotary-funded program in Washington, DC, and the larger program operated by Higher Achievement, Reading Partners works in (large) part with volunteers, which helps in keeping costs down. The evaluation of Reading Partners was conducted in 2012-13 in a subset of the schools where the program operates. Results suggest gains in reading proficiency. While this evaluation was not about STEM, it suggests again that tutoring programs can make a difference.

Policy and What You Can Do

From a policy point of view, there are legitimate questions about the cost effectiveness of some tutoring programs. This cost effectiveness issue must be looked at carefully on a case by case basis. But when the programs are staffed in part or fully by volunteers, they are more likely to be cost effective. Tutoring may also in some cases – especially when it is profit-motivated, act as a substitute for good quality teaching. This may be a serious problem in some developing countries (as an example, see this paper on Nepal), but probably much less so in developed countries. In most situations, tutoring is likely to lead to positive changes.

For those who care about helping disadvantaged students better succeed in schools, the good news is that there are many ways to contribute. If you have or can take the necessary time to do so, you can get personally involved like Don and his fellow Rotarians are doing, going every week to a school and working with a few students. But if you do not have the time, you can still help by contributing funding to organizations that are doing a great job on the ground.

STEM Education and Tutoring in the Capital City: Part 2 – Measuring Gravity

Note: This post is part of a series of three on tutoring for science and mathematics among disadvantaged students. Part 1 looks at needs. Part 2 and part 3 give examples of successful programs.

by Quentin Wodon

Innovative Tutoring

Imagine a group of elementary school students gathering in a school gymnasium as part of a tutoring session. The students are trying to estimate the gravitational acceleration force on an object at sea level, where Washington, DC, is located. The students throw a golf ball in the air in the gymnasium. They record the time it takes for the ball to fall from apogee to the floor using a simple stop watch. They repeat the exercise 25 times. They also estimate the distance from apogee to the top of the ceiling, which is done by first measuring the distance from floor to ceiling and next by guessing by how much the ball misses the ceiling. The students’ estimate of ‘g’, the gravitational acceleration due to the force exerted by the earth on the golf ball, turns out to be within three percent of the accepted value for Washington, DC, even though each of the 25 individual computations per throw varied widely. This showed to the students how approximate values, when averaged, may converge on true values with reasonable accuracy.

Two students perform the gravity experiment
Two students perform the gravity experiment

Another experiment used a hygrometer, an instrument for measuring humidity or moisture content as well as temperatures. This was coupled with water and iced water in cans. Students had to figure out the temperature at which beads of water formed on the outside of the tin cans, which was followed by a discussion of what fog is, how temperature affects relative humidity, why clouds form and sometimes rain or snow is produced.

Two students work with a hygrometer
Two students work with a hygrometer

Program Characteristics

These scenes are not from a movie, but from a volunteer-based tutoring program run in a public school located in Anacostia, the poorest area of the city. Until recently, few children at the school passed standardized mathematics and reading tests, but things have improved. The tutoring program has now been in existence for six years. It is run by Dr. Don Messer a member of the Rotary Club of Washington, DC in District 7620. Together with teachers, school administrators, and a half dozen other tutors from his Rotary club Don designed the program in an innovative way.

The program focuses on mathematics and reading, and on the types of questions asked in standardized tests. This is not to “teach to the test”, but to ensure that children understand potential test questions well. Tutors work with students in small groups of three or four to generate interactions and more learning. The groups meet once or twice a week for the entire school year. The goal is not only to help the students learn, but also to help them understand that there is a future for them that often they didn’t know existed.

Tutoring can work to improve learning – this is why so many parents who have the means to do so invest in tutoring (there is a rather large literature on private tutoring – as just one recent example see this paper on Vietnam). But children from disadvantaged backgrounds do not have such opportunities, which is why volunteer-run programs are so important for those children.

To work well, tutoring sessions should be active, varied, and even fun. Sessions should combine structured and unstructured instruction, as well as individual and collective work, and they should focus on specific skills. In Don’s small but effective volunteer program the first part of each tutoring session focuses on prior test problems from DC standardized tests. These tests are augmented by problems that tutors or teachers prepare to emphasize special themes. In mathematics for example, a package would contain around 80 problems, ranging from routine arithmetic operations to data analysis (histograms, bar charts, tables), basic geometry, and problems that require reading to make sense of what is to be done. The problem set is paced by student progress, not by a time schedule. Tutors make sure that if a problem is difficult to understand for one or more of the students, all students understand what the problem is driving at before they start to work on the problem. Students work on the problem until all have finished, but if the tutor sees that at least one student remains confused, a group discussion is launched to help the students get the correct solution. The tutors also try to interject simple science illustrations within the problems to be solved, as illustrated earlier with the gravity constant and hygrometer experiments.

Impact and Recognition

How successful has Don’s program been? No impact evaluation is available to say for sure, but success rates at standardized tests have been systematically higher for tutored than non-tutored students year after year. The results, albeit not based on a randomized study, are encouraging. In part thanks to this program, the Rotary club of Washington, DC, was recognized two years ago as Volunteer Group of the Year by Chancellor Henderson of the District of Columbia Public Schools. For the Rotarian tutors, the experience has been highly rewarding. And in Don’s case, there was no better reward than having a fifth-grader tell him: “You know Dr. Messer, you’re my grandpa.”

In the third and last post in this series, I will discuss results from several programs that operate in Washington, DC, and have been rigorously evaluated, including Higher Achievement and Reading Partners.

Note: Part of this blog post is adapted from a section in a book published by the author entitled Membership in Service Clubs: Rotary’s Experience (Palgrave Macmillan, 2014).

 

STEM Education and Tutoring in the Capital City: Part 1 – The Problem

Note: This post is part of a series of three on tutoring for science and mathematics among disadvantaged students. Part 1 looks at needs. Part 2 and part 3 give examples of successful programs.

by Quentin Wodon

Brandon was a quiet student enrolled in a primary school located in one of the poorest areas of Washington, DC, the capital city of the United States. Students in that area tend to have very low scores on standardized tests. Upon the recommendation of his teachers Brandon started to participate in the school’s tutoring program. He said little, but it was clear that he was absorbing the material being taught like a sponge. When the results from the District of Columbia’s comprehensive assessment system (DC-CAS) tests were announced, Brandon achieved proficiency in both mathematics and English. For his efforts and success, Brandon received a well-deserved award during the fifth grade graduation ceremony!

Brandon receives an award for his hard work
Brandon receives an award for his hard work

Tutoring and other supplemental education programs have received renewed attention in the United States. Under the much debated ‘No Child Left Behind’ Act adopted a dozen years ago, public schools that have not made enough progress in learning assessments for two consecutive years are in principle required to provide tutoring services to children. This makes sense given that there is scientific evidence that tutoring programs can make a difference in learning achievement if they are well implemented.

Series of Three Posts

This series of three posts on STEM (Science, Technology, Engineering, and Mathematics) education and tutoring in the capital city is written in recognition of World Science Day for Peace and Development celebrated each year on November 10. The day raises awareness of the importance of science and aims to bridge the gap between science and societies. The focus of World Science Day celebrations this year is about quality science education.

Improving science education is needed not only in developing countries, but also in developed countries, and especially so in the capital city of Washington, DC. This first post in the series documents the state of science education in the United States and in the District of Columbia. The second post will show how as individuals we can make a difference. That post will tell the story of Rotarians who have been actively involved in mathematics and science tutoring in one of the city’s schools for several years. The third post will argue that tutoring can be brought to scale and be part of the solution. That post will report on the impact of a tutoring program implemented in Washington, DC, and a few other cities by Higher Achievement.

Performance of the US

When Brandon received his award, he was enrolled in one of the worst performing public schools in Washington, DC (the schools has since made substantial progress under new management). The District of Columbia itself is one of the worst performing areas in the United States according to national assessment data. And the performance of the United States is one of the lowest among OECD and other developed countries according to international assessment data. Before talking about the potential promise of tutoring programs, providing a few statistics and basic facts about the performance of the United States, the District of Columbia, and schools within the District may be useful to underscore the magnitude of the problem we face.

Consider first the performance of the US as a nation. International comparable data on the performance of school systems in science, mathematics, and reading are available from PISA (Program for International Student Assessment). PISA measures skills for reading, mathematics and science literacy among 15 year olds. The test has been conducted every three years among a sample of students in each participating country since 2000. The latest round of data collection took place in 2012 with 65 countries participating. Results were released in December 2013.

Among 34 OECD countries, the US ranked 27th in mathematics, 17th in reading, and 20th in science, with no statistically significant improvement over time. This is despite the fact that the U.S. spends more per student than most other countries (only Austria, Luxembourg, Norway, and Switzerland spend more, but these countries do much better). More than one in four US students did not show basic mathematics proficiency on the test. The US also had a below-average share of top performers, and (not surprisingly) students from disadvantaged backgrounds performed worse on average.

Performance of the District of Columbia

Consider next the performance of the District of Columbia within the US. Comparable data on state-level performance are available from the 2013 National Assessment of Educational Progress (NAEP). Data on performance in mathematics are available in 4th and 8th grades.

Nationally, the average score for fourth-graders in mathematics was 242 in 2013. For the District of Columbia, the average was 229, the lowest score in the nation. Nationally, 83 percent of students performed at or above basic level. In the District, that share was 66 percent, again the lowest in the nation. Some 42 percent of students showed proficiency nationally, but in the District the proportion was only 28 percent. Only two states (Louisiana and Mississippi) performed worse. Gaps between the District and the nation are also large in eighth grade.

Whether those gaps are due to poor teaching or the fact that many children come from disadvantaged background is beyond the scope of this blog post (for an analysis of teacher value added in the district, see this recent paper). But whatever the reasons, the fact remains that many students in the District do not perform well.  Furthermore, within the District of Columbia, inequalities in student performance also tend to be high between the well-to-do and the less fortunate.

Mentioning this inequality in performance between groups is just another way to emphasize how beyond broad averages, for the poor the likelihood to perform well on standardized tests in the District is really low. One way to show this inequality at work is to share a little known fact about the NMSQT/PSAT test administered each year in 11th grade by the College Board. For the high school class of 2015, the District (together with New Jersey) had the highest required qualifying scores for students to become National Merit Semifinalists. Students in the District had to obtain a score of 224 out of a maximum of 240 to qualify, a much higher threshold than in many other states. This is because while many students do poorly in the Districts, a few do very well, and the threshold to become a National Merit Semifinalist is state-specific and percentage based.

To sum up, the District of Columbia tends to be at the bottom in terms of average performance in mathematics (as well as science and reading) within the United States, with the United States also faring poorly internationally. That’s the problem. In the next two posts, I will discuss part of the solution – whether tutoring could help make a difference.

Early Childhood Development Free Online Course: Part 3 – How to Implement ECD Interventions?

Note: This post is part of a series of three explaining what is covered under the why (part 1), what (part 2), and how (part 3) modules of the World Bank’s online course.

by Quentin Wodon

Following up on the posts of yesterday and the day before, today’s post is about the “how” of early childhood development (ECD). This is the question asked in the third and last module of the World Bank’s new online course on ECD.

ecd

The first question – the “why” of ECD – focused on individual child-level outcomes: how children grow and develop, the powerful positive impact that ECD interventions can have for them, the high economic returns from such interventions, and the risks incurred without them. Next, with the “what matters” question, we took a step back and looked at ECD systems as a whole at the country level – are country policies and programs structured in such a way that they can deliver ECD interventions for young children. The third question is about “how”. The “how” question looks at what lies in between country systems and ECD interventions at the child level. It deals with how ECD program and policies can be designed, implement, and evaluated.

How to implement ECD interventions is again a fairly broad area of inquiry. The World Bank course focuses on six different topics related to the “how” question. Each is briefly discussed in this post. As before, the hope is that together with the two previous posts, this post will enable readers to decide whether it would be worthwhile for them to actually take the online course, and if they do not, to at least provide them with a few useful resources on ECD.

The first topic on the “how” of ECD focuses on what the World Bank has done in this field in terms of support to governments. This is after all a World Bank course! Key findings from a 12-year review of the World Bank’s experience with ECD projects are provided. The good news is that lending and grants as well as analytical work for ECD has increased dramatically in the last few years after a number of policy document recognized the importance of investments in ECD. Examples of projects implemented in countries such as Mexico, Eritrea, and Indonesia are also provided.

The second topic considers the issue of inter-sectoral coordination between agencies and Ministries, as well as between the wide range of providers of ECD services. It discusses what a comprehensive EDC system could look like and how coordination helps in achieving synergies between interventions as well as cost savings though the combination of interventions in service delivery. Again, examples of coordination mechanisms from Africa, Latin America, and the Caribbean are provided.

The third topic is devoted to ECD diagnostics, whether for countries or projects. The topic discusses the inputs that are needed for ECD interventions, and how such interventions can be funded. It reviews some of the broad indicators (including those related to the Millennium Development Goals) that have been used to monitor ECD outcomes at the country level, as well as some of the more detailed child assessment tools that can be used for more detailed evaluations, including by servicer providers such as NGOs.

These assessment tools are then described in more details in the fourth topic in order to help practitioners chose the right type of child assessment to measure whether they are indeed achieving the outcomes that they are trying to achieve. The issue of how to adapt such assessment tools to local conditions – an assessment tool for the United States may not work in Bolivia – is also discussed. The topic finally presents briefly the area of focus of on-going impact evaluations being implemented by the World Bank, in part using these tools, in ten countries in Africa, Latin America, East Asia, and South Asia.

The fifth topic looks in more details at the cost of ECD interventions and their financing. The topic shows how to estimate the overall cost of specific interventions, as well as the cost per beneficiary. While some costs may be borne by families, many of the costs should be paid for by governments and/or other funders, especially when programs targeted low-income beneficiaries. Guidelines as to how much countries should invest in ECD programs are provided and examples of combinations of funding sources, allocation mechanisms, and coverage rates for different types of ECD interventions from Columbia, Denmark, and Indonesia are documented.

The sixth and last topic considers issues related to project design, implementation, and monitoring and evaluation. The discussion is based in part on a case study for a large project implemented by the World Bank in Indonesia. After providing a snapshot of the project, the concepts of results chain and results framework are introduced to link inputs with outputs and intermediate as well as final outcomes. While the case study is based on a large project, the concepts presented also apply to smaller initiatives led by NGOs and other service providers.

That’s it for the structure of the course! As mentioned in the first post of this series, the course takes about four to five hours to complete. The course was initially designed for World Bank task team leaders in education, health, and social protection who are involved in the design and implementation of ECD projects or in policy dialogue on ECD with governments and other organizations. While these staffs have broad-based program and project-related responsibilities, they may not be familiar with some more detailed aspects of ECD interventions and policies. The e-learning course provides them with a resource to broaden their knowledge base.

But it is also hoped that the course, which is available free of charge, should be of interest to policy makers in countries as well as staffs from donor and other agencies, including non-governmental organizations. The course may also be of interest to undergraduate and graduate university students. The hope is that many of you, the readers, will take the course to learn more about this fascinating and very important topic.

Note: This series of three posts is based on the e-learning course mentioned above, which was developed by a World Bank team comprising of Amina Denboba, Monica McLin, Michelle Neuman, Rebecca Kraft Sayre, Yidan Wang, and the author.