Evaluation is essential to assess what works and share stronger stories

Readers of this blog know that I have emphasized for some time the need to strengthen a culture of evaluation in Rotary. Evaluations should be undertaken not only for our service projects, but also to assess how our clubs meet, work, and grow – or wither away. This post is about a recent evaluation of an education project supported by my club, and how the evaluation is proving to be useful not only for the local nonprofit we worked with, but also for our club and more generally for practitioners and policy makers working in the field of education.

OWED celebration
Photo of OWEd scholarship winners with Brian Pick, Chief of Teaching and Learning for the District of Columbia Public Schools, and Dave Paris, member of the Board of OWEd.

For several years my club has supported One World Education (OWEd), a great nonprofit based in Washington, DC. OWEd runs the largest argumentative writing program in public and charter schools in the city. The nonprofit reached 5,800 middle and high school students this past school year. The aim of the program, which runs for 4-5 weeks in the schools, is to improve the research, writing, and presentation skills of the students, many of whom are from disadvantaged backgrounds and do not do very well in school.

In previous years, our support to OWEd consisted in providing a bit of funding and volunteering at some of their events. This year, we provided college scholarships for some of the high school students (seniors) who participated in the program and worked especially hard. But we also did more. Together with a team at American University, we designed an evaluation of the program to better measure its impact. For more than 550 students, teachers collected essays written in class before and after the program. The essays were graded by professors and instructors in the Department of Literature at American University. This enabled us to assess whether the program made a difference in the writing skills of middle and high school students.

The evaluation demonstrated that the program has a positive impact. The program generates statistically significant gains in writing quality, especially for students who performed worst on the initial pre-program assignment.  The positive impact of the program was confirmed through data on the perceptions of teachers and students about the program. Two summary briefs about those evaluation results have been written and are now available for public schools and for charter schools separately.

It is clear that this type of evaluation is beneficial for the nonprofits whose programs are evaluated, as the evaluations enable the nonprofits to measure their impact, and take corrective action when needed.  The evaluations are also beneficial for our club in reassuring members that we are investing in worthwhile initiatives.

But there is more. Many others are interested in such evaluations and may learn from them, possibly generating larger impacts beyond the specific programs being evaluated. And these evaluations provide for great stories to be featured in local newspapers or magazines as well as social media, giving more visibility not only to the nonprofits and programs being evaluated, but also to the Rotary clubs that supported those evaluations.

This is what we are focusing on now – making sure that the positive results obtained by OWEd through its program are better known in Washington, DC, and beyond. We are writing short articles that document those results, and some of the stories of the students who benefited from the program.  We have secured already two placements for stories in the local media and we hope to write additional articles for national publications about the results of the evaluation. In addition, we will also prepare technical papers for academic journals. It remains to be seen whether we will be successful, but we now have a stronger story to tell thanks to the evaluation.

Finally, as mentioned, the evaluation has been summarized in two easy-to-read briefs. The two briefs, together with briefs about the work of other nonprofits operating in the field of education and skills for youth in the city, will be included in a small brief series on innovations in education in Washington, DC to be published by the World Bank. We hope that this simple brief series will help attract attention to the nonprofits doing great work in the city, while also helping practitioners and policy makers learn from the experience of successful programs.

In summary, evaluation is essential not only to help improve service projects, whether implemented by Rotary clubs or nonprofits, but also to tell stronger stories about ways to improve the lives of the less fortunate. Investing more in evaluation seems to be a win-win for nonprofits as well as service clubs.  And for Rotary as a whole, as I mentioned it in a previous series of posts on this blog, focusing more on partnerships, innovation, and evaluation seems key to achieve larger impacts.

Pro Bono Rotarian Initiative

Rotary is about fellowship and service work. How do we increase the impact of our service work in order to achieve higher impact in our communities while also fostering fellowship among Rotarians and others committed to making a difference in the life of the less fortunate? One potential response is the concept of the pro bono Rotarian or Rotaractor.

In my (limited) experience, many clubs engage in service projects that do not really build on the professional expertise of their members. Beautifying a school before the start of the school year, serving food for the homeless, helping in the renovation of a house for a vulnerable family, distributing dictionaries to third graders, or even joining a polio vaccination drive for a short period of time are all worthwhile activities. Such activities should continue and they often enable many members in a club to be involved in the service projects of the club.

But these one-shot activities typically do not build on the expertise that Rotarians have developed over many years in their professional career. In addition to traditional (local) service projects, Rotarians should probably also engage in more extensive pro bono work, for example to provide advice to nonprofits as consultants would. While the term pro bono is often associated with free legal advise, pro bono work can be done in many other areas, building on a wide range of expertise that volunteers may have. The value of the volunteer time that Rotarians would allocate to pro bono consulting could be very high for local nonprofits, with potentially larger beneficial impacts for communities than is the case with traditional projects. Again, the idea is not to pitch one form of service work against another, but to expand on what clubs currently do in their service work.

Importantly, I believe that a pro bono consulting model may also be beneficial for fellowship among Rotarians. While for some issues faced by nonprofits pro bono consulting can be done effectively in a short period of time, for more complex issues analyzing the challenges faced by a nonprofit and suggesting a solution takes a few months. For these challenges, pro bono consulting is typically done by a small team of 3-5 volunteers who commit to dedicating a bit of their time for several months in order to provide in-depth professional and free advice to local nonprofits. As Rotarians work together on such pro bono projects, stronger fellowship and friendships will emerge, and the vitality of clubs will improve as well. The pro bono Rotarian concept can really be a win-win for local nonprofits, Rotary clubs, and the communities we serve.

This coming Rotary year, I will help my club explore in a systematic way pro bono consulting opportunities with local nonprofits in our area (Washington, DC). You will hear more about this in coming weeks and months through this blog. We will start small, and we will assess the value of our pro bono work along the way. But we hope that the idea will grow and strengthen our club, as well as other clubs that may adopt this model.

If you would like to move in this direction in your club as well or if you would like to discuss similar ideas you may have, don’t hesitate to comment on this blog or to send me if you prefer a private email through the Contact Me page. I will be happy to help if I can, and I look forward to learning from you if you have already adopted a pro bono consulting model in your own Rotary or Rotaract club.

Providing Education for Girls and Employment Opportunities for Women: Deepa Willingham at the World Bank

If you want to provide more opportunities to girls, you shouldn’t only provide them with an education – you also need to change perceptions of gender roles so that, when they grow up, girls can (among other things) fully contribute to the household’s livelihood. To achieve this, combining education with interventions for entrepreneurship and employment is the right way to go.  This messages emerges not only from impact evaluations, but also from experiences on the ground and case studies of non-governmental organizations.

PACE home-banner
PACE Universal Website Photo – Program Participants

In celebration of International Women’s DayDeepa Willingham, a Rotarian and the founder of a program called Promise of Assurance to Children Everywhere or PACE, participated in a World Bank event on March 8 about inspiring women who made a difference in the world through innovative programs in the areas of education and health.

PACE is educating girls ages three to twenty-three in a village in West Bengal, India. The school started and remains small, with a total of about 350 girls enrolled since 2003. But retention rates are at 90 percent and almost 100 girls have now completed primary school. The schools currently admits 25 students each year, well below the demand as the school receives 100 applications each year.  Admission is need-based in order to give priority to the most disadvantaged families.

What I find especially interesting is the fact that, based on community feedback, PACE has started to help women in the village find decent work through various initiatives. To help expand employment opportunities in the village, PACE is providing literacy and vocational training courses for women, many of whom go on to craft jewelry products sold locally and in the US thanks to a micro-loan.

Additional income generating activities include planting 10,000 fruit bearing trees and providing cycle-vans. Recently, an organic garden was initiated on the school’s grounds as a training facility for local farmers.  PACE has also been actively upgrading water and sanitation facilities by installing 35 tube wells and 400 sanitation units. Without safe water at home, deworming children in the school did not work as well.

Deepa explained to me that when the project started, family incomes in the village were extremely low. There are signs that this has changed for the families that have benefited from the NGO’s programs, with many families making three to four times more than what they used to  bring in (according to the families’ applications for their children to enroll in the school).

The attitudes of fathers towards their daughters have changed, as measured for example by their presence during the school’s cultural activities. Also, in the past many newborn girls in the village did not get birth certificates. This is changing simply because an official birth certificate is required for admission in the school.

Is the project cost effective? The cost for the package of services provided to girls is $375 and paid mostly through grants and other resources raised by the NGO. This package of services includes not only schooling (following a board approved curriculum), two meals per day, school supplies and uniforms, access to health care as needed, and after school enrichment programs in music, art, theater, yoga and life-skills training.

How does this compare to public schools? This is not an easy question to answer, because of complex funding by federal, state, and municipal governments for basic services as well as complementary programs (such as school meals for example). Estimates from various studies can be found through a rapid search on the web. It seems that overall PACE’s programs may be more costly than a typical public school, and also more costly than the programs implemented by low cost private schools. But the range of services provided by PACE is clearly broader, and quality is likely to be (much) higher.

As for PACE’s services for women, their cost is estimated at $175 per year. This includes the cost of the adult literacy program for reading, writing in Bengali, and accounting, as well as the vocational training program for jewelry making and tailoring.

On the occasion of International Women’s Day, at least two important lessons emerge from PACE. The first lesson is that we can learn from the experience of NGOs like PACE on how to combine multiple interventions – in education, but also in vocational training and basic health, in order to make a larger impact in the life of girls by changing attitudes towards gender roles. The second, and most important lesson is that beyond the important role of the state that we often emphasize in development work, committed individuals such as Deepa can truly make a difference in the life of girls and women.

A recording of the event at the World Bank in celebration of International Women’s Day will soon be available here in case you could not watch it online on March 8.

This post is adapted with minor changes from a post published on March 8 on the World Bank’s Education for Global Development blog.

Rotary’s Family Health Days Featured at the World Bank

Today, in celebration of International Women’s Day, Marion Bunch, the founder of Family Health Days, participated in a well-attended event at the World Bank about inspiring women who made a difference in the world through innovative programs in the areas of education and health. The event was streamed online through World Bank Live, and a recording will soon be available if you missed it. What are family health days? Do these days make sense from a policy point of view? And who is Marion Bunch? This blog post answers these three questions.

Family Health days

What are family health days?

Imagine hundreds of thousands of families and individuals getting free health care for two or three days. Family Health Days makes this feasible. Individuals and families can be tested for HIV/AIDs as well as receive counseling. Testing and/or screening for tuberculosis, diabetes, high blood pressure, and other conditions such as cervical and breast cancer is provided. Children receive immunizations as well as Vitamin A supplements. Some sites also include dental clinic and hygiene education as well as eye examinations. Health counseling is also provided.

The program was launched in 2011 and is now active in a half dozen countries in sub-Saharan Africa and South Asia. The program is led by Rotary in partnership with the Coca-Cola Africa Foundation, the U.S. Mission – including the Centers for Disease Control, USAID and the health service delivery expertise of their implementing partners – and Ministries of Health that provide services and supplies at the sites. Media partners promote the days in each of the countries. Thousands of volunteers in each country, including many Rotarians, help in various ways.

Do family or child health days make sense from a policy point of view?

Intrigued by the concept of family health days, I conducted a rapid search of the literature and found that for purposes such as screening for illnesses and child immunization, holding such days appears to make a lot of sense. This type of programs appear to be beneficial and cost effective if implemented well.

For example, Fiedler and Chuko looked at the reach and cost of child health days in Ethiopia in 2008. They found that the program reached more than 10 million children at an average cost of half a dollar per child (one dollar when including measles). This made the program cost-effective, with potential additional benefits to be reaped through economies of scope in increasing coverage at delivery sites.

Palmer and others note in a 2010 article that child health days are becoming increasingly popular, implemented in more than 50 countries at the time of the study. They conclude that the days are helping countries achieve high and equitable coverage of essential health and nutrition services. Child health days were also recognized as beneficial in a 2013 World Health Organization report on essential nutrition actions for improving maternal, newborn, infant, and young child health and nutrition.

Improvements could be made, however. In a 2012 review of experiences integrating the delivery of maternal and child health services with childhood immunization programs, Wallace, Ryman, and Dietz emphasize the importance of proper planning and awareness when implementing these interventions, among other actions to reduce the risks of logistical difficulties, time-intensive interventions ill-suited for campaign delivery, concerns about harming existing services, and overlap of target age groups with other service delivery mechanisms. The review also revealed gaps in information about costs and impacts.

On costs, a 2014 paper by Fiedler and Semakula suggests that part of the reason why costs may appear so low in some countries (US$0.22 per child in their analysis for Uganda) is because of underpayment for Ministry of Health staff and volunteer allowances. Still, the authors find that child health days are successful in saving lives, making them highly cost-effective. Vijayaraghavan and others in a case study for Somalia also find child health days to be cost-effective in addressing leading causes of child mortality in a conflict settings. They rate child health days as one of health sector’s best buys in sub-Saharan Africa.

Who is Marion Bunch?

What is also interesting – especially as we celebrate International Women’s Day and the power of women to make a difference in the life of the less fortunate, is the personal story of Marion Bunch. Marion is the Chief Executive Officer of the Rotarian Action Group Rotarians for Family Health & AIDS Prevention. She has received numerous awards on behalf of her work for AIDS. But first and foremost, she simply considers herself a mother who helps in representing the human face of AIDS. She started her work after losing her son to the disease in 1994. As she explained it in an interview. “I never thought I’d do anything about it until one day, three years after his death, I felt a tap on the shoulder, and a voice in my ear said, ‘Mom, get up and get going; you haven’t done anything, and it’s been three years.’” Since then, the reach of Family Health days has expanded every year, in close partnership with Ministries of Health in the countries where the program is running. Marion’s story has been told in a documentary film that won two 2015 Telly Awards.

What can be learned from Marion’s project?

On the occasion of International Women’s Day, at least two important lessons emerge her project. The first lesson is that we can expand access to health care for children and families through Child and Family health Days, and we should invest in the expansion of these programs. But the second and most important lesson is that beyond the important role of the state that we often emphasize in development work, committed individuals can truly make a difference in the life of the less fortunate. The theme of the event at the World Bank was “Inspiring Women of Action”. There is no doubt that Marion is such a woman.

If you were not able to watch the event live, a recording will soon be made available here.

This post is adapted from a post published today on the World Bank’s Investing in Health blog.

World Bank Live To Feature Rotarian Women of Action

 

On March 8, in celebration of International Women’s Day, World Bank Live will feature a discussion with inspiring Rotarians who have made a difference in the world. Hosted and sponsored by the World Bank Group Staff Association, the session will illustrate the power of women to change the world and improve the lives of the less fortunate through innovative and impactful projects in the areas of education and health.

The event will take place from 2 PM to 3 PM in the World Bank Preston Auditorium in Washington DC. The event will also be streamed online, so you can watch from wherever you are. Please do not hesitate to share this blog post with friends and other Rotarians who might be interested in this event, and maybe even ask you Club or District leaders to spread the word. This promises to be a great event.

iwd

What Is World Bank Live and How Do I Connect?

The World Bank Group aims to eradicate extreme poverty within a generation by 2030. It consists of five organizations: (1) the International Bank for Reconstruction and Development which lends to governments of middle-income and creditworthy low-income countries; (2) the International Development Association which provides interest-free loans and grants to governments of the poorest countries; (3) the International Finance Corporation which finances investment, mobilizes capital in international financial markets, and provides advisory services to businesses and governments; (4) the Multilateral Investment Guarantee Agency which offers political risk insurance (guarantees) to investors and lenders; and finally (5) the International Centre for Settlement of Investment Disputes which provides international facilities for conciliation and arbitration of investment disputes.

World Bank Live is the web streaming platform used by the World Bank Group to enable citizens worldwide to participate in high level events online. The platform enables viewers not only to follow the event, but also to post comments online as part of an interactive discussion. In order to connect to the March 8 event “Inspiring Women of Action: A Celebration of International Women’s Day”, simply click here.

Who Will Be the Speakers?

Three speakers will be featured.

Marion jennifer deepa

 

 

 

Marion Bunch is the Chief Executive Officer of the Rotarian Action Group Rotarians for Family Health & AIDS Prevention. Marion has received numerous awards on behalf of her work for AIDS, and considers herself a mother who represents the face of AIDS because she started her work after losing her son to the disease in 1994. One of her signature programs has been the organization of Family Health Days in several developing countries where families receive free consultations and health care.

Jennifer Jones is the President and CEO of Media Street Productions Inc., a television production company. She is also a Director on Rotary International’s global board. Through Rotary, she has successfully transferred her professional skills into her volunteer life including several Rotary missions where she has created documentaries and taught journalism and ethics classes in Brazil, Tanzania and Haiti. She has also participated in Rotaplast medical missions in Venezuela and Peru.

Deepa Willingham is also a Rotarian and the Founder and Chair of Promise of Assurance to Children Everywhere (PACE). Born and raised in India, Deepa worked in the United States, and then returned to India. PACE Universal is an organization dedicated nurturing the educational, health, nutritional, social and cultural development of girls in impoverished areas of India and other parts of the world.

The panelists will be introduced by Daniel Sellen, the Chair of the World Bank Group Staff Association. Daniel has been in the World Bank for twenty years, the last twelve of which based in Delhi, Abidjan, and Bogotá. He is the proud father of two daughters, who give him extra reason to celebrate International Women’s Day.

The event is organized by a team led by Eva Ruby de Leon, Christian Bergara, and Clara Montanez.

May I Attend the Event in Person if I Live near Washington, DC?

If you are watching the event online, no registration is needed.

If you would like to attend in person, limited seating is available. To attend in person, unless you are a World Bank staff, spouse, or retiree, you need to register ahead of time so that a security pass can be prepared for you. To register for the event in person, please click here.

We hope many of you will be able to attend in person if you are in the area, or watch the event online. Please don’t hesitate to send me an email through the Contact Me page of this blog if you have any question.

Open Access World Bank Publications on Education (Resources Series No. 5)

Getting a good education is one of the best ways to escape poverty in the developing world. This post, the fifth in a series on open access World Bank publications, provides easy access to a selection of 50 books and reports published since 2010 by the World Bank on education and (to a lower extent) on WASH in schools. The publications were compiled as a resource for participants at the 2016 Rotary International Presidential Conferences on literacy and WASH in schools in Kolkata, India, and on WASH in schools in Pasay near Manila, Philippines. The list of publications is available here.

KolkataManilla

Rotary International has long recognized the importance of basic literacy and education, as well as WASH (water, sanitation, and hygiene). These areas have been recognized as two of the six areas of focus of the Rotary Foundation. Many clubs and districts around the world are implementing projects in those areas.

How can clubs and districts contribute to efforts towards literacy and education, including through WASH in schools? These are the questions that will be discussed at the Kolkata and Pasay Conferences. The conferences are part of five flagship conferences organized by Rotary International in 2015-16. The other conferences are on peace and conflict resolution in Ontario (California), disease prevention and treatment in Cannes, and economic development in Cape Town.

The compilation of recent World Bank publications on education made available here is provided as a service to Rotarians and others working on those areas without any endorsement of the World Bank as to which publications should be featured. Access is provided through the World Bank’s Open Knowledge Repository. In order to keep the list manageable, the focus is on books and reports published since 2010 as opposed to other publications. Only publications from the World Bank are listed simply because covering (many) other organizations would be a rather complex task. At the same time, focusing on World Bank has the advantage of being able to go global with a single organization.

The hope is that the publications listed, and more generally the World Bank’s open access knowledge resources, will be useful to conference participants and others working on education and WASH in schools.

 

Open Access Publications from the World Bank: Introduction (Resources Series No. 1)

This post is the first in a series on open access resources from the World Bank that could be useful to Rotarians as well as others involved in service work and development projects around the world. Probably more than any other development organization, the World Bank is making available a wealth of resources on topics related to development, including a large number of books and reports. The focus of most World Bank open access knowledge resources is on developing countries, but data and publications are also available for developed countries, and often lessons learned from the developing world have implications for service projects and social policy in developed countries as well.

In coming weeks, this blog will feature selections of recently published World Bank books and reports by topic, considering in priority the areas of focus of the Rotary Foundation (TRF), namely promoting peace, fighting disease, providing clean water, saving mothers and children, supporting education, and growing local economies apart from eradicating polio. The hope is that the featured publications will be beneficial not only to researchers, but also to practitioners and policy makers.

Why a Focus on Open Access Resources?

The inspiration for this series of posts on open access resources came in part from the fact that Rotary is organizing between January and March 2016 five conferences on the core areas of focus of the Rotary Foundation. The first will be the Rotary Presidential Conference on Peace and Conflict Prevention/Resolution or “World Peace Conference” to be held in January 2016 in Ontario, California. The other conferences are on disease prevention and treatment in Cannes, economic development in Cape Town, literacy and WASH (water, sanitation, hygiene) in schools in Kolkata, and WASH in schools in Manila. The dates of the five conferences are listed in the table below together with their websites.

Dates Topic Location Website
15-16 January Peace and conflict prevention/resolution Ontario, California, USA Click here
19-20 February Disease prevention & treatment Cannes, France Click here
27 February Economic development Cape Town, South Africa Click here
12-13 March Literacy & WASH in Schools Kolkata, India Click here
18-19 March WASH in Schools Pasay City, Philippines Click here

The conferences are sponsored jointly by Rotary International President K.R. Ravindran and TRF Trustee Chair Ray Klinginsmith. Each conference will be led by local Rotary districts and are open to all, whether Rotarians or not. The conferences will feature plenary sessions with world class speakers as well as parallel sessions on topics of interest and hands-on workshops.

The hope for this series of posts on open access resources is that selecting relevant publications on the topics to be discussed at the above five conferences could be useful not only to conference participants, but also to many others working or implementing service projects in those fields.

Why Focusing on World Bank Resources?

Only resources available from the World Bank will be included in this series even though many other organizations provide highly valuable open access resources. Restricting the focus on resources provided by the World Bank is driven by practicality. Including other organizations would yield a rather unwieldy list of relevant publications due to the scope of what would need to be included. At the same time, focusing on the World Bank has the advantage of being able to go global with a single organization, since the World Bank is engaged with the developing world as a whole. By contrast, many other organizations, including regional development banks, tend to have more of a regional focus.

In order to keep the list of publications and other resources highlighted through this series manageable, the focus in most cases will be on open access books and reports as opposed to other publications such as working papers, articles, and briefs. Even when restricting resources to books, a large number of World Bank publications directly relevant to the topics of the five Rotary conferences can be listed. In the case of the first conference on promoting peace for example, several dozen recent books and reports published since 2010 that relate closely to the topics of the conference can be listed.

Topics for Consideration

To keep things simple, the series of posts will consider in priority the six areas of focus of TRF, which also correspond to the topics selected for the five Rotary Presidential conferences (to a large extent, the conference on disease prevention and treatment also implicitly covers the area of focus of TRF devoted to saving mothers and children).

But the series will also feature a few cross-sectoral topics that are highly relevant to multiple areas of focus of TRF. One example is that of early childhood development, for which interventions are needed from virtually all six areas of focus of TRF. The series could also cover some topics in more depth than others, for example allocating more than one post to a single area of focus of TRF if this appears to be warranted.

So please, do not hesitate to share your views as to what should be covered by providing a comment on this post, so that your views can inform the final selection of topics and open access resources to be provided.

Increasing the Impact of Rotary (Partnerships Series No. 9)

This post is the last in a series of nine posts on partnerships, innovation, and evaluation in Rotary. The rationale for the series was my conviction that if Rotary is to have a larger impact globally, it must rely more than has been the case so far on partnerships, innovation, and evaluation (and in some areas advocacy, as has been the case with polio). Seven different projects or investments that have relied on partnerships, were innovative, and were evaluated at least in some way, were showcased. A compilation of the case studies together with a brief introduction is available here. Separate briefs are also available for each of the projects here.

TRF_Centennial_logo_lockup

As I mentioned it in the introduction to the series, partnerships help to implement larger projects and benefit from the expertise of organizations that are among the best in their field. Rotary’s Foundation was created almost 100 years ago (the Centennial is next year) and it has about $1 billion in assets. This is respectable, but in the world of development projects, which is in practice where Rotary is investing most of its funds, this remains small. Without innovation, the contribution of Rotary is an important drop, but still a drop in the development assistance bucket.

By contrast, if Rotary clubs and district innovate, successful pilots can then be scaled up by other organizations with deeper pockets, thereby potentially achieving much larger impact. However, for innovative projects to be recognized as such, proper evaluations are needed. We must be able to demonstrate the impact of pilot projects. Innovation and evaluation are like twins: they work best in pairs. Together, partnerships, innovation, and evaluation are the key to larger impact.

To encourage clubs and districts to think bigger and more strategically, stories of great projects were shared: an innovative financing mechanism for polio eradication; an award winning project fighting malaria and Ebola in Mali; a teacher training program that is transforming teaching and learning in Nepali classrooms; a project on obstetric fistula saving the lives of mothers and children in Nigeria; a program to invest in the writing skills of disadvantaged youth in the United States; a project to improve access to water and sanitation in Uganda; and a global network of Peace Centers and Peace Fellows to help promote peace.

Some of these programs and projects are large. Others are small. Most were implemented through global grants, but one was implemented through a district grant. All these projects have been in one way or another innovative. They have all leveraged partnerships not only to crowd in financial resources, but also – and even more importantly – to build on great expertise. And they have all relied on monitoring and evaluation mechanisms to assess their impact, at least partially.

Putting together great projects requires work. Fundraising is often time consuming in Rotary given the funding model of the Rotary Foundation that requires raising funds from many clubs and districts first before getting a match from the Foundation. Planning, implementing, and in addition evaluating projects also takes time, especially when one tries to do this in a professional way. Finally, in order to be innovative, Rotarians leading projects need to be aware of where the frontier is in their field, and what could be innovative. This also takes some time.

There is nothing wrong with clubs and districts funding and implementing traditional Rotary projects. Most projects will continue to be fairly simple, with funds provided to worthy charitable causes. These projects, as well as the volunteer time often contributed by Rotarians when implementing them, serve an important purpose. The beneficiaries of these projects are better off thanks to them. These projects help communities, and they also benefit Rotary through the goodwill that the projects create.

But if we want to raise the bar and achieve larger impact, we also need to do more innovative projects. Rotary needs to be bolder, more ambitious. It needs to better learn from its projects, both the great and not so great ones, and make sure that lessons learned are shared broadly, well beyond the Rotary family. The launch of the Future Vision model, despite some challenges, was a step in the right direction. As we celebrate the Centennial of the Rotary Foundation next year, let’s make sure that we have the right vision for what Rotary and its Foundation could accomplish in the next 100 years.

Improving Teaching and Learning in Nepal (Partnerships Series No. 8)

Many developing countries have made substantial progress towards improving education attainment (the level of education attained by students) over the last two decades. At the same time, the instruction provided by teachers to students often remains of limited quality. This results in less than stellar education achievement (how much students actually learn). While students may do well enough on portions of examinations that rely for the most part on memorization, they tend to do less well when asked to think creatively or solve complex problems. This post, which is part of a series on partnerships, innovation, and evaluation in Rotary, tells the story of an innovative teacher training program in Nepal that has the potential of improving student learning substantially.

NTTI

Importance of Teacher Training

Outstanding in-service teacher training programs can make a major difference in how teachers teach, how much students learn, and how much they learn, especially among disadvantaged groups. Many factors influence student achievement, including factors that are beyond the control of schools such as a student’s socio-economic context. But teachers are the most important factor under the control of education systems to improve learning. Teachers also account for the bulk of public spending for education in developed and developing countries alike. For these reasons, there is increasing interest in finding ways to attract, retain, develop, and motivate great teachers.

The tasks of attracting, retaining, and motivating teachers fall squarely within the scope of the mission of Ministries of Education. Developing teachers is also a key responsibility and priority for the Ministries, but in this area there is also scope for nonprofits and organizations such as Rotary to play a role by helping to create great in-service training programs for teachers. The importance of in-service training and professional development to improve instruction is recognized by practitioners and policy makers. Three lessons emerge from the literature.

First, opportunities for teacher training and professional development should be made available. But not all programs achieve the same results. When in-service programs focus on changing pedagogy, the evidence suggests that they can improve teaching and as a result student achievement. By contrast, programs that merely provide additional teaching materials for teachers do not generate substantial gains.

Second, the contents of training programs aiming to change pedagogy matter as well. In-service training programs that expose teachers to best practices in instruction and actually show teachers how to implement these practices are more likely to generate positive change. Promoting collaboration between teachers, among others through teacher networks where teachers can exchange ideas is useful. Mentoring programs whereby junior teachers benefit from the guidance of experience teachers also tend to be effective. Other approaches tend to be less successful.

Third, it is important that in-service training and professional development programs target in priority the teachers who need help the most. Teachers who are struggling may benefit more than already great teachers from various programs. Similarly, students from disadvantaged backgrounds or living in poor areas also tend to benefit more from a higher quality instruction than better off students who have more help from their family at home. Identifying priority pockets of needs is most beneficial when implementing and teacher training programs.

Innovative Program in Nepal

Traditional instruction in Nepal relies on lecturing by teachers and memorization by students. Together with the Nepali NGO PHASE, NTTI (Nepal Teacher Training Innovations) has implemented innovative teacher training programs in Nepal for several years.  NTTI aims to train public schools teachers to make the classroom more interactive by coaching them on how to lead classroom discussions, facilitate group work, and ask questions to students to encourage individual thought. Instead of relying on punishment and at times shaming in the classroom to control student behavior, teachers are trained to use dynamic inquiry-based instruction methods and provide positive encouragement to motivate the students to learn. As the classroom becomes more participatory, students engage in their own learning.

The PHASE-NTTI model does not rely on one-off training. Instead it involves a cumulative cycle of trainings and intensive follow-up support to individual teachers. The aim is to help teachers move from an awareness of effective teaching practices to actual implementation of the practices in their own classrooms. The training model includes a series of teacher development courses: Introduction to Best Teaching Practices; Girls’ Sensitivity Training; and a Training of Trainers for those selected as Mentor Teachers.

The model includes pre- and post-training classroom observations, individual feedback received by teachers from Master Trainers, and follow up individual teacher support by Mentor Overall, the program is implemented over a two-year period in each school.

While no impact evaluation is yet available to measure the impact of the program, quantitative data obtained through pre- and post-training classroom observation are encouraging. In contrast to teacher-driven and student-silent classrooms, classrooms with trained teachers seem to be closer to functioning as hubs of learning.

Instead of only lecturing trained teachers lead classroom discussions, facilitate group work, and ask questions to encourage individual thought. Students learn how to make their own novel connections and think critically about what they hear and read. Qualitative data suggest that the program is appreciated by teachers and students.

Remaining Challenges and Conclusion

There have been challenges to which the program has had to adapt. The program did not work as well in secondary schools, so it now focuses on primary schools. Support from principals for teachers changing their pedagogical approach is needed, but not guaranteed. Distances to schools in rural areas make it hard to maintain regular contact after initial trainings. Lack of time for teachers to prepare lessons as advocated by the program is also a constraint. The structure of classroom time may limit creativity and inquiry-based teaching. The persistence of traditions harmful to girls in parts of the country is a major challenge to keep girls in school.

The PHASE-NTTI program does not have all the answers to these challenges, but it does have the key features that tend to be associated with successful in-service training programs. The program is also a great example of partnership (with the Ministry of Education and public schools), innovation (in teacher training), and evaluation (at least through monitoring of teacher pedagogy). A Rotary global grant proposal has been submitted to help develop the PHASE-NTTI program further and implement it in additional areas.

Fighting Malaria and Ebola in Mali (Partnerships Series No. 7)

As part of this series of posts on increasing Rotary’s impact through partnerships, innovation, and evaluation, I had to include Muso, a nonprofit that has successfully fought malaria and Ebola in Mali, in part with support from Rotary. As some readers may remember it, I talked about Muso in this blog previously, and this post is based in large part on previous posts. But at the risk of repetition, it is worth emphasizing again that Muso is a great example of an innovative approach that has been evaluated and has the potential of being replicated at scale thanks to partnerships.

Muso2

A Successful Pilot

Muso started in Mali as a project to provide basic care to communities, focusing initially in part on malaria. Two years ago a Harvard/University of California San Francisco study documented a tenfold difference in child mortality rates after the rollout of the Muso model in the program’s catchment area. Muso relies on professionalized community health workers to diagnose illnesses quickly in its catchment area, refer patients to clinics and hospitals as needed, and reduce financial barriers to care for families.

One needs to be careful in inferring causality between the intervention and the reduction in child mortality since the evaluation was based on repeated cross-sectional data as opposed to a randomized control trial. Still, the impact appears to have been major and obtained at relatively low cost. The Muso model was recently recognized as a finalist for two major prizes – the GSK Global Healthcare Innovation Award and the Caplow Children’s Prize.

Support to the Ministry of Health

Some successful pilot interventions in health are implemented without ever being scaled up, so that their benefits for a country’s population as a whole may be limited. This is not the case for Muso. In addition to implementing and evaluating an innovative model, Muso worked closely from the start with the Malian Ministry of Health as well as other partners to explore opportunities to expand the model nationally.

In November last year, based in part on the Muso model, Mali’s Ministry of Health Division of Community Health Systems announced a strategic plan to scale up professionalized community health workers throughout the country. How did this transformation happen? As just mentioned, Muso and other partners have been actively working with the Ministry of Health for seven years. The partnership was launched in 2008. The collaboration has been not only operational, but also scientific, with support from university researchers. Though this partnership, Muso has been able to provide technical assistance to help the Ministry develop a national plan for scaling-up the community health worker model.

Factors contributed to the success of this partnership and to the scaling-up announcement according to Dr. Ari Johnson, the co-founder of Muso, who was interviewed last year. “First, we started the partnership early on. By setting an operational research partnership at the design phase of the project, we were able to ensure that the pilot would focus on the priorities of the Ministry of Health, which meant a focus on child survival and disease-specific targets, including early effective treatment for malaria.”

A second factor for success was the ability to work with other NGOs to test the robustness of the community health worker model in different parts of the country. “The Malian Ministry of Health worked simultaneously with several NGOs on operational research to test community health workers models. This included, but was not limited to Muso with the operational research study in Yirimadjo and Doctors without Borders (Médecins sans Frontières) with another study in Kanbaga. These experiences with multiple partners in several locations provided the Ministry of Health with converging evidence for scaling up paid, professionalized community health workers,” explains Dr. Johnson.

The third factor for success was the support of international organizations. Multilateral and bilateral global health institutions are all trying to strengthen health systems in Africa. In Mali, Dr. Johnson explains that UNICEF and the Global Fund provided important support for the adoption and expansion of the community health workers model by the Ministry of Health.

Finally Dr. Johnson insists that operational research partnerships must be long-term to succeed. “Longitudinal operational research partnerships take time to implement, but over time, they help build relationships between public sector policy makers and hubs of research. These relationships become avenues for translating research into evidence-based policy change at scale.” Dr. Johnson adds that “the long view is critical, and often neglected in global health work. Short-term funding cycles push organizations to move on after a few years and abandon the foundations of a strong partnership. A long term partnership for iterative and ongoing research is crucial to support Ministries in their strategic plans and thereby achieve scale.”

Many questions remain. What should be the health care financing system for community health workers? How can those workers be deployed across both urban and rural areas? How is the supervision of the workers to be worked out? These and other questions will need to be answered. But progress is being made.

Relevance for Ebola

How does all this relate to the Ebola crisis that recently hit West African countries? Apart from their role in preventing and treating malaria as well as other common illnesses, community health workers can be essential in the fight against Ebola. Ebola arrived in Mali through a two-year-old girl who had traveled with her grandmother from Guinea died.  Mali became the sixth West African country with a confirmed Ebola case.

Why are community health workers so important for the fight against Ebola? They are crucial in part because they tend to be trusted members of their communities. They can not only help in providing information about Ebola and promoting appropriate behaviors, but they can also help to trace and monitor those who have been in contact with the virus. This must be done for at least 21 days – the period during which symptoms do not yet emerge, and it must be continued after that period if individuals become sick. In the other countries affected by the virus – including Guinea, Liberia, and Sierra Leone, community health workers already play that vital role.

As Dr, Johnson explained it, “there is huge potential for community health workers to accelerate the effort to stop Ebola across West Africa, by supporting epidemiologic surveillance, contact monitoring, returning traveler monitoring, community engagement, and prevention counseling.”

Conclusion

Muso has piloted an innovative new model of health care delivery that appears to have contributed to reducing child mortality in its area of intervention. The model has also proved valuable in fighting the Ebola epidemics. Rotary provided crucial support to Muso when the NGO was still small and not as well-known as it is today, with fewer resources. Rotarians and Rotaractors have volunteered with Muso in Mali, and others have contributed to making the project better known internationally.

Today, Muso is scaling up, aiming to raise substantial funds to expand its program. It is also launching a rigorous impact evaluation through a randomized controlled trial to measure its effectiveness. While many organizations and individuals have contributed to Muso’s success, especially the Muso team working on the ground, at times taking substantial risks to help the population as was the case during the Ebola epidemics, Rotary and Rotarians have played a small supporting role as well.