Free ebook 2 – Partner, Innovate, Evaluate: Increasing Rotary’s Impact

The second ebook in the Rotarian Economist Short Books Series has been published. Partnerships, innovation, and evaluation can increase the quality, scope, and reach of Rotary’s service work in communities. The book suggests with case studies how this can be done. All books in the series are free and available here in multiple formats.  Please share this link widely with others for them to be able to benefit from this resource. And if you like the books in the series, please consider writing a quick review at Smashwords!

2-partner-innovate-evaluate

Rotary International’s New Website and its Story on Literacy

Rotary International launched a much anticipated new website this month. The Rotary Leader January newsletter mentions five reasons to check out the new site, but I must admit that my main reason to look at it right away was the story on literacy because I knew it would feature a great Nepal project.

NTTI

The literacy story talks about the importance of training teachers. It features several great projects, but the one I know well is implemented by NTTI (Nepal Teacher Training Innovations) and PHASE, two great NGOs my club is working with together with the Rotary Club of Kathmandu Mid-Town. The aim of the project, which will hopefully soon benefit from a global grant, is to improve instructions in Nepal’s primary schools.

Below is a brief excerpt of the story as it relates to NTTI. The story is featured on the “supporting education” section of the new website:

“Before taking part in the NTTI program, one teacher relied heavily on memorization, having her students copy words off the blackboard. After training, the teacher made her lesson on animate and inanimate objects more interactive, says Ashley Hager, NTTI’s director. The teacher asked children to point to objects and describe how they were different. She then listed the differences on the board and paired students up to discuss them. As a final exercise, the class went outside to find examples in nature.

One student approached the teacher with a live ant in her hand and inquired, “This is an animate object, yes?” The teacher agreed. The child then squashed the ant and asked, “Is it still an animate object now?” Caught by surprise, the teacher asked the rest of the students what they thought, and a lively conversation followed. 

Other teachers agree that the training taught them the value of interactive teaching. “It’s transformed my way of teaching and given me brilliant ideas to employ the best teaching practices I have learned,” says Goma Khada, who teaches fourth grade at Shrijana Higher Secondary School in Thumpakhar.”

The literacy story is available at the following link (https://www.rotary.org/en/teaching-teachers-key-literacy). If you would like to contribute to the global grant, let me know!

This story is just one of many great stories on the new website. Stories are provided for each area of focus of the Rotary Foundation. Other stories focus on what it means to be a Rotarian or other topics of interest to clubs. So please check the new website out!

And just for the sake of completeness, let me end by mentioning that the five reasons highlighted in the Rotary Leader January newsletter to visit the new website are (1) Better organization; (2) Improved readability; (3) Compelling storytelling (what I just focused on); (4) Prominent calls to action; and (5) Enhanced metrics.

 

Growing the Membership and Serving the Community: Example of a Strategic Plan for a Rotary Club

On July 1, at the start of each new Rotary year, new club Presidents elected by the membership of more than 34,000 Rotary clubs worldwide take on the responsibility to lead their club for a year.  New elected leaders are also in place, again for a year, at the level of Rotary Districts and even Rotary International.

Rotary has long called on clubs and districts to adopt strategic plans. This is good practice for any organization, but especially so for an organization with new leaders every year. It is not clear exactly how many clubs adopt such plans, given that many clubs are small and may not feel the need to put a strategy plan down on paper. Yet strategic plans can be helpful, especially when clubs or districts try new innovative approaches to strengthening their membership and achieving a larger impact on their community.

Starting this year, my club – the Rotary Club of Capitol Hill, has adopted a number of important and hopefully innovative changes in the way it will function. The changes range from how many times the club will meet each month to the type of service work it will engage in, and how it will aim to strengthen its membership.

As this may be useful for other clubs, I thought I should share on this blog a strategic note that describes these changes and what the club hopes to achieve in the coming year.  Maybe the note can help other clubs think about their own options.

Please do not hesitate to share feedback on the strategic note of my club available here. You can do so by commenting/leaving a reply to this blog post. Over the year I will report occasionally through the blog on the progress (or lack thereof!) made towards our objectives for the 2016-17 Rotary year.

 

Increasing the Impact of Rotary (Partnerships Series No. 9)

This post is the last in a series of nine posts on partnerships, innovation, and evaluation in Rotary. The rationale for the series was my conviction that if Rotary is to have a larger impact globally, it must rely more than has been the case so far on partnerships, innovation, and evaluation (and in some areas advocacy, as has been the case with polio). Seven different projects or investments that have relied on partnerships, were innovative, and were evaluated at least in some way, were showcased. A compilation of the case studies together with a brief introduction is available here. Separate briefs are also available for each of the projects here.

TRF_Centennial_logo_lockup

As I mentioned it in the introduction to the series, partnerships help to implement larger projects and benefit from the expertise of organizations that are among the best in their field. Rotary’s Foundation was created almost 100 years ago (the Centennial is next year) and it has about $1 billion in assets. This is respectable, but in the world of development projects, which is in practice where Rotary is investing most of its funds, this remains small. Without innovation, the contribution of Rotary is an important drop, but still a drop in the development assistance bucket.

By contrast, if Rotary clubs and district innovate, successful pilots can then be scaled up by other organizations with deeper pockets, thereby potentially achieving much larger impact. However, for innovative projects to be recognized as such, proper evaluations are needed. We must be able to demonstrate the impact of pilot projects. Innovation and evaluation are like twins: they work best in pairs. Together, partnerships, innovation, and evaluation are the key to larger impact.

To encourage clubs and districts to think bigger and more strategically, stories of great projects were shared: an innovative financing mechanism for polio eradication; an award winning project fighting malaria and Ebola in Mali; a teacher training program that is transforming teaching and learning in Nepali classrooms; a project on obstetric fistula saving the lives of mothers and children in Nigeria; a program to invest in the writing skills of disadvantaged youth in the United States; a project to improve access to water and sanitation in Uganda; and a global network of Peace Centers and Peace Fellows to help promote peace.

Some of these programs and projects are large. Others are small. Most were implemented through global grants, but one was implemented through a district grant. All these projects have been in one way or another innovative. They have all leveraged partnerships not only to crowd in financial resources, but also – and even more importantly – to build on great expertise. And they have all relied on monitoring and evaluation mechanisms to assess their impact, at least partially.

Putting together great projects requires work. Fundraising is often time consuming in Rotary given the funding model of the Rotary Foundation that requires raising funds from many clubs and districts first before getting a match from the Foundation. Planning, implementing, and in addition evaluating projects also takes time, especially when one tries to do this in a professional way. Finally, in order to be innovative, Rotarians leading projects need to be aware of where the frontier is in their field, and what could be innovative. This also takes some time.

There is nothing wrong with clubs and districts funding and implementing traditional Rotary projects. Most projects will continue to be fairly simple, with funds provided to worthy charitable causes. These projects, as well as the volunteer time often contributed by Rotarians when implementing them, serve an important purpose. The beneficiaries of these projects are better off thanks to them. These projects help communities, and they also benefit Rotary through the goodwill that the projects create.

But if we want to raise the bar and achieve larger impact, we also need to do more innovative projects. Rotary needs to be bolder, more ambitious. It needs to better learn from its projects, both the great and not so great ones, and make sure that lessons learned are shared broadly, well beyond the Rotary family. The launch of the Future Vision model, despite some challenges, was a step in the right direction. As we celebrate the Centennial of the Rotary Foundation next year, let’s make sure that we have the right vision for what Rotary and its Foundation could accomplish in the next 100 years.

Improving Teaching and Learning in Nepal (Partnerships Series No. 8)

Many developing countries have made substantial progress towards improving education attainment (the level of education attained by students) over the last two decades. At the same time, the instruction provided by teachers to students often remains of limited quality. This results in less than stellar education achievement (how much students actually learn). While students may do well enough on portions of examinations that rely for the most part on memorization, they tend to do less well when asked to think creatively or solve complex problems. This post, which is part of a series on partnerships, innovation, and evaluation in Rotary, tells the story of an innovative teacher training program in Nepal that has the potential of improving student learning substantially.

NTTI

Importance of Teacher Training

Outstanding in-service teacher training programs can make a major difference in how teachers teach, how much students learn, and how much they learn, especially among disadvantaged groups. Many factors influence student achievement, including factors that are beyond the control of schools such as a student’s socio-economic context. But teachers are the most important factor under the control of education systems to improve learning. Teachers also account for the bulk of public spending for education in developed and developing countries alike. For these reasons, there is increasing interest in finding ways to attract, retain, develop, and motivate great teachers.

The tasks of attracting, retaining, and motivating teachers fall squarely within the scope of the mission of Ministries of Education. Developing teachers is also a key responsibility and priority for the Ministries, but in this area there is also scope for nonprofits and organizations such as Rotary to play a role by helping to create great in-service training programs for teachers. The importance of in-service training and professional development to improve instruction is recognized by practitioners and policy makers. Three lessons emerge from the literature.

First, opportunities for teacher training and professional development should be made available. But not all programs achieve the same results. When in-service programs focus on changing pedagogy, the evidence suggests that they can improve teaching and as a result student achievement. By contrast, programs that merely provide additional teaching materials for teachers do not generate substantial gains.

Second, the contents of training programs aiming to change pedagogy matter as well. In-service training programs that expose teachers to best practices in instruction and actually show teachers how to implement these practices are more likely to generate positive change. Promoting collaboration between teachers, among others through teacher networks where teachers can exchange ideas is useful. Mentoring programs whereby junior teachers benefit from the guidance of experience teachers also tend to be effective. Other approaches tend to be less successful.

Third, it is important that in-service training and professional development programs target in priority the teachers who need help the most. Teachers who are struggling may benefit more than already great teachers from various programs. Similarly, students from disadvantaged backgrounds or living in poor areas also tend to benefit more from a higher quality instruction than better off students who have more help from their family at home. Identifying priority pockets of needs is most beneficial when implementing and teacher training programs.

Innovative Program in Nepal

Traditional instruction in Nepal relies on lecturing by teachers and memorization by students. Together with the Nepali NGO PHASE, NTTI (Nepal Teacher Training Innovations) has implemented innovative teacher training programs in Nepal for several years.  NTTI aims to train public schools teachers to make the classroom more interactive by coaching them on how to lead classroom discussions, facilitate group work, and ask questions to students to encourage individual thought. Instead of relying on punishment and at times shaming in the classroom to control student behavior, teachers are trained to use dynamic inquiry-based instruction methods and provide positive encouragement to motivate the students to learn. As the classroom becomes more participatory, students engage in their own learning.

The PHASE-NTTI model does not rely on one-off training. Instead it involves a cumulative cycle of trainings and intensive follow-up support to individual teachers. The aim is to help teachers move from an awareness of effective teaching practices to actual implementation of the practices in their own classrooms. The training model includes a series of teacher development courses: Introduction to Best Teaching Practices; Girls’ Sensitivity Training; and a Training of Trainers for those selected as Mentor Teachers.

The model includes pre- and post-training classroom observations, individual feedback received by teachers from Master Trainers, and follow up individual teacher support by Mentor Overall, the program is implemented over a two-year period in each school.

While no impact evaluation is yet available to measure the impact of the program, quantitative data obtained through pre- and post-training classroom observation are encouraging. In contrast to teacher-driven and student-silent classrooms, classrooms with trained teachers seem to be closer to functioning as hubs of learning.

Instead of only lecturing trained teachers lead classroom discussions, facilitate group work, and ask questions to encourage individual thought. Students learn how to make their own novel connections and think critically about what they hear and read. Qualitative data suggest that the program is appreciated by teachers and students.

Remaining Challenges and Conclusion

There have been challenges to which the program has had to adapt. The program did not work as well in secondary schools, so it now focuses on primary schools. Support from principals for teachers changing their pedagogical approach is needed, but not guaranteed. Distances to schools in rural areas make it hard to maintain regular contact after initial trainings. Lack of time for teachers to prepare lessons as advocated by the program is also a constraint. The structure of classroom time may limit creativity and inquiry-based teaching. The persistence of traditions harmful to girls in parts of the country is a major challenge to keep girls in school.

The PHASE-NTTI program does not have all the answers to these challenges, but it does have the key features that tend to be associated with successful in-service training programs. The program is also a great example of partnership (with the Ministry of Education and public schools), innovation (in teacher training), and evaluation (at least through monitoring of teacher pedagogy). A Rotary global grant proposal has been submitted to help develop the PHASE-NTTI program further and implement it in additional areas.

Technology in Nepal’s classrooms: Using impact evaluation as a learning device

Students use laptops and digital resources provided by OLE Nepal
Students use laptops and digital resources provided by OLE Nepal

Impact evaluations are becoming essential in the way we think about development and service projects. Pilot programs suggesting statistically significant impacts are hailed as breakthroughs and as candidates for scaling up. Programs without such clear impact tend to be looked down upon and may be terminated. This may not be warranted. A primary function of impact evaluations should be to improve existing programs, especially in fields where evidence of positive impacts remains scarce. The experience of OLE Nepal, which is part of the OLE network and aims to improve learning and teaching through technology, is instructive in this regard.

Last week, Rabi Karmacharya shared his experience at OLE Nepal at a seminar co-sponsored by the World Bank, the Global Partnership for Education and the Rotary Club of Capitol Hill, a supporter of Rotary Club of Kathmandu Mid Town in expanding OLE Nepal’s programs. After a successful career in engineering and technology in California, he launched OLE Nepal in 2007. Now a social entrepreneur, Rabi wants to use technology to transform the way children learn through engagement, exploration and experimentation.

OLE Nepal has achieved quite a bit over the last eight years: deploying 5,000+ laptops in 100+ schools, training 600+ teachers on integrating ICT in the classroom teaching-learning process, developing 600+ learning modules for use by teachers, and creating a digital library with 6000+ books and other items used in schools and community libraries. Its ultimate objective is to help transform and improve Nepal’s education system with technology, working closely with the Ministry of Education and other partners.

What I found most interesting about Rabi’s presentation is how OLE Nepal – as well as donors that support the NGO such as the World Food Program, responded to an evaluation of its programs in 2009-11. The evaluation used a quasi-experimental design: English and mathematics tests administered to students, with collection of additional data through student, teacher and household surveys.

The results were disappointing, with no effect on student test scores even though both teachers and students reported liking the digital contents and finding them useful. The results contradicted field observations and other evidence from teachers and students on the fact that the program was making a difference.

Several factors may have led to an apparent lack of impact on student learning, according to Rabi. First, the program had been implemented only for a relatively short period of time, and improving student learning takes time. Second, it turned out that not all teachers reported to the one-week intensive training session that was held before launching the program, which may have decreased overall impacts in the schools that benefitted from the program. Third, some teachers may not have used as the available digital resources as much as expected due to the increase in workload that this entailed. Fourth, after the initial training, the support provided to teachers was limited. Fifth, it could be that the digital content, while following the curriculum taught in school, was too difficult for students to grasp.

These and a number of factors (including questions about variations in exam difficulty between the baseline and endline tests in the evaluation, and the possibility that some students spent more time on games available through the digital libraries rather than on study for examinations) may have led to the results.

What is important is that the evaluation did not lead to the demise of the program. Witnessing firsthand the changes in classroom dynamic and student engagement brought about by the program, donors continued to fund the NGO, which has been able to grow with support from the World Food Program, the Embassies of Denmark and Finland, and most recently the Air Asia Foundation.

In response to the evaluation, OLE Nepal introduced a number improvements in its programs:

  • It now promotes a shared model between grades for a more effective use of laptops and digital resources by schools and teachers;
  • The use of resources by students is better supervised and teachers receive more extensive support after the initial one-week training. In addition, a volunteers program is available for additional support;
  • The educational contents have been revised to follow the curriculum even more closely; and
  • More emphasis will be put on enhancing the ability of students to read in early grades, since this is a prerequisite for them to be able to learn subsequently and use digital resources effectively.

OLE Nepal’s experience is a great lesson in social entrepreneurship, humility, and resilience from an innovative NGO that strives to help teachers and students harness the power of technology in the classroom.

Note: this post is reproduced with minor changes from a post by the author for the World Bank’s Education for Global Development blog, available at http://blogs.worldbank.org/education/.

Webinar with Rotarian Founder of OLE Nepal: Learning from Pilots and Scaling Up

Established in September 2007, OLE (Open Learning Exchange) Nepal has pioneered the integration of technology in classroom teaching-learning process. OLE Nepal partnered with Nepal’s Department of Education to launch laptop-based integrated learning initiative in 2008, and has gradually expanded the program to more schools over the years. It has also designed and developed digital learning materials, trained teachers to use them to enhance student learning, and set up appropriate infrastructure in schools to enable learning using technology.

On August 4, 2015 at 12:30 PM (EST), Rabi Karmacharya will present at the World Bank a seminar/webinar on Learning from Pilots and Scaling Up: Integrating Technology in Classrooms in Nepal. Rabi will discuss successes and challenges in implementing OLE’s programs from their pilot phases to scaling up.

Rabi is a social entrepreneur who helped launch OLE Nepal with the vision to use technology to improve the quality of primary education in Nepal’s public schools, and to transform the way children learn through engagement, exploration and experimentation. He has extensive experience in technological innovation and management, and a conviction that young educated Nepalis have a critical role to play in nation building.

Prior to launching OLE Nepal in 2007, he headed HimalayanTechies, one of the first successful software outsourcing companies that he co-founded in Nepal in 2001. He currently serves as the Chair of the company’s Board of Directors. Rabi holds M.Eng and B.Sc. Degrees in Electrical Engineering & Computer Science from the Massachusetts Institute of Technology (MIT). He worked at 3Com Corporation in Santa Clara, California for three years as design engineer prior to his return to Nepal. Rabi is Asia Society’s Asia 21 Young Leader (2010) and Asia 21 Fellow (2011). He is also an active Rotarian from the Rotary club of Kathmandu Mid Town.

The seminar/webinar will be held at the World Bank “G” building (701 18th Street, NW, Washington D.C.) in room J7-044, It will be accessible remotely through WebEx (please send me an email through the Contact Me page if you would like to get the web link). If you live in the Washington DC area and would like to attend this seminar in person, please let me know as well through the Contact Me page. Light Lunch will be served.

This seminar and webinar is sponsored by the Education Global Practice at the World Bank, the Global Partnership for Education, and the Rotary Club of Capitol Hill that is helping to put together a global Rotary grant to expand OLE’s digital library program.