Technology in Nepal’s classrooms: Using impact evaluation as a learning device

Students use laptops and digital resources provided by OLE Nepal
Students use laptops and digital resources provided by OLE Nepal

Impact evaluations are becoming essential in the way we think about development and service projects. Pilot programs suggesting statistically significant impacts are hailed as breakthroughs and as candidates for scaling up. Programs without such clear impact tend to be looked down upon and may be terminated. This may not be warranted. A primary function of impact evaluations should be to improve existing programs, especially in fields where evidence of positive impacts remains scarce. The experience of OLE Nepal, which is part of the OLE network and aims to improve learning and teaching through technology, is instructive in this regard.

Last week, Rabi Karmacharya shared his experience at OLE Nepal at a seminar co-sponsored by the World Bank, the Global Partnership for Education and the Rotary Club of Capitol Hill, a supporter of Rotary Club of Kathmandu Mid Town in expanding OLE Nepal’s programs. After a successful career in engineering and technology in California, he launched OLE Nepal in 2007. Now a social entrepreneur, Rabi wants to use technology to transform the way children learn through engagement, exploration and experimentation.

OLE Nepal has achieved quite a bit over the last eight years: deploying 5,000+ laptops in 100+ schools, training 600+ teachers on integrating ICT in the classroom teaching-learning process, developing 600+ learning modules for use by teachers, and creating a digital library with 6000+ books and other items used in schools and community libraries. Its ultimate objective is to help transform and improve Nepal’s education system with technology, working closely with the Ministry of Education and other partners.

What I found most interesting about Rabi’s presentation is how OLE Nepal – as well as donors that support the NGO such as the World Food Program, responded to an evaluation of its programs in 2009-11. The evaluation used a quasi-experimental design: English and mathematics tests administered to students, with collection of additional data through student, teacher and household surveys.

The results were disappointing, with no effect on student test scores even though both teachers and students reported liking the digital contents and finding them useful. The results contradicted field observations and other evidence from teachers and students on the fact that the program was making a difference.

Several factors may have led to an apparent lack of impact on student learning, according to Rabi. First, the program had been implemented only for a relatively short period of time, and improving student learning takes time. Second, it turned out that not all teachers reported to the one-week intensive training session that was held before launching the program, which may have decreased overall impacts in the schools that benefitted from the program. Third, some teachers may not have used as the available digital resources as much as expected due to the increase in workload that this entailed. Fourth, after the initial training, the support provided to teachers was limited. Fifth, it could be that the digital content, while following the curriculum taught in school, was too difficult for students to grasp.

These and a number of factors (including questions about variations in exam difficulty between the baseline and endline tests in the evaluation, and the possibility that some students spent more time on games available through the digital libraries rather than on study for examinations) may have led to the results.

What is important is that the evaluation did not lead to the demise of the program. Witnessing firsthand the changes in classroom dynamic and student engagement brought about by the program, donors continued to fund the NGO, which has been able to grow with support from the World Food Program, the Embassies of Denmark and Finland, and most recently the Air Asia Foundation.

In response to the evaluation, OLE Nepal introduced a number improvements in its programs:

  • It now promotes a shared model between grades for a more effective use of laptops and digital resources by schools and teachers;
  • The use of resources by students is better supervised and teachers receive more extensive support after the initial one-week training. In addition, a volunteers program is available for additional support;
  • The educational contents have been revised to follow the curriculum even more closely; and
  • More emphasis will be put on enhancing the ability of students to read in early grades, since this is a prerequisite for them to be able to learn subsequently and use digital resources effectively.

OLE Nepal’s experience is a great lesson in social entrepreneurship, humility, and resilience from an innovative NGO that strives to help teachers and students harness the power of technology in the classroom.

Note: this post is reproduced with minor changes from a post by the author for the World Bank’s Education for Global Development blog, available at http://blogs.worldbank.org/education/.

Webinar with Rotarian Founder of OLE Nepal: Learning from Pilots and Scaling Up

Established in September 2007, OLE (Open Learning Exchange) Nepal has pioneered the integration of technology in classroom teaching-learning process. OLE Nepal partnered with Nepal’s Department of Education to launch laptop-based integrated learning initiative in 2008, and has gradually expanded the program to more schools over the years. It has also designed and developed digital learning materials, trained teachers to use them to enhance student learning, and set up appropriate infrastructure in schools to enable learning using technology.

On August 4, 2015 at 12:30 PM (EST), Rabi Karmacharya will present at the World Bank a seminar/webinar on Learning from Pilots and Scaling Up: Integrating Technology in Classrooms in Nepal. Rabi will discuss successes and challenges in implementing OLE’s programs from their pilot phases to scaling up.

Rabi is a social entrepreneur who helped launch OLE Nepal with the vision to use technology to improve the quality of primary education in Nepal’s public schools, and to transform the way children learn through engagement, exploration and experimentation. He has extensive experience in technological innovation and management, and a conviction that young educated Nepalis have a critical role to play in nation building.

Prior to launching OLE Nepal in 2007, he headed HimalayanTechies, one of the first successful software outsourcing companies that he co-founded in Nepal in 2001. He currently serves as the Chair of the company’s Board of Directors. Rabi holds M.Eng and B.Sc. Degrees in Electrical Engineering & Computer Science from the Massachusetts Institute of Technology (MIT). He worked at 3Com Corporation in Santa Clara, California for three years as design engineer prior to his return to Nepal. Rabi is Asia Society’s Asia 21 Young Leader (2010) and Asia 21 Fellow (2011). He is also an active Rotarian from the Rotary club of Kathmandu Mid Town.

The seminar/webinar will be held at the World Bank “G” building (701 18th Street, NW, Washington D.C.) in room J7-044, It will be accessible remotely through WebEx (please send me an email through the Contact Me page if you would like to get the web link). If you live in the Washington DC area and would like to attend this seminar in person, please let me know as well through the Contact Me page. Light Lunch will be served.

This seminar and webinar is sponsored by the Education Global Practice at the World Bank, the Global Partnership for Education, and the Rotary Club of Capitol Hill that is helping to put together a global Rotary grant to expand OLE’s digital library program.