Buying Down Polio (Partnerships Series No. 2)

By partnering with the World Bank in an innovative way, Rotary has successfully leveraged  its funding for polio eradication, contributing to success towards one year without polio in Nigeria and in Africa. This post, the second in a series on partnerships, innovation, and evaluation, explains how the innovative polio buy-down mechanism has worked.

PIC 3. FROM LEFT: PRESIDENT MUHAMMADU BUHARI VACCINATING HIS THREE MONTHS OLD GRAND-DAUGHTER, ZULEIHA BELLO ABUBAKAR WITH ORAL POLIO VACCINE TO MARK ONE YEAR OF FREE POLIO CASE IN NIGERIA AT THE PRESIDENTIAL VILLA ABUJA ON SATURDAY (25/7/15). WITH HIM ARE: EXECUTIVE DIRECTOR, NATIONAL PRIMARY HEALTH CARE DEVELOPMENT AGENCY (NPHCDA) DR ADO MUHAMMAD AND THE INCIDENT MANAGER, POLIO EMERGENCY OPERATION ABUJA CENTRE, DR ANDREW ETSANO 028/JULY2015/ICE/STATE-HOUSE
Nigeria’s President vaccinates his granddaughter – Photo courtesy of Dr. Etsano.

Last month, Africa achieved a key milestone towards polio eradication, with no case of polio observed for a full year. It will still take a few weeks for the World Health Organization to officially certify this milestone, and for the region to be declared polio-free, no polio cases should be observed for a period of three years. Still, tremendous progress towards polio eradication has been accomplished. Just a few years ago, hundreds of cases of polio were observed annually in Nigeria. The country achieved its first full year without polio on July 24, 2015. This will leave only Afghanistan and Pakistan on the list of polio-endemic countries.

As noted in a recent post on the World Bank health blog, achieving one year without polio in Nigeria required persistence and courage. In some areas, professionals and volunteers who led the polio campaigns risked their life: Boko Haram assassinated nine polio vaccinators two years ago in the north of the country. Vaccinators had to rely on “hit and run” tactics to reduce exposure to risk, vaccinating children quickly in the morning and leaving the area by the afternoon. (For an understanding of the role of a wide range of people at the heart of polio eradication (in the case of Afghanistan), see the great slide show provided by the Global Polio Eradication Initiative.)

The polio campaigns also required great effort and creativity from multiple agencies, including through an innovative buy-down mechanism implemented by the World Bank and funded by the Bill and Melinda Gates Foundation, as well as Rotary International and the U.S. Centers for Disease Control via the U.N. Foundation. (The Gates Foundation and Rotary International are the two largest donors worldwide towards polio eradication over the last 30 years.) Partnership with the government of Nigeria, the World Health Organization (WHO), and UNICEF, among others, was also crucial to the success of the campaigns.

How did the polio buy-down mechanism work? The basic idea was for the World Bank to fund polio eradication projects through concessional IDA (International Development Association) loans. In the case of Nigeria, two projects worth $285 million, including additional financing, were implemented over the last dozen years. The projects included clauses that allowed loans to Nigeria to become grants if the country achieved a high level of polio immunization coverage. In other words, if the immunization targets indicated in the loans were achieved and verified independently through in-depth audits, the government would receive grant funding for polio eradication without the need to repay the loans.

For the government of Nigeria, this was potentially a great deal. And for the Gates Foundation and the Rotary Foundation of Rotary International, this was also a pretty good investment. In general, investments towards polio eradication have been shown to be fairly cost-effective. But with the buy-down mechanism, these investments were especially cost-effective.

Due to the concessional nature of IDA loans (long-term zero or low-interest loans which grace repayment periods), for every dollar contributed to the buy-down, the actual amount of resources that could be transferred to the government for the polio campaigns was two times larger. The buy-down funds were transferred by the Gates Foundation and Rotary International (in the case of Rotary in partnership with the United Nations Foundation) to the World Bank at the start of the project, and used to repay the loan at the end of the project if the target immunization rates had been achieved.

Through this buy-down mechanism, the Gates Foundation and Rotary International were able to offset all future loan repayment obligations with a much smaller amount of funding to pay back IDA than the face value of the loans granted to Nigeria. Again, one dollar invested by these private donors generated about $2 for polio eradication in Nigeria, with a similar mechanism in place for Pakistan. The mechanism also had built-in incentives to encourage strong implementation performance by the government of Nigeria since the loans would be transformed into grants only if the specific immunization targets were to be achieved.

At the time of the first buy-down mechanism for polio, then-World Bank President James. D. Wolfensohn stated, “The partnership to buy-down loans to grants on the basis of good performance is an example of the innovative thinking occurring in the private sector and the World Bank about how to increase finances for the fight against global diseases. This financial innovation is bringing the goal of a polio-free world one large step closer to becoming reality.”

Could similar buy-down mechanisms be applied in other areas? That was probably the hope when this innovative mechanism was created for polio a dozen years ago. It seems however that with few exceptions the idea has not yet been replicated much in other development areas, even if it has been mentioned in a number of reports, including in a Results for Development report on education.

A number of conditions have to be met for this type of buy-down mechanism to be successful. But in the case of polio, it has been successful, enabling the Gates Foundations, individual Rotarian donors through the Rotary Foundation, the United Nations Foundation, and the World Bank to achieve higher impact towards polio eradication than would have been the case otherwise.

A brief on polio in Africa and the buy-down mechanism is available here.

This post is reproduced with minor changes from a post published by the author on September 2, 2015 on the World Bank’s Financing for Development blog at http://www.fin4dev.org/.

Partnerships, Innovation, and Evaluation, 1: Introduction

This post is the first in a series on increasing the impact of Rotary. The series will feature case studies of great service projects that have achieved larger impact through partnerships, innovation, and evaluation. The hope is that the case studies will encourage clubs and districts to think bigger in their service work.  The series will cover each of the areas of focus of the Rotary Foundation, as well as polio.

Service work through volunteering or projects is at the heart of what Rotary is all about. Membership surveys suggest that the main reason why members join and remain in Rotary is the opportunity to serve (see my recent book on Rotary). Fellowship and networking are also very important, but service is first.

Rotary is a fairly decentralized organization with at its core the Rotary club. Rotarians come in many shapes and forms, beliefs and passions. There is amazing diversity in the types of service work that Rotarians engage in. This is a strength as members choose to contribute to the causes they are most passionate about.

Most of the service work that Rotarians engage in is done through volunteering, not through service projects that benefit from financial support from the Rotary Foundation (TRF). In adition, many projects implemented with TRF support are small and based on local opportunities identified by clubs. These projects may not rely on partnerships, they may not be especially innovative, and they may not be evaluated in depth. As long as it is clear to clubs and local communities that the projects are helpful, a lack of partnership, innovation or evaluation is not necessarily a major drawback. One straitjacket does not fit all in Rotary.

At the same time however, if Rotary is to have a larger impact globally, there is also a need to put together more and larger projects that do rely on partnerships, are innovative, and are monitored and evaluated properly.

Partnerships help to implement larger projects and benefit from the expertise of organizations that are among the best in their field. Partnerships may also generate visibility and media coverage for Rotary (polio is the best example). Partnerships have a cost since effort is required for collaborations to work. But if partnerships deliver scale, expertise, or visibility, gains outweigh the costs.

Innovation is even more important than partnerships to achieve larger impact and discover better ways to serve communities. Without innovation, the contribution of TRF is a drop in the development assistance bucket. TRF does have a respectable size, but in comparison to development funding, it is very small.

Total annual giving by the foundation represents less than half a percent of what the World Bank provides in development assistance every year, and this is just one of a number of development agencies. But if Rotary experiments and innovates, pilots that prove successful can be scaled up by other organizations with deeper pockets, thereby achieving larger impact.

Without serious monitoring and evaluation, innovation does not help much because impact on the ground must first be demonstrated at the pilot stage for a promising intervention to be scaled up. Innovation and evaluation are like twins: they work best as a pair. Evaluation is also needed for Rotary to learn internally from both successes and mistakes.

All three ingredients ̶ partnerships, innovation, and evaluation, can help increase the impact of Rotary’s service work. In order to encourage clubs and districts to move in that direction, this series will show how partnerships, innovation, and evaluation can be harnessed to serve Rotary’s mission of service above self.

The series will tell the story of projects in each of the areas of focus of TRF: promoting peace, fighting disease, providing clean water, saving mothers and children, supporting education, growing local economies, and eradicating polio.

You will learn about an innovative financing mechanism for polio eradication; an award winning project reducing under five mortality in Mali; a program that is transforming teaching and learning in Nepali classrooms; a project to save the life of mothers and children in Nigeria; a program to invest in the writing skills of disadvantaged youth in the United States; projects and initiatives to improve access to water and sanitation in Uganda; and the work done by Rotary with Peace Centers.

All these projects are in one way or another innovative. They all leverage partnerships. And virtually all build on solid monitoring and evaluation mechanisms. Hopefully, the series will give you additional insights into some of the great projects that clubs and districts are implementing around the world.

Please do not hesitate to send me an email through the Contact Me page of this blog if you believe other projects should be featured (perhaps in another series), and feel free to post comments on the projects that you find particularly inspiring.

 

 

How Can this Blog Be Useful To You? Priorities for 2015-16

This blog was launched almost nine months ago on world polio day. I took a short break from the blog over the last few weeks due to work and a holiday break, but I am now back and fresh to start blogging again. With the new Rotary year starting, I thought it would be interesting to share a few thoughts about my priorities for the blog, trying to make sure that the blog is useful to you – the readers. Please don’t hesitate to let me know if you think that these are the right priorities!

Priority 1: Helping Clubs and Districts Design and Evaluate Projects

A number of other blogs on Rotary and service clubs – including Rotary-managed blogs such as Rotary Voices and Rotary Service Connections – regularly feature stories about successful service projects and initiatives. Information on projects is also available in Rotary showcase. All these are highly valuable resources, but there is also space for a different type of blog that would provide more in-depth analysis of successful projects, why they have been successful (or not), and how we know that this is the case. This last point matters: in order to be able to know whether projects have been successful or not, some form of evaluation is needed.

One of the priorities for the blog this coming year will therefore be to feature and analyze more successful projects implemented by service clubs as well as other organizations, discuss why the projects have been successful, and document how we know that they indeed have been successful. One of my convictions is that while Rotary is rightfully implementing many different types of projects, we could also progressively invest more in innovative projects that could be properly evaluated and expanded by others with deeper pockets if successful.

In addition, I also hope to make available through this blog a range of open access resources from different sources – including from my employer, the World Bank – that can help service clubs (and nonprofits more generally) think through the design and evaluation of their projects. Specifically, by the end of this new Rotary year, I hope that the blog will feature such resources in an easily accessible and organized way for most or perhaps all key areas of focus of the Rotary Foundation (promoting peace, fighting disease, providing clean water, saving mothers and children, supporting education, growing local economies, and eradicating polio).

Priority 2: Making the Contribution of Service Clubs Better Known

Rotary and other service club organizations are not always as good as they should be at explaining clearly what they do, and measuring their contribution to local communities and society. Consider just one example. We know how much the Rotary Foundation of Rotary International is contributing to projects around the world, but we do not have good estimates of how much clubs are contributing through their own small foundations and projects that do not benefit from Rotary Foundation funding. I have a few ideas about how this could be estimated, and will try them out. Also important is the value of the time and expertise that Rotarians are contributing to many different types of projects. These are all areas that I plan to investigate this year, with the hope that some of the results will be of use to clubs, districts, and perhaps even Rotary International.

Priority 3: Discussing Constraints and Opportunities for Growth

A year ago I published a book on membership in service clubs based on Rotary’s experience. The data collected for the book, as well as other data, can shed light on some of the constraints faced by clubs as well as opportunities for growth. Similar assessments could also be done for what is referred to in Rotary as “New Generations” (Interact and Rotaract clubs). This is another area where I hope to be able to invest a bit of time and share results as well as examples of good practice through the blog.

While the blog will continue to touch on other topics and will also welcome guest bloggers, these three areas are my tentative priorities for this coming year. Don’t hesitate to let me know what you think by commenting on this post or contacting me privately (if you prefer) through the Contact me page.

Trying a Different Type of District Conference: Does It Work?

by Quentin Wodon

For the past four years, I have conducted evaluations of our district 7620 conferences using surveys administered through the web. This year our conference was different. It was shorter than previous conferences and cheaper to attend. It included on the first day several opportunities to participate in community service projects with local NGOs. It had substantially higher attendance (425 registrations) than previous conferences. It focused largely on fun and fellowship, with only a few sessions on Rotary matters. And it involved multiple locations with transportation provided from one location to the other. Because the conference was located in an area with several Rotary clubs nearby, many participants were also able to attend without having to book a hotel night.

Did the new format of the conference work? A total of 155 participants responded to the evaluation survey, which makes the results reliable. Overall, the conference was clearly a success. As shown in Figure 1, almost half of participants rated the conference as better than previous conferences. This is slightly below the result for last year at 60 percent, but still impressive given that for the previous two years (2012 and 2013) most respondents rated the conferences on par with previous conferences. We are getting better at organizing these events!

One Pager District Conference 2015_Page_1

Figure 2 provides data on satisfaction rates with the facilities and various aspects of the conference. The number of respondents for each question and ratings are provided. The ratings look good with most respondents rating most aspects of the conference as very good or good. Fewer responses are provided for hotel rooms because as mentioned many participants did not need to book a room, which is a good thing to keep costs down. The organization of the conference and the opportunities for fellowship were well rated. The categories on learning about Rotary and meeting with the district leadership were less well rated, probably in part because few sessions at the conference focused on Rotary business and training, but even in past conferences, these ratings have not been high either. Importantly, the cost of the conference was much better rated than in previous years – the conference was affordable!

One Pager District Conference 2015_Page_2

Some 25 different sessions were individually rated with at least nine respondents per session (this is a minimum number of respondents to ensure some reliability in the assessment). Six of the 25 sessions got 75 percent or more “very good” ratings: two of the service project sessions, the high school 4-way speech contest, the Interact session, the Saturday evening dinner with Dean Rohrs as speaker, and the subsequent Rock Tenor music performance. In other words, service projects, interactions with youth, and the Saturday capstone events stole the show in terms of approval ratings. Another nine sessions got between 60 percent and 75 percent “very good” ratings.

What could still be improved in future years? When asked what types of sessions they would like to see more off, sessions on successful projects and debates/discussions on Rotary and its future were mentioned the most. There were few of these sessions this year, and we should probably have more next year. In terms of speakers, participants would like more motivational and entertaining speakers. Participants would like the conference to remain short at two days. As to whether it is better to have one or more districts present at the conference, the feedback was split between the two options. All of those results were similar in previous years.

To sum up, attendance at the conference was high and most participants were highly satisfied with the event. The conference was affordable and fun to attend. At the same time, a number of areas for improvements were identified. Many of these recommendations are not new: they had already emerged from the evaluation of the past three conferences. The good news is that we seem to be getting better at organizing these events, and now at making sure that they are affordable for more Rotarians to participate.

Organizing Great District Conferences: Lessons Learned

by Quentin Wodon

April-May is a busy time for many Rotary districts as this is often the period during which districts organize their annual conference. How can districts organize great conferences combining learning and fun at an affordable cost for participants? A few months ago, I ran a series of three posts on preparing and evaluating great conferences. The posts were based on a detailed evaluation of the conferences organized by my district over the last three years. The evaluation is available here. Given that we are entering conference season in full swing, let me summarize in this post some of the key points I made in the three-part series on this topic a few months ago (the links to the series are Part 1, Part 2, Part 3).

What Feedback Did Conference Participants Give?

In my district, our evaluations suggested that participants were often fairly happy with most aspects of the conferences. But they also had suggestions. When asked what types of sessions they would like to see more off in future conferences, they suggested having more sessions on successful projects and debates/discussions on Rotary and its future. In terms of the types of speakers, participants would like more motivational and entertaining speakers, as well as more speakers from the business world versus nonprofits. Participants would also like less time spent on award ceremonies.

Participants would like the conferences to be shorter (two days). Shorter conferences would also help reduce the cost of attending the conference, which is often a complaint. This in turn may make it easier to attract more Rotarians to these events, including some of the younger Rotarians for whom cost may be a more serious issue. As to whether it is better to have one or more districts present at a conference, feedback was split between the two options – some participants prefer to have only their own districts, while others like the opportunity to meet members from other districts. Virtually all participants like opportunities for discussions with Interactors and Rotaractors, and would like more such opportunities.

While some of the feedback received in your district may be different, it seems to me that quite a bit of what we learned in my district about what was great and what could be improved in district conferences is likely to apply in many other districts as well.

Is It Difficult to Evaluate Conferences?

It is not. Evaluating district conferences in a serious way is feasible at virtually no cost, as illustrated by the work we did in our district. The surveys for the evaluation were administered through the web and by sending an email to participants a few days after the conferences took place. Using web surveys reduced the time needed to tabulate data, and ensures that there is no waste of information, say from legibility issues often encountered with printed surveys. Participation rates can be strong, so that the surveys are representative statistically. You can even monitor changes in the evaluation of conferences over time – as we did – by fielding similar surveys year after year.

Our latest survey for 2014 survey had a total of 24 questions, some with multiple sub-questions. The questionnaires were designed to take about 15’ to complete, so that substantial information can be captured without taxing too much the time of respondents. Two emails (one initial email and one reminder email) were sent to participants to ask them to fill the survey – this was enough to generate fairly good response rates.

In terms of the structure of the questionnaire, a first set of questions were asked to respondents about their profile (age, gender, Rotary status, length of membership, club affiliation, past attendance at district conferences, attendance rate at club meetings, positions of leadership in the organization, etc.). A second set of questions were asked for participants to evaluate all of the conference sessions to which they participated one by one, as well as their general appreciation of the conference along a number of characteristics and some of their preferences for future sessions. Finally, a last set of questions were open-ended to elicit qualitative feedback on the conferences. The questionnaire of the 2014 evaluation is available in the report on the evaluation.

If your district is one of many that are organizing their conference in the last quarter of the Rotary year, good luck! And if you would like help with evaluating your conference, please let me know by sending me an email through the Contact Me page of the blog.

Preparing and Evaluating Great Conferences: Part 3 – Lessons Learned

by Quentin Wodon

In the first post of this series, a simple argument was made for the importance of evaluating annual conferences – whether for Rotary districts or other organizations. Major investments are made in those conferences in terms of time and money. They are highlights of the life of their organizations, and essential to build friendships and teamwork among members. In the second post, summary results from the evaluation of the latest annual conference of Rotary district 7620 were provided to show how simple evaluations can provide valuable insights. In this last post,  more information is shared on how the evaluations for the last three conferences of the district were designed, and what some of the recommendations of participants were for future conferences.

Rotarians pack meals for the homeless at a district conference session
Rotarians pack meals for the homeless at a district conference session

Design of the evaluations

The questionnaire of the surveys implemented among conference participants were administered through the web (Survey Monkey) a few days after each conference. Using web surveys reduces the time needed to tabulate data, and ensures that there is no waste of information – for example from qualitative feedback – due to legibility issues often encountered with printed surveys.

In 2014, a total of 100 Rotarians responded to the survey, generating a response rate of about 40 percent, which is fairly good for a web survey and is likely to provide a good level of representativeness. Response rates for the two previous surveys for 2012 and 2013 were good as well. However, it may be that Rotarians who respond are those who tend to be more involved in the activities of their clubs and districts.

The conference evaluation surveys have been implemented for three years. Very similar questionnaires were fielded in the three years to maximize comparability. In 2013 and 2014 however, additional questions were added versus 2012 to better capture preferences from participants for future conferences.

The 2014 survey had a total of 24 questions, some with multiple sub-questions. The questionnaires were designed to take about 15’ to complete, so that substantial information could be captured without taking too much time for respondents. Two emails (one initial email and one reminder email) were sent to participants to ask them to fill the survey – this was enough to generate good response rates.

In terms of the structure of the questionnaire, a first set of questions were asked to respondents about their profile (age, gender, Rotary status, length of membership, club affiliation, past attendance at district conferences, attendance rate at club meetings, positions of leadership in the organization, etc.). A second set of questions asked participants to evaluate all of the conference sessions to which they participated one by one, as well as their general appreciation of the conference along a number of characteristics and some of their preferences for future sessions. Finally, a last set of questions were open-ended to elicit qualitative feedback on the conferences. The questionnaire of the 2014 evaluation is available in the report Evaluating Rotary District Conferences: Lessons from District 7620).

Suggestions from Respondents

Key results from the evaluation of the 2014 survey were already provided in the second post in this series. But it may be useful to summarize some of the feedback received for future conferences. As mentioned earlier, while the results are strictly speaking valid only for Rotary district 7620, they probably have broader relevance for other districts and service organizations.

When asked what types of sessions they would like to see more of in future conferences, participants suggested having more sessions on successful projects and debates/discussions on Rotary and its future. In terms of the types of speakers, participants would like more motivational and entertaining speakers, as well as more speakers from the business world versus nonprofits. Participants would also like less time spent on award ceremonies. Having at least one session devoted to a service project – like packing meals for people who are homeless in the picture above, is highly appreciated.

Participants would like the conferences to be shorter (at two and a half days, the 2014 conference was shorter than the 2012 and 2013 conferences, but even shorter conferences would be better). Shorter conferences would also help reduce the cost of attending the conference. This in turn may make it easier to attract more Rotarians to these events, including some of the younger Rotarians for whom cost may be a more serious issue.

As to whether it is better to have one or more districts present at a conference, the feedback was split between the two options – some participants prefer to have only their own district, while others like the opportunity to meet members from other districts. Virtually all participants like opportunities for discussions with Interactors (high school members of Interact clubs) and Rotaractors (young professionals in Rotaract clubs).

Conclusion

Evaluating district conferences in a serious way is feasible at virtually no cost, as illustrated in the case of Rotary district 7620 in this series of three posts. The results suggest that most participants are highly satisfied with the events. The hotels are often great, as is the organization. Yet areas for improvement include the need to hold the cost of the conferences down and to organize the conferences in such a way that more learning on the future of Rotary and successful service projects can take place. Many of these recommendations have been observed for three years in a row in the evaluations of the conferences implemented by district 7620. The good news is that by learning from these evaluations, the district has been able to further increase satisfaction rates with the conferences.

Next year’s district conference promises to be a bit different from the past three – with more of an emphasis on being financially friendly to new members. The goal, as in previous years, will be to have as many new members in the district attend as possible. But the conference committee is exploring – among other ideas – the possibility of relying on the hospitality of Frederick Rotarians to open their homes for an overnight stay for attendees. With about 400 Rotarians in four clubs living in the Frederick area where the conference will take place, this could be very successful.

 

Preparing and Evaluating Great Conferences: Part 2 – A Case Study

by Quentin Wodon

For the past three years, Rotary district 7620 has conducted evaluations of its annual district conferences using web surveys. As mentioned in the first post in this series, conducting such evaluations is important. Millions of hours and tens of millions of dollars are invested every year by Rotarians in attending district conferences, yet these conferences are rarely evaluated thoroughly.

This post shows how such evaluations can be useful by summarizing results for the (highly successful) 2014 conference (the report for all three conferences combined is entitled Evaluating Rotary District Conferences: Lessons from District 7620). The third post will provide lessons learned on what Rotarians would like to see in future conferences. While the analysis is specific to district 7620, it probably has broader relevance as well.

The "See Something Say Something" theater performance was the highest rated session at the conference
The “See Something Say Something” theater performance was the highest rated session at the conference

Success of the Conference

The evaluations of the previous two conferences of the district (in 2012 and 2013) suggested that while participants enjoyed these conferences, they could have been shorter and less expensive with more engaging speakers and more learning opportunities. Conferences should also be fun.

Did the district succeed in organizing a great conference in 2014? To a large extent, the answer is yes. The conference was shorter, and had lots of fun, but its cost for participants remained relatively high. The conference was better attended than in previous years. Most participants were seasoned Rotarians, but many Interactors and Rotaractors participated as well, thanks in part to an Interact Leadership Conference organized as a smaller sub-conference within the main conference.

Almost 60 percent of participants rated the conference as better than previous conferences, which is a major achievement (in the previous two years most respondents rated the conference as on par with previous conferences). A total of 100 Interactors, Rotaractors, and Rotarians responded to the web-based evaluation survey for the conference, which makes its results reliable.

Evaluation by Category and Session

Data on satisfaction rates with the facilities and various aspects of the conference were obtained and are shown in the Figure below. Most of the ratings look great with large shares of respondents rating various aspects as very good or good. The hotel rooms as well as the conference and hotel facilities and the convenience of the location ranked at the top. The organization of the conference and the opportunities for fellowship were also well rated. Even the category on learning about Rotary was well rated, but as in previous years only one in five participants said that they had learned a lot of new information that is likely to be useful to them as Rotarians. Apart from the issue with the quality of the food served by the hotel, the cost of the conference was the category with the lowest ratings. This is in fact an issue that has been identified for three years in a row.

Selected Results from the 2014 D-7620 Conference Evaluation
Selected Results from the 2014 D-7620 Conference Evaluation

The evaluation provided feedback on all conference sessions. For 26 sessions the sample size was large enough to tabulate responses (a minimum of 10 respondents per session was required to assess a session individually). The conference was focused on youth (on Friday and Saturday) and Wounded Warriors (for the Sunday brunch). Nine of the 26 sessions got 60 or more “very good” ratings.

By and large these were the sessions on youth, including the traditional Four Way Test competition for high school students, a See Something Say Something theater performance on bullying by Interactors from North Carolina, a speech by Jack Andraka – the winner of the prestigious Intel high school competition, and a speech by Teresa Scanlan – a recent Miss America and founder of an orphanage in Haiti. Also highly rated were the Interact Leadership Conference and the Mother’s Day brunch with wounded warriors on Sunday. Hospitality suites as always also fared very well.

What Worked, What Could be Improved

For those who did not attend this specific conference, all this may seem a bit abstract. But these data and results are provided to make the point that by evaluating conferences through simple web surveys, you can know exactly which sessions were great, and which ones not so great. This obviously can be useful for preparing future conferences. In addition, for the 2014 conference, it was clear that the focus on youth and wounded warriors was a hit, something to keep in mind.

The evaluation – including feedback from open-ended qualitative questions included in the web survey, also suggested areas for improvement, together with data on preferences regarding the type of speakers to invite, the length and cost of these conferences, and some of their other features. All of this will be discussed in the third and last post on this topic.

Note: Part of the analysis in this post is updated from a section in a book published by the author entitled Membership in Service Clubs: Rotary’s Experience (Palgrave Macmillan, 2014).

Preparing and Evaluating Great Conferences: Part 1 – The Need

Note: This post is the first in a series of three on preparing and evaluating conferences. Part 1 is about the need to evaluate, part 2 about a case study doing so, and part 3 on some of the lessons learned.

by Quentin Wodon

Many organizations, including firms, NGOs, professional associations, as well as service clubs, organize annual conferences. These conferences are essential not only for running the business of the organization, but also for building team work among the members of the organization. It is not clear however whether these annual conferences are evaluated properly.

Past RI President Bill Boyd at the D-7620 2014 conference
Past RI President Bill Boyd at the D-7620 2014 conference

In the case of Rotary, a global service club organization founded in 1905 which has today 1.2 million members, a global annual convention is organized typically in June of each year. But what may matter more are the conferences organized by each of the 530 or so districts in the world. These district conferences are annual events to which Rotarians from the district are invited to participate. They represent the main event where Rotarians from different clubs belonging to a common geographic area can meet each other and exchange their experiences.

It is important for an organization such as Rotary International to pay attention to district conferences not only because they are essential events in the life of clubs and districts, but also because the resources invested in the conferences are substantial, in terms of both time and money. Right now is the time when many Rotary districts are actively preparing their next annual conference, since these events are typically held in April or May before the end of the Rotary year.

This first post in a series of three on preparing and evaluating great conferences provides quick and dirty “back of the envelope” calculations of the potential investments made in district conferences every year. Because this investment is rather large, the conferences should be evaluated seriously.

Investments in Time

Consider first the allocation of time. To provide a very rough estimate of the time that may be invested every year in preparing and attending district conferences, data from the last three conferences organized by District 7620 can be used. The first conference was organized by district 7620 jointly with district 7630 in 2012. Together the two districts cover the states of Maryland and Delaware, as well as the District of Columbia (that is, the capital city Washington, DC). For the second conference, district 7610 in Northern Virginia joined in as well. The third conference was organized solely by district 7620.

The total attendance at the three conferences combined was estimated at about 1,050 Rotarians and guests. If one considers that activities at the conference go on for up to ten hours per day, and if we assume that participants on average spend 2.5 days attending the conferences (including travel time), with 1050 participants this would represent an investment in time of 26,250 hours. Together, the two districts that organized the joint conference in 2012 have about 4,000 Rotarians, and when the three districts are included for the 2013 conference, membership is above 6,000. For district 7620 in 2014, the membership was at about 2,350. Over the three years, this represents a potential attendance at the conference of close to 12,400 Rotarians. Therefore the amount of time allocated on average for the two conferences was then approximately 2.12 hours per Rotarian per year.

Multiplying this estimate by the number of Rotarians in the world yields an allocation of about 2.54 million hours per year for Rotary worldwide just for Rotarians to attend district conferences. This estimate may well be on the low side given that the attendance rate at the two conferences evaluated in this report, at about 8.5 percent on average for the three years using the approach just outlined, may be lower than is typical in other districts.

In addition, one should include the time spent by district leadership teams and conference committee members as well as district executive secretaries in preparing the conferences. It is not unrealistic to suggest that for any given district at least 30 Rotarians will volunteer time to help prepare various aspects of the conferences, including preparing their own presentations. Some of these Rotarians will spend a very large amount of time on the conferences, especially if they are part of the core organizing committee, while others will spend less time.

Just for the sake of the argument, consider an average of 25 hours spent by each of the 30 Rotarians for preparing various aspects of the conference (this is probably a rather low estimate). With some 530 Rotary districts worldwide, this would generate another 400,000 volunteer hours for preparation. Thus, possibly three million hours are allocated to district conferences every year by Rotarians, and this may well be a conservative estimate. The bottom line is that clearly a large amount of time is allocated to prepare and attend these conferences, so that making these conferences a success does matter for all those involved.

Financial Investments

Consider next the question of cost. Most of the costs of district conferences are borne by participants who pay their hotel bill as well as a fee for attending the conference to cover meals and other costs. A typical fee in the United States to attend a district conference will run at a few hundred dollars including meals and hotel rooms.

District conferences tend to be organized in major hotels or resorts, at least in the United States, so that even when special rates are obtained, room fees will typically be in the $100 to $150 per night. Some Rotarians stay for three or more nights, others for only one or two nights, and some just come for a day. Guests are likely to have additional expenses both at the site of the conference and to get to the conference site.

Assume, again for the sake of the argument, that on average the total cost of the conference per attendee is of the order $600 – it will often be  higher for those staying three nights, but it will be lower for those coming just for one day without staying any night in the hotel or resort. Even based on the somewhat low attendance rate of about 8.5 percent mentioned above for the three conferences considered here, with 1.2 million Rotarians worldwide, this would generate a cost of $61 million per year.

This might be too high because the cost of district conferences may be lower in other countries than it is in the United States. But on the other hand, this does not include special costs for the attendance of high level Rotary officials (whether those are paid for by districts or Rotary International) and other invited speakers. This estimate does not include the cost at Rotary headquarters to oversee and monitor district conferences and it also does not include the cost of the annual Rotary convention that typically welcomes more than 20,000 Rotarians from all over the world each year and could in a way be considered as a super district conference.

This cost estimate also does not factor in the opportunity cost of the time allocated by Rotarians to prepare and attend the conferences (that opportunity cost could be very high if estimated as is normal practice at the wage rate of those involved). Overall, the costs of district conferences are likely to be substantial, and could be higher than those indicated here depending on conference participation rates.

Implication

Major investments in time and money are made every year by Rotarians to prepare and attend district conferences. These conferences are the highlight of the Rotary year in each district. They should be evaluated well. The second post in this series will show how this can be done professionally at very low cost with a case study, and the third post will discuss lessons from the evaluation of the last three conferences of district 7620.

Note: Part of this blog post is adapted from a section in a book published by the author entitled Membership in Service Clubs: Rotary’s Experience (Palgrave Macmillan, 2014).