by Quentin Wodon
In the first post of this series, a simple argument was made for the importance of evaluating annual conferences – whether for Rotary districts or other organizations. Major investments are made in those conferences in terms of time and money. They are highlights of the life of their organizations, and essential to build friendships and teamwork among members. In the second post, summary results from the evaluation of the latest annual conference of Rotary district 7620 were provided to show how simple evaluations can provide valuable insights. In this last post, more information is shared on how the evaluations for the last three conferences of the district were designed, and what some of the recommendations of participants were for future conferences.
Design of the evaluations
The questionnaire of the surveys implemented among conference participants were administered through the web (Survey Monkey) a few days after each conference. Using web surveys reduces the time needed to tabulate data, and ensures that there is no waste of information – for example from qualitative feedback – due to legibility issues often encountered with printed surveys.
In 2014, a total of 100 Rotarians responded to the survey, generating a response rate of about 40 percent, which is fairly good for a web survey and is likely to provide a good level of representativeness. Response rates for the two previous surveys for 2012 and 2013 were good as well. However, it may be that Rotarians who respond are those who tend to be more involved in the activities of their clubs and districts.
The conference evaluation surveys have been implemented for three years. Very similar questionnaires were fielded in the three years to maximize comparability. In 2013 and 2014 however, additional questions were added versus 2012 to better capture preferences from participants for future conferences.
The 2014 survey had a total of 24 questions, some with multiple sub-questions. The questionnaires were designed to take about 15’ to complete, so that substantial information could be captured without taking too much time for respondents. Two emails (one initial email and one reminder email) were sent to participants to ask them to fill the survey – this was enough to generate good response rates.
In terms of the structure of the questionnaire, a first set of questions were asked to respondents about their profile (age, gender, Rotary status, length of membership, club affiliation, past attendance at district conferences, attendance rate at club meetings, positions of leadership in the organization, etc.). A second set of questions asked participants to evaluate all of the conference sessions to which they participated one by one, as well as their general appreciation of the conference along a number of characteristics and some of their preferences for future sessions. Finally, a last set of questions were open-ended to elicit qualitative feedback on the conferences. The questionnaire of the 2014 evaluation is available in the report Evaluating Rotary District Conferences: Lessons from District 7620).
Suggestions from Respondents
Key results from the evaluation of the 2014 survey were already provided in the second post in this series. But it may be useful to summarize some of the feedback received for future conferences. As mentioned earlier, while the results are strictly speaking valid only for Rotary district 7620, they probably have broader relevance for other districts and service organizations.
When asked what types of sessions they would like to see more of in future conferences, participants suggested having more sessions on successful projects and debates/discussions on Rotary and its future. In terms of the types of speakers, participants would like more motivational and entertaining speakers, as well as more speakers from the business world versus nonprofits. Participants would also like less time spent on award ceremonies. Having at least one session devoted to a service project – like packing meals for people who are homeless in the picture above, is highly appreciated.
Participants would like the conferences to be shorter (at two and a half days, the 2014 conference was shorter than the 2012 and 2013 conferences, but even shorter conferences would be better). Shorter conferences would also help reduce the cost of attending the conference. This in turn may make it easier to attract more Rotarians to these events, including some of the younger Rotarians for whom cost may be a more serious issue.
As to whether it is better to have one or more districts present at a conference, the feedback was split between the two options – some participants prefer to have only their own district, while others like the opportunity to meet members from other districts. Virtually all participants like opportunities for discussions with Interactors (high school members of Interact clubs) and Rotaractors (young professionals in Rotaract clubs).
Evaluating district conferences in a serious way is feasible at virtually no cost, as illustrated in the case of Rotary district 7620 in this series of three posts. The results suggest that most participants are highly satisfied with the events. The hotels are often great, as is the organization. Yet areas for improvement include the need to hold the cost of the conferences down and to organize the conferences in such a way that more learning on the future of Rotary and successful service projects can take place. Many of these recommendations have been observed for three years in a row in the evaluations of the conferences implemented by district 7620. The good news is that by learning from these evaluations, the district has been able to further increase satisfaction rates with the conferences.
Next year’s district conference promises to be a bit different from the past three – with more of an emphasis on being financially friendly to new members. The goal, as in previous years, will be to have as many new members in the district attend as possible. But the conference committee is exploring – among other ideas – the possibility of relying on the hospitality of Frederick Rotarians to open their homes for an overnight stay for attendees. With about 400 Rotarians in four clubs living in the Frederick area where the conference will take place, this could be very successful.