Trying a Different Type of District Conference: Does It Work?

by Quentin Wodon

For the past four years, I have conducted evaluations of our district 7620 conferences using surveys administered through the web. This year our conference was different. It was shorter than previous conferences and cheaper to attend. It included on the first day several opportunities to participate in community service projects with local NGOs. It had substantially higher attendance (425 registrations) than previous conferences. It focused largely on fun and fellowship, with only a few sessions on Rotary matters. And it involved multiple locations with transportation provided from one location to the other. Because the conference was located in an area with several Rotary clubs nearby, many participants were also able to attend without having to book a hotel night.

Did the new format of the conference work? A total of 155 participants responded to the evaluation survey, which makes the results reliable. Overall, the conference was clearly a success. As shown in Figure 1, almost half of participants rated the conference as better than previous conferences. This is slightly below the result for last year at 60 percent, but still impressive given that for the previous two years (2012 and 2013) most respondents rated the conferences on par with previous conferences. We are getting better at organizing these events!

One Pager District Conference 2015_Page_1

Figure 2 provides data on satisfaction rates with the facilities and various aspects of the conference. The number of respondents for each question and ratings are provided. The ratings look good with most respondents rating most aspects of the conference as very good or good. Fewer responses are provided for hotel rooms because as mentioned many participants did not need to book a room, which is a good thing to keep costs down. The organization of the conference and the opportunities for fellowship were well rated. The categories on learning about Rotary and meeting with the district leadership were less well rated, probably in part because few sessions at the conference focused on Rotary business and training, but even in past conferences, these ratings have not been high either. Importantly, the cost of the conference was much better rated than in previous years – the conference was affordable!

One Pager District Conference 2015_Page_2

Some 25 different sessions were individually rated with at least nine respondents per session (this is a minimum number of respondents to ensure some reliability in the assessment). Six of the 25 sessions got 75 percent or more “very good” ratings: two of the service project sessions, the high school 4-way speech contest, the Interact session, the Saturday evening dinner with Dean Rohrs as speaker, and the subsequent Rock Tenor music performance. In other words, service projects, interactions with youth, and the Saturday capstone events stole the show in terms of approval ratings. Another nine sessions got between 60 percent and 75 percent “very good” ratings.

What could still be improved in future years? When asked what types of sessions they would like to see more off, sessions on successful projects and debates/discussions on Rotary and its future were mentioned the most. There were few of these sessions this year, and we should probably have more next year. In terms of speakers, participants would like more motivational and entertaining speakers. Participants would like the conference to remain short at two days. As to whether it is better to have one or more districts present at the conference, the feedback was split between the two options. All of those results were similar in previous years.

To sum up, attendance at the conference was high and most participants were highly satisfied with the event. The conference was affordable and fun to attend. At the same time, a number of areas for improvements were identified. Many of these recommendations are not new: they had already emerged from the evaluation of the past three conferences. The good news is that we seem to be getting better at organizing these events, and now at making sure that they are affordable for more Rotarians to participate.

Organizing Great District Conferences: Lessons Learned

by Quentin Wodon

April-May is a busy time for many Rotary districts as this is often the period during which districts organize their annual conference. How can districts organize great conferences combining learning and fun at an affordable cost for participants? A few months ago, I ran a series of three posts on preparing and evaluating great conferences. The posts were based on a detailed evaluation of the conferences organized by my district over the last three years. The evaluation is available here. Given that we are entering conference season in full swing, let me summarize in this post some of the key points I made in the three-part series on this topic a few months ago (the links to the series are Part 1, Part 2, Part 3).

What Feedback Did Conference Participants Give?

In my district, our evaluations suggested that participants were often fairly happy with most aspects of the conferences. But they also had suggestions. When asked what types of sessions they would like to see more off in future conferences, they suggested having more sessions on successful projects and debates/discussions on Rotary and its future. In terms of the types of speakers, participants would like more motivational and entertaining speakers, as well as more speakers from the business world versus nonprofits. Participants would also like less time spent on award ceremonies.

Participants would like the conferences to be shorter (two days). Shorter conferences would also help reduce the cost of attending the conference, which is often a complaint. This in turn may make it easier to attract more Rotarians to these events, including some of the younger Rotarians for whom cost may be a more serious issue. As to whether it is better to have one or more districts present at a conference, feedback was split between the two options – some participants prefer to have only their own districts, while others like the opportunity to meet members from other districts. Virtually all participants like opportunities for discussions with Interactors and Rotaractors, and would like more such opportunities.

While some of the feedback received in your district may be different, it seems to me that quite a bit of what we learned in my district about what was great and what could be improved in district conferences is likely to apply in many other districts as well.

Is It Difficult to Evaluate Conferences?

It is not. Evaluating district conferences in a serious way is feasible at virtually no cost, as illustrated by the work we did in our district. The surveys for the evaluation were administered through the web and by sending an email to participants a few days after the conferences took place. Using web surveys reduced the time needed to tabulate data, and ensures that there is no waste of information, say from legibility issues often encountered with printed surveys. Participation rates can be strong, so that the surveys are representative statistically. You can even monitor changes in the evaluation of conferences over time – as we did – by fielding similar surveys year after year.

Our latest survey for 2014 survey had a total of 24 questions, some with multiple sub-questions. The questionnaires were designed to take about 15’ to complete, so that substantial information can be captured without taxing too much the time of respondents. Two emails (one initial email and one reminder email) were sent to participants to ask them to fill the survey – this was enough to generate fairly good response rates.

In terms of the structure of the questionnaire, a first set of questions were asked to respondents about their profile (age, gender, Rotary status, length of membership, club affiliation, past attendance at district conferences, attendance rate at club meetings, positions of leadership in the organization, etc.). A second set of questions were asked for participants to evaluate all of the conference sessions to which they participated one by one, as well as their general appreciation of the conference along a number of characteristics and some of their preferences for future sessions. Finally, a last set of questions were open-ended to elicit qualitative feedback on the conferences. The questionnaire of the 2014 evaluation is available in the report on the evaluation.

If your district is one of many that are organizing their conference in the last quarter of the Rotary year, good luck! And if you would like help with evaluating your conference, please let me know by sending me an email through the Contact Me page of the blog.

Preparing and Evaluating Great Conferences: Part 3 – Lessons Learned

by Quentin Wodon

In the first post of this series, a simple argument was made for the importance of evaluating annual conferences – whether for Rotary districts or other organizations. Major investments are made in those conferences in terms of time and money. They are highlights of the life of their organizations, and essential to build friendships and teamwork among members. In the second post, summary results from the evaluation of the latest annual conference of Rotary district 7620 were provided to show how simple evaluations can provide valuable insights. In this last post,  more information is shared on how the evaluations for the last three conferences of the district were designed, and what some of the recommendations of participants were for future conferences.

Rotarians pack meals for the homeless at a district conference session
Rotarians pack meals for the homeless at a district conference session

Design of the evaluations

The questionnaire of the surveys implemented among conference participants were administered through the web (Survey Monkey) a few days after each conference. Using web surveys reduces the time needed to tabulate data, and ensures that there is no waste of information – for example from qualitative feedback – due to legibility issues often encountered with printed surveys.

In 2014, a total of 100 Rotarians responded to the survey, generating a response rate of about 40 percent, which is fairly good for a web survey and is likely to provide a good level of representativeness. Response rates for the two previous surveys for 2012 and 2013 were good as well. However, it may be that Rotarians who respond are those who tend to be more involved in the activities of their clubs and districts.

The conference evaluation surveys have been implemented for three years. Very similar questionnaires were fielded in the three years to maximize comparability. In 2013 and 2014 however, additional questions were added versus 2012 to better capture preferences from participants for future conferences.

The 2014 survey had a total of 24 questions, some with multiple sub-questions. The questionnaires were designed to take about 15’ to complete, so that substantial information could be captured without taking too much time for respondents. Two emails (one initial email and one reminder email) were sent to participants to ask them to fill the survey – this was enough to generate good response rates.

In terms of the structure of the questionnaire, a first set of questions were asked to respondents about their profile (age, gender, Rotary status, length of membership, club affiliation, past attendance at district conferences, attendance rate at club meetings, positions of leadership in the organization, etc.). A second set of questions asked participants to evaluate all of the conference sessions to which they participated one by one, as well as their general appreciation of the conference along a number of characteristics and some of their preferences for future sessions. Finally, a last set of questions were open-ended to elicit qualitative feedback on the conferences. The questionnaire of the 2014 evaluation is available in the report Evaluating Rotary District Conferences: Lessons from District 7620).

Suggestions from Respondents

Key results from the evaluation of the 2014 survey were already provided in the second post in this series. But it may be useful to summarize some of the feedback received for future conferences. As mentioned earlier, while the results are strictly speaking valid only for Rotary district 7620, they probably have broader relevance for other districts and service organizations.

When asked what types of sessions they would like to see more of in future conferences, participants suggested having more sessions on successful projects and debates/discussions on Rotary and its future. In terms of the types of speakers, participants would like more motivational and entertaining speakers, as well as more speakers from the business world versus nonprofits. Participants would also like less time spent on award ceremonies. Having at least one session devoted to a service project – like packing meals for people who are homeless in the picture above, is highly appreciated.

Participants would like the conferences to be shorter (at two and a half days, the 2014 conference was shorter than the 2012 and 2013 conferences, but even shorter conferences would be better). Shorter conferences would also help reduce the cost of attending the conference. This in turn may make it easier to attract more Rotarians to these events, including some of the younger Rotarians for whom cost may be a more serious issue.

As to whether it is better to have one or more districts present at a conference, the feedback was split between the two options – some participants prefer to have only their own district, while others like the opportunity to meet members from other districts. Virtually all participants like opportunities for discussions with Interactors (high school members of Interact clubs) and Rotaractors (young professionals in Rotaract clubs).

Conclusion

Evaluating district conferences in a serious way is feasible at virtually no cost, as illustrated in the case of Rotary district 7620 in this series of three posts. The results suggest that most participants are highly satisfied with the events. The hotels are often great, as is the organization. Yet areas for improvement include the need to hold the cost of the conferences down and to organize the conferences in such a way that more learning on the future of Rotary and successful service projects can take place. Many of these recommendations have been observed for three years in a row in the evaluations of the conferences implemented by district 7620. The good news is that by learning from these evaluations, the district has been able to further increase satisfaction rates with the conferences.

Next year’s district conference promises to be a bit different from the past three – with more of an emphasis on being financially friendly to new members. The goal, as in previous years, will be to have as many new members in the district attend as possible. But the conference committee is exploring – among other ideas – the possibility of relying on the hospitality of Frederick Rotarians to open their homes for an overnight stay for attendees. With about 400 Rotarians in four clubs living in the Frederick area where the conference will take place, this could be very successful.

 

Preparing and Evaluating Great Conferences: Part 2 – A Case Study

by Quentin Wodon

For the past three years, Rotary district 7620 has conducted evaluations of its annual district conferences using web surveys. As mentioned in the first post in this series, conducting such evaluations is important. Millions of hours and tens of millions of dollars are invested every year by Rotarians in attending district conferences, yet these conferences are rarely evaluated thoroughly.

This post shows how such evaluations can be useful by summarizing results for the (highly successful) 2014 conference (the report for all three conferences combined is entitled Evaluating Rotary District Conferences: Lessons from District 7620). The third post will provide lessons learned on what Rotarians would like to see in future conferences. While the analysis is specific to district 7620, it probably has broader relevance as well.

The "See Something Say Something" theater performance was the highest rated session at the conference
The “See Something Say Something” theater performance was the highest rated session at the conference

Success of the Conference

The evaluations of the previous two conferences of the district (in 2012 and 2013) suggested that while participants enjoyed these conferences, they could have been shorter and less expensive with more engaging speakers and more learning opportunities. Conferences should also be fun.

Did the district succeed in organizing a great conference in 2014? To a large extent, the answer is yes. The conference was shorter, and had lots of fun, but its cost for participants remained relatively high. The conference was better attended than in previous years. Most participants were seasoned Rotarians, but many Interactors and Rotaractors participated as well, thanks in part to an Interact Leadership Conference organized as a smaller sub-conference within the main conference.

Almost 60 percent of participants rated the conference as better than previous conferences, which is a major achievement (in the previous two years most respondents rated the conference as on par with previous conferences). A total of 100 Interactors, Rotaractors, and Rotarians responded to the web-based evaluation survey for the conference, which makes its results reliable.

Evaluation by Category and Session

Data on satisfaction rates with the facilities and various aspects of the conference were obtained and are shown in the Figure below. Most of the ratings look great with large shares of respondents rating various aspects as very good or good. The hotel rooms as well as the conference and hotel facilities and the convenience of the location ranked at the top. The organization of the conference and the opportunities for fellowship were also well rated. Even the category on learning about Rotary was well rated, but as in previous years only one in five participants said that they had learned a lot of new information that is likely to be useful to them as Rotarians. Apart from the issue with the quality of the food served by the hotel, the cost of the conference was the category with the lowest ratings. This is in fact an issue that has been identified for three years in a row.

Selected Results from the 2014 D-7620 Conference Evaluation
Selected Results from the 2014 D-7620 Conference Evaluation

The evaluation provided feedback on all conference sessions. For 26 sessions the sample size was large enough to tabulate responses (a minimum of 10 respondents per session was required to assess a session individually). The conference was focused on youth (on Friday and Saturday) and Wounded Warriors (for the Sunday brunch). Nine of the 26 sessions got 60 or more “very good” ratings.

By and large these were the sessions on youth, including the traditional Four Way Test competition for high school students, a See Something Say Something theater performance on bullying by Interactors from North Carolina, a speech by Jack Andraka – the winner of the prestigious Intel high school competition, and a speech by Teresa Scanlan – a recent Miss America and founder of an orphanage in Haiti. Also highly rated were the Interact Leadership Conference and the Mother’s Day brunch with wounded warriors on Sunday. Hospitality suites as always also fared very well.

What Worked, What Could be Improved

For those who did not attend this specific conference, all this may seem a bit abstract. But these data and results are provided to make the point that by evaluating conferences through simple web surveys, you can know exactly which sessions were great, and which ones not so great. This obviously can be useful for preparing future conferences. In addition, for the 2014 conference, it was clear that the focus on youth and wounded warriors was a hit, something to keep in mind.

The evaluation – including feedback from open-ended qualitative questions included in the web survey, also suggested areas for improvement, together with data on preferences regarding the type of speakers to invite, the length and cost of these conferences, and some of their other features. All of this will be discussed in the third and last post on this topic.

Note: Part of the analysis in this post is updated from a section in a book published by the author entitled Membership in Service Clubs: Rotary’s Experience (Palgrave Macmillan, 2014).