The Education Initiative Report released by the Association of Canadian Pension Management (ACPM ) in May 2010 is a response to plan sponsor concerns regarding the level of plan members’ general financial literacy. They were specifically concerned about plan members’ awareness of the need to save adequately for retirement, along with their ability to do so.
The report was based on information gleaned at a series of six roundtable discussions across the country with both DB and CAP sponsors in 2008 and early 2009.
How to measure the effectiveness of communications methods was a hot topic, and there was considerable discussion around what “effective” or “successful” meant.
- For some plan sponsors, it simply meant getting positive feedback from plan members that the information they received was useful.
- Others were more intent on seeing behaviour change: increased plan participation, fewer members in the default fund, more members increasing their own contributions over time.
Whatever definition is adopted, those who did measure effectiveness used both direct (in terms of asking plan members for feedback) and indirect methods.
Direct Methods of Measuring Effectiveness
- Members are surveyed to ask whether they are satisfied with the level of information about the plan and whether they understand its structure.
- Members are asked to complete an evaluation at the end of information sessions.
- Members are asked about their preferred means of communication. The predominant answer becomes the method used most often to communicate with employees.
- Telephone surveys may be conducted to cut down on the use of paper, and as a means of reaching those who are not technologically inclined.
- All-member survey may be conducted with open-ended questions, rather than multiple choice so as to gather more precise feedback.
- Focus groups are held with representative groups of plan members
- Questions about the retirement program may be included in an annual or biannual employee engagement survey
- Surveys may also be held to specifically ask about service received by and communication to both active members and retirees. The data gathered forms a quality-of-service index.
- Members’ recall of key messages may be tested following an information session, or they may be asked about their perceived value of the retirement plan. Feedback cards may be delivered with the annual statement. Plan members may be provided with incentives to complete and submit them, if anonymity is not an issue.
Indirect Methods of Measuring Effectiveness
- Analyze what funds the members are in and see if they’re taking an active role in their investments.
- Compare enrollment pre- and post-information seminar.
- Compile statistics of online communication: who opened email, who left it open long enough to read it, who deleted it without reading it, profiles of recipients, etc.
- Monitor participation in plan.
- Analyze members’ decisions and actions; compare them to those from other years and draw conclusions
- Track the number of hits on the Web site to see if they increase
- Measure attendance at meetings to see if it increases.
Plan sponsors will also be particularly interested in the following sections of the report:
- Page 11: Successful communications efforts
- Page 13: Key challenges
- Pages 15-27: Case studies of Caesars Windsor; The Economical Insurance Group; Enbridge ; McMaster University; GlaxoSmithKline; and the Ontario Teachers’ Pension Plan.