ICI Comment Letter to DOL Regarding New Retirement Survey Project (pdf)

April 29, 2016 Submitted electronically G. Christopher Cosby Department of Labor, Employee Benefits Security Administration 200 Constitution Avenue NW, N-5718 Washington, DC 20210 Re: Proposed Information Collection Request Submitted for Public Comment; On the Road to Retirement Surveys Dear Mr. Cosby: The Investment Company Institute1 appreciates the opportunity to provide comments on the Department of Labor’s proposed Information Collection Request (ICR)2 regarding the On the Road to Retirement Surveys. The ICR is intended to gather information on how people make planning and financial decisions before and during retirement, and how those retirement planning strategies and decisions evolve over time. As an organization with established research capabilities, ICI is in a unique position to comment on this matter.3 ICI strongly supports efforts to promote retirement security for American workers and agrees that understanding how individuals progress along the road to retirement will inform retirement savers, plan sponsors, financial services providers, and policymakers on ways to make that road easier to traverse. ICI has serious reservations, however, about the survey effort as currently designed. According to the ICR, the Department’s goal in undertaking the On the Road to Retirement Surveys is to gain insights and data that can be used to guide future retirement policy and research. In 1 The Investment Company Institute (ICI) is a leading global association of regulated funds, including mutual funds, exchange-traded funds (ETFs), closed-end funds, and unit investment trusts (UITs) in the United States, and similar funds offered to investors in jurisdictions worldwide. ICI seeks to encourage adherence to high ethical standards, promote public understanding, and otherwise advance the interests of funds, their shareholders, directors, and advisers. ICI’s U.S. fund members manage total assets of $16.9 trillion and serve more than 90 million U.S. shareholders. 2 81 Fed. Reg. 10280 (February 29, 2016), at www.gpo.gov/fdsys/pkg/FR-2016-02-29/pdf/2016-04315.pdf. 3 One of the major roles the Institute serves is as a source for statistical data on the investment company industry. With a research department comprising about 40 professionals, including seven PhD-level economists, the Institute conducts public policy research on fund industry trends, shareholder characteristics, the industry’s role in U.S. and international financial markets, and the retirement market. For example, the Institute publishes reports focusing on the overall U.S. retirement market, fees and expenses, and the behavior of defined contribution plan participants and IRA investors. In its research on mutual fund investors, IRA owners, and 401(k) plan participants, the Institute conducts periodic household surveys that connect directly with investors. G. Christopher Cosby April 29, 2016 Page 2 of 30 explaining its rationale, the Department notes that “[r]elatively little is known about how people make planning and financial decisions before and during retirement” and that a “major hurdle to retirement research is the lack of data on how people make these decisions related to retirement.”4 While we agree that more research in this area would indeed be helpful, we encourage the Department to consider the Survey of Consumer Finances (SCF)5 and the Health and Retirement Study (HRS),6 which provide helpful insight into the life cycle of retirement saving and motivation for some retirement planning activities. Academic research has provided extensive insight into households’ retirement saving utilizing these surveys, other household survey data, and proprietary recordkeeper or plan data.7 ICI also has conducted extensive research on retirement savers’ decisionmaking, whether surveying households on changes in their plans in the wake of the financial market crisis;8 or recently retired households on their defined contribution (DC) plan distribution decisions at retirement;9 or traditional IRA–owning households to understand the sources of information consulted when making an IRA rollover decision and the reasons for rolling over retirement plan accumulations to traditional IRAs.10 We applaud the Department for including participant testing as part of its efforts to develop the On the Road to Retirement Surveys, as well as allowing for public comment on the proposed ICR study materials.11 Our review of the ICR study materials, however, raised several serious concerns. In particular, as discussed in Part I of the letter, we have general concerns about certain aspects of the survey design with regard to the samples, timing, and cost of the surveys. Part II of the letter identifies specific concerns with the wording used in the questionnaire in certain areas, which we believe will hinder understanding by survey respondents and prevent the Department from being able to conduct 4 See 81 Fed. Reg. 10280 at page 10281. 5 Materials related to the SCF are available at www.federalreserve.gov/econresdata/scf/scfindex.htm. 6 Materials related to the HRS are available at hrsonline.isr.umich.edu. 7 For mention of some of this literature, see Brady, Burham, and Holden, The Success of the U.S. Retirement System (December 2012), at www.ici.org/pdf/ppr_12_success_retirement.pdf. In addition, see research published through the National Bureau of Economic Research (www.nber.org), the Pension Research Council at the Wharton School at the University of Pennsylvania (www.pensionresearchcouncil.org), the Social Security Administration’s Retirement Research Consortium (www.ssa.gov/policy/rrc/), and the Employee Benefit Research Institute at www.ebri.org. 8 See Holden, Burham, Bogdan, and Schrass, “American Views on Defined Contribution Plan Saving, 2015,” ICI Research Report (February 2016), at www.ici.org/pdf/ppr_16_dc_plan_saving.pdf. 9 See Sabelhaus, Bogdan, and Holden, “Defined Contribution Plan Distribution Choices at Retirement: A Survey of Employees Retiring Between 2002 and 2007,” ICI Research Report (Fall 2008), at www.ici.org/pdf/rpt_08_dcdd.pdf. 10 See Holden and Schrass, “The Role of IRAs in U.S. Households’ Saving for Retirement, 2015,” ICI Research Perspective (February 2016), at www.ici.org/pdf/per22-01.pdf. 11 These comments are in reference to the supporting ICR materials (PDF files labeled Appendix A through P, and “Supporting Statement for On the Road to Retirement Surveys: Investigating Retirement Planning Decision-Making and the Evolution of Individuals’ Retirement Planning Decisions over Time”) provided to ICI in an email from Lynn Johnson at DOL, EBSA, acting on behalf of Mr. Cosby, dated March 3, 2016. G. Christopher Cosby April 29, 2016 Page 3 of 30 meaningful analysis of the survey results. Finally, Part III raises concerns regarding how the Department may interpret the information gleaned from the survey effort. We hope that these comments are helpful as the Department embarks on this extensive survey effort. I. Samples, Timing, and Cost of the Surveys The ICR says that “[t]he Department is planning to undertake a long-term research study to develop a panel that will track U.S. households over several years in order to collect data and answer important research questions on how retirement planning strategies and decisions evolve over time.”12 Supporting materials13 indicate that the project will initially use an address-based sampling methodology to develop a panel of potential respondents that will be asked to complete multiple surveys over the period of at least two years. The frequency of the surveys will range from quarterly to annual. The primary mode of data collection will be online (similar to the RAND ALP14), although mixed-mode (mail and telephone) surveying also will be used. The research project includes the following components: 1. Pretesting: The pretesting will consist of cognitive interviews, pilot surveys, and respondent debriefings to the pilot surveys. The data collected during these pretests will be used solely to improve the quality of the survey instruments. 2. Pre-Survey: A national address-based mail survey will be sent to 25,000 households to recruit members for the panel. The Department expects a 40 percent response rate so the end result will be a pool of about 10,000 respondents. In addition, since the sample is restricted to households with a member who is at least 25 years old, it is estimated that the final number of respondents eligible for the panel will be about 9,500. 3. Initial Participant Survey (IPS): This survey has two parts, the IPS baseline and the IPS follow- up interview. The expected panel size is about 4,500, drawn from the anticipated pool of 9,500, with target sub-samples based on age. The Department estimates that it will take between 20– 30 minutes for the respondent to complete the baseline interview. Information collected in this survey includes retirement plan participation, financial literacy measures, use of a professional financial adviser, and ownership of IRAs. The second part, the IPS follow-up interview, will be 12 See 81 Fed. Reg. 10280 (February 29, 2016) at page 10281. 13 See note 11. 14 Materials related to the RAND American Life Panel (ALP) are available at https://alpdata.rand.org/. G. Christopher Cosby April 29, 2016 Page 4 of 30 administered two weeks after the IPS baseline interview and will take respondents about 15–25 minutes to complete. 4. Advice Interaction Survey (AIS): This survey will be conducted two months after the IPS baseline interviews and fielded to every IPS respondent. The expected sample size is about 4,250. There are three follow-up AIS interviews scheduled during the first year of the AIS interviews. 5. Annual Participant Survey (APS): This survey will be similar to the IPS survey and has two parts, the APS baseline and the APS follow-up. Data from the IPS survey will be preloaded into the APS instrument. The expected sample size is 4,075. 6. After the APS, the quarterly AIS surveys will be administered again. Survey respondents will receive substantial monetary incentives to complete the surveys. The DOL believes that by offering large incentives, the pre-survey and IPS will achieve response rates of about 40 percent and the AIS and APS will have response rates of more than 90 percent (Figure 1). Figure 1 Respondent Universe and Expected Response Rates: On the Road to Retirement Panel Data collection effort Respondent universe Expected response rate Expected number of participants Eligible participants Pre-Survey 25,000 40% 10,000 9,500 Initial Participant Survey (IPS) 9,500 47% 4,500 4,500 Advice Interaction Surveys Year 1 (AIS1) 4,500 94% 4,250 4,250 Annual Participant Survey (APS) 4,500 91% 4,075 4,075 Advice Interaction Surveys Year 2 (AIS2) 4,500 90% 4,035 4,035 Source: Table 9 in ICR supplemental material, “Supporting Statement for On the Road to Retirement Surveys” The supporting materials spell out the samples, timing, and cost of the surveys (and provide the multiple questionnaires involved). G. Christopher Cosby April 29, 2016 Page 5 of 30 A. Survey Samples Given the focus of the research on detailed decisions surrounding job change, the transition into retirement, and the management of retirement plan accumulations, the survey sampling should be designed to generate enough respondents with these experiences to allow for statistically meaningful analysis of the data. It is not clear that the contemplated sampling strategy that draws from respondents age 25 or older and only targets sub-samples by age will result in enough respondents with the specific experiences being explored. ICI survey research finds that about one-quarter of U.S. households own traditional IRAs and about half of traditional IRA–owning households have rollovers in their traditional IRAs, which means only one in eight U.S. households have such rollover experience.15 The DOL survey will be asking a broader question, about the closing of DC accounts,16 but the sample may not provide enough respondents with this experience for detailed analysis. With regard to retirees, ICI research finds that between one-fifth (using the Current Population Survey [CPS])17 and one-quarter (using the ICI Annual Mutual Fund Shareholder Tracking Survey)18 of U.S. households have a head of household that is retired and not working. With regard to the use of financial advisers, ICI survey research finds that about three in 10 U.S. households report an ongoing relationship with a professional financial adviser.19 Whereas the sampling procedure only has age targets, it is not clear that it will provide enough respondents to analyze the specific decisions regarding account closing, job change, and retirement transition, and the use of financial advice. Figure 2 reports the target sample sizes by age, with a survey design that oversamples households in their fifties and sixties. 15 See Holden and Schrass, “The Role of IRAs in U.S. Households’ Saving for Retirement, 2015,” ICI Research Perspective (February 2016). 16 As will be discussed in Part II below, it is not clear why the Department has limited this question to DC accounts. In many DB plans it is possible to take a lump-sum distribution. In addition, it is not clear why only closing of accounts is considered, when leaving an account in a former employer’s plan often is an option at job separation. 17 ICI tabulation of the March 2015 CPS data, analyzing heads of households age 25 or older regarding their employment status over the prior year. In the CPS, respondents are only asked about retirement status if they are not working. 18 The ICI Annual Mutual Fund Shareholder Tracking Survey asks respondents about retirement status and work status separately (see Figure 6 in this letter). For a description of the survey, see Burham, Bogdan, and Schrass, “Ownership of Mutual Funds, Shareholder Sentiment, and Use of the Internet, 2015,” ICI Research Perspective (November 2015), at www.ici.org/pdf/per21-05.pdf. 19 ICI tabulation from the ICI Annual Mutual Fund Shareholder Tracking Survey. See note 18 and Figure 3 in this letter. G. Christopher Cosby April 29, 2016 Page 6 of 30 Figure 2 Target Sample Sizes by Age: On the Road to Retirement Panel Age of oldest member of the household Target sample size 25–34 700 35–44 850 45–54 1,050 55–64 1,050 65 or older 850 Total 4,500 Source: Table 8 in ICR supplemental material, “Supporting Statement for On the Road to Retirement Surveys” Figure 3 takes the data for the initial DOL panel sample of 4,500 respondents by age (Figure 2) and applies incidence of IRAs, DC plans, and DB plans from the SCF (Figure 3, column 1) to estimate the number of observations with each type of retirement accumulation experience that might be included in the first interview (column 2). As illustrated by Figure 3, with the 90 percent retention rate assumed by the DOL, the sample sizes decline further (column 3) by the end of the survey period. If the retention rate were a bit lower, for example, 75 percent, the samples, particularly among the youngest respondents could get too small for detailed analysis within the groups (column 4). Similarly, considering advice use, data from ICI’s Annual Mutual Fund Shareholder Tracking Survey (column 1) applied to the DOL samples provides insight into the number of respondents that may have that experience, whether related to retirement planning or not (columns 2 and 3). This analysis considers the activity in question in isolation from, rather than in combination with, the other activities. Focusing on certain combinations of activities may reduce the sample being analyzed further. For these reasons, we urge the Department to reevaluate their sampling procedure and determine whether the samples need to be increased, or whether target quotas for retirement plan coverage, IRA ownership, advice use, job change status, or retirement status need to be considered.20 Given the life cycle of saving and investing,21 the Department may want to consider limiting the survey effort to households age 35 or older. 20 Attrition may be impacted by the surveys’ design regarding the availability of a “refused” option or the ability of a respondent to skip questions. It is not clear whether these options will be available to respondents. 21 Analysis of SCF data on households’ primary reasons for savings finds that younger households tend to save for a home purchase, education, or the family, while the focus primarily on retirement rises with age. See Figure 7.2 in Investment Company Institute, 2015 Investment Company Fact Book: A Review of the Trends and Activities in the U.S. Investment Company Industry, at www.icifactbook.org. G. Christopher Cosby April 29, 2016 Page 7 of 30 Figure 3 Expected Sample Sizes May Get Small for Some Activities 1Investment Company Institute tabulations of the Federal Reserve Board 2013 Survey of Consumer Finances. 2Investment Company Institute tabulations of the 2015 ICI Annual Mutual Fund Shareholder Tracking Survey. Note: Incidence for all in the samples may not match incidence for all reported in column 1 because of differing age compositions. Sources: Investment Company Institute tabulations of the 2013 Survey of Consumer Finances, 2015 ICI Annual Mutual Fund Shareholder Tracking Survey, and Table 8 in ICR supplemental material, “Supporting Statement for On the Road to Retirement Surveys” Incidence Percent Initial panel DOL assumption: 90% retention Alternative assumption: 75% retention (1) (2) (3) (4) Households owning IRAs1 25–34 17.6 123 111 92 35–44 25.2 214 193 161 45–54 28.1 295 266 221 55–64 38.1 400 360 300 65 or older 34.3 292 262 219 All (25 or older) 29.4 1,324 1,192 993 Households owning DC accounts1 25–34 39.5 277 249 207 35–44 46.6 396 356 297 45–54 46.2 485 437 364 55–64 41.7 438 394 328 65 or older 11.1 94 85 71 All (25 or older) 35.6 1,690 1,521 1,267 Households with DB1 25–34 14.4 101 91 76 35–44 18.8 160 144 120 45–54 26.3 276 249 207 55–64 40.0 420 378 315 65 or older 54.1 460 414 345 All (25 or older) 32.6 1,417 1,275 1,062 Households with an ongoing relationship with a professional financial adviser2 25–34 21.0 147 132 110 35–44 25.7 218 197 164 45–54 31.4 330 297 247 55–64 32.3 339 305 254 65 or older 33.9 288 259 216 All (25 or older) 29.4 1,322 1190 992 Expected sample size for each age group in DOL panel survey G. Christopher Cosby April 29, 2016 Page 8 of 30 B. Survey Timing The ICR says that the “investigation will explore a set of research questions on retirement savings, investment, and drawdown behavior by conducting a study that tracks retirees and future retirees over an extended period.”22 The supporting materials,23 however, map out research deliverables only over two years of survey time. The supporting materials indicate that the survey work is anticipated to occur between the spring of 2016 and the fall of 2018 (although its cost is annualized over six years). It is unclear whether additional surveys will be fielded to the panel, or when they will be fielded. Two years of survey work will not meet the stated goal of tracking retirees and future retirees over an extended period. C. Survey Costs The Department should clarify whether the survey effort only spans two years, or if it will continue through six years; and when and at what cost the surveys will be conducted. From the Schedule of Data Collection Efforts,24 we could not determine the extent or the cost of follow-up surveys in years three through six of the project. The supporting materials indicate: “Overall, the contract and oversight costs are estimated to total $7,453,432 over the course of the contract. The annualized cost is about $1,242,239 ($7,453,432 / 6 years = $1,242,239).”25 The supporting materials, which specified survey work occurring over two years (spring 2016 to fall 2018), do not mention if there will be additional surveys in years three through six. If such additional surveys will occur, it is not clear what their cost will be, and whether that cost has been included in the contract described. The only cost that appears to occur over the entire 6-year contract is for federal worker time, which is estimated without regard to the possibility of federal wages increasing over the next six years.26 Assumptions in the survey design raise additional concerns. It was not indicated whether the contractor may change the cost of the surveys contingent on results obtained once in the field. For example, if response rates are not as high as assumed, the cost of fielding the survey could increase. If the study is to be extended beyond two years, the longer time frame may result in difficulties with panel attrition. 22 See 81 Fed. Reg. 10280 at page 10281. 23 See note 11. 24 See supporting materials mentioned in note 11, “Supporting Statement for On the Road to Retirement Surveys,” at pages 14–15. 25 Ibid, at pages 13–14. 26 The cost of federal workers is assumed to be $51.43 per hour, 400 hours per year, over six years, which totals $123,432. This does not allow for wage increases. (Ibid.) G. Christopher Cosby April 29, 2016 Page 9 of 30 II. Survey Questionnaires The following five areas of questioning contemplated by the survey questionnaires are in need of significant improvement: (A) the specification of “advice,” and the role of other information and educational materials; (B) the order and wording of questions related to determining retirement plan status at his or her current job for the respondent and his or her spouse/partner; (C) the identification of rollovers and other dispositions of retirement accumulations and the determination of actions after “closing” accounts; (D) the categories for work status and retirement; and (E) other wording changes necessary to move the surveys toward consistency. 27 We address each of these areas in detail below, followed in Part III by an analysis of the related issue of the importance of ensuring that answers to the questions are not misinterpreted. A. Use of “Advice” Encompasses Too Many Things, Yet Misses Others Gaining insight into how households make decisions related to job change, rollovers, and retirement planning is a key goal of this survey project. Yet, the questions designed to research these events are poorly and sometimes inconsistently worded. For example, the questions include lists of possible sources of financial “advice” that the survey respondent may have used, but the sources vary from question to question. DOL should standardize the lists throughout to allow for consistent comparison. But, before doing that, DOL should make sure that the choices on the lists will provide meaningful insight into what sources and types of information, educational materials, and advice people use. The survey is fielded to the entire panel, and gathering the full range of resources consulted would provide greater insight into the retirement planning process. The following changes to the list are the most crucial to meeting the goals of the survey: 28 • Self-help information and educational materials should be separated out from personalized advice. 27 These comments are in reference to supporting ICR materials; see note 11. 28 Note that our suggestions are supported by the Department’s final rule defining who is a fiduciary as a result of giving investment advice (81 Fed. Reg. 20946 (April 8, 2016)). The Department makes clear that “general communications” are not investment advice. General communications are defined as “general communications that a reasonable person would not view as an investment recommendation, including general circulation newsletters, commentary in publicly broadcast talk shows, remarks and presentations in widely attended speeches and conferences, research or news reports prepared for general distribution, general marketing materials, general market data, including data on market performance, market indices, or trading volumes, price quotes, performance reports, or prospectuses.” Regulation section 2510.3–21(b)(2)(iii). Many of the items on the survey list would likely fall into this category, including television programs, books/newspapers/magazines, Internet sources, seminars or group meetings such as video presentations or webinars. Under Department’s rule, “investment education,” which includes certain general information (including plan information, financial, investment, and retirement information, asset allocation models, and interactive investment materials) that does not include a recommendation, is also not considered advice. Regulation section 2510.3–21(b)(2)(iv). G. Christopher Cosby April 29, 2016 Page 10 of 30 • The use of generic online calculators needs to be separated from “personalized advice via websites.” • The role of the employer should be separately identified. Using the suggestions described above, the Department should create a more complete list for these financial decisionmaking questions. We have developed a sample list (Figure 4) as an example that provides a more complete list of sources of information individuals might consult when making financial decisions. It groups responses to help respondents recall the sources they may have consulted, and then drills down into the specific materials, information, or advice received. Figure 4 Suggested Expansion of List of Sources of Information and Advice Please check off all that you consulted. _________________________________________________________________________ Communication with your household's professional financial adviser One-on-one in-person meeting One-on-one phone conversation One-on-one email exchange Personalized advice via website Information from financial services firms Browsed firm website Used retirement calculator on firm website E-chatted with representative on firm website Called phone center rep Read printed material or newsletter Attended a webinar Attended a seminar/presentation Information/educational materials from employer-sponsored retirement plan(s) Printed materials, e.g. newsletters A seminar or workshop sponsored by your employer Online materials from your retirement plan Called phone center rep G. Christopher Cosby April 29, 2016 Page 11 of 30 Other sources of information IRS rules or publications Government websites Spouse or partner Other members of your household Friends and family outside of your household Coworkers Books/Newspapers/Magazines Television programs Other, please specify: _______ __________________________________________________________________________ ICI has extensive experience with gathering such information from traditional IRA–owning households with rollovers. About half of traditional IRA–owning households have rollovers in their traditional IRAs, and ICI, in an annual survey, asks those households about their most recent rollover and the sources of information they consulted when making their most recent rollover decision. Figure 5 presents the results of the ICI’s survey questions, which first ask respondents which sources of information were consulted, followed up by what was the primary source of information. The sources of information include a financial professional, official government material, materials from financial services firms, materials from an employer, and personal resources. We also believe it is important to explore whether the source was hardcopy/printed, online, or in-person. And, given the responsibility of the employer/plan sponsor to provide certain notifications to terminating participants, ICI included in its question three information sources available from the employer: a seminar or workshop sponsored by your employer, printed materials provided by your employer, and online materials from your employer.29 Respondents also were permitted to volunteer other resources consulted. 29 For the answers related to the use of each of these components, see Figure 15 in Holden and Schrass, “The Role of IRAs in U.S. Households’ Saving for Retirement, 2015,” ICI Research Perspective (February 2016). G. Christopher Cosby April 29, 2016 Page 12 of 30 Figure 5 Sources of Information Consulted for the Rollover Decision Percentage of traditional IRA–owning households with rollovers, mid-2015 *Multiple responses are included. Note: Employer-provided information includes workshops, seminars, and printed or online materials. Other responses given included: myself, other online information, bank, books and magazines, and seminar or workshop from a financial services firm. Source: Investment Company Institute IRA Owners Survey; see Holden and Schrass, "The Role of IRAs in U.S. Households' Saving for Retirement," ICI Research Perspective (February 2016) In summary, the DOL should re-evaluate their lists of “advice” consulted. We have several recommendations. First, the question wording teeing up the list needs to be expanded to indicate that it is information and advice. Second, the lists should include employer-provided materials, given the role of the employer in the retirement plan process. Third, the lists need to keep separate those items that are information or self-help options as opposed to personalized advice whether from a professional financial adviser or personalized website that makes specific recommendations. While, through its surveys, ICI did not explore the details of the materials or advice provided by financial professionals, the DOL certainly could do so. In which case, additional activities could be included under professional financial adviser. Finally, the financial planning questions throughout the surveys should employ the same lists as the “advice” questions to the extent that the items are relevant for the various decisions.30 30 For example, compare IPS baseline question HF3 with IPS baseline question HF4 and IPS baseline question RP2 (similarly for the APS baseline). In addition, AIS question FA1 (retirement advice or information received by the respondent) inquires about a longer list of sources (including an other/specify option) than AIS question FA6 (retirement advice or information received by the spouse). 6 8 11 4 4 7 5 49 43 23 38 32 38 29 30 63 Spouse or partner Coworker, friend, or family member Employer-provided information Phone representative from financial services firms Printed materials from financial services firms Online materials from financial services firms IRS rules or publications Financial professional Source* Primary source G. Christopher Cosby April 29, 2016 Page 13 of 30 B. Retirement Plan Status at Current Job Follows a Confusing Path In section (5), “Your Current Employer-Provided Retirement Plan,” of the IPS Baseline and APS baseline, questions EPR6 through EPR16 elicit information about employer-provided retirement plans at the respondent’s current employer.31 These questions are confusing and do not allow for the possibility of more than one retirement plan at the individual’s current employer. As a result, depending on the worker’s situation, the respondent likely will be confused and the answers will not be useful. According to the March 2015 CPS, 15 percent of workers age 25 or older reported employment by federal, state, or local government.32 Government employees typically have DB plans and voluntary DC plans. In addition, in the private sector, 31 percent of active DC plan participants are at employers that also sponsor DB plans.33 An individual with more than one retirement plan at his or her current employer will have difficulty with the questionnaire. Consider a worker who has a DB plan and a DC plan at work (for simplicity, for this example, we will not even consider whether a union is involved or not, which avoids another layer of confusing questions). Our hypothetical worker (who could be a private-sector worker at a larger employer or a typical government employee) will proceed as follows through the IPS baseline interview.34 EPR6. Do you have a retirement plan at your current job? Yes, skip to EPR7, No, Don’t know Our hypothetical worker selects “yes,” and proceeds. EPR7. …Is your retirement plan of the “Defined Benefit” type or of the “Defined Contribution” type? 1. Defined Benefit – benefits paid out based on a formula Also, if the type of investment professional is important to the Department, then that information should be collected consistently. For example, AIS question FA12 asks the respondent to select from among "Broker; Investment Adviser; Bank; Personal finance website; and Other, please specify," while IPS baseline and APS baseline question RP2 asks the respondent if s/he "consult[ed] a financial planner or advisor or an accountant." 31 Additionally, workers who have multiple employers may have difficulty moving through these questions. 32 ICI tabulation of March 2015 CPS, workers reporting on their employment status in 2014. 33 ICI tabulation of DOL Form 5500 Research File data for plan year 2013. 34 Similar difficulties will occur in section (5), Your Current Employer-Provided Retirement Plan, in the APS baseline, and in section (8), Your Spouse’s/Partner’s Current Employer-Provided Retirement Plan, in the IPS baseline and APS baseline, as well. G. Christopher Cosby April 29, 2016 Page 14 of 30 2. Defined Contribution – money accumulates in a retirement account for me 3. Both types 8. Don’t know Although the question asks about a retirement plan, we will assume that our worker answers “both types” to report on both plans, and proceeds to EPR7b. EPR7b. Were you automatically enrolled in the plan(s) by your employer/union or did you have to actively choose to enroll?35 • I was automatically enrolled by my employer/union • I actively chose to enroll • Don’t know The instructions do not indicate whether the respondent may choose multiple answers here. Generally if that is not specified, the respondent is expected to choose one answer. For example, our hypothetical worker would be automatically enrolled in the DB plan, but his or her DC plan does not have automatic enrollment.36 In such circumstance, our worker may not know how to answer this question. Indicating “yes” to automatic enrollment would give the correct answer for the DB, but not tell us about the design of the DC plan, and would be in conflict with the worker’s experience in the DC plan. If constrained to one answer, our worker might respond “don’t know” because no single answer covers his or her situation. Similarly, the questions on choice of investment37 pose the same difficulty because the investment experience in the DB plan (i.e., no investment choice) is inherently different from the investment experience in the DC plan (i.e., where there is investment choice). The remaining questions about the retirement plan continue to cause difficulties for the respondent because contributions,38 loans, and withdrawals are different between the DB and DC plans. 35 In this series of questions, this is the only one that allows for the possibility of more than one plan in the wording of the question, though likely not in the answer. 36 Even if the DC plan had automatic enrollment, if our hypothetical worker actively enrolled in the plan then he or she would not indicate automatic enrollment. This question may not capture automatic enrollment as a plan design feature. 37 There are two investment choice questions: EPR7c. Do you have any choice over how the money in this retirement plan is invested? EPR7d. Have you made any choices about how the money in this retirement plan is invested? 38 For example, the next battery of questions creates similar difficulties for workers with both DB and DC plans. EPR8. Do you currently make contributions to this plan? EPR8a. Did you choose the amount of money to contribute or was the amount automatically selected for you? EPR9. What is the amount [that] you contribute currently per pay period or per year? G. Christopher Cosby April 29, 2016 Page 15 of 30 There are several approaches that could be used to improve this part of the survey. We will describe one approach for illustration. Because DB plan participants typically do not make participation or investment management decisions, DB plans can be addressed with one simple question. The first question in this section (EPR7) could ask “At your current job, do you have a defined benefit plan?” The respondent would answer “yes,” “no,” or “don’t know.”39 The next question could read “At your current employer, do you have a defined contribution plan?” The respondent would answer “yes,” “no,” or “don’t know.” The details about participation, contribution, investment, loans, and withdrawals then follow naturally from that question and would be limited to only the DC plan. If there is the possibility of multiple DC plans, a question asking about a second DC plan may be needed.40 The DOL also needs to determine its interest in the role that a union might be playing in any of the plans, and ask accordingly. At a minimum, the vendor needs to code conditional question wording so that a person without a union does not have to stumble over employer/union language. Additionally, union involvement may vary across plans and thus the questions on employer versus union actions in the retirement plan(s) should be determined separately for each plan, if the Department determines this is an important line of questioning. We recommend that these changes be incorporated into the spouse/partner questions and throughout the ensuing surveys. In addition, the Department should check the survey wording to be sure that new information, reflecting new activities or events since the prior survey, is collected.41 Respondent burden could be reduced if previous answers were preloaded and presented and the respondent could confirm or modify the responses. The APS baseline is administered one year after the IPS baseline and only a month after the most recent AIS, thus much information could be provided for confirmation. For example, the APS baseline appears to collect the full battery of retirement plan information again. In addition, changes in the respondent’s answers may be difficult to interpret because they may reflect an actual change or a correction to what the respondent had reported previously. EPR10. Does your employer/union make contributions to this plan? EPR11. What amount does your employer/union contribute currently per pay period or per year? EPR12. Does the amount of the employer/union contribution depend on the amount that you contribute? That is, does the employer/union make “matching contributions”? 39 In this simple approach, respondents would not be asked whether they make contributions to the DB plan. While participant contributions do exist in DB plans (for example, in some union plans and for federal government employees), they do not apply to the great majority of DB plans. The Department could consider whether additional detail on DB plan design should be collected and ask follow-up questions accordingly. 40 It might be instructive to explore how the SCF handles collecting such information. 41 For example, see note 45. G. Christopher Cosby April 29, 2016 Page 16 of 30 C. Wording Around Rollovers and “Closing Plans” Needs Refinement The surveys include several questions about closed-out accounts. These questions need further refinement: (1) to consider disposition of DB accumulations as well as DC accumulations, because both types of plans can allow distributions; (2) to allow consideration of other events, such as disposition of retirement plan accumulations after job change, termination, or retirement, not only closed-out accounts; (3) to not confound multiple disposition events into one list of actions; and (4) to treat the special circumstance of retirement plan changes at the same employer with a different list of options. There are several areas in the surveys where questions about closing accounts and the disposition of the money are included42 and they cover three different circumstances: (1) in the IPS baseline, question EPR17e, if the individual has “ever closed out a retirement account;” 43 (2) in the IPS follow-up, question EIR1b, if the individual has changed employment and had a DC account at their prior job; (3) in the IPS follow-up, question EIR3a, if the individual is still at the same job/employer, but no longer has the DC account; (4) in the APS baseline, question EPR17e, if the individual has “ever closed out a retirement account;” (5) in the APS follow-up, question EIR1b, if the individual has changed employment and had a DC account at their prior job; and (6) in the APS follow-up, question EIR3a, if the individual is still at the same job/employer, but no longer has the DC account. The next 42 The AIS has an abbreviated question that does not determine the disposition of the account(s), merely that account(s) recently have been closed or rolled over (again not directly considering the possibility of money coming from a DB plan accumulation). RF5. In the past three months, have you (or your spouse/partner) closed or rolled over any dedicated retirement savings accounts, such as 401(k) plans, IRAs, Keogh accounts, and the like? RF6. What kind of account(s) did you (or your spouse/partner) close or roll over? Table with checklist of possible account types closed or rolled over for R and for spouse/partner {IRA, Roth IRA, 401(k), 403(b), SIMPLE 401(k), SIMPLE IRA, SEP, SARSEP, Keogh, Payroll Deduction IRA, money purchase plan, profit sharing plan, other (specify)} 43 For example, section (6) of the IPS baseline and APS baseline includes the following questions: EPR17e. Have you ever closed out a retirement account that you obtained through a plan offered by your business, employer, or union? Yes, No, or Don’t know. EPR17f. What happened to the plan(s) you closed? Check all that apply. • Withdrew all of the money/received cash settlement/lump-sum • Rolled over to new employer’s plan • Rolled over into IRA • Converted to or purchased an annuity • Took out/withdrew some of the money • Left money in the account • Lost money that was not yet vested • Other, please specify: ____ G. Christopher Cosby April 29, 2016 Page 17 of 30 question on the IPS baseline and the APS baseline is “[w]hat happened to the plan(s) you closed?” and on the IPS follow-up and APS follow-up interviews, the next question is “[w]hat happened to the plan(s)?” The wording identifying closing of accounts/plans and the subsequent disposition of the money is confusing. None of the questions asks the respondent to contemplate the disposition of DB plan accumulations, with two questions explicitly referencing DC plans only. The surveys, therefore, do not encourage the respondent to consider DB as well as DC plans as a possible source of such funds, and by using the words “closed out a retirement account” might discourage individuals from thinking of DB accumulations. Survey respondents should be encouraged to think of all of their various retirement accumulations as possibly the source of a distribution, whether from DB or DC plans. Many traditional DB plans allow lump-sum distributions and cash balance plans typically do as well. DOL Form 5500 data for 2013 show that 25 percent of private-sector DB plans are cash balance plans, covering 33 percent of private-sector DB plan active participants and 33 percent of private-sector DB plan assets.44 The list of possible dispositions of the closed account contains a perplexing choice: “left money in the account.” Given that the premise for the question is that the individual took all of the money, i.e. closing out the relationship with the plan, it is confusing that one of the choices in the next question, which is trying to determine where the money went, allows the respondent to indicate they “left money in the account.” It also is confusing that it suggests that the individual closed a “plan,” rather than took money out of a plan or closed an account. Whereas leaving employment—whether due to job change, termination, or retirement—does not necessarily lead to the distribution of the account, the individual should have as a choice, leaving the money behind. But, the question was not based in the context of changing employers, but of closing the account. The AIS, question EIR1b, asks respondents who said they had a DC account, and then indicated a change of employment, “what happened to the plan(s)?” While this question does not mention the closing of the account and is, therefore, more open-ended, it still only focuses on the DC plan accumulation. The questions across the surveys that are based on any change in employment (whether recent or further in the past) should consistently, though separately, cover DB and DC accumulations; be based solely on an employment change and not require an account closing; and questions about disposition of accumulations should be segregated by plan type (i.e., DB and DC). It is important to gather information for different plans and different disposition events separately. Use of the word “ever” may capture multiple events and the survey will not be able to sort them out. The IPS baseline and APS baseline ask the respondent if they “ever” closed a retirement account, which is an open-ended time frame and may involve the respondent having to remember multiple distributions. This might be okay for the initial question, but makes the second question about 44 See Table A1 in U.S. Department of Labor, Employee Benefits Security Administration, Private Pension Plan Bulletin: Abstract of 2013 Form 5500 Annual Reports (2013 Data Release Version 1.0; September 2015), at www.dol.gov/ebsa/pdf/2013pensionplanbulletin.pdf. G. Christopher Cosby April 29, 2016 Page 18 of 30 the disposition of the account hard to answer and the resulting data collected difficult to interpret. Consider an individual with two job changes who may have cashed out a small account when young on their first job change, but rolled over the entire account to an IRA on their most recent job change. It will not be possible to know whether the person did this as a combined one-time action (spend some, roll over some) versus the reality of a life cycle change in activity (spending all, rolling over all).45 The Survey of Income and Program Participation (SIPP) has a battery of questions related to retirement account disposition at job change, termination, or retirement, and it might be instructive to consult their approach to the questions.46 ICI handles the question of treatment of rollovers in its Annual IRA Owners Survey by asking about the most recent rollover, finding out when it occurred, the reason for the rollover, and the percentage of the assets in the employer-sponsored retirement plan that was rolled over.47 The questions for the individual who is still at the same employer, but no longer has “a retirement plan in which money was accumulated in an account for you,” warrant a different approach. It is unclear what circumstance the DOL is trying to capture and whether the answers to the distribution questions as currently formulated will be readily interpretable. In this respect, if one is still employed, DC account access typically is limited to hardship withdrawals, in-service withdrawals, and possibly loans. In the normal course of any of these three activities, it is unlikely that the account will be closed. So, the likely scenario is the employer closed the plan.48 In this case, it is important first to document that this is what has occurred before inquiring about the ultimate disposition of the accumulation. The two steps should be identified so as to understand that the individual was forced to do something with their retirement plan accumulation, rather than decided on their own to make a move. A further difficulty here will relate to identification of the prior retirement plan given the confusing treatment of participation in more than one retirement plan (discussed above). D. Individuals May Still Work in Retirement The questions on work and retirement situation need refinement. Question EPR1 in section (4), and question EPSP1 in section (7), of the IPS baseline are intended to determine the employment 45 In addition, as a panel survey, the APS baseline will gather the same information as the IPS baseline by asking about any account closing “ever,” just with one additional year of experience (the time since the IPS baseline was administered). 46 For example, see the retirement topical module of the SIPP, wave 11 in the 2008 panel at www.census.gov/content/dam/Census/programs- surveys/sipp/questionnaires/2008/SIPP%202008%20Panel%20Wave%2011%20- %20Topical%20Module%20Questionnaire.pdf. 47 See Figures 12 and 13 in Holden and Schrass, “The Role of IRAs in U.S. Households’ Saving for Retirement, 2015,” ICI Research Perspective (February 2016). 48 Depending on how the individuals interpret the questions, a frozen DB plan may be called to mind, which also may pose difficulties in interpreting the results. G. Christopher Cosby April 29, 2016 Page 19 of 30 status of the respondent and his or her spouse.49 When studying retirement planning and saving activity, it is important to know whether the individual considers him/herself retired. An individual may be retired from his or her lifetime occupation, but still working. ICI’s latest annual survey of 6,000 U.S. households conducted in mid-2015 found that 14 percent of retired households were working in retirement—that is the head of the household indicated that he or she was retired from his or her lifetime occupation and was currently working part-time or full-time (Figure 6).50 Also, given the extensive documentation of the differences in retirement plan coverage for part- time versus full-time workers and the differing degrees of financial stress among part-time versus full- time workers,51 “working now” should at least capture whether the individual is “now working full- time,” or “now working part-time.” Additionally, it might be instructive to explore whether the individual’s work status over the past year is more relevant, in which case, the CPS questions might provide useful insight.52 49 For example, section (4) in the IPS baseline has the following question: EPR1. What is your current employment situation? 1 Working now 2 Temporarily laid off and awaiting recall 3 On sick or other leave 4 Unemployed and looking for work 5 Disabled 6 Retired 7 Homemaker 8 Other, please specify:__________ 50 For more information on the ICI Annual Mutual Fund Shareholder Tracking Survey, see Burham, Bogdan, and Schrass, “Ownership of Mutual Funds, Shareholder Sentiment, and Use of the Internet, 2015,” ICI Research Perspective (November 2015). In addition, ICI tabulation of the 2013 SCF finds that 12 percent of retired households were working. 51 See statistics and discussion in Brady and Bogdan, “Who Gets Retirement Plans and Why, 2013,” ICI Research Perspective (October 2014), at www.ici.org/pdf/per20-06.pdf. 52 The CPS allows for categorization of workers into full-time, full-year; full-time, part-year; part-time, part-year; and part- time, full-year. For example, see Brady and Bogdan, “Who Gets Retirement Plans and Why, 2013,” ICI Research Perspective (October 2014). In addition, in a similar vein, the Department may want to add questions to elicit “normal income” to see whether the household’s current work and income are unusual. The SCF contains two questions to get at this idea. Specifically, after the data on actual income are collected, respondents are asked, “Is this income unusually high or low compared to what you would expect in a ‘normal’ year, or is it normal?” If the respondent answered that their current income was unusually high or low, then this follow-up question is asked: “About what would your total income have been if it had been a normal year?” See Bricker, Dettling, Henriques, Hsu, Moore, Sabelhaus, Thompson, and Windle, “Changes in U.S. Family Finances from 2010 to 2013: Evidence from the Survey of Consumer Finances,” Federal Reserve Bulletin 100, No. 4 (September 2014), at www.federalreserve.gov/pubs/bulletin/2014/pdf/scf14.pdf. G. Christopher Cosby April 29, 2016 Page 20 of 30 Figure 6 Retired Households Also May Be Working Percentage of U.S. households age 25 or older, mid-2015 Note: Employment status is based on the head of household (sole or co-decisionmaker for saving and investing). The head of household was considered retired if he or she responded affirmatively to the question: "Are you retired from your lifetime occupation?" Source: Investment Company Institute Annual Mutual Fund Shareholder Tracking Survey E. Surveys Need Other Wording Consistency Checks The On the Road to Retirement project contemplates multiple surveys (i.e., the IPS baseline and IPS follow-up, the AIS, and the APS baseline and APS follow-up). The survey questions would benefit from a close comparative read to be sure that questions are worded consistently throughout the surveys. In addition, survey wording should be scrutinized for clarity. Below, we highlight only a few such areas of concern around consistency of inclusion or question language: Social Security, IRA contributions, and stock investments. (These issues are in addition to the difficulties with survey questions discussed in Part II, A through D, above.) Social Security. Given the important role that Social Security plays in Americans’ retirement preparedness and retirees’ income,53 the surveys should consistently inquire about the role of Social 53 For example, see Brady and Bogdan, “A Look at Private-Sector Retirement Plan Income After ERISA, 2013,” ICI Research Perspective (October 2014), at www.ici.org/pdf/per20-07.pdf; and Congressional Budget Office, CBO’s 2015 Long-Term Projections for Social Security: Additional Information (December 2015), at www.cbo.gov/sites/default/files/114th-congress- 2015-2016/reports/51047-SSUpdate-2.pdf. 49 6 2 2 25 16 U.S. households (age 25 or older) Not employed, not retired Retired & not employed Retired & employed part-time Retired & employed full-time Employed part-time, not retired Employed full-time, not retired G. Christopher Cosby April 29, 2016 Page 21 of 30 Security in the household’s finances. The IPS baseline and the APS baseline each include only one minor mention of Social Security. This mention appears in question I1,54 in a long list of items to be included in the household’s estimate of total combined income of all members of the family. By contrast, in the IPS follow-up and APS follow-up interviews, separate questions are devoted to Social Security benefits: EIR6. Did you personally receive any income in [year] from Social Security retirement benefits? EIR6a. What was the total of your Social Security retirement benefit income in [year]? EIR6b. Did you earn more than X? SPEIR6. Did your spouse/partner receive any income in [year] from Social Security retirement benefits? SPEIR6a. What was the total of his/her Social Security retirement benefit income in [year]? SPEIR6b. Did he/she earn more than X? For the purposes of consistency and the changing roles of different components in retirement planning, and because Social Security is such a key component of retirement, these two separate questions on Social Security should be included in all versions of the various surveys. In addition, the collection of Social Security benefits data in this survey will allow for checking of its role for respondents in this survey against other surveys, such as the CPS. Furthermore, Social Security is not the only important retiree income resource. Given the rising role of IRAs and DC plans, distributions from these accounts should be collected as separate components of income. In the total combined income question, “pensions” are mentioned, but DC plan and IRA distributions are not separately brought to mind.55 IRA contributions. The questions relating to IRA contributions need to be reworked. Contributions should not be characterized as “put[ting] any money into an IRA account,” in order to distinguish between contributions and rollovers (IPS baseline, questions DR1a and DR2a). The questions seeking the amount should limit the answer to the prior year (likely, tax year 2015) as limits change over time, eligibility for catch-up changes over time, and households should not have to figure 54 IPS baseline and APS baseline, question: I1. Which category represents the total combined income of all members of your family (living here) during the year [year]? This includes money from jobs, net income from business, farm or rent, pensions, dividends, interest, Social Security payments and any other money income received by members of your family who are 15 years of age or older. 55 See note 54. G. Christopher Cosby April 29, 2016 Page 22 of 30 out an average contribution amount or rate (IPS baseline, questions DR1b and DR2b). And, the amount contributed should be tied to the time period of the yes/no contribution question. Stock investments. The questions related to amounts invested in stocks (IPS baseline and APS baseline, questions DR4 and DR5)56 would benefit from more generic wording, by referring to “investment funds” rather than “mutual funds,” and adding mutual funds to the list of possibilities in the main question. In addition to substituting “investment funds” for “mutual funds,” question DR4 should be revised to make it clear that all types of investment funds, including ETFs, should be considered. This would be accomplished by revising the second sentence in question DR4 to read as follows: “Please include any type of investment fund based on stocks, such as stock mutual funds, stock index funds or stock ETFs.” An additional wrinkle that perhaps should be tested during the pre-test period, is whether respondents already are including stock investments through target date funds, lifecycle funds, balanced funds, asset allocation funds, or lifestyle funds in their responses, or can handle mentioning the share held in such funds separately as the next question. These funds have an equity component–on average about 60 percent of such funds are invested in equities—but younger participants in target date funds will have a higher allocation to equities than that average.57 While ICI has never asked survey respondents to do this calculation, ICI routinely does this calculation when analyzing recordkept data. The EBRI/ICI 401(k) database finds that young 401(k) plan participants are likely to be highly concentrated in equities, and often it is through balanced funds, particularly target date funds (Figure 7).58 To miss that component would bias downward the retirement savers’ exposure to equities. The Department should consider an additional question that asks the respondent to report the share of their retirement assets invested in target date, lifecycle, balanced, asset allocation, or lifestyle funds. 56 IPS baseline and APS baseline, questions: DR4. Is any of the money in these accounts invested in stocks, either directly in stocks or in mutual funds that hold stocks? Please include any type of investment fund based on stocks, such as Index Funds or ETFs. DR5. About what percent is invested in stocks or stock mutual funds? Range: 0..100 57 Morningstar reports the asset allocation of target date funds. See Morningstar, “Morningstar Lifetime Allocation Indexes,” (2015), at http://corporate.morningstar.com/us/documents/Indexes/AssetAllocationsSummary.pdf. For the variation in asset allocation across 401(k) plan participants by age, see Holden, VanDerhei, Alonso, and Bass, “401(k) Plan Asset Allocation, Account Balances, and Loan Activity in 2014,” ICI Research Perspective (April 2016), at www.ici.org/pdf/per22-03.pdf. 58 See Holden, VanDerhei, Alonso, and Bass, “401(k) Plan Asset Allocation, Account Balances, and Loan Activity in 2014,” ICI Research Perspective (April 2016). G. Christopher Cosby April 29, 2016 Page 23 of 30 Figure 7 Younger 401(k) Plan Participants Hold Greater Concentrations in Equities Percentage of 401(k) plan account balances by participant age, 2014 Note: The equity portion of balanced funds includes the equities held through target date funds. Components may not add to the total because of rounding. Percentages are dollar-weighted averages. Source: Tabulations from EBRI/ICI Participant-Directed Retirement Plan Data Collection Project; see Holden, VanDerhei, Alonso, and Bass, “401(k) Plan Asset Allocation, Account Balances, and Loan Activity in 2014,” ICI Research Perspective (April 2016) The foregoing represent a few of the more significant areas that merit revisiting the wording of the survey questions, but the survey instruments generally would benefit from additional scrutiny with a goal of improving consistency.59 59 In addition, other minor fixes such as changing the income questions to gather 2015/2016 data rather than 2014 data (e.g., IPS baseline, 2015 data for questions I1, I2, I3, and I4; similarly 2016 data for APS baseline, questions I1, I2, I3, and I4). The wording in IPS baseline and APS baseline, question RP3 may need to be adjusted for respondents who are retired already (e.g., did you develop a plan for saving for retirement?). In the IPS follow-up interview, section (9b) Response to Positive /Negative Shocks, it might make more sense to ask the first-order effect questions first, then follow with the financial investment questions (reorder questions to: RPNS2, RPNS1, RPNS4, RPNS3). And, the surveys should consistently offer the option of having a spouse or partner (e.g., the IPS baseline and APS baseline, question DHC11 does not mention “or living with a partner”). In the AIS, question FA14. “Thinking about your communications over the past 32 43 48 44 38 43 41 28 19 14 11 16 5 6 7 8 7 7 78 77 73 65 56 66 20s 30s 40s 50s 60s All Company stock Equity portion of balanced funds Equity funds G. Christopher Cosby April 29, 2016 Page 24 of 30 III. Expectations on Insights to Be Gleaned from the Surveys We have concerns regarding how the Department may interpret the information gleaned from the survey effort. The Department should consider what answers they expect to the questions they are posing. It also should consider how deviation from the expected answers will be interpreted and whether the interpretation provides meaningful insight. We are concerned that some of the questions may be not be helpful in achieving the goal of the surveys. This is particularly true with regard to the questions regarding financial capabilities; market expectations, labeled “mutual fund expectations” in the surveys; and investment choices and risk. Finally, there is the additional concern that respondents’ participation in the surveys may change their subsequent behaviors. These points are each described in detail below. A. Financial Capabilities One example of where the survey results may be misinterpreted relates to the questions regarding financial capabilities, section (7) of the AIS, questions FC1 through FC6.60 It may be difficult to judge some of the self-assessments because familiarity with a certain activity or product type may be low when an individual is not at a point in time that they believe they need to understand the activity or product. For example, a 25-year-old may not be well-versed in annuities, and may not need to be until he or she is closer to deciding whether to purchase an annuity. Understanding these likely variations in knowledge levels based on respondent age is crucial to an assessment of the survey results. Two of the capability questions inquire as to the respondent’s knowledge of “the tax benefits of different types of retirement plans” and his or her “understand[ing of] the different types of retirement accounts … [to] choose the one that is best for me…” DOL should be cautious about its expectations that respondents will have confidence that they can know or understand the array of rules regarding three months with this financial services provider, what kind of advice were you seeking?” also should include on the list, “determining amount to withdraw from an IRA or DC plan account.” ICI’s survey research finds that traditional IRA– owning households often consult professional financial advisers to determine their traditional IRA withdrawal amounts. See Holden and Schrass, “The Role of IRAs in U.S. Households’ Saving for Retirement, 2015,” ICI Research Perspective (February 2016). 60 Section (7) of the AIS proceeds as follows: On a 5 point scale (1=Strongly disagree, 5=Strongly agree): Please indicate the degree to which you agree with the following statements: FC1. I am confident that I have a good grasp of how risk changes when choosing a different mix of investments. FC2. I am confident in my knowledge of annuities. FC3. I am confident in my knowledge of the tax benefits of different types of retirement plans. FC4. I understand annuities well enough that I can choose a suitable annuity for myself. FC5. If I need to make an investment decision, I can select a mix of investments that are in line with how much risk I want to take on. FC6. I understand the different types of retirement accounts well enough that I can choose the one that is best for me and my financial situation. G. Christopher Cosby April 29, 2016 Page 25 of 30 participation, limits, Roth versus traditional contributions, and withdrawals, for the different retirement plans—401(k), 403(b), 457, Keogh, SIMPLE IRA, SEP IRA, SAR-SEP IRA, traditional defined benefit, cash balance—or accounts—traditional and Roth IRAs. An individual typically does not need to know or understand all of the different plans or accounts available, just the ones offered at his or her employer(s) and traditional and Roth IRAs. A person reading this question literally may be inclined to conclude that they don’t know much about all of the different plans/accounts, but they may have enough knowledge about savings options generally to effectively save for retirement. Interestingly, the IPS baseline interview contains questions very similar to those included in many U.S. and foreign surveys administered by Dr. Annamaria Lusardi at The George Washington University School of Business, Global Financial Literacy Excellence Center, to assess financial literacy,61 in section (3), Saving and Spending. We urge the Department to use the exact wording from the Lusardi survey efforts, so as to facilitate comparison of respondents in the DOL survey to the results obtained in the other large body of work. The AIS is administered to the panel on a quarterly basis. Part of the AIS collects information on activities engaged in during the prior three months, which updates actions taken along the road to retirement. But, DOL should consider whether its data would be as relevant and useful if it did not measure financial capability at a quarterly frequency. B. “Mutual Fund Expectations” The survey questions in the AIS, IPS follow-up, and APS follow-up described as probing on “mutual fund expectations,” actually are asking respondents to contemplate the expected returns on an asset class—large company stocks—not mutual funds.62 While we agree that mutual funds are an investment that will be familiar to many survey respondents, the questions in this section do not provide insight into mutual funds, but, rather, into expectations about stock market returns. Mutual funds invest in a variety of financial securities including large-cap stocks, small-cap stocks, corporate bonds, open-market paper, state and local government bonds, and U.S. Treasury bonds and bills. In addition, mutual funds also may invest in foreign securities, both private and government. In fact, there are mutual funds that invest in Treasury inflation-protected securities (TIPS) which presumably are meant to be the investment to which “mutual funds” are compared in the later questions. 61 See Lusardi, The George Washington University School of Business, Global Financial Literacy Excellence Center (GFLEC), at gflec.org; and “How Can Financial Literacy Improve Retirement Planning?” presentation at ICI 2015 Retirement Summit, April 8, 2015, Washington, DC, at www.ici.org/pdf/ppt_15_summit_lusardi.pdf. 62 See AIS, section (9) and IPS follow-up and APS follow-up, section (4), which purport to be probing on “mutual fund expectations,” (AIS questions MFE2 and MFE3; IPS follow-up and APS follow-up questions MFEC2 and MFEC3). The questions are focused on the underlying stock investment held by the “fund” rather than the specific structure of mutual funds. G. Christopher Cosby April 29, 2016 Page 26 of 30 The wording of these questions needs to be changed. AIS question MFE2, and IPS follow-up and APS follow-up question MFEC2 ask a respondent to “[s]uppose you just won a lottery prize of $10,000 in mutual fund shares invested in a broad range of stocks of well-known U.S. corporations operating in a wide range of industries.” We urge the Department to rewrite the question to delete the term “mutual fund shares.” That is, the question should read: “…$10,000 invested in a broad range of stocks of well-known U.S. corporations operating in a wide range of industries.” The reference to “mutual fund shares” is not relevant to the questions being asked and may confuse the respondent. The respondent is then asked to provide the percent chance that the investment will be worth more than $10,000 at age 65/70/75. Even corrected to avoid the mention of mutual funds, the questions (AIS, MFE2 and MFE3, and IPS follow-up and APS follow-up, MFEC2 and MFEC3) on the probability of such a stock investment being worth more in the future (in nominal and in inflation- adjusted terms) are unlikely to provide any analytically useful information. At the very least, the DOL should explain what the “correct” answers to the questions would be and how the Department would interpret responses that were not the “correct” answers before they field the survey. The answers to the questions can only be interpreted relative to historical experience. It is, of course, possible that future investment returns will not be like historical experience. However, if that were to be the case, there is no way to judge the answers to these questions. That is, there would be no way to discern between respondents who expected that future investment returns would be similar to historical returns but had a poor understanding of history, from those who understood history but thought that future investment returns would be substantially better or substantially worse than historical returns. As shown in Figure 8, based on the annual returns on large-cap stocks from 1926 to 2014,63 a 100 percent chance that investment returns would be positive, on either a nominal or inflation-adjusted basis, would be at least roughly accurate for anyone with at least a 10-year investment horizon and— depending on one’s view of Americans’ understanding of probabilities—arguably not far off for any respondent.64 Given that, the “correct” answers to these questions based on historical returns only range from 69 percent (for the probability annual returns on large-cap stocks exceed inflation) to 100 percent (for investment horizons over 14 years for nominal returns and more than 17 years for real returns), the average respondent would need to possess detailed knowledge of historical returns over various time spans and at least a passing understanding of how to calculate probabilities for the questions to elicit any analytically useful information. 63 Annual returns were taken from Morningstar, Ibbotson SBBI 2015 Classic Yearbook: Market Results for Stocks, Bonds, Bills, and Inflation 1926–2014, Chicago, IL: Morningstar (2015). 64 For a 25 year-old respondent, the future day he or she is being asked to hypothesize about is 40 years away. G. Christopher Cosby April 29, 2016 Page 27 of 30 Figure 8 Probability That Returns Are Positive by Asset Class and Holding Period Based on annual total return data from 1926 to 2014 from Ibbotson SBBI 2015 Classic Yearbook *The balanced portfolio is assumed to be 55 percent large-cap stocks; 5 percent small-cap stocks; 20 percent long-term corporate bonds; 10 percent long-term government bonds; 7.5 percent intermediate-term government bonds; and 2.5 percent U.S. Treasury bills. Source: Investment Company Institute tabulations using Ibbotson SBBI 2015 Classic Yearbook C. Investment Choices and Risk After asking respondents their predictions on future stock market values, the IPS follow-up and APS follow-up interviews continue with additional questions asking the respondent to compare different portfolios. It is not clear what information the IPS follow-up and APS follow-up surveys are hoping to elicit in the questions (MFEC4, MFEC5, and MFEC6) on the choice between $10,000 invested in large company stocks and $10,000 invested in a risk-free investment. If the questions are aimed at eliciting responses about tradeoffs that investors are likely to face in the market, then the questions are based on a misunderstanding of market risk premiums. All else equal, the level of the risk- free rate of return should not affect the decision to invest in risky assets. The risk-return tradeoff. Economists assume that individuals generally do not like to take on risk. Because of this, theory predicts that riskier assets will have a higher expected rate of return to compensate investors for taking on risk. Historical data on investment returns show that, consistent G. Christopher Cosby April 29, 2016 Page 28 of 30 with this theoretical prediction, asset classes with more volatile returns from year to year tend to have higher average returns over time (Figure 9). Figure 9 The Tradeoff Between Risk and Return Based on annual total return data from 1926 to 2014 from Ibbotson SBBI 2015 Classic Yearbook Simple average of annual returns Standard deviation of annual returns Geometric average of annual returns Nominal returns (percent) U.S. Treasury Bills 3.5% 3.1% 3.5% Intermediate-Term Government Bonds 5.4 5.6 5.2 Long-Term Government Bonds 6.1 10.0 5.7 Long-Term Corporate Bonds 6.5 8.3 6.2 Large-Cap Stocks 12.3 20.1 10.3 Small-Cap Stocks 16.7 32.1 12.2 Balanced portfolio* 10.0 12.9 9.2 Inflation-adjusted returns (percent) U.S. Treasury Bills 0.5% 4.0% 0.4% Intermediate-Term Government Bonds 2.4 7.0 2.1 Long-Term Government Bonds 3.1 11.3 2.5 Long-Term Corporate Bonds 3.5 9.9 3.1 Large-Cap Stocks 9.3 20.5 7.2 Small-Cap Stocks 13.7 32.2 9.1 Balanced portfolio* 7.0 13.7 6.1 *The balanced portfolio is assumed to be 55 percent large-cap stocks; 5 percent small-cap stocks; 20 percent long-term corporate bonds; 10 percent long-term government bonds; 7.5 percent intermediate-term government bonds; and 2.5 percent U.S. Treasury bills. Source: Investment Company Institute tabulations using Ibbotson SBBI 2015 Classic Yearbook The “risk-free” asset. Short-term U.S. Treasury bills are typically considered to be the “risk- free” asset. That is, it is assumed that, regardless of market conditions, the U.S. government will never default on the payment of interest or principal on Treasury debt. And, because they are short-term, they will not vary greatly in value when interest rates change. The yield on U.S. Treasury securities typically increases with time to maturity (that is, the interest rate on a 10-year bond is typically higher than the interest rate on a three-month note) because the value of longer-term securities is more sensitive to changes in the interest rate subjecting the owner to more risk if he or she decides to sell the security prior to maturity. G. Christopher Cosby April 29, 2016 Page 29 of 30 The risk-free rate and risk taking. The risk-free rate should not affect risk taking. The difference between the expected rate of return on a risky investment and the return on the risk-free asset is known as the risk premium. Investments provide a risk premium to compensate investors for taking on risk. To the extent one believes that future investment returns will be similar to historical investment returns, it is the risk-premium on an investment that should be stable, not the expected rate of return on an investment, whether expressed nominally or on an inflation-adjusted basis. That is, risky investments offer a risk premium (higher expected returns than the risk-free asset) as compensation for taking on risk. All else equal, if the risk-free rate of return increases or decreases, the expected rate of return on a risky asset should adjust to maintain the same premium relative to the risk-free rate.65 If a respondent interprets the assumed risk-free rate of return in questions MFEC4, MFEC5, and MFEC6 to all be rates that were determined in the market, then changing that rate should have no impact on the individual’s choice of investment. The risk-free alternative investment described in the questions appears to be a TIPS. All else equal, the decision on whether to invest in risky assets or riskless assets should not be affected by whether the TIPS yield is 0.0 percent, 2.0 percent, or 4.0 percent. It is the risk premium, or the expected return on an investment relative to the risk-free investment, that impacts the decision to take on risk. If the assumed rates of return used in the questions are not intended to be interpreted as market-determined rates, then the questions need to be revised. That is, if the questions are trying to elicit how much of a return bonus or return penalty, relative to the risk-free rate, would be needed to get respondents to change their investment choice, then question MFEC4 should use the market rate one could earn on the riskless investment described (which, presumably, would be a rate derived from the TIPS yield curve) at the time the survey is in the field. The risk-free rate in question MFEC5 should then be a rate that is either 2.0 percentage points higher or lower than the current market rate, and the risk-free rate in question MFEC6 should then be a rate that is either 4.0 percentage points higher or lower than the current market rate. Of course, it is not clear how useful this information would be, as the market does not typically offer rates of return that are two to four percentage points above or below the market rate. D. Impact of Repeat Surveys Finally, the DOL should be cognizant of the possibility that the repeated surveys pointing out to respondents the array of information, advice, educational materials, and retirement savings 65 Among things held equal is the perceived riskiness of the risky asset. If a decline in the risk-free rate is associated with an increase in the perceived riskiness of an asset, then the risk premium on the asset will increase. Once prices have adjusted to the new expectations, however, the increase in the risk premium should not encourage risk taking, as it is simply compensating for the increase in perceived risk. G. Christopher Cosby April 29, 2016 Page 30 of 30 opportunities at their fingertips may impact the respondents’ behavior in connection with retirement saving and planning. Research finds that survey respondents’ future actions can be impacted by having taken a survey.66 Therefore, results surrounding changes in respondents’ activities over time may not be generalizable. * * * The Institute agrees with the importance of understanding how American workers manage their accounts and steward their retirement assets to and through retirement, but has serious reservations about the Department’s proposed survey project. The On the Road to Retirement Surveys need to change the line of questioning on sources of information and advice consulted by retirement savers and retirees, retirement plan participation, activities when changing employers, and investment risk, in addition to other refinements spelled out above. Furthermore, the full scope and complete cost of the project should be clarified. If the survey effort only occurs over two years, a main goal of the project–“long-term research…over several years” to determine “how retirement planning strategies and decisions evolve over time”67 will not have been met. We welcome the opportunity to discuss these comments further or to provide additional information to you and your staff as you work on this survey project. In the meantime, please contact the undersigned, David Abbey (david.abbey@ici.org or 202/326-5920) or Sarah Holden (sholden@ici.org or 202/326-5915), if you have any questions. Sincerely, /s/ David Abbey /s/ Sarah Holden David Abbey Sarah Holden Deputy General Counsel Senior Director Retirement Policy Retirement & Investor Research 66 For example, see Zwane, Zinman, Van Dusen, Pariente, Null, Miguel, Kremer, Karlan, Hornbeck, Giné, Duflo, Devoto, Crepon, and Banerjee, “Being surveyed can change later behavior and related parameter estimates,” Proceedings of the National Academy of Sciences of the United States of America 108, no. 5 (2011): 1821–1826, at www.pnas.org/content/108/5/1821; or Bach and Eckman, “Changes in Labor Market Behavior Due to Panel Conditioning in a German Panel Study,” presented at the American Statistical Association, Joint Statistical Meetings, August 2015. Bach and Eckman conclude that the survey induced changes in behavior among respondents. 67 See 81 Fed. Reg. 10280 (February 29, 2016) at page 10281.