ICI Submits Comment Letter on Information Collection Request - On the Road to Retirement Surveys (pdf)

May 30, 2017 Submitted Electronically Office on Information and Regulatory Affairs Attn: OMB Desk Officer for DOL–EBSA Office of Management and Budget Room 10235 725 17th Street, NW Washington, DC 20503 US Department of Labor – OASAM Office of the Chief Information Officer Attn: Department Information Compliance Management Program Room N1301 200 Constitution Avenue, NW Washington, DC 20210 Re: Information Collection Request - On the Road to Retirement Surveys; OMB ICR Reference Number 201412-1210-007 Ladies and Gentlemen: The Investment Company Institute (ICI)1 appreciates the opportunity to provide comments on the Department of Labor’s Information Collection Request (ICR)2 regarding the On the Road to Retirement Surveys, a panel survey project. The ICR is intended to gather information on how people make planning and financial decisions before and during retirement. The Department hopes that the insight gained from the surveys “will provide policy-makers and the research community with valuable 1 The Investment Company Institute (ICI) is a leading global association of regulated funds, including mutual funds, exchange-traded funds (ETFs), closed-end funds, and unit investment trusts (UITs) in the United States, and similar funds offered to investors in jurisdictions worldwide. ICI seeks to encourage adherence to high ethical standards, promote public understanding, and otherwise advance the interests of funds, their shareholders, directors, and advisers. ICI’s members manage total assets of US$19.4 trillion in the United States, serving more than 95 million US shareholders, and US$1.6 trillion in assets in other jurisdictions. ICI carries out its international work through ICI Global, with offices in London, Hong Kong, and Washington, DC. 2 82 Fed. Reg. 19391 (April, 27, 2017). The survey includes the following components: (1) pretesting, (2) pre-survey, (3) initial participation survey (IPS), (4) advice interaction survey (AIS), and (5) annual participant survey (APS). For a description of the survey project, see pp. 3–4 of ICI’s letter to DOL dated April 29, 2016, described at footnote 5, infra. Information Collection Request 201412-1210-007 May 30, 2017 Page 2 of 14 information that can be used to guide future policy and research.”3 As an organization with established research capabilities, ICI is in a unique position to comment on this matter.4 ICI strongly supports efforts to promote retirement security for American workers and generally agrees that understanding how individuals progress along the road to retirement can inform retirement savers, plan sponsors, financial services providers, and policy-makers on ways to make that road easier to traverse. We believe that the utility of this survey and the quality of the information collected, however, would significantly benefit from the improvements described below. ICI previously recommended numerous changes to the proposed ICR survey materials in a letter to the Department dated April 29, 2016 (2016 ICI Letter).5 Although the Department made several of the changes we suggested to the survey material now submitted to the Office of Management and Budget (OMB),6 some of the more fundamental issues have not been addressed, and thus our concerns remain. We believe it is important to reiterate these unresolved concerns to OMB. In addition to the comments we previously provided to the Department, we include two new comments in this letter that we believe would further increase the utility of the ICR. Our comments are offered to help OMB proceed in its 3 81 Fed. Reg. 10280, at 10281 (February 29, 2016). 4 One of the major roles ICI serves is as a source for statistical data on the investment company industry. With a research department comprising more than 40 professionals, including seven PhD economists, ICI conducts public policy research on fund industry trends, shareholder characteristics, the industry’s role in US and international financial markets, and the retirement market. For example, ICI publishes reports focusing on the overall US retirement market, fees and expenses, and the behavior of defined contribution plan participants and individual retirement account (IRA) investors. In its research on mutual fund investors, IRA owners, and 401(k) plan participants, ICI conducts periodic household surveys that connect directly with investors. Our research team also has extensive experience engaging in survey design, administrative data collection, and data archiving. This expertise includes prior work at the US Bureau of Labor Statistics in the Division of the Consumer Expenditure Surveys focused on questionnaire development and data processing; and prior work constructing a three-year panel data collection engaging a large mid-western R-1 university system’s full sample of 17,000 graduate students. We currently contribute a member to the Internal Revenue Service’s Statistics on Income Consultants Panel (www.irs.gov/uac/soi-tax-stats-soi-consultants-panel). ICI researchers also field on-going annual and higher frequency data collection activities, and engage in ongoing collaborative data research collection and analysis efforts with the Employee Benefit Research Institute (EBRI; www.ebri.org) and BrightScope, a Strategic Insight Business (www.brightscope.com). Our comments are motivated by the perspective gained from our research and expertise in these activities. 5 See letter from David Abbey and Sarah Holden, to G. Christopher Cosby, Employee Benefits Security Administration, US Department of Labor, available at www.ici.org/pdf/29881.pdf. Our letter responded to the Department’s proposed ICR published at 81 Fed. Reg. 10280 (February 29, 2016), at www.gpo.gov/fdsys/pkg/FR-2016-02-29/pdf/2016-04315.pdf. Those comments were in reference to the supporting ICR materials (PDF files labeled Appendix A through P, and “Supporting Statement for On the Road to Retirement Surveys: Investigating Retirement Planning Decision-Making and the Evolution of Individuals’ Retirement Planning Decisions over Time”) provided to ICI in an email from Lynn Johnson at DOL, EBSA, acting on behalf of Mr. Cosby, dated March 3, 2016. 6 The new ICR materials (PDF files labeled Appendix 1 through 17, “Supporting Statement Part A – Road to Retirement 4_10_2017”, and “Supporting Statement Part B – Road to Retirement 4_10_2017”), are available at www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=201412-1210-007 and www.reginfo.gov/public/do/PRAICList?ref_nbr=201412-1210-007. Information Collection Request 201412-1210-007 May 30, 2017 Page 3 of 14 review in a way most likely to result in a valuable resource, for both the Department and the broader research and policy communities. Following a summary of our comments in Section I, Sections II through VI offer comments on specific elements of the current ICR. I. Introduction and Summary of Key Points. We appreciate that the Department’s goal in undertaking the On the Road to Retirement Surveys is to collect data that can be used to gain insights on households’ preparation for retirement, so as to guide future retirement policy and administration. We agree that more research in this area would be beneficial.7 In the 2016 ICI Letter, we identified several concerns, particularly regarding the intended samples, timing, and cost of the surveys. We also devoted considerable time to the survey instrument and offered specific recommended changes to the wording used in the questionnaire in certain areas. We appreciate the Department’s efforts to address the points made by all commenters.8 We also appreciate the Department’s acknowledgement that while, “[m]any notable improvements have been made in response to the public comments… [t]he team recognizes, however, that the design work is still not complete.”9 While the Department has made some changes in the past year, we continue to have meaningful concerns about the merit of the current design. We encourage the Department to make additional changes at this time, ahead of instrument testing, so that the participant testing stage of the survey development process has the best chance of resulting in a robust instrument, sample and data collection from which to yield reliable information for the benefit of American households. Our goal in submitting this comment letter is to help make the most of any opportunity that the granting of a mandate to collect Road to Retirement data would represent for the Department and for American families, writ broadly. We believe that the changes we suggest below would increase the practical utility, as well as the quality and clarity, of the information collected. As discussed in greater detail below, we encourage the Department to: • Extend the survey project beyond two years. We continue to be concerned with the ICR because, as currently planned and budgeted (2017), only two years of the budget are devoted to data collection, making for a panel of very short duration. The Department should retool and expand the data collection to support an extended effort beyond two years, so that the benefit 7 In Section A.8 of Supporting Statement Part A, the Department wrote of “strong expressions of support made by each commenter, … , including whether the information will have practical utility,” at p. 7. We believe this statement does not represent the main thrust of the 2016 ICI Letter. We continue to have fundamental concerns about the merit of the ICR based on the Department’s current (unchanged) research sample design. 8 In addition to the 2016 ICI Letter, the Securities Industry and Financial Markets Association (SIFMA), TIAA, and the US Chamber of Commerce also submitted comments. 9 See Supporting Statement Part A, Section A.8, at p. 8. Information Collection Request 201412-1210-007 May 30, 2017 Page 4 of 14 of a panel design can be realized. The current data collection framework does not support meaningful panel evolution, and is not likely to contribute much on its own. • Increase the size of the sample. It appears that the sample size remains quite small and unchanged from the originally proposed ICR, making it particularly vulnerable to attrition. The Department should expand or reshape the initial sample to accommodate research on meaningful challenges to retirement security. • Create an advisory panel. The creation of an advisory panel of experts in the retirement savings field will help with ongoing survey design challenges, including completing the remaining initial design work. • Make the data public, and promote the use of the data by outside researchers. To maximize the usefulness of the ICR, we strongly suggest that the full micro data be distributed in an organized and easy to use manner. This is because the data collection will only be valuable to the extent it can be used by the research community. We believe that the number of researchers outside the Department with expertise and interest in this research area promises to yield a number of research projects that is several multiples of what would likely be generated by the Department alone. • Improve the wording and design of the survey instruments. The Department should revise the survey questions, as described in Section VI below, to: (1) reduce the amount of specific identifiable account and personal data being requested to protect the sample and maintain the integrity of survey participants’ privacy rights, (2) better leverage comparability with other survey efforts used in the retirement research community, and (3) better reflect the financial landscape Americans have to work with as they travel the road to retirement. In the 2016 ICI Letter, we encouraged the Department to consider the existing research in this area.10 To maximize the utility of the ICR, the Department should determine what the data gathered from each element of the ICR would contribute above and beyond the already available data on US households. We note that the Department has distinguished the ICR effort from those that had been suggested as alternatives.11 Relative to these efforts, the ICR proposes a relatively high frequency panel. 10 See 2016 ICI Letter at p. 2. In particular, we noted the Survey of Consumer Finances (SCF) and the Health and Retirement Study (HRS), which provide helpful insight into the life cycle of retirement saving and motivation for some retirement planning activities. 11 See Supporting Statement Part A, Section A.4, at p. 5–6. Information Collection Request 201412-1210-007 May 30, 2017 Page 5 of 14 Academic research has provided extensive insight into households’ retirement saving utilizing household survey data, and proprietary record keeper or plan data.12 ICI has conducted extensive research on retirement savers’ decision making, including surveying households on changes in their plans in the wake of the financial market crisis;13 recently retired households on their defined contribution (DC) plan distribution decisions at retirement;14 and traditional IRA–owning households to understand the sources of information consulted when making an IRA rollover decision and the reasons for rolling over retirement plan accumulations to traditional IRAs.15 ICI also has conducted more aggregated analysis of the US retirement landscape.16 ICI researchers have written books on US government programs and tax expenditure supports for retirement.17 ICI researchers have contributed to peer-reviewed journals in the fields of economics, finance, public policy and law on the topics of: integrated social security program reforms;18 the relative efficacy of financial literacy survey instrument questions;19 the measurement of retirement resource adequacy;20 the role of employer plan features in developing household savings and investment habits;21 and the impacts of recessionary shocks on labor 12 For mention of some of this literature, see Brady, Burham, and Holden, The Success of the US Retirement System (December 2012), at www.ici.org/pdf/ppr_12_success_retirement.pdf. In addition, see research published through the National Bureau of Economic Research (www.nber.org), the Pension Research Council at the Wharton School at the University of Pennsylvania (www.pensionresearchcouncil.org), the Social Security Administration’s Retirement Research Consortium (www.ssa.gov/policy/rrc/), and the Employee Benefit Research Institute (www.ebri.org). 13 See Holden, Schrass, and Bogdan, “American Views on Defined Contribution Plan Saving, 2016,” ICI Research Report (February 2017), at www.ici.org/pdf/ppr_17_dc_plan_saving.pdf; and Holden and Schrass, “Defined Contribution Plan Participants’ Activities, First Three Quarters of 2016,” ICI Research Report (February 2017), at www.ici.org/pdf/ppr_16_rec_survey_q3.pdf. 14 See Sabelhaus, Bogdan, and Holden, “Defined Contribution Plan Distribution Choices at Retirement: A Survey of Employees Retiring Between 2002 and 2007,” ICI Research Report (Fall 2008), at www.ici.org/pdf/rpt_08_dcdd.pdf. 15 See Holden and Schrass, “The Role of IRAs in US Households’ Saving for Retirement, 2016,” ICI Research Perspective (January 2017), at www.ici.org/pdf/per23-01.pdf. 16 See Brady, Burnham, and Holden, “The Success of the US Retirement System,” ICI Research Report (December 2012), at www.ici.org/pdf/ppr_12_success_retirement.pdf. 17 See Brady, How America Supports Retirement, Washington, DC: Investment Company Institute, 2016, at www.ici.org/whobenefits. 18 See Fichtner and Seligman, “Enhancing US Retirement Security through Coordinated Reform of Social Security Disability and Retirement Insurance Programs,” Journal of Retirement, 4(1): 131–147. 19 See Schmeiser and Seligman, “Using the Right Yardstick: Assessing Financial Literacy Measures & Validating Future Financial Well-Being,” Journal of Consumer Affairs, 47(2): 243–262. 20 See Brady, “Measuring Retirement Resource Adequacy,” Journal of Pension Economics and Finance, 9(2): 235–262, 21 See Seligman and Bose, “Learning by Doing: Active Employer Sponsored Retirement Savings Plan Participation and Household Wealth Accumulation,” Quarterly Review of Economics and Finance, 52(2): 162–172. Information Collection Request 201412-1210-007 May 30, 2017 Page 6 of 14 force participation and retirement preparation, to name a few of our active research areas.22 Our detailed comments, below, draw on experience we have gained in building this research portfolio. We continue with more detailed comments in order to offer constructive suggestions regarding the survey instruments, in light of the changes incorporated by the Department since last year. II. To be of value, the survey must extend beyond two years. The Department expressed its intention “to undertake a long-term research study that will track US households over several years in order to collect data and answer important research questions on how retirement planning strategies and decisions evolve over time.”23 We support the Department’s intended goals. We have significant concerns, however, that a two-year frame is inadequate to achieve these goals. In the 2016 ICI Letter we explained, “Two years of survey work will not meet the stated goals of tracking retirees and future retirees over an extended period.”24 In other comments received by the Department, the Chamber of Commerce recommended a minimum five-year frame.25 The Department responded, “EBSA intends to extend the longitudinal survey past… two years.”26 However, the project design continues to allocate budget resources to data collection for only two years. We assume that the limited sample size (described in Section III below) and brief time in field for this ICR are meant to keep costs low. Based on ICI’s significant experience in developing and implementing surveys, we have significant doubts that the proposed budget of $7.3 million will build sustained value, based on the current design. The budget, as proposed, is for a “six year contract,”27 but only funds two years of survey work. We suggest that it would be beneficial to rework the budget so that more resources can be devoted to the data collection. We believe an exercise of only two years would impose significant costs on taxpayers while producing little in the way of benefits to the Department or the broader research and policy communities. 22 See Seligman and Wenger, “Asynchronous Risk: Unemployment, Equity Markets, and Retirement Savings,” Journal of Pension Economics and Finance, 5(3): 237–255. See also Seligman, “Involuntary Retirement, US Social Security Program Participation and the Great Recession,” Public Finance and Management 14(3): 329–356. 23 82 Fed. Reg. 19391. 24 2016 ICI Letter, at p. 8. 25 See letter from Randel Johnson and Ronald Bird, to G. Christopher Cosby, Employee Benefits Security Administration, US Department of Labor (April 29, 2016) (2016 Chamber of Commerce Letter), available at www.reginfo.gov/public/do/DownloadDocument?objectID=68469700, at p. 2. 26 See Supporting Statement Part A, Section A.8.a, at p. 10. 27 See Supporting Statement Part A, Section A.14, at p. 25. Information Collection Request 201412-1210-007 May 30, 2017 Page 7 of 14 III. The Department should increase the size of the sample to accommodate analyses of meaningful challenges to retirement security. In the 2016 ICI Letter, we raised concerns regarding sample sufficiency.28 Comments from SIFMA similarly suggested the Department expand the sample size from age 56+, and offered examples of research of interest deriving from this age group which would inform the Department as well as the retirement research community while helping to shape the expectations of those in younger age groups.29 The Department, however, did not make changes to the research sample design. Rather, in response to expressions of concern about sample size, the Department stated, “it is difficult to ensure that large sample sizes would be obtained for each possible experience of interest, especially for events that are relatively rare.”30 The Department also noted that “the research team strongly believes that a ‘statistically meaningful analysis of the data’ does not and should not imply any specific minimum acceptable threshold for the size of sample subgroups.”31 While we appreciate the Department’s observation regarding the lack of rigid rules on sample size, we note that such a rationalization has limits. After all, one cannot analyze data one does not have. We also note that the Department appears to place significant reliance on the usefulness of estimates from simple hypothesis tests,32 which rely on the statistical significance they claim is too often misunderstood and misused. Clearly, sample size is important to justify in an ICR devoted to building a new panel dataset, and we encourage OMB not to dismiss the concerns we are documenting here. We believe that a benchmark for sample size that is tied to at least a few, more narrowly targeted, analyses is a reasonable goal for the Department. Our thinking on this matter is informed by our research, as described below. Our research generally finds that the majority of Americans currently are doing a good job of preparing for retirement. In light of this, understanding differences between various sub-groups becomes crucial for understanding and targeting the challenges faced by those who are having difficulty planning for retirement or maintaining their standard of living in retirement. We believe that sample 28 We wrote “It is not clear that the contemplated sampling strategy that draws from respondents age 25 or older and only targets sub-samples by age will result in enough respondents with the specific experiences being explored.” See page 5 of the 2016 ICI Letter. We also wrote, “If the study is to be extended beyond two years, the longer time frame may result in difficulties with panel attrition.” See page 8 of the 2016 ICI Letter. The small sample size makes the survey particularly vulnerable to attrition. 29 See letter from Lisa J. Bleier and Bernard V. Canepa, to G. Christopher Cosby, Employee Benefits Security Administration, US Department of Labor (April 29, 2016) (2016 SIFMA Letter), available at www.reginfo.gov/public/do/DownloadDocument?objectID=68469900, at p. 3. 30 See Supporting Statement Part A, Section A.8.a, at p. 9. 31 See Supporting Statement Part A, Section A.8.a, at p. 9. 32 See Supporting Statement Part B, Section B.1.b, Table 10, at p. 11. Information Collection Request 201412-1210-007 May 30, 2017 Page 8 of 14 size should be evaluated in light of whether it allows the survey to capture some meaningful risks along the “Road to Retirement.” Our concern is illustrated by the following example. Consider, among vulnerable groups, those in prime savings years who become unemployed and then exit the labor force. We suggest that understanding the following factors would be of significant interest in terms of policy and administration of both public and ERISA regulated retirement programs: (i) how permanent or temporary a labor force exclusion is, among the middle-aged-and-older working age sample, (ii) the reasons behind why the exclusion occurs, and, (iii) what remedial steps these households might take to catch up in terms of retirement preparation if/as they reenter work. When we consider the information within Table 6 of the ICR supporting materials,33 where sample criteria are detailed, it appears that the Department proposes to initially survey roughly 25 households aged 41–55 with a member not in the labor force and not retired. It appears that another 25 households in this circumstance that are past age 56 will also be surveyed. Following the proposed pretesting and pre-survey phases, when we at ICI scale up the matrix to estimate the Year 1 Initial Participant Survey (IPS) and Advice Interaction Survey (AIS1) sample sizes, we do not see a robust sample. Based on the Department’s estimate for a sample size of 4,500, ICI estimates for these cells are less than 250 observations. In Year 2 survey efforts, the Department projects sample attrition that yields even smaller cell counts for these groups of interest. It is unclear that a two-year effort with such small samples provides any opportunity to yield much insight on paths for those out of the labor force. That noted, these are likely the households that deserve some focus by policy-makers, because the ability to save is fundamentally tied to employment that yields more income than is needed for immediate consumption. Understanding paths that led to a return to employment, and therefore back onto the road to retirement would be of great benefit. Thus we feel that the Department’s effort would be able to make a helpful contribution to our collective knowledge base, regarding elder poverty as a retirement outcome, if scaled and targeted appropriately. At the same time, we feel that the current effort (considering the two-year time frame and the small sample size) is not likely to support the kind of targeted research suggested by SIFMA last year (mentioned above), or by the example above. For the foregoing reasons, we encourage OMB and the Department to consider the responsibility of launching a multi-wave, multi-cohort design type of survey effort. One might, for example, scale the initial sample size to afford the capacity to persist over a 10-year frame (10 years would offer a look at a meaningful but still relatively brief window over the life-cycle). Such a design is much more likely to inform research and understanding than is a small, short panel, regardless of the novelty of its higher frequency. Assuming the panel becomes a used and valuable resource, it might be refreshed in Year 5 or 6. If by some chance a well-scaled data set addressing a key area of concern to policy-makers and families alike does not meet these criteria, in those intervening years, it will nonetheless be more justifiable to have planned for a properly scaled exercise. 33 See Supporting Statement Part B, Section B.1.a, Table 6, at p. 5. Information Collection Request 201412-1210-007 May 30, 2017 Page 9 of 14 To summarize our first two general areas of comment, panel size and duration, we respectfully note that early in its “Supporting Statement Part A,” the Department seems to acknowledge the limitations of the proposed panel size and duration, noting that “tradeoffs must be made when working within a budget.”34 We respectfully offer that in the absence of meeting a suitable scale, the budget is essentially being wasted in light of the lack of meaningful data that the effort will achieve. That half of those interested parties providing comments last year conveyed these same concerns is relevant to any serious consideration of how best to proceed with this work. We offer as a constructive alternative to expanding this effort that one might instead use Department funding to support a higher frequency collection of an already established survey, perhaps with an additional module targeting questions regarding details of interest to the Department and the research community. Given the notable start-up costs listed in the design and pretesting efforts, which comprise roughly 20 percent of the total budget,35 this option might make greater sense to consider, in as much as the current budget might provide a larger sample of the data which the entire research and policy communities would value. IV. The Department should create an advisory panel of experts in the retirement savings field that will help with ongoing survey design changes, including the remaining initial design work. The initiation of an effort like this provides an advisory component through the comment process, including these comments, but that is a temporary occurrence. For an ICR of this nature to truly provide an ongoing contribution to the retirement research field, it would likely benefit from more consistent and ongoing peer review. Last year, the Chamber of Commerce commented that the Department should consider creating an advisory panel to provide ongoing guidance to the effort.36 We agree with the Chamber of Commerce on this point. Among other things, an advisory panel can help the Department to establish targets for research, adding to the one we offered above, in Section III that would benefit sample panel construction. Further, over the two-year data collection contemplated by the ICR, such a committee might help address findings of analyses of non-response along the way. In Section B.3 of the ICR supporting material,37 the Survey of Consumer Finances (SCF) is referenced as a potential additional data benchmark, in this case. We suggest that SCF experts, along with those who have offered comments along this process, and those with expertise collecting and analyzing data from academic and US government agencies would form a reasonable pool from which to select a panel that would contribute significantly to this effort. They are the likely audience for the work and have 34 See Supporting Statement Part A, Section A.8.a, at p. 9. 35 See Supporting Statement Part A, Section A.14 at p. 25: $1.5 million of the budget is earmarked for design of the study and data collection experiments. 36 See 2016 Chamber of Commerce Letter, at p. 2. 37 See Supporting Statement Part B, Section B.3, at p. 22. Information Collection Request 201412-1210-007 May 30, 2017 Page 10 of 14 considerable expertise. The creation and involvement of such an advisory panel would not be cost prohibitive. ICI would, as we assume would other interested parties, be willing to contribute our expertise to such a panel. Working with a constructive and critical panel of experts from outside the Department can protect a project from mishaps and unlock a larger pool of expertise in the retirement research community if or when it is most needed. Over a longer period of time, consistent with our comments under Sections II and III, we are concerned that without an advisory component, the effort proposed by the Department risks becoming irrelevant, and thus may not be worth its budgetary cost. In lieu of a panel of experts, we offer that the opportunity to build on another established and ongoing panel data collection effort, would allow the benefit of established expertise, quite possibly at lower cost. In this light, one might further consider allowing other ongoing efforts to bid to offer the Department’s survey instrument and frequency preferences to their samples, to evaluate the merit of this alternative. V. The Department should make the data public, and promote their use by outside researchers. The establishment of a valuable data panel should include consideration of how best the research community can be engaged to value and make use of these data. Last year, the Chamber of Commerce commented that the Department should make public use data files available and that the Department should include a unique identifier to accommodate researchers’ interest in merging panel waves in order to construct useable panel data over time.38 We agree with the Chamber of Commerce on this point as well. The potential contribution these data might offer is more likely to be realized if they are made public. The Department briefly mentions the fact that it will make the data public, but it does not describe how it will do so. A single sentence in Supporting Statement Part A notes, “[g]iven that the survey results will be widely available, the broader public will benefit as well,”39 without outlining any distribution mechanism. Later, in Table 5 of the ICR supporting material, the schedule of deliverables includes “Database: De-Identified Database for Research.”40 The Department should develop a plan to facilitate the usability of the micro data, including software intake files, data documentation, data archive distribution, and for facilitating the use of the data among established groups such as the Social Security Administration’s Retirement Research Consortium research centers.41 The full micro data should be distributed in an organized and easy to use manner, as with the RAND Health and Retirement Study (HRS) files, or at least the basic modular HRS files. These too would likely enhance EBSA’s deeper understanding of retirement issues. These 38 Ibid. 39 See Supporting Statement Part A, Section A.6, at p. 6. 40 See Supporting Statement Part A, Section A.16.b, Table 5, at p. 28. 41 Information on the consortium is available at www.ssa.gov/policy/rrc/. Information Collection Request 201412-1210-007 May 30, 2017 Page 11 of 14 resource investments, however, do not appear to be in the budget.42 Established panel data facilities such as the HRS have significant institutional capacity in this regard. The Inter-university Consortium for Political and Social Research (ICPSR),43 at the University of Michigan is another possible institution to consider for advice in this area. The Road to Retirement Surveys will only be considered valuable to the extent they are able to be used by both the Department and the research community. We believe that the number of researchers outside of the Department with expertise and interest in this research area promises to yield several multiples of the research produced by the Department alone. There are many who wish to better understand lifecycle planning processes by US households, and therefore, the public availability of the data is crucial to the utility of the ICR. VI. The Department should improve the wording and design of the survey instrument. Last year all four commenters offered several suggestions for improvements to the survey instruments being proposed by the Department of Labor.44 We acknowledge the many refinements the Department made to the survey instruments since last year. Having reflected on those, and the other comments the Department received, we suggest a number of further improvements to the survey instrument. We suggest that the Department work with the following three goals in mind: (1) to reduce the amount of specific identifiable account and personal data being requested in order to protect the sample and maintain the integrity of privacy rights, (2) to better leverage comparability with other survey efforts used by the retirement research community, and (3) to better reflect the financial landscape Americans have as potential resources as they travel the road to retirement. We respect that most of the data that would be collected under the ICR are, or can be made of value to, the work of the Department as well as the broader research and policy communities. However, we suggest that the Department reduce the amount of specific identifiable account and personal data being requested in order to protect Americans from unintended damages from data breach, the weakening of social norms regarding safe keeping of personal financial information, and the development of survey fatigue and the resultant increase in sample attrition. Throughout the surveys currently proposed, the Department repeatedly asks the respondent for highly sensitive personal information. For example, in the IPS and APS surveys, the Department asks the respondent for the Employer Identification Numbers (EINs) of the employers of both the respondent and spouse or partner. At the end of the IPS follow-up interview, the Department requests 42 See Supporting Statement Part A, Section A.14, at p. 25. 43 The ICPSR is a central data management and curation facility that acts as a hub for most university data archiving efforts. 44 See the 2016 ICI Letter, the 2016 SIFMA Letter, the 2016 Chamber of Commerce Letter, and the letter from Jonathan E. Feigelson, to Employee Benefits Security Administration, US Department of Labor (April 29, 2016), available at www.reginfo.gov/public/do/DownloadDocument?objectID=68470700. Information Collection Request 201412-1210-007 May 30, 2017 Page 12 of 14 that the respondent send in their employer’s Summary Plan Description document that describes the details of the respondent’s retirement plan. Perhaps most concerning, the Department asks for a recent retirement account statement that will list the account numbers and balances in each allocated investment. Finally, at the end of the AIS survey, the Department requests that the respondent send in copies of investment recommendations, customer account agreements, and disclosures of compensation arrangements from any financial professional from which the respondent received personalized information or advice within the last three months. Both the level of reporting burden and the sensitivity of what is being requested push beyond what we feel is acceptable. We cannot see the merit for these requests, especially when, at small sample sizes, the opportunity to generalize from plan and account specifics is essentially non-existent. We suspect that the burden such invasive requests will impose on respondents will likely increase sample attrition, further aggravating the existing small sample problem (as described in Section III above). This request could further be seen to set a precedent for the level of personal information requested by non-US government surveys. Thus we respectfully suggest that OMB consider disallowing this portion of the ICR. In the case that it be allowed, we suggest requiring a significant justification and the incorporation of a very compelling data security plan. Second, we suggest that the Department’s instrument be further refined to better leverage comparability with other survey efforts used in the retirement research community, such as the HRS, the SCF, the Survey of Income and Program Participation (SIPP), the Current Population Survey (CPS) March supplement, and other cross sectional and panel data collection efforts. There exist meaningful deviation between the instruments being proposed and other large panel datasets we employ in our research at ICI. We suggest that the Department send the survey to administrators of other salient ongoing data collections listed above, and request comments targeted to help identifying question pattern matches. Such collaboration is symbiotic, should be valued, and might be of help in identifying good candidates for the panel we recommend under Section IV above. Third and finally, consistent with the 2016 ICI Letter, we suggest that the Department’s fielded instrument should better reflect the financial landscape Americans have to work with as they travel the road to retirement. (In a few places we suggest question pattern changes that we think work better as well.) Having these tenants in mind, the following comments regard the design and wording of the questions in the: Initial Participant Survey (IPS), Advice Interaction Survey (AIS), and the Annual Participant Survey (APS). They are intended to improve utility and quality of the Department’s data collection effort in the On the Road to Retirement Surveys.
Questions EPR7A1 and EPSP7A1 of the IPS and APS would benefit from having the words “at your (his/her) current job” added to the question text. The question should read “Now we have some questions about the plan at your (his/her) current job that is most important to you Information Collection Request 201412-1210-007 May 30, 2017 Page 13 of 14 (him/her).” Unless compatibility with another survey effort would offer more improvement, we suggest this language be incorporated.
In the IPS and APS, question EPR16m should be revised to reflect the question text for EPSP16m. The word “next” should be added to the question text. The question should read “The following questions concern the plan at this job that is the next most important to you, other than the plan at this job you just told us about. Unless compatibility with another survey effort would offer more improvement we would suggest this language be incorporated.
In the IPS and APS, we recommend restricting the maximum amount allowed to be input for the IRA contribution questions (DR1b and DR2b) to be equal to the IRA contribution limits from the IRS.
In the retirement finances section in the AIS, we think there would be value to adding a question to account for Roth conversions. Currently the questions ask about withdrawals from dedicated retirement accounts “such as 401(k) plans, IRAs, Keogh accounts, and the like.” Later the survey asks whether or not any of the amount withdrawn was rolled over into an IRA, however there is no opportunity to distinguish a Roth conversion. We suggest the Department take advantage of this opportunity to collect useful information on whether Americans are engaging in this activity. We would suggest that such a Roth conversion question could be added after RDF3d and RDF3f, if “IRA” is chosen by respondents. Roth IRAs have been gaining in attention, and conversions are a relatively recent innovation, so other data panels may offer limited opportunity for improvement. In fact the right wording by the Department might be considered for adoption by others. Again, a panel of experts could facilitate this exchange, one of potential value to all parties.
In the retirement finances section in the AIS, we recommend replacing questions RFD3, RFD4, and RFD5 with one question to collect information on the type of withdrawal made. This question could be asked after RFD2 and would be a ‘mark all that apply’ type of question. We suggest the following response categories be considered: {lump sum distribution, loan, something else, DK, RF}. This would simplify this section by replacing three questions with just one. The subsequent questions could have text fills based on the answers input into the new question.
In the AIS, add “accounts” when talking about 401(k) plan accounts. Currently questions RF1 and RF5 ask about “401(k) plans.” Employees can open up accounts, but not plans.
We think the AIS, question RF7 should be a ‘mark all that apply’ type of question. In addition, we think this question might be improved if it did not only ask about increasing or decreasing contributions to employer-sponsored retirement plans or payroll deduction IRAs. This would allow responses signaling increased or decreased regular contributions to a traditional and/or Roth IRA’s.
In the AIS, we recommend changing the answer categories for questions FA15a and FA22a to: {I plan to follow all of the advice, I plan to follow most of the advice, I plan to follow some of the advice, I plan to follow a very small portion of the advice and I do not plan to follow any of the advice, DK, RF}. At the very least we recommend dropping the choice of “I plan to follow a Information Collection Request 201412-1210-007 May 30, 2017 Page 14 of 14 little of the advice” since that answer may be confused with “I plan to follow some of the advice.” The Department may also want to consider using the same answer categories for questions FA15a, FA15c, FA22a, and FA22c.
Following the question above we suggest adding a question along the lines of, “No matter the extent to which one follows any particular advice, some value the opportunity to discuss their savings and investment choices with a professional. On a scale from 1 – 5, with 1 being of no value, and 5 being , of very high value, how much did you/do you value the opportunity to meet with an investment adviser: {1,… ,5, DK, RF}
In the AIS, we recommend removing the statement in FC2 since FC4 will provide the same information. * * * ICI agrees with the importance of understanding how American workers manage their accounts and steward their retirement assets to and through retirement, but has serious reservations about the Department’s proposed survey project. If the Department fails to clarify the full scope and complete cost of the project and the survey effort only occurs over two years, the main goal of the project–“to undertake a long-term research study…over several years” to determine “how retirement planning strategies and decisions evolve over time”45–will not have been met. We welcome the opportunity to discuss these comments further or to provide additional information to you and your staff as you work on this survey project. In the meantime, please contact the undersigned, David Abbey (david.abbey@ici.org or 202/326-5920) or Sarah Holden (sholden@ici.org or 202/326-5915), if you have any questions. Sincerely, /s/ David Abbey /s/ Sarah Holden David Abbey Sarah Holden Deputy General Counsel Senior Director Retirement Policy Retirement & Investor Research 45 See 82 Fed. Reg. 19391 at p. 19391.