Why Probabilities Have No Place in Retirement Planning
A young man has just scored a date with the woman of his dreams. He wants their first date to be romantic and memorable, so he arranges a picnic in a field of wildflowers atop a scenic hilltop. Because he is careful in his planning, he checks the forecast. The weatherman calls for a beautiful sunny day with just a 10% chance of rain. Delighted, the young man sets his plan in motion. Midway through the picnic, the sky darkens and the clouds burst forth a torrent of rain. Alas, the date is memorable, but not in a good way. From the young man’s perspective the Weatherman was wrong. Had he known that there was actually a 100% chance of rain, he would have planned differently.
Fast forward 40 years, the young man in our story (still single) is now 65 years old and planning his retirement. He meets with his financial advisor who enters his income needs and portfolio information into a retirement planning application that employs Monte Carlo simulations to guide decision making. “Will my retirement savings be enough to last throughout retirement?” our protagonist asks anxiously. “I have great news,” proclaims his advisor, “Your portfolio has a 90% probability of sustainability!” Our man immediately breaks into a cold sweat…
This parable highlights the problems of making important life decisions based upon inherently unpredictable outcomes. It also highlights the folly of applying probability software to retirement planning analysis. The assignment of so-called “probabilities of success” to investor portfolios implies that the application has predictive ability. In truth, even the most popular retirement planning software used by financial professionals is far, far from a crystal ball.
THE PROBLEM WITH PROBABILITIES
The failings of probability-based retirement software, particularly those that apply Monte Carlo simulation techniques, are reasonably well known in professional circles. One of the first academic papers to raise the issue was a 2006 article written by renowned retirement researcher and York University of Toronto Professor Moshe Milevsky entitled, “ Will the True Monte Carlo Number, Please Stand Up?” As Professor Milevsky notes in the introduction,
Of course, as most investment advisors have known for years, a retirement number – if it actually exists – is vague and imprecise, since it depends on many economic unknowns, especially future equity market returns. After all, this “number” must be invested somewhere in order to produce income – and the portfolio return process is inherently random.
In addition to the unpredictability of future returns, Milevsky goes on to document how “probabilities” produced by popular retirement software applications vary from one app to the next, depending upon the applications’ internal assumptions and design parameters.
Anohter academic study study published last month titled, The Efficacy of Publicly-Available Retirement Planning Tools, concluded that “the advice provided from a majority of these tools is extremely misleading to households.”
Ironically, as a baseline reference point for the study, the authors compared the output from 36 free consumer retirement planning applications to a type of professional app that has, itself, been challenged for producing overly optimistic and unreliable results. These publications have caused some to question whether retirement planning software offers any real value to consumers at all. So what’s the consumer to do?
STRESS –TESTING VS. PREDICTING
Both the weatherman and the financial advisor use probability software to determine the likelihood of positive outcomes. Perhaps a better way to approach retirement planning is from a glass half-empty perspective. Instead of attempting to predict “probabilities of success,” the role of good retirement software should not be as a crystal ball, but rather as a tool for showing consumers how their portfolios might hold up under adverse conditions.
What consumers really need to know is not how they may fare if things go well, but what will happen to them if the aforementioned 10% probability of rain turns into a 100% probability of a thunderstorm. Speaking less metaphorically, consumers desperately need and want to know, “If things go badly in the investment markets, will I still be okay?”
Traditionally, historical back-testing software has been used for this purpose. By entering one’s retirement profile into a back-testing app, a consumer can test how his portfolio may have fared if he retired prior to previous bear markets. While such information is useful and interesting to consumers, back-testing also has significant limitations.
Specifically, past returns are unlikely to be repeated in the exact same sequence again, and there is absolutely no guarantee that future returns may not be worse than historical experience. Further, suppose a person wanted to test how his portfolio might hold up over a 30 year retirement horizon if he had retired at the end of 1999 (i.e., just before the 2000-2002 and 2007-2009 bear markets). Since we are only in 2016, it is obviously not possible to back test the future!
BOOTSTRAPPING – A BETTER RETIREMENT PLANNING MOUSETRAP?
One solution to the limitations of back-testing is to apply a Monte Carlo simulation technique called bootstrapping. While the simulation engine under the hood of most Monte Carlo-based retirement apps requires the program designer to make assumptions about expected mean rates of return and volatility for various asset classes, bootstrapping requires no such assumptions. Simulations are produced instead by randomly sampling historical returns.
If enough simulations are generated (typically a minimum of 5,000), the median result may be expected to be in line with the historical averages. By considering the range of results below the median, bootstrapping programs may illustrate scenarios representing below average investment returns, with the Value at Risk (VaR) statistics (bottom 1%, 5%, and 10% results) representing scenarios that may be as bad or worse than the historical record.
To illustrate this concept by example, the following table presents the bootstrapping simulation results for a 65 year old investor with a 25 year retirement horizon, a $1,000,000 initial portfolio value and a 70:30 stock:bond retirement allocation. In this example, the investor requires a $50,000 (5%) initial withdrawal rate and a 3% annual cost-of-living increase. He estimates his annual investment expense at 1% and has stated that he expects to withdraw proportionately from each asset class each year and rebalance to maintain his 70:30 allocation.
Remaining Balance…
| 80% | $1,212,308 | $1,358,150 | $1,439,849 | $1,513,529 | $1,483,135 |
| 60% | $1,091,368 | $1,127,568 | $1,108,806 | $1,004,560 | $796,054 |
| Median | $1,038,653 | $1,040,195 | $977,559 | $833,761 | $535,366 |
| 40% | $988,481 | $958,058 | $864,393 | $671,558 | $316,435 |
| 20% | $886,511 | $789,407 | $615,265 | $329,948 | $0 |
| 10% | $818,595 | $685,467 | $466,587 | $129,937 | $0 |
| 5% | $763,903 | $601,042 | $353,836 | $0 | $0 |
| 1% | $675,021 | $472,024 | $190,510 | $0 | $0 |
| Worst | $545,910 | $259,541 | $0 | $0 | $0 |
Simulation results generated by Nest Egg Guru.
By focusing on the bottom of half of the results (particularly, the VaR results) and displaying the simulation range in five year increments over the illustrated time period, the consumer can gain a much more tangible sense of whether and how long his savings may last. What’s more, by presenting the data in this format, it is easy for the consumer to then test how changing factors that are within his control (spending rate, withdrawal strategy, asset allocation, expenses, etc.) may impact the outcomes.
SUMMARY
The objective of this essay has been to highlight the flaws and limitations of probabilistic/ predictive retirement analysis and to suggest that a better, more realistic methodology may be to approach retirement planning from the perspective of stress-testing. To be clear, there is absolutely nothing predictive in the simulation results in the accompanying table. The percentiles illustrated should not be viewed as probabilities. Instead, the bottom half of the results merely represent potential scenarios that may be used to give consumers a clearer picture of what may happen if things go badly.
While bootstrapping offers a neat way to illustrate this data, it is not without its flaws and limitations too. In this example, bootstrapping was applied only to historical stock market data from 1970-2014. The bond portion of the portfolio was assumed to be a constant 2% per year, which reasonably reflects the return an investor might realistically earn today on a 5 year CD or 10-year treasury. The fact that bootstrapping simulations were not applied to historical bond data reflects a limitation with most retirement apps today in that the return environment on bonds (and cash) today are near the bottom of the historic extreme. As a result, any Monte Carlo application that is generating randomized returns based on either mean historical returns or random sampling of historical returns, may be overoptimistically biased.
Consumers should also be aware that, regardless of a retirement calculator’s underlying methodology, the design of the app may also have an impact on its applicability and reliability. For instance, applications that fail to properly account for the impact of investment expenses or inflation, or, as noted, the current low interest rate on bonds, may be overly optimistic in their results reporting.
Alas, with any retirement planning app, the devil is in the details. Consumers and advisors alike would do well to take the time to understand the assumptions and limitations inherent in any retirement planning application.
“A version of this article was originally published on Nerd Wallet and has also appeared on NASDAQ.com and NewsOK.”
John H. Robinson is the owner of Financial Planning Hawaii and a co-founder of Nest Egg Guru, a retirement planning software application for financial professionals.