When investing in non-index mutual funds, investors make two critical assumptions: 1) that skillful managers exist, and 2) that they have the ability to recognize them. If an investor is not willing to make these two assumptions, they should invest in non-active funds like index funds or exchange traded funds (ETFs). Mutual fund analysis, both qualitative and quantitative, attempts to identify skillful active managers. Qualitative analysis looks at factors such as the background and experience of the manager and the mutual fund company. Here, we look only at the quantitative factors such as manager performance, style, style consistency, risk, risk-adjusted performance, etc.

What is the best way to analyze, and ultimately select, mutual funds?

Financial journalists are not equipped to analyze mutual funds. In most cases they are simply reporting the performance figures they received from the managers themselves or the marketing/public relations people. Mutual fund rating services are good data collectors but lack any real sophistication in fund analysis. These services are oriented toward the retail fund investor. Consequently sophisticated advisors, plan sponsors and consultants must perform their own mutual fund analysis.

The two biggest mistakes in quantitative mutual fund analysis are improper benchmarking and end point bias. How can you avoid these mistakes?

Benchmarking
The most common error made when measuring a manager’s performance is the selection of an improper benchmark. Morningstar’s star ratings, for example, are based on fund’s performance relative to a broad group of fund returns, as opposed to a more specific benchmark that reflects the manager's true style. Because of this, on February 28, 2000, at the very peak of the growth stock bubble, most of Morningstar’s five star funds were growth funds while there were no five star value funds. Two years later, after the value funds did well and the growth funds crashed, most of the five star funds were value funds.¹ Due to the importance of proper benchmarking, we devote a special section to it (see Benchmarks).

End Point Bias
The other common mistake made in performance analysis is called “end point bias.” Most of the funds recommended by various financial publications are ones that recently performed well. When looking at cumulative statistics, recent performance above the benchmark creates the illusion that the fund has consistently outperformed. Cumulative statistics are calculated through the most recent time period. Annualized return for one, three, five, and seven years, for example, is often used to evaluate mutual funds. Notice that the most recent year is included in all of these periods. Due to the nature of these statistics, recent performance often “hides” past performance.

Here is an example. The September 15, 2003 Forbes magazine heralded the Mairs & Power Growth fund as one of the three best funds to own based on its long term record. In Figure 1 the long term annualized returns for this fund look quite good.

Figure 1: Mutual Fund Analysis - Long-Term Annualized Returns for Mairs & Power Growth Fund

Because this fund outperformed its benchmark (the S&P 500 is a reasonable benchmark for this fund) for 2, 3, 5, 7, 10, 15, 20, and 25 year periods, one would think that the fund is a consistently good performer. Now look at Figure 2 below. The red and green shaded area at the bottom of the performance graph shows the cumulative return relative to the benchmark. If you had purchased this fund twenty five years ago you would have spent all but the last couple of years below your benchmark. It has only been the very good performance in the last few years that give it the high annualized rates of return found in Figure 1. This is what we mean by an “end point bias.” We could also call it the broken clock syndrome (a broken clock will be right twice a day). Similarly, if a manager has been managing money for twenty five years, even with no skill, there is likely to be several years of good performance. If your end point (the date on which your analysis ends) is particularly good, cumulative statistics may create the illusion of consistently good performance.

Figure 2: Mutual Fund Analysis - Example of End-Point Bias

Would Forbes have recommended this fund three years earlier? We doubt it. Figure 3 shows us that through February 2000 the fund had under performed its benchmark by 831 percentage points.

Figure 3:Mutual Fund Analysis - Mairs & Power Growth Fund Performance up to 2000

Figure 4 shows that the fund under performed the benchmark for every one of the periods shown. The only difference between Figure 1 and Figure 4 is the end point.

Figure 4: Mutual Fund Analysis - Mairs & Power Growth Fund Returns for Specific Periods

One way to avoid end point bias is to look at rolling time periods. Figure 5 shows rolling three year periods of excess returns. Here you can see an almost equal amount of time underperforming and outperforming the benchmark. To have confidence that a manager is skillful and that the skill will likely result in beating the benchmark in the future, we prefer managers that consistently outperform.

Figure 5:Mutual Fund Analysis - Mairs & Power Growth Fund Excess Return

Let’s take a look at a consistent outperformer. Figure 6 shows the performance of the Fidelity Low Priced Stock Fund. It outperformed its benchmark by 6.58% annually!

Figure 6:Mutual Fund Analysis - Fidelity Low Priced Stock Fund Performance

Figure 7 shows the same three year rolling excess return chart. Notice that there weren’t any three year periods where the fund underperformed its benchmark.

Figure 7:Mutual Fund Analysis - Fidelity Low Priced Stock Fund Excess Return

The bottom panel of Figure 6 contains some useful statistics. One statistical measure of consistency is tracking error, which is the volatility (standard deviation) of excess return. All things equal, the less volatile the excess returns the greater the chance the manager is skillful rather than lucky. Tracking error is used to calculate a risk-adjusted measure of performance called the “information ratio.” The information ratio is the annualized excess return divided by the tracking error. The information ratio for the Fidelity fund is a very high at 1.48. What are the chances that a manager could have achieved this information ratio by being lucky? Part of the answer will depend on how long she achieves a high information ratio. The longer the good performance persists, the less chance of luck and the more chance of skill. StyleADVISOR’s “Significance Level” statistic measures the probability of luck vs. skill. To have confidence that the manager was skillful and not just lucky the significance level should be at least 95%. For the Fidelity Fund it is 100% (see the bottom panel of Figure 6).

Fortunately there are sophisticated software programs like Zephyr’s StyleADVISOR to help investors perform useful and accurate mutual fund analysis. The most important first step is to select the proper benchmark. If that is not done all of the fancy statistics we have discussed will be meaningless. Investors can accurately measure a manager’s performance, evaluate the consistency of the performance, and determine the probability that the manager’s performance is the result of skill. Such an analysis dramatically improves the likelihood that our second assumption – our ability to pick skillful managers – is true and in doing so that our selections may lead to superior future performance.

End Notes:

¹A recent change in Morningstar’s methodology narrowed the peer group comparisons. Unfortunately, peer groups have none of the characteristics of a good benchmark.