Investors often try to gauge a manager’s skill by comparing the manager’s performance to the performance of a group of similar managers. This is typically called a “universe” or “peer group” analysis. Many consider a universe or universe composite to be a good benchmark for a manager. In fact, manager universes do not have the qualities that make a good benchmark. They are not investable, they are not specified in advance, and they are usually too broadly defined. For more information about what makes a good benchmark, see Benchmarks.

Universes also suffer from what is known as “survivor bias” because many of the poor performing managers drop out of the universe. Poorly performing mutual funds, for example, are often merged with more successful funds. When this happens only the successful fund’s track record is maintained, so the poor performance is not represented in any universe that includes the fund. Once a product no longer exists, for whatever reason, it is dropped out of the universe.

Despite these limitations, peer group comparisons continue to be popular with investors. Zephyr Associates developed an improved way of presenting universe groups. Figure 1 is a peer group analysis using the standard “floating bar” chart. Figure 2 is a graph that Zephyr developed years ago to provide more information. Notice that the floating bar graph shows where Fidelity Magellan ranked in the universe for only six time periods. The bottom graph shows where Magellan ranked every month for the last twenty five years. Looking at the top graph one might conclude that Magellan had never been in the bottom quartile of the universe. With the bottom graph we can see that there are two such periods. Managers may not want to show all the periods, so for them the floating bar graph might be best. Sponsors, consultants, and advisors should always use the more comprehensive graph where no time periods can be hidden.

Figure 1

Figure 2

Another thing to be aware of is the difference between cumulative time periods (last one year, three years, five years etc) and rolling time periods. This relates to “end point bias,” which is discussed in Mutual Fund Analysis. When a manager’s recent performance is good, cumulative analysis tends to make its longer term performance look good, regardless of past performance. Beware of this illusion. Figure 3 shows the cumulative rank of RS Emerging Growth Fund Ending December 1999. This tech heavy fund was up 182% in 1999. That means that if you owned that fund at the end of 1999 and purchased it anytime in the last twelve years your fund’s performance would have put it in the top one percentile of all small cap growth funds. This does not mean that the fund was consistently in the top of the universe. Figure 4 shows a rolling three year window. Here we see that this fund was in the bottom quartile of the universe for a number of three year periods.

Figure 3

Figure 4

To give you an idea of how important 1999 was for the high cumulative rank we see in Figure 3, look at Figures 5 through 8 below. In all four cases we are looking at cumulative rankings (last one, three year etc.) Only Figure 5 includes 1999. Figure 6 ends December 1998; Figure 7 ends December 1997; and, Figure 8 ends December 1996. In the first example we could see that rolling returns removed the illusion of a consistent high ranking. But in this second example we see that even the cumulative returns were erratic if we remove the one standout ending year, 1999. I believe that the great majority of professional investors and advisors get fooled into thinking that managers were consistently good performers by this kind of analysis.

Figure 5

Figure 6

Figure 7

Figure 8

There is no reason to limit peer group comparisons to returns. StyleADVISOR generates 27 statistics that can be used to compare a manager to his peers. Figures 9 through 12 provide several examples. These graphs can be viewed for cumulative time periods (last one year, three years, five years etc.) or for various rolling periods. You can display rank, as shown in these four graphs, or the raw statistics (not shown).

Figure 9

Figure 10

Figure 11

Figure 12

A discussion of peer groups wouldn’t be complete without some discussion about universe construction. We believe that the Zephyr universe construction methodology is the best in the business. Zephyr does style analysis on all the domestic equity separate account managers and mutual funds as well as domestic fixed income managers. (For a complete discussion on how these are constructed see Zephyr Universes). Our methodology creates universes of managers with similar styles. The broader the universes, the greater the style difference among managers. The smaller the universe, the less significant the results. Look at Figure 13. This is the mid-cap growth mutual fund universe. Managers that plot near the border of two or more universes may share behavioral characteristics with managers in the bordering universe(s). The Legg Mason fund could almost be classified as a large cap core fund. If the time period used for an analysis favored value this fund would most likely look good relative to its “peers” despite the manager’s skill or lack thereof. If growth is in favor the opposite would be true. One way to minimize this distortion is to create a custom universe where the manager analyzed is placed in the center of the universe.

Figure 13

Figure 14 is Zephyr’s unique scan search graph. Here we selected all US equity funds with at least a five year history on a style map. Next, we drew a circle around the funds that fell close to Legg Mason. We can make this circle as large or small as we like depending on how large we want the universe to be. Those funds represented by the red dots will be designated as a custom universe for Legg Mason.

Figure 14

Peer group and manager universe comparisons are a poor substitute for a good benchmark. Nevertheless they seem to be extremely popular among investors and advisors. Zephyr has redesigned the typical peer group analysis to give investors more information and better understanding of how a manager has ranked against his peers. We have also strived to create better universes and to give our clients the ability to create better custom peer groups.