Performance Analysis Introduction Are you winning or losing? Why? Performance measurement will answer the first question and can help to answer the second. A sophisticated performance analysis system can provide valuable feedback for an active manager. The manager can tie his or her decisions to outcomes and identify success, failure, and possible improvements. Performance analysis has evolved from the days when the performance goals were vague, if not primitive: "Don't steal any money!" "Don't lose any money!" "Do a good job!" "Do as well as I could at the bank!" "Beat the market!" "Justify your active management fee!" The goal of performance analysis is to distinguish skilled from unskilled investment managers. Simple cross-sectional comparisons of returns can distinguish winners from losers. Time series analysis of the returns can start to separate skill from luck, by measuring return and risk. Time series analysis of returns and portfolio holdings can go the farthest toward analyzing where the manager has skill: what bets have paid off and what bets haven't. The manager's skill ex post should lie along dimensions promised ex ante.
Page 478 The drive for sophisticated performance analysis systems has come from the owners of funds. Investment managers have, on the whole, fought an unsuccessful rear-guard action against the advance of performance analysis. This is understandable: The truly poor managers are afraid, the unlucky managers will be unjustly condemned, and the new managers have no track record. Only the skilled (or lucky) managers are enthusiastic. Of course, these owners of funds make several key assumptions in using performance analysis: that skillful active management is possible; that skill is an inherent quality that persists over time; that statistically abnormal returns are a measure of skill; and that skillful managers identified in one period will show up as skillful in the next period. The evidence here is mixed, as we will discuss in this chapter and Chap. 20. Performance analysis is useful not only for fund owners, but also for investment managers, who can use performance analysis to monitor and improve the investment process. The manager can make sure that the active positions in the portfolio are compensated, and that there have been no unnecessary risks in the portfolio. Performance analysis can, ex post, help the manager avoid two major pitfalls in implementing an active strategy. The first is incidental risk: Managers may like growth stocks, for example, without being aware that growth stocks are concentrated in certain industry groups and concentrated in the group of stocks with higher volatility. The second pitfall is incremental decision making. A portfolio based on a sequence of individual asset decisions, each of them wise on the surface, can soon become much more risky than the portfolio manager intended. Risk analysis can diagnose these problems ex ante. Performance analysis can identify them ex post. The lessons of this chapter are: • The goal of performance analysis is to separate skill from luck. Cross-sectional comparisons are not up to this job. • Returns-based performance analysis is the simplest method for analyzing both return and risk, and distinguishing skill from luck. • Portfolio-based performance analysis is the most sophisticated approach to distinguishing skill and luck along many different dimensions.
Page 479 • Performance analysis is most valuable to the sponsor (client) when there is an ex ante agreement on the manager's goals and an indication of how the manager intends to meet those goals. • Performance analysis is valuable to the manager in that it lets the manager see which active management decisions are compensated and which are not. Skill and Luck The fundamental goal of performance analysis is to separate skill from luck. But, how do you tell them apart? In a population of 1000 investment managers, about 5 percent, or 50, should have exceptional performance by chance alone. None of the successful managers will admit to being lucky; all of the unsuccessful managers will cite bad luck. We present a facetious analysis of the market in Fig. 17.1. We have divided the managers along the dimensions of skill and luck. Those with both skill and luck are blessed. They deserve to thrive, Figure 17.1 Skill and luck.
Page 480 and they will. Those with neither skill nor luck are doomed. Natural selection is cruel but just. But what about the two other categories? Those managers with skill but no luck are forlorn. Their historical performance will not reflect their true skill. And, finally, there is the fourth category. These managers have luck without skill. We call them the insufferable. Most managers can easily think of someone else they believe is in this category. Fortunately or unfortunately, we observe only the combination of skill and luck. Both the blessed and the insufferable will show up with positive return histories. The challenge is to separate the two groups. The simple existence of positive returns does not prove skill. Almost half of all roulette players achieve positive returns each spin of the wheel, but over time they all lose. The existence of very large positive returns also does not prove skill. How much risk was taken on in generating that return? Performance analysis will involve comparing ex post returns to ex ante risk in a statistically rigorous way. Chapter 12 included brief mentions of the standard error of the information ratio. The approximate result is where Y measures the number of years of observation.1 The number of years enters because we define the information ratio as an annualized statistic. Equation (17.1) implies that to determine with 95 percent confidence (t statistic = 2) that a manager belongs in the top quartile (IR = 0.5) will require 16 years of observations.2 It is a fact of investment management life that proof of investment prowess will remain elusive. 1This assumes that all the error arises from the estimated mean residual return. If we also account for the error arising from the estimated residual risk, we find where Δt is, e.g., 1/12 if we observe monthly returns. See Problem 3 for more details. 2See Problem 4 for a discussion of why changing the information ratio from an annualized number to a monthly number does not improve our ability to statistically verify investment performance.
Page 481 We can view the basic predicament from another angle. What if you are truly a top quartile manager, with an information ratio of 0.5? What is the probability that your monthly, quarterly, annual returns are positive? Figure 17.2 shows the result as the horizon varies. Over one month, you have only a 56% chance of positive realized alpha. Over a 5-year horizon, this rises to 87%. This implies that over the standard 5-year horizon, 13% of skilled managers will have negative realized alphas. Given the horizons for careers, and for ideas in the investment business, luck will always play a role. The efficient markets hypothesis suggests that active managers have no skill. In its strong form, the hypothesis states that all currently known information is already reflected in security prices. Since all information is already in the prices, no additional information is available to active managers to use in generating exceptional returns. Active returns are completely random. The semistrong version states that all publicly available information is already reflected in security prices. Active management skill is really insider trading! The weak form of the hypothesis claims only that all previous price-based information is contained in current prices. This rules out technical analysis as skilled active management, but would allow for skillful active management based on fundamental and economic analysis. There have also been many academic studies of active managers' performance. These studies have focused on three related questions: • Does the average active manager outperform the benchmark? • Do the top managers have skill? • Does positive performance persist from period to period?