Beta Backtesting
Beta, or beta exposure, is a prediction about how much a fund will make or lose, on average, based on a market move. For example, if your firm’s beta exposure to the market is 24%, then we would predict that your firm would make 2.40% if the market was up 10%, and lose 0.24% if the market was down 1%.
We can measure the beta and beta exposure to any risk factor. In what follows we are going to talk about the market beta and market returns, but the same logic could be applied to betas for any factors.
On any given day, your firm may make more or less than what was expected based on its beta and the move in the market. For example, if your beta was 24% and the market was up 1%, we would expect you to make 0.24% due to the market move. If you made 0.64%, the 0.40% difference would be attributed to alpha. Based on one day, then, it is impossible to say if the forecasted beta was accurate or not.
Over time, your firm will generate positive alpha on some days and negative alpha on others. Hedge funds that perform well will add more positive alpha than negative alpha in the long run. To test beta, then, we don’t want to look at the sum of the alphas. What we want to look at is the correlation of the alpha with market returns. If the beta forecast is accurate then the daily alpha time series should be uncorrelated with the market.
To see why this is the case, imagine that the forecasted beta was always too high. If this was the case, then the model would expect you to generate more P&L on days when the market was up and generate greater losses on days when the market was down. Because of this, you would be more likely to generate negative alpha on days when the market was up, and positive alpha on days when the market was down. The total alpha could be positive or negative; the bias would show up in the correlation.
If the alpha is uncorrelated with the market returns then a graph of alpha versus the market returns should have zero slope. Alpha should not show any tendency to be higher or lower on days when the market is higher or lower. It might sound a bit confusing, but if our beta forecast is accurate then the beta of the alpha series to the market should be zero.
The application actually reports two tests, “Slope Test (t)”, and “Slope Test (t-1)”. If your firm trades securities in other time zones, or illiquid securities, then the prices of those securities may react with a lag to the market return. For example, the S&P 500 may be up today due to positive news, but the impact on your Asia holdings may not be seen until the following day. Our default beta calculations takes into account this potential lag, and we test that component of beta it as well.
The accompanying graph show the alpha series versus the market return on the same day. For most firms this is very close to “Slope Test (t)”, but it is not exactly the same. The real tests requires a multivariate regression, which, in this case, would require three dimensions to display.
Each test has a value for the slope and a probability. If the value is positive this indicates that your alpha was positively correlated with market returns. This would happen if the beta forecast tended to be too low. The opposite is true when the value is negative.
Positive slope => Realized beta was higher than what was forecast
Negative slope => Realized beta was lower than what was forecast
Of course, the slope will rarely be exactly zero. What we want to know is: is the slope significantly different than zero? The probability associate with each slope is the p-value from the regression. It is the probability of observing a given slope value this different than zero by chance. This number is always less than 50%. The closer the number is to 0% the more statistically significant it is. If the value is less than 5% then the value should be investigated further.
Similar to the VaR backtest, you are more likely to see statistically significant slopes if you look at lots of positions or sub-portfolios separately. Also, just as with the VaR forecast, the beta forecast assumes no trading. A statistically significant slope could indicate bad data, or an inappropriate model, but it could also come about naturally as the result of high trading volume.
Further Reading
For more on the importance of lagged data in financial time series see our white paper, Asynchronous Data and Serial Correlation in Financial Time Series.