Can you Predict Stock Market Returns with Short Interest?

Can you Predict Stock Market Returns with Short Interest?

February 23, 2015 Research Insights, $SPY
Print Friendly
(Last Updated On: January 18, 2017)

Short Interest and Aggregate Market Returns

Abstract:

We show that aggregate short interest is one of the strongest known predictors of the equity risk premium. High aggregate short interest predicts lower future equity returns at monthly, quarterly, semi-annual, and annual horizons. In addition, aggregate short interest outperforms a host of popular predictors from the extant literature both in sample and out of sample. We investigate the source of this predictability and find that high aggregate short interest anticipates negative future industrial production growth, aggregate earnings growth, and cash flow innovations. Overall, our results are consistent with theoretical models of informed trading by short sellers. In other words, our results suggest that short sellers are informed traders who are able to anticipate changes in macroeconomic conditions and associated changes in equity market returns.

Alpha Highlight:

Goyal and Welch (2008) take a comprehensive look at various popular market predictor variables from the existing literature, including such favorites as dividend-price ratios (yield), dividend-earnings ratios (payout ratios), book-to-market ratio, volatility and others. In reviewing the evidence, both in-sample and out-of-sample, Goyal and Welch conclude that none of these variables seems to robust.

Yet hope springs eternal in the market-timing literature.

This paper argues that there is a powerful market indicator, which performs better than 14 popular predictor variables considered in Goyal and Welch (2008). This market premium indicator is aggregate “Short Interest”:

  • Short interest (defined by investopedia):  The total number of shares of a particular stock that have been sold short by investors but have not been covered or closed out. This can be expressed as a number or as a percentage… When expressed as a percentage, short interest is the number of shorted shares divided by the number of shares outstanding.

This paper indicates that increases in aggregate short interest are associated with lower future market returns as well as declines in future economic activity.

Short Interest Index (SII):

To test short interest’s ability to predict the market, the authors take samples of monthly firm-level aggregate short interest data from 1973 to 2012 from Compustat. The paper generates a variable called, “Short Interest Index (SII)” to measure overall market pessimism.

  • Short Interest Index (SII): First, the authors detrend the raw aggregate short interest series to exclude this secular change factor; Then, the authors standardize this detrended series. Thus, SII variable is the standardized, detrended log aggregate short interest.
Short Interest and Aggregate Market Returns_1
The results are hypothetical results and are NOT an indicator of future results and do NOT represent returns that any investor actually attained. Indexes are unmanaged, do not reflect management or trading fees, and one cannot invest directly in an index. Additional information regarding the construction of these results is available upon request.

Panel A shows the raw log aggregate short interest, which has an upward trend; and panel B shows the detrended series, which appears fairly stable over time. Notice in particular that the SII increases substantially in the middle of 2008, just before the worst part of the financial crisis, and falls sharply during the later stages of the crisis. This provides a hint that there is potentially some correlation between short interest and market performance.

The results show that while the 14 market predictors exhibit strong correlations with each other, SII is largely unrelated to previous predictive variables! That is, the SII variable seems to contain information that is substantially different from other popular market predictors.

To compare the predictive powers of these indicators, the authors run regressions on SII and the 14 predictors in Goyal and Welch. The other 14 popular predictors are listed below:

  • Log dividend-price ratio (DP): difference between the log of dividends and the log of prices.
  • Log dividend yield (DY): difference between the log of dividends and the log of lagged prices.
  • Log earnings-price ratio (EP): difference between the log of earnings and the log of prices.
  • Log dividend-payout ratio (DE): difference between the log of dividends and the log of earnings.
  • Excess stock return volatility (RVOL): computed using a 12-month moving standard deviation estimator.
  • Book-to-market ratio (BM): the ratio of book value to market value for DJIA.
  • Net equity expansion (NTIS): the ratio of 12-month moving sums of net issues by NYSE listed stocks divided by the total end-0f-year market capitalization of NYSE stocks.
  • Treasury bill rate (TBL): interest rate on a 3-month treasury bill.
  • Long-term yield (LTY): yield on long-term US government bonds.
  • Long-term return (LTR): return on long-term US government bonds.
  • Term Spread (TMS): difference between the long term yield on government bonds and treasury bills.
  • Default yield spread (DFY): difference between BAA and AAA-rated corporate bond yields.
  • Default return spread (DFR): difference between long-term corporate bond and long-term government bond returns.
  • Inflation (INFL): calculated from the CPI for all urban consumers.

Key Findings:

In-sample test: results show that increases in the SII strongly predict lower future S&P 500 premiums.

  • A one-standard-deviation increase in the SII corresponds to an 47 basis point decrease in future monthly premiums.
  • Predictive power is significant: R-square of SII regression is over 1% at the monthly predict horizon and is almost 12% at the annual predict horizon. (Thompson 2008 and Zhou 2010 argue that a monthly R-square statistic of approximately 0.5% represents economic significance)
  • Among the remaining 14 predictors, only RVOL, TMS, and DFR show t-stat significance.
Short Interest and Aggregate Market Returns_2
The results are hypothetical results and are NOT an indicator of future results and do NOT represent returns that any investor actually attained. Indexes are unmanaged, do not reflect management or trading fees, and one cannot invest directly in an index. Additional information regarding the construction of these results is available upon request.

Asset Allocation: Part 4 in this paper measures the economic significance of SII for asset allocation. It considers a mean-variance investor who following the SII predictive strategy to determine equity allocations.

Let’s see an example of how SII prediction works during ’08 financial crisis.

Panel A shows that during the early stage of great recession, the SII portfolio increases its short position. Then it subsequently increases its long position from 2009 to 2012, riding the recovery rally. Panel B illustrates that cumulative wealth grows substantially from the end of 2007 to the end of 2012!

Short Interest and Aggregate Market Returns_3
The results are hypothetical results and are NOT an indicator of future results and do NOT represent returns that any investor actually attained. Indexes are unmanaged, do not reflect management or trading fees, and one cannot invest directly in an index. Additional information regarding the construction of these results is available upon request.

SII is associated with macroeconomic conditions.

Paper further points out that a higher SII is associated with lower future industrial production and aggregate real earning growth. Also, a higher SII strongly predicts lower subsequent cash flow shocks. The authors believes that these relationships suggest that short sellers have some ability to forecast future slowdowns in economic activity.

Conclusions:

We have done extensive research around the question of whether short interest can be a useful tool for predicting equity market premiums. Our conclusion is that this measure–like many others–is not robust to small changes. That said, the indicator is interesting and worth a look for financial economists.

 


Note: This site provides NO information on our value investing ETFs or our momentum investing ETFs. Please refer to this site.


Join thousands of other readers and subscribe to our blog.


Please remember that past performance is not an indicator of future results. Please read our full disclosures. The views and opinions expressed herein are those of the author and do not necessarily reflect the views of Alpha Architect, its affiliates or its employees. This material has been provided to you solely for information and educational purposes and does not constitute an offer or solicitation of an offer or any advice or recommendation to purchase any securities or other financial instruments and may not be construed as such. The factual information set forth herein has been obtained or derived from sources believed by the author and Alpha Architect to be reliable but it is not necessarily all-inclusive and is not guaranteed as to its accuracy and is not to be regarded as a representation or warranty, express or implied, as to the information’s accuracy or completeness, nor should the attached information serve as the basis of any investment decision. No part of this material may be reproduced in any form, or referred to in any other publication, without express written permission from Alpha Architect.


Definitions of common statistics used in our analysis are available here (towards the bottom)




About the Author

Wesley R. Gray, Ph.D.

After serving as a Captain in the United States Marine Corps, Dr. Gray earned a PhD, and worked as a finance professor at Drexel University. Dr. Gray’s interest in bridging the research gap between academia and industry led him to found Alpha Architect, an asset management that delivers affordable active exposures for tax-sensitive investors. Dr. Gray has published four books and a number of academic articles. Wes is a regular contributor to multiple industry outlets, to include the following: Wall Street Journal, Forbes, ETF.com, and the CFA Institute. Dr. Gray earned an MBA and a PhD in finance from the University of Chicago and graduated magna cum laude with a BS from The Wharton School of the University of Pennsylvania.


  • Ryan

    I’m concerned by the linear de-trending used to produce the SII indicator. It makes sense historically, since short interest increased over time as markets became more sophisticated. Today, however, there are no structural reasons why short interest should increase indefinitely.

    Frankly it looks like short interest decoupled from the linear trend ten years ago. I’m not convinced this indicator is useful going forward since the detrended series will be negative for the next 40 years.

  • can’t say I disagree…but an interesting concept nonetheless

  • dan knight

    growth in hedge funds generally lead to more short selling – convert arb, L/S, etc. more hedging. dubious of robustness of the indicator

  • Drew Dickson

    The Cohen, Deither, and Malloy (2005) is one of the best on this topic. Rigorous, and methodologically accurate. Their illuminating trick was to identify shifts in supply and demand curves separately – thus eliminating the noise of dividend grossing, convertible arbitrage, and other non-informational trading activities – and revealing that there indeed is fundamental informational value in changes in short interest, especially when that change is motivated by the demand side. It basically suggests that hedge funds or shorts are less behaviorally biased than the average investor, and are able to process information more accurately (or, of course, that they have “inside” information).

    Bottom line is this (in my view) – even if people don’t like a stock, or it has fallen precipitously already; if borrow availability is still decreasing, and this is accompanied by higher borrow costs, then it doesn’t pay to be a contrarian. The knife is still falling.

  • Drew, nice highlight. Here is a write up on CDM 2005 we did a while ago: http://www.alphaarchitect.com/blog/2011/06/02/follow-the-short-sell-demand/

  • Drew Dickson

    Thx Wesley. Think the CDM results were fairly robust, even outside the tech bubble window – but you are right, the cost of implementation is high – and not just stock loan fees and transaction costs, but the price-making nature of exiting an illiquid, short loser (while the basket itself might be a winner, a few of them are bound to suck the life out of you). You just can’t capture that “what if” in an analysis like that, because of the recursive nature of the argument. The painful squeeze might never happen, without you making it happen.

  • Eric

    Where does one find the current aggregate short interest level of the stock market?

  • Probably BBERG would be best. Here are some quick stats on NYSE listed stocks.

  • Eric

    for those without a bloomberg terminal – is there any place to go periodically to see short interest levels?

  • not that I know of