Dealing with Misleading Resesarch

Let me start with an old story -- a cartoon that used to hang on my office door.  There is a Congressman sitting at his desk saying "My mail is running 3-1 against the proposal."  The cartoon shows two "stacks" of envelopes -- one with three and the other with one.

The realm of probability, sampling, and statistical inference is probably the weakest suit for the supposed Wall Street experts.  Here is a current case of some importance to investors.

Background

Getting a job at Morgan Stanley is exceedingly difficult.  They seem to understand the secret hiring rules that I use at NewArc:

1. Find the very smartest people.   You cannot coach speed.
2. Hire those who want to be team players and who create a positive environment.
3. Rely less on knowledge and more on coach-ability.  I know what they need to know, so the question is whether they are willing to learn.

Morgan Stanley uses non-traditional criteria involving the ability to calculate risk/reward rapidly in game situations.  So how could their team get this research so wrong?

The Research Finding

The key question is whether a peak in the the Economic Cycle Research Institute (ECRI) leading indicators suggests a decline in stocks over the next six months.

The Morgan Stanley study, highlighted by the Pragmatic Capitalist via Abnormal Returns (thanks to both valued sources for highlighting the question) is a bearish scream:

Based on the study, which goes back over 30 years, the equity markets

are down 80% of the time 6 months after a peak in the WLI growth rate.

My Observations

The errors in this approach are manifest.  I expect that regular readers had the radar on high without any help.

• You should not use percentages  when dealing with a small number of cases.  It implies more data.  Better to say "in five of six cases."
• This should be a case study analysis.  You only have a few instances and they are all very different.  You do not have enough cases for quantitative analysis.  This is elementary research design in any university curriculum.
  
• The frequency analysis does not really support the conclusion.  I looked up the cases, which took only a few minutes.  One is positive, two are neutral and three are negative.  It is not 80% unless you jigger the timing to fit a pre-conceived notion.
• The ECRI forecasts the economy, not the stock market.  The ECRI is adamant about future economic strength.  It all comes down to whether investors believe that the current market is priced for a brisk rebound.
  
• The most recent case is - -quite obviously -- without precedent.  The chart below makes this apparent.

This is a classic.  There is no precedent.  You have to reach back 35 years to find anything even close.  How could anyone think that this instance could be forecast from the preceding six cases?

Conclusions

The most obvious conclusion is that one should accord some respect to the ECRI analysis of their own data.

Equally important is the analysis of recessions and post-recessions.  There is an overwhelming desire to draw inferences from slender reeds.

When there is little data, the wise researcher accepts reality.

What is needed is a comparative case study of recessions. If a grad student came to a prof with a proposal with this problem in mind, that would be the only acceptable methodology.

Meanwhile, I encourage young readers to apply for a job at Morgan Stanley!

They need some help.