Sentiment arguments abound

As I am relatively new to the market, with this being my experience with a downtrend. It is also my first exposure to people relying on contrarian market sentiment arguments for calling bottoms. I think I remember Jim Cramer screaming about how the New York Times had a story about the bad market on its front page so everyone should run out and buy, buy, buy. This was around May 15th, oops. Jim isn't the only offender though. I keep reading how market sentiment is so bearish we must be at a bottom, but still the market goes down.

Here is a post from Helen Meisler on June 16th.

Yesterday I showed the chart of the Investor's Intelligence percentage of bulls and how it was in an area of a bottom. Today I want to show the American Association of Individual Investors chart. What I find fascinating is that the percentage of bulls did not fall from last week to this week (26% both weeks), but those folks who are bearish jumped ship from the correction camp, as bears jumped 10% to 55%. The first one came days before the July 2002 low. The second one came at the October 2002 low, and the third one came in February 2003, three weeks before the March 2003 low. Obviously each time led to a rally that lasted longer than a few days. I'm sure some of you will question Thursday's trading volume and do what I normally do, noting how light it was compared to the down days. But I am going to point out that while it might have been light, it was surely heavier than volume during March and April, when I was complaining about volume on a daily basis and the bulls thought I was crazy to be doing so So if you liked the market this past spring, and didn't mind the volume being so low, then how can you complain about 1.9 billion shares traded on the NYSE yesterday vs. 1.5 traded in April? (And let's face it, the NYSE was where all the action was this spring, not in the Nasdaq, which couldn't keep up with it or with the S&P 500.) It is entirely possible we still need to do more retesting next week -- in fact, I would like to see some retesting early in the week -- but the indicators all say we've got a bottom for now.

Here is a post from Helen Meisler on July 20th.

I have been harping away about not having enough bearish sentiment, so I wanted to share with you that the American Association of Individual Investors just reported the results of its weekly survey, and finally we have a sentiment gauge that is extreme! The bulls are at 23.85%, the lowest percentage since April 2005. The bears are at 57.9%, the highest percentage since February 2003. Sometime later this morning, the charts should be available for you to see the statistic in graphic form.

I really enjoy reading Mark Hulbert and really appreciate the service he provides on MarketWatch.com. He looks at a different sentiment indicator obtained from the newsletters he covers, the Hulbert Stock Newsletter Sentiment Index (HSNSI). In his latest post, Mark comes to a very different conclusion than Helen.

...But recent trends are worrisome: So far in July, the HSNSI has increased by 6.2 percentage points while the Dow Jones Industrial Average has declined by 222 points. It is rare, and bearish, for advisers to become more bullish in the face of a market decline. These recent trends reflect a marked shift in mood on the part of the average newsletter editor. Most newsletters quickly exited from stocks in the wake of the market's decline that began in mid-May, suggesting that this decline was more likely a mere correction than the beginning of a major bear market. But rather than continuing to view the market's glass as half empty, editors are gradually beginning to view that glass as half full. If this trend persists, it would suggest that the stock market must decline a lot more before there is enough skepticism to support a more sustainable rally....

To be sure, the top performers are not more bearish than the bottom performers, so this contrast by no means supports an outright sell signal. Again, however, as with the first factor discussed in this column, the trends are not encouraging. I hope my discouraging assessment is proven unfounded.

How can two people, both using contrarian sentiment indicators come to different conclusions? Is the indicator really that useful? Why do I see sentiment arguments all over the place and why are they almost always wrong?

Like the title of my blog implies, I'm not very experienced with the market. I'm a trained scientist, a molecular spectroscopist to be specific, so I've spent many days of my life just staring at data and trying to pull unambiguous information from it, while still making sure that such information can stand up to peer review. So far I've been fairly successful in getting my papers published in quality journals and there are several more on the way. (Some papers are highlighted in the professional links section)

I decided to use my scientific background to look at the data. It occurred to me that I might be able to bring a different perspective to the interpretations of the AAII sentiment indicators.

I created the figure using the actual AAII data (I had to pay for a 1 year membership). The dotted line is the actual weekly survey results and the smoothed curve is a Fast Fourier Transform (FFT) smoothing that removes Fourier components with frequencies higher than 1/n(dt) . I used n=20. All this does is remove some of the high frequency noise.

The first thing that really jumps out at me when I examine the correlation between the AAII sentiment survey (Percent Bearish) and a market bottom is the fact that there is absolutely no relationship between the magnitude of the market correction/bear market and the actual peak in the percent bearish survey. Consider the market bottom in 1990, which represented a correction of about 14%. The percent bearish was about 48%. Now look at the market bottom of 2002, which represented a correction of over 50%, the percent bearish was less than the 1990 high.

Ok, so what? If there is no correlation between the correction and the percent bearish peak, what use is the indicator? Well, we certainly can't say, "Oh there are extreme readings in the AAII indicator, so we are at a bottom" because there is nothing to say that the indicator can't go higher. As Helen found out above. The only thing the sentiment indicator is good for is being able to pinpoint the exact moment in the past when the market bottomed and kick ourselves for not buying. Looks to me that William O'Niel's method of spotting bottoms is still the best and most robust, as it relies on market action and volume. He does recommend looking at the Put/call ratio and I'll post a quick analysis on that later.

Look at the percent bearish now! It is higher than the bottom of 2002. I doubt we are done going down since the economy is just beginning to show signs of a slowdown. But I guess if you keep claiming that we have hit bottom everytime the sentiment spikes you'll get right eventually.

Finally, let me point out that there is nothing to stop the indicator from having percent bearish readings into the 80%-90% range. It is unlikely, but considering how screwed up this country and world is getting, being bearish on the stock market might soon be par for the course.

Update July 25, 2006

I had another thought about the AAII sentiment survey last night. Remember when some Boston Red Sox hacker tried to flood the 1999 All-Star ballots with votes for Nomar Garciaparra? He cast some 25,000 votes just before the close of voting, trying to guarantee a victory over Derek Jeter. I know this is probably not happening, but with so many people relying on this sentiment survey, what is to stop someone from trying to manipulate the market by flooding the AAII survey with overly bearish sentiment?

Just a thought. And another thing, why would anyone use the AAII sentiment survey over the newsletter sentiment? The newsletter people are the experts and presumably control the flow of more money than the average retail investor.