Monday, October 08, 2007

Correlating the Correlation

For several months now I have been producing a regular post entitled “Conspicuous Correlation” that demonstrated an apparent relationship between the decline in home prices and a near simultaneous pullback in the retail sales of the most discretionary items.

While that post generally “eyeballed” what seemed to be a fairly obvious relationship, I’d like to expand the analysis (hattip Deejayoh for proposing the concepts found below) a bit in order to present a more complete representation of the correlation as well as setting the stage for future “Conspicuous Correlation” posts.

First, in order to get a sense of the original presumed correlation, take a look at the chart below (click for larger version) showing the year-over-year percentage change to both the S&P/Case-Shiller Composite Index (measuring home price change) versus my “discretionary” retail sales index (measuring retail sales change).

Note that the chart shows that during the period from January 2001 to June 2003 retail sales faltered a little as the effects of the dot-com recession worked to dampen spending but that housing remained at exceptional rates of appreciation.

Notice also that from July 2003 to January 2006 both measures show exceptional, and possibly related or correlated growth that then seemed to also decline in tandem in early 2006 with the retail sales component having generally remained either negative or dampened similar to the rates seen during the last recession.

So this frames the heart of the matter.

Did the exceptional growth of housing values spill over to the consumption of discretionary items and then as the housing tide turned did it spill over again but in reverse?

Anecdotally, I think, we would probably all be pretty comfortable with assuming this correlation exists as we have all likely witnessed some aspect of the cash out refinancing and other related activities resulting from the housing boom but let’s see if a more complete statistical correlation exists to hang our hat on.

Next, let’s expand the chart a bit (click for much larger version) to include the full range of available data starting in January 1993 and running all the way through the latest month where both series have data points in July 2007.

Notice that in general, the rate of change of the two series do not appear to be very well correlated and, in fact, that during the tail end of the last housing recession in 1993, retail sales was growing strongly while home appreciation floundered along the bottom until 1997.

Also note that while growth (or lack thereof) of retail sales was clearly dampened during the dot-com recession, home price appreciation seems to have been little effected.

The next chart (click for larger version) shows the full range of series data (Not year-over-year change… simply the underlying data) for both the “discretionary” retails sales and the S&P/Case-Shiller Composite index starting from 1993, as well as introducing a Pearson’s correlation that calculates the degree of correlation between the two data series.

Note that in this chart, the correlation set for each successive month cumulative staring from January 1993 so although the correlation starts out nonexistent initially, the two measures begin to correlate strongly for the remainder of the dataset.

It’s important to note that I included this chart mostly as an exercise to set up the next several charts and I don’t think that this correlation shows anything of any great importance other than that, over a long period of time, both home prices and retail spending increased.

There is nothing in the chart to suggest that the home price appreciation influenced the increase to retail spending, only that they both increased over the same period.

The next chart (click for larger version) betters the prior by using a 12 month “moving” correlation whereby each successive correlation data point indicates the degree of correlation between the last 12 months of series data.

This chart clearly shows periods of variation where the two series range from closely correlated to no correlated at all.

One feature to note is that the correlation was strong during the mid to late nineties prior to the dot-com recession where both home prices and retail sales grew consistently.

Then during the dot-com recession the correlation fell apart as retail sales were clearly flattened by the downturn and home prices seemed largely unaffected.

Also note that as of late, the correlation has fallen apart again as home prices have declined and retails sales, although having flattened, have not declined.

The final chart (click for larger version) takes a different angle, instead attempting to correlate the year-over-year rates of change of the underlying data series rather than the series themselves.

As you can see there is little consistent correlation between the rates of change of these series but that having been said, the best correlation to date has been in the trend seen since 2006.

Although the correlation has weakened a little as home prices continue to slide and retail sales have at worst flattened, this current existing correlation would be interesting to watch over the coming months.

If a measurable pullback continues to occur in discretionary retail sales, this correlation will persist, leaving us to possibly conclude that there is a direct effect between the latest decline in home values and consumption of discretionary items.

Ill update the “Conspicuous Correlation” post to reflect this better assessment of the correlation so be sure to check back as the new retail sales numbers are being released on Friday and they will definitely make an impact one way or the other.