Lies, damned lies and statistics

One of the many and varied things that annoy me, and believe me, it's a very long list, is the use of faulty statistics.  This practice is very widespread, it's almost as pervasive as the use of logical fallacies.  So widespread is it that I'm not going to list every example, it's a very long list, and my blood pressure won't stand it; instead I'll concentrate on one particular application of statistics that is constantly in the news, and constantly manipulated: house prices.

It's been two days since Halifax announced that, according to their resources, house prices rose 1.9% in January; as I stated at the time, I believe this to be mostly a sampling error due to their methodologies more than anything else (although that will come with time).  This has opened the floodgates, today's papers were full of the usual tripe "gazumping is back", "prices to double by year X", and so on; exactly the same sort of tripe we used to get every week from 2000 to mid-2008.

The main source of my property sceptisism stem from these pieces of "research", this is what made me think "no, stop, this is madness", none of it made sense or stood up to reason.

While I'm not quite old enough to remember the details of the last property crash, I was at school at the time, so I do remember it happening, but it had zero practical impact to me personally; the fact that it happened, however, was definitely firm in my mind.  My own direct concern with property began after graduating from university (2000), I was no longer in a parent-home/student-flat bubble; property seemed expensive, but there was no great media broo-ha-ha about it.  Of course, there were the home makeover shows, those have always existed, but their tone was mostly one of "making the most of that back bedroom" rather than "how to add £30,000 onto your house".

It was in 2001 that the measured house prices got back to "normal" - i.e. they reverted to the trendline.  At that time there was a large and vocal group of economists claiming that a minor house price crash was imminent, after all the signs of the economy entering a recession were quite large.  There was also a smaller group claiming all the usual estate agent spiel about "this is your last chance to buy".  As we know now, despite the faltering economy (Britain didn't quite enter a recession, but anyone job hunting in those years will tell you things were pretty crap), prices accelerated up, defying all previously known logic.  The crash warning economists shut up and went home; the pseudo estate agents started to believe their own hype and they would continue to spread more and more nonsense.

The key issue was ignored by both.  Well, the economists kept their heads down, not wanting to appear foolish after getting the last call wrong; and the estate agents were just blissfully unaware of it, they believed the story of house prices having their own independent momentum and didn't look into it.  The key issue, as we're now all too aware, was the international carry trade and lowering interest rates; money could be transported from Japan (0% rates) to the U.K. (5% rates - sounds high now, but was low by previous standards).  There was an unprecedented wall of money looking for returns, and the U.K. (and U.S., and New Zealand, and Iceland) couldn't get enough of it.  The "credit crunch", as much as the politicians and media are blaming the banks for, was caused by the stopping of this flow; and, what's more, this flow isn't coming back any time soon, and it definitely won't happen with interest rates of only 1%!  (Yes, I am saying the housing market would be healthier with higher interest rates.)

This post isn't supposed to be another diatribe on economics, I've written far too many of them already even though I have no training or experience on the subject, so I'll get on to the point.  The point is that most of the post-2000 boom happened in the early years: the record was 25% growth in 2002, by 2004 it had moderated slightly; house price growth hit a big flat zero in 2005; 10% in 2006, etc.  The growth had been plateuxing for years, but yet all the talk was of every increasing house prices, and ever faster growth.  The unexpected boom in 2000 blew all reality out of the window; people who'd seen the future and knew the boom was temporary were ignored or ridiculed, nothing would convince the evangalists of their wrongness.

Part of this reason (as well as sheer human stupidity, lack of self-awareness, and logical fallacies) is faulty statistics.  Some of them are deliberately faulty, others are faulty by accident.

They fall into two categories: 1) measures of house prices; 2) measures of market activity.

Measures of house prices

First things first: it's impossible to measure house prices.  Houses are not a commodity, there is no exchange with prices quoted.  Each house is different, even those gigantic estates of identical houses are all different - each has got a slightly different view, access to amenities, etc.  Attempting to produce reliable microstatistics to be crawled over like ticker-tape is at best foolish and at worst completely insane, yet that's what happens.  The problem is that each house price index has some significant flaws, so lets have a look!

Self-published estate agents

Occasionally a newspaper will produce a set of numbers that come direct from an estate agent.  The self-invalidation of such things should be obvious, but to spell it out it's invalid for two reasons: 1) one agency can be dramatically out-of-step from the market as a whole; and 2) even the agents acknowledge that they make it up "well three, I think it was three, nice houses on XYZ Street sold for £1,000,000, so there".

Conclusion: There's just no scientific method here, just ignore it.

Rightmove

There was a brief period of time in 2006-2007 when Rightmove were the most quoted index going, which annoyed me immensely.  It was a cold morning in 2006 when Rightmove published a "The average house in the U.K. is now over £200,000" - the other index have never come close to this figure, Halifax and Nationwide peaked at £186,000.  It's not that Rightmove are ahead of the curve (they are based on asking prices rather than achieved prices), it's that they are so far ahead they're no longer attached and have long since flew off on a tangent.

More suspiciously is Rightmove's decision to only take into account new listings, this conveniently leaves out listings which have been reduced in price.  This is also open to abuse since if a property is removed from the system, and re-entered at a higher price, the index shows rising prices even though no transactions have actually occurred.  (The good people at housepricecrash.co.uk called Rightmove on this years ago.)  In their defence, Rightmove claim to remove "obvious errors" from their index, but that doesn't change the fact it's only an average of asking prices, and asking prices differ wildly from achieved prices.

Conclusion: it's just an average of numbers picked from thin air, only useful to gague the arrogance of its member agencies.

Halifax and Nationwide

These are two of the longest running indexes, and their methodology is sound (relative to estate agents and Rightmove).  They use their own mortgage approval data to measure house values, nothing is made-up.  However is still a very long list of critisims that can be laid at their door:

They both apply fiddle factors, like mix adjusting.  This is, in principal, a good thing to do since house prices differ massively depending on the type of house; if prices were just averaged regardless the statistic would lose all meaning.  It is the means by which this adjusting occurs that is worrying and introduces it's own biases.  If no mix adjusting occured then the index would vary greatly simply due to more flats than houses being sold, or vice versa, but by weighting based on type of house they introduce the risk of sampling errors - if only one mansion is sold (at an increase), but ten thousand flats are sold (having crashed) the index will report nothing.  So what should they use then?  An established statistical principal, when dealing with a sample where some members are significant multiples of others is to use a geometric mean rather than arithmetic mean; this would reduce the need to guess property "mix", and would remove both types of bias.

The other fiddle factor is seasonal adjustment, this makes month-on-month figures meaningless; but over a whole year should have no affect.

Since both are measured on mortgage approvals, and since both specialise in relatively boring mortgages, the index is completely ignorant of cash buyers (rich people) and "specialised lending" (poor people), this introduces a certain conservative bias too.

Conclusion: they are the longest serving index, so are useful for historical comparisons, but it should be noted that Halifax don't even use the Halifax index when they value a house... I'll let that sink in for a moment.

Land Registry

The Land Registry produce two index: a quarterly one which is a simple average of every single last transaction made in the previous three months, and a monthly one which uses a more sophisticated methodology.  Both of them benefit from the fact that it's the law to register property sales with the Land Registry, there are no missed transactions, it's all included.

The quarterly index, however, does no mix adjusting (of either the good or bad variety) so can be tipped either way due to shifts in volumes of sales between different types of property.

The monthly index was introduced only a couple of years ago to counter that.  It uses "repeat sales regression analysis" to calculate the rate of change; in other words it measures price changes based on the difference in price achieved by the same house.  And, of course, they use a geometric mean.

However, even this index is not foolproof.  There have been cases of mortgage fraud which would distort the index, and other dishonest cases (have you ever seen a new build with a cashback offer?  Want to know why you hand over £220,000 then they give you £20,000 right back?  That's right, it's so £220,000 goes in the Land Registry stats); and, for some inexplicable reason, they exclude auctions.

Conclusion: it's by far and away the most accurate index, it exceeds all the others by virtue of: a) having the largest sample of raw data, and most diverse; b) having the most robust statistical methodology; and c) the Land Registry are probably the most honourable, there are no mixed messages, no "it's all good please take one of our mortgage" undertones, no discernible political bias.

Conclusion

(I'll save my debunking of the non-price property market statistics - e.g. mortgage approvals, sales numbers, etc. - to another day as this post is getting quite long.)

I, on more than one occasion, during the boom years was told "my property has gone up £30k already" by people who had bought a flat only one year previously.  Depending on how much I liked the person I would either politely smile and nod, or take issue with it (my default was smile and nod, I try not to debate property or economics in the "real" world).  If I really hated a person I would puncture their bubble, if I liked them I'd also sound a warning; it was the vast majority of humanity I simply said "that's nice" and carried on.  How would I correct/warn them?  I'd direct them to a site such as House Prices, and they would enter their postcode... "Bastard! Bloke next door moved in last week, he only paid £500 more!"

That site is fantastic, the law that allows the publication of all house sale prices was a very good one, it was designed to break the monopoly of estate agents being the arbiter of house price statistics - on that score it failed to stop them spouting bollocks, but it allows those with a clue to know what they're getting themselves in for.  Despite all the shouting in the press "Prices have doubled in the last ten years", etc., I can go to that site and find out that a flat at the other end of my street sold two months ago for only 20% than it fetched nine years ago.

House prices have obviously risen alot, and they've come back down alot but still show a healthy profit; you would think that would be enough for people, but since the idiots won the war of minds in 2001 reality has been overruled with faulty microstatistics which resulted in this incongruity we've all seen.

The Land Registry figures are by far the most reliable, but never got any mainstream mentions (except for two paragraphs in the business section).  The front pages and lifestyle sections of the newspapers couldn't reprint Rightmove's version fast enough.  The reason is fairly clear, when Rightmove were claiming 15% year-on-year rises, the Land Registry claimed 6% - boring, who wants to read about that.

But this school of nonsense hasn't gone yet, people are still in denial (which, as we all know, is a river in Egypt); but this is now manifesting itself in unusual ways.  The Land Registry figures have begun to be quoted by the mainstream press.  Why is this?  Because it's just as conservative on the downside as it is on the up, while Halifax and Nationwide are reporting -18%, the Land Registry are claiming -13.5% (figures for the calendar year of 2008).  Cherry picking statistics is still alive and well, it's just that the most reliable one happens to be the most palatable to them.

The media make the faulty statistics even worse, they do this by mixing reports in invalid ways.  "After ten years of double digit growth (according to the Nationwide, and still omitting 2005 for no adequately explained reason), prices are falling but only slightly (according to the Land Registry)."  You idiots... if you're going to use one index, stick with it; despite their problems comparing Halifax now with Halifax then is valid, comparing it with a radically different index isn't!

Anyway, the moral of this story:  Last week's Halifax press release should be taken with a pinch of salt, especially as the Nationwide report for the same period came to completely different conclusions - given the similarity of the two they should be in sync if there was a genuine turn around - the working hypothesis should be that Halifax is suffering line noise.

If the same is repeated for several months running, or is corroborated by the Land Registry, then it's a different story.

[ Part two here ]