It’s Not My Fault, We Had Bad Data

This is a recent article from Computerworld about Alan Greenspan’s testimony before Congress concerning his analysis of the causes behind the financial meltdown. His conclusion was that,

Business decisions by financial services firms were based on "the best insights of mathematicians and finance experts, supported by major advances in computer and communications technology," Greenspan told the committee. "The whole intellectual edifice, however, collapsed in the summer of last year because the data inputted into the risk management models generally covered only the past two decades — a period of euphoria."

A few things popped into my mind as I read this article and I wanted to share them with my readers.

Blaming a massive failure on bad data is hardly new. I’m sure the Egyptian who was in charge of building the Sphinx blamed bad estimates on the amount of resources needed to construct it which is why she has no nose. More recently, government contractors write these failures due to data into the contracts themselves. “If we don’t get the most ridiculously accurate data from across the entire government in the first two weeks of the period of performance then we can’t be held accountable for the conclusions of this study or for the reliability of the product we’re making.” It is often called the assumption of Garbage In, Garbage Out. So blaming bad data is not shocking to me, nor anyone else who’s ever dealt with models before. The reason I think it surprised Wall Street and their analysts is because they are used to having an abundance of data at their fingertips down to the minute details, yet never addressed their assumptions about the original accuracy of the data itself. The other aspect to this point is that the Wall Street analysts probably thought that because the free market financial industry was so transparent that there was nothing that was not already known about the situation and factored into the current price. Boy, we they wrong.

This financial meltdown finally revealed to me how much we rely on computers and machines for everything, and that a series of bad data fed into the computers (it wasn’t even the machines fault but our own) caused the entire system to collapse and we only averted disaster because the government bought everything. The financial analysts build computer models that buy and sell just about everything according to certain rule-sets. Buy low and sell high is a traditional rule-set easy to remember for layman investors. Sell 1000 Eurodollar double-butterfly spreads at 3 ticks versus 2 ticks makes the rule-sets that are generated a little more complicated. (BTW, I have no idea what the previous sentence means, I just remember my friend using those words as a commodities trader.) These rule-sets then function at the speed of light. Don’t think so? This same commodities trader indicated to me how much more money he could make by decreasing his program’s run-time by 2 milliseconds. Then you add in the data that may or may not be tainted because of bad assumptions, under- or over-regulation, or biases in the market due to government policies (the Fannie/Freddie policy of making a house available to every American who wanted one). This creates a system that is like a runaway Mack truck with a driver that has periodic chances to adjust his driving every mile or so and only using his review mirror as a sight adjustment. The bottom line is that people are no longer the driving factors: the programs we create are and based on their underlying rule-sets they drive the market in certain directions and to certain levels.

Greenspan’s idea that including "historic periods of stress, capital requirements” in the risk models “would have been much higher and the financial world would be in far better shape today” is ridiculous. This framework is exactly the same way my bosses tried to create a computer model of 21st century conflict. You review how well you did and then fill the holes with whatever obvious parts are missing and parts that you possibly have data for. Why bother trying to fill a model deficiency with a concept that doesn’t have any associated data with it (people’s fear, irrational behavior, networks of contagion, etc.). Thus the computer models would be much better by including elements that we have a lot of data for; “historic periods of stress.” The fundamental problem with computer models is that they are a simplification of reality. They only include a finite number of variables about how reality occurs. And each of these computer models is being used in the reality it is trying to model, thus necessitating its own actions as one of the variables it ought to consider which affects the model which will affect the behavior variables ad infinitum. Combine that infinite regression thought with the irrational behavior and fear/greed of everyday people and all of a sudden, trying to model financial reality using computers and finite rule-sets will inevitably lead to situations like the one last month. I suppose you could say it is the inexorable outcome of complex adaptive systems.