Why the free market is like quantum mechanics

If we were all omniscient and had infinitely fast and perfect computers, maybe we could use quantum mechanics to explain chemistry and biology. In reality, no amount of quantum mechanics can completely explain the chemistry that goes into treating a disease with a drug, baking a cake or making a baby. Why? Because the system is too complicated, and we don't even know all the factors that make it tick yet.

Now imagine someone who has started out with the honest and admirable goal of trying to apply quantum mechanics to understand the behavior of a biological system like a large protein. He knows for a fact that quantum mechanics can account for (a far better word than explain) all of chemistry- the great physicist Paul Dirac himself said that. He has complete confidence that quantum mechanics is really the best way to get the most accurate estimates of various properties for his system.

But as our brave protagonist actually starts working out the equations, he starts struggling. After all, the Schrodinger equation can be solved exactly only for the hydrogen atom. We are dealing with a system that’s infinitely more complex. The complexity forces our embattled savant to make cruel approximations at every stage. At some point, not only is he forced to commit the blasphemy of using classical mechanics for simulating his system, but he has to actually stoop to using empirical data for parameterizing many of his models. At one point he finds himself fighting against the Uncertainty Principle itself!

In the end our hero is chagrined. He started out with the lofty dream of using quantum mechanics to capture the essence of his beloved protein. He ended instead with a set of approximations, parameters from experiments, and classical mechanics-derived quantities just for explaining his system. Prediction was not even an option at this point.

But his colleagues were delighted. This patchwork model actually gave fairly useful answers. Like most models in chemistry, it had some explanatory and predictive value. Even though the model was imperfect and they did not completely understand why it worked, it worked well enough for practical purposes. But this modest degree of success held no sway for our bright young scientist. He stubbornly insisted that if, just if, we had a perfectly fast computer with unlimited accuracy and an infinite amount of time, quantum mechanics indeed would have been spectacularly successful at predicting every property of this system with one hundred percent accuracy. Maybe next time he should just wait until he gets a perfectly accurate computer and has an infinite amount of time.

The preceding parable was narrated to describe what I think is a rather unwarranted swathe of criticism coming from libertarians about the financial crisis during the last few years. For instance see Amit Varma's criticism here which sums up many of the major points. The reasons for the financial crisis are many, probably more complex than quantum mechanics, and society will surely keep on debating them for years. But one of the most common reasons cited by libertarians (usually in the form of a complaint) for the failure of the economy is that we should not blame the free market for what happened because we never got a chance to actually have a free market. If only we got a chance to have a perfect free market, things would be lovely.

Notwithstanding the fact that this argument inches uncomfortably close to arguments made by the most vocal proponents of socialism in the twentieth century (“There was nothing wrong with the system per se, only with its implementation”), I think it’s a little nutty. Maybe a perfect free market wouldn’t have led to the crisis, but that’s like our young chemist saying that infinitely accurate computers and quantum mechanics would not have led to the kind of imperfect models that we get. The problem is really that there are so many practical obstacles in the application of quantum mechanics to a real-life chemical system, that we are simply forced to abandon the dream of using it for explaining such systems. Unless we come up with a practical prescription for how quantum mechanics is going to address these real-life obstacles without making approximations, it seems futile to argue that it can really take us to heaven.

To me it seems that libertarians are ignoring similar obstacles in the way of implementing a perfect free market. What are these obstacles? Most of them are well known. There’s imperfect competition because of the existence of inherent inequalities, leading to monopolies. There’s all that special interest lobbying, encouraged by politicians, which discourages true competition and allows monopolies to get a head start. There’s information asymmetry, which simply keeps people from knowing all the facts.

But all these problems are really part of a great problem- human nature itself. All the obstacles described above are basically the consequence of ingrained, rather unseemly human qualities- greed, the lust for power, the temptation to deceive, and a relentless focus on short term goals at long term expense. I don’t see these qualities disappearing from our noble race anytime soon.

Now sure, I think we can completely agree that the free market was invented to curb some of the worst manifestations of these qualities, and it has worked remarkably well in this regard. Remarkably well, but not perfectly so. Maybe libertarians need to understand that the last vestiges of the dark side of humanity cannot be done away with, since they are an indelible part of what makes us human. So unless they come up with practical ways in which they can surmount these obstacles, in which they can solve the problem of human nature itself- a difficult goal to put it mildly- it’s rather futile to keep on chanting that all our problems would be solved if only we could somehow make these inherently human qualities disappear.

The final argument that libertarians usually make is; just because there are obstacles in the way of a goal (the perfect free market) that may seem even insurmountable, that does not mean we should not keep on striving towards the goal. I think that’s perfectly laudable. But the problem is, unless you come up with a practical solution for all the problems that you face on the way, your goal is just going to remain an abstract and unworkable ideal, not exactly the kind of solution that's desirable in the practical fields of politics and economics. More importantly, all this striving towards the goal may create problems of its own (the science analogy would be unimaginably expensive calculations, scientists laid-off because of the lack of results, overheating of the computers leading to fires etc.). We have all seen these problems. There’s the well-known problem of externalities, there’s the problem of unregulated firms getting ‘too big to fail’, there’s the problem of growing income inequality. Surely we have to admit that these are real problems too.

So what should libertarians do? Well, didn’t our intrepid quantum mechanic grudgingly accept the intervention of approximations and parametrizations? These seemed ugly, but he had no option but to use them, since quantum mechanics simply could not solve all the obstacles in his way. Similarly, perhaps free marketers could realize that at least in some cases, government intervention, no matter how ugly it may seem, may be the only way to reach a workable goal. Sure, it may not be the best of all goods, but it could be the least of all evils. What would have happened if our bright young scientist had kept on insisting that he wouldn’t budge an inch if he is forced to use anything other than quantum mechanics? He would have ended up with nothing.

And in economics even more than in chemistry, a model that partly works is better than a model that does not exist. “Sometimes it’s not enough to do our best; we need to do what’s necessary” (W. Churchill)...

The writings of John Cassidy

For the last several weeks I have been enjoying John Cassidy's "How Markets Fail". I am almost done with the volume and have to say that it is one of the best and most balanced critiques of markets that I have read. Cassidy who was educated at Oxford and certainly knows his economics carefully documents the history of how academic mathematical theories like Arrow's impossibility theorem and Robert Lucas's theory of rational expectations came to be mistaken as practical rules for application to the free market when they were really supposed to be not much more than ideal mathematical constructs. Quants fell into the same trap (incidentally I saw a book today by Scott Patterson named "The Quants" which looked quite engaging).

Cassidy also documents very well how most free market theorists did not include the behavioral economic approaches later pioneered by psychologists like Daniel Kahneman and Amos Tversky. One of the important points that Cassidy makes is that a lot of these people who came up with models for finance and the free market thought that anomalies would not persist for long and that the market would iron them out, except that it doesn't, usually as a result of the (rather obvious) failure of models to forecast human behavior.

Even more enlightening is Cassidy's series of interviews with Chicago school economists like Richard Posner, Gary Becker and Eugene Fama in the New Yorker. It is heartening to see how most of these people who were once die hard free marketeers are now taking a more moderate stance towards the world and accepting the limitations of things like rational expectations and the efficient market hypothesis. One reason for the decline of the Chicago school has been the death of Milton Friedman, but another reason seems to be the genuine flaws that at least some of the practitioners seems to have acknowledged. All except Eugene Fama, who in his interview appears to be as much of a stubborn free market "fundamentalist" as anyone else; the last man manning the fort, keeping a brave face and clinging to the flag known as the efficient market hypothesis, with smoke and mirrors being his main weapon of combat. Behavioral economists who were once despised at Chicago are now part of the establishment there.

Very enlightening and more than a little gratifying. I would strongly suggest especially the interviews and also the book.

A WRECK

An interesting and witty talk by Thomas Frank which I attended yesterday showed how profoundly undemocratic conservatism in America has become. His recent book "The Wrecking Crew" How Conservatives Rule" shows how conservatives have essentially turned the country into a totalitarian stronghold that deeply despises democracy and freedom under the guise of promoting "morality" and free markets. Concomitant reading of J. Peter Scoblic's "U.S. vs Them" shows how this breed of conservatism, far from being restricted to the Bush administration, has been nurtured and furthered since the end of World War 2 when rabidly anticommunist conservatives honed threat inflation, pseudomoral philosophizing and downright bullying to a fine art. Their principles were clear; anyone who advocated any form of government intervention was automatically a godless communist, anyone who advocated negotiation with the enemy was a coward. In fact Scoblic's book makes it clear how dogmatic and totalitarian today's conservatives are by deftly narrating how their forebearers had a deep disdain even for Eisenhower and Nixon!

I hope that just as communists gave those who would argue for government intervention a bad name, so would these modern American conservatives give a necessarily and well-deserved bad name to those who keep on advocating rampant and totalitarian capitalism. Interestingly as Frank quipped yesterday, the same conservatives who keep on opposing government intervention now have the government working around the clock to save the country. Irony would find this ironic. But that should not be surprising; the stated goal of today's conservatives is to be in bed (literally) with big business. If government intervention is necessary to appease big business, so be it. And yes, did Bush tell you that your taxpayer money is going to be on the line now? So much for the "read my lips- no taxes" motto that trickled down from his father. People's hard-earned money be damned if appropriating it is going to help Uncle Care Bear. If this does not convince anyone of the hypocrisy of Republicans, it's hard to see what will. They care about lowering taxes only as far as that can keep big business fat and happy.

Let's just hope that the recent market failure and many similar incidents expose the evils of dogmatic thinking of any kind- whether socialist or laissez faire capitalist- and teach people to take a more nuanced and moderate stand that advocates different policies and actions based on the situation and times. The point is that it is not possible to sustain one stock, extreme philosophy for too long, a fact that naturally does not go down the prickly gullets of idealogues in administrations such as this one. They want to see the world in black and white. No matter that it's not just shades of gray but gray that often changes into other colours.

Politics should be flexible and should be able to choose and draw from a wide variety of options in different proportions based on requirements for that age. There is no such thing as a completely socialist or completely libertarian or completely anarchist or completely nihilist philosophy that would work for all times and all people. A full century of extremism- from Stalin's totalitarianism to the Chicago Boys' laissez faire support for totalitarian regimes in Central America- should more than warn us about constantly dealing in absolutes and monolithic economic models. Brendon Barber put it very well for the BBC:

This is not the final crisis of capitalism, but it ought to be the end of the road for a particular version of it.

The truth is that there is no single economic model of capitalism.

There has always been a considerable difference between Europe's social market and the Anglo-Saxon version in the US - not to mention the wild-west excesses of post-communist Russia or the strange hybrid that now operates in China.

What we have seen in the last few days should rather be the last gasp of the belief that the way to secure prosperity is to let free markets rip and tear up any regulation that gets in the way of short-term profit.

Already the view that the state has no role to play has gone with the nationalisation of Freddie, Fannie and AIG in the US and Northern Rock in the UK.

The lessons that we need to draw are that we have given far too much importance to the City and neglected other sectors - not just traditional manufacturing, but new environmental jobs and the creative sector, to give just a few examples. This should be neo-liberalism RIP.

But the paradox that has yet to work its way through is what this means for politics.

Voters are shifting to the right in their voting intention, but shifting to the left in many policy preferences as they expect government to tax the rich more, regulate the energy sector and restore sensible regulatory structures in the City.

As Frank said yesterday, one good thing should undoubtedly come out of the recent sham- Hoover should not get his second term.