I recently picked up a copy of Tyler Cowen and Eric Crampton's 2002 edited volume Market Failure or Success: The New Debate (now only in print with the Independent Institute, though it was originally published by Edward Elgar) and have really enjoyed what I've read so far. The book is a collection of essays by prominent IO scholars organized into four sections: a fantastic introduction by the editors, four essays that form the foundation of the "new" market failure theories based on information problems, four theoretical critiques of said theories, and 8 essays providing empirical and experimental evidence of the editors' thesis: that information-based market failure theory is often merely a theoretical possibility not borne out in real life and that economic analysis of knowledge often provides us with the reasons why.
Two pieces by Stiglitz are featured in the first theoretical section: one on information asymmetries and wage and price rigidities and the other on the incompleteness of markets. Akerlof's famous "lemons" paper and Paul David's paper on path dependence are also included. I was happy to see that Demsetz's "Information and Efficiency; Another Viewpoint" was the first essay in the theoretical critique section as it sets the stage for the other chapters in that section. The empirical and experimental section features Liebowitz and Margolis' response to Paul David on path dependence in technology, Eric Bond's direct test of Akerlof's "lemons" model, and an essay I've never ready by Gordon Tullock entitled "Non-Prisoner's Dilemma."
The introduction provides a short summary of the arguments presented in the following 3 sections and includes a great discussion of the editors' views of the core problems with information-based market failures. Here's the conclusion of the intro chapter:
Our world is a highly imperfect one, and these imperfections include the workings of markets. Nonetheless, while being vigilant about what we will learn in the future, we conclude that the 'new theories' of market failure overstate their case and exaggerate the relative imperfections of the market economy. In some cases, the theoretical foundations of the market failure arguments are weak. In other cases, the evidence doe snot support what the abstract models suggest. Rarely is analysis done in a comparative institutional framework.
The term 'market failure' is prejudicial - we cannot know whether markets fail before we actually examine them, yet most of market failure theory is just theory. Alexander Tabarrok (2002) suggests that 'market challenge theory' might be a better term. Market challenge theory alerts us to areas where market might fail and encourages us to seek out evidence. In testing these theories, we may find market failure or we may find that markets are more robust than we had previously believed. Indeed, the lasting contribution of the new market failure theorists may be in encouraging empirical research that broadens and deepens our understanding of markets.
We believe that the market failure or success debate will become more fruitful as it turns more to Hayekian themes and empirical and experimental methods. Above, we noted that extant models were long on 'information' - which can be encapsulated into unambiguous, articulable bits - and short on the broader category of 'knowledge,' as we find in Hayek [Hayek's 1945 article The Use of Knowledge in Society can be read here for free. A short explanation of the main theme of the article can be found here. - LR]. Yet most of the critical economic problems involve at least as much knowledge as information. Employers, for instance, have knowledge of how to overcome shirking problems, even when they do not have explicit information about how hard their employees are working. Many market failures are avoided to the extent we mobilize dispersed knowledge successfully.
It is no accident that the new market failure theorists have focused on information to the exclusion of knowledge. Information is easier to model, whereas knowledge is not, and the economics profession has been oriented towards models. Explicitly modeling knowledge may remain impossible for the immediate future, which suggests a greater role for history, case studies, cognitive science, and the methods of experimental economics.
We think in particular of the experimental revolution in economics as a way of understanding and addressing Hayek's insights on the markets and knowledge; Vernon Smith, arguably the father of modern experimental economics, frequently makes this connection explicit. Experimental economics forces the practitioner to deal with the kinds of knowledge an behavior patterns that individuals possess in the real world, rather than what the theorist writes into an abstract model. The experiment then tells us how the original 'endowments' might translate into real world outcomes. Since we are using real world agents, these endowments can include Hayekian knowledge and not just narrower categories of information.
Experimental results also tend to suggest Hayekian conclusions. When institutions and 'rules of the game' are set up correctly, decentralized knowledge has enormous power. Prices and incentives are extremely potent. The collective result of a market process contains a wisdom that the theorist could not have replicated with pencil and paper alone.