A Mencius Moldbug has written a confused and rambling 7400 word critique of futarchy. But since Mencius seems to have passion and potential, let me try to communicate. Most readers may prefer to skip this post; it will get tedious.
So if PM good, DM bad, your complaints should focus on features that distinguish decision and prediction markets, right?
OK, except that morons may largely cancel each other, in which case you don't need as many non-morons. But this issue applies equally to prediction and decision markets, doesn't it?
You can ask for a full probability distribution. If speculators know they don't know anything, then they will give you a broad distribution that expresses a lot of uncertainty. This is them telling you they don't know much. In your nose example, they may just give you the distribution over nose sizes for elite Chinese. And how is this issue different for prediction vs. decision markets?
Trader experience does seem to improve their collective accuracy. But this hardly means accuracy was zero initially. By now we have a great deal of experience in field and lab prediction markets where accuracy was valuable from day one. And how does this issue distinguish prediction vs. decision markets?
Play money markets do often fail badly, but many of them have done very well. All else equal I prefer real money markets, but "complete waste of time" is way too strong. And how does this distinguish prediction vs. decision markets?
There are several possible concerns you might have here. First, traders might sabotage events in the real world to improve their forecast accuracy. This can be a problem for any forecasting mechanism that rewards accurate contributions, and there are several ways to deal with it. Also, traders might try to distort the market price because that price influences decisions. Theory, lab data, and field data suggest this is not much of a problem. And how do these issues distinguish prediction vs. decision markets?
Er, in pretty much every speculative market that has ever existed, traders have had differing relevant info. Identical info is a very rare situation. Some traders had more total info, and the traders who had less would have been well advised to leave. And how does this distinguish prediction vs. decision markets?
After the fact it is quite easy to test for forecast accuracy. And even if initial accuracy was poor, why not begin if we expect accuracy to greatly improve? And how does this distinguish prediction vs. decision markets?
We can sensibly compare small differences in conditional expectations even when conditional variances are high. For example, even when you cannot predict the date of your death well, you can reasonable expect a longer lifespan if you exercise today than if you do not exercise today. The difference in these two expectations can be tiny compared to the variance in your realized lifespan, and yet you can have reasonable beliefs about this difference, and act on them. See also the presidential nomination decision markets which have been run for many years by IEM and Intrade as proof that decision markets are mechanically feasible, even in situations of great uncertainty.
There is now a prediction market industry, where I do a lot of consulting, most of which is to private companies. Most of this consulting is on prediction markets, but some are decision markets.
For firms I recommend forecasting profits or stock price, not revenue. For nations I suggest a full national welfare; GDP is only a first step to such a measure.
Yvain,
All I can say is that if everyone approached prediction markets with the same skepticism as you, they would be truly useful tools. If they weren't basically illegal, that is.
There is no philosopher's stone. Good government is government by good people. In essence, an iterated prediction market is just a very clever way to employ good people, in a very unusual organizational structure which is unusually tolerant to the presence of bad people in the population. (Not to mention unusually inexpensive from the management perspective.) Treat the tool as what it is, and you can't go wrong. Treat it as a philosopher's stone...
Posted by: Moldbug | May 24, 2009 at 08:52 PM
Moldbug, you do not understand what you read. Our conversation is over.
Posted by: Robin Hanson | May 24, 2009 at 08:57 PM
rafal,
Note that if the decision is welfare-diminishing compared to the status quo, there will be a group of people losing more money than the bears gain. They would have a strong incentive to buy the decision back, wouldn't they?
I think that's the cleverest argument on this whole thread! Note, however, that this would create a kind of policy-control contest very familiar to, say, the citizens of the late Roman Republic. I'm not sure it passes Schneier's test. Well, okay, I'm sure it doesn't. On the other hand, that doesn't mean it wouldn't be an improvement on Washington's present policy process...
Posted by: Moldbug | May 24, 2009 at 08:57 PM
Note that if the decision is welfare-diminishing compared to the status quo, there will be a group of people losing more money than the bears gain. They would have a strong incentive to buy the decision back, wouldn't they?
They have the same incentive to buy the decision back as the consumers do to buy an open market in foreign sugar. Unfortunately, corn producers and sugar producers have stronger incentive to buy the market closed.
Concentrated benefits, diffuse costs.
Posted by: Steve Johnson | May 24, 2009 at 09:23 PM
Well! I feel it's only appropriate to give Professor Hanson the last word.
Conveniently, I finished the aforementioned paper - and discovered that it concludes with an admirably-motivated, and quite plausibly complete, list of all its spherical cows. (In case it isn't obvious, my objection above was to the last-named cow.) If this doesn't place Professor Hanson in the 99th percentile of academic honesty, I myself am a spherical cow:
Of course the fact that we have a particular model illustrating these results hardly implies that these results always hold in every context. Our model assumes risk-neutrality, normally distributed values and signal errors, interior choices of information quantity, only quantal-response-type irrationality, no meta-signals about the signals of other agents, no transaction costs of trading, no budget constraints, and a single rational manipulator with quadratic manipulation preferences and a commonly known strength of desire to manipulate. However convenient these assumptions may have been for solving the model, one can reasonably question the empirical relevance of models based on them.
The prosecution rests - and calls for leniency.
Posted by: Moldbug | May 24, 2009 at 09:42 PM
Whether they're "accurate" depends in large part on definitions of accurate, and who controls those definitions. They'll be different in decision markets (where accuracy depends on policy outcomes) than prediction markets (which simply predict, or not, what outcome will occur).
We face this today and I do not believe we have *truly* objective criterion for evaluating whether the implementation of a policy on any subject of import the least bit controvercial was accurately predicted. That is to say, we already have a lot of data on policy implementations and outcomes, and it's much harder to agree on whether their effects were accurately predicted in advance than with a market that simply predicts whether A or B will happen ("Will Barak Obama's Stimulus Package pass and be signed into law" is a much simpler question than "what will the effect of Stimulous Package A be, will it be better than Stimulous Package B, and will either be better than doing nothing, and over what time frame, and with or without other ripple effects" - CBO's predictions, for example, are considerabily different than the White House's).
In the abstract, people propose Policy A, others say Policy B will be better, some say the status quo is better than either. The advocates of Policy A say it will cost $100 Billion Woolongs and create 1.21 gigajobs. Critics say that it will cost four times that much in the end and produce half as many jobs.
Policy A is enacted, it ends up costing several multiples of what was predicted and net job growth is undetectable. Advocates say it was because it was underfunded at the beginning, and that other factors impacted it, and given how the economy performed, conditions would have been worse without it: It saved jobs. Critics say their criticisms were accurate and that the policy made things worse ultimately, particularly because of unintended consequences (Brawndo consumption went up 400%, causing crop failures and an increase in illiteracy and out of wedlock childbirth, and these things aren't included in the advocate's data regarding whether the policy outcome was good or bad). Adocate's rejoider is that well B, C, and D happened in the interval as well, and without Policy A in place, those things would have excacerbated the situation, so A really saved us from catastrophy.
This is much more open to manipulation than a prediction market in that sense alone.
Also, I think Hanson is well too dismissive of the ability of "Bears" to manipulate outcomes. In a world where George Soros & friends could cause a run on the Pound, it's not hard to foresee that a subset of interested parties who will benefit disproportionately from the implementation of a policy will have more interest in investing enough to make sure it happens than everyone else. Will "Wolves" follow that money, because it is more likely to occur (and thus pay out), or will they vote with the "sheep" (those whose direct interest in the policy choice is less strong, and indeed who may pay the costs and receve no benefit)?
"Smart Money" is more likely to go with the "Bears" on this, in my opinion. Their market behavior will be based on an evaluation of what will "win" rather than which will be the best policy to implement, and everyone else in the market will then be "sheep" (AKA the "marks").
For some reason a lot of the exchanges on "Wolves" vs. "Sheep" reminded me of Phil Helmuth going on rants about "idiots" who have offsuit Queen-Ten and raise. Phil's "idiots" win hands, get to final tables, even win tournaments, despite the fact that though over time - if your time horizon is long enough - the pros do much better than "dead money". When interest and passion is high, a lot of sheep enter the market who otherwise might not be there - and this will make the outcomes different than in the smaller-scale running experiments Hanson foresees as testing and building the case for Decision Markets.
But in this situation the way the sheep will be separated from their money by the wolves is not directly tied to which policy outcome was accurately predicted to be preferable. The decision market will be dominated by those "Wolves" who are most skilled in knowing which decision will dominate, not which one is good policy and which one(s) are worse, and by players with an interest in the outcome (Bears) regardless of it's overall impact (the "I'm All Right, Jack" set). The wolves will prosper in the market from investing correctly in which policy alternative will win, not which one will have the most benefitial impact.
It "may" (to use Professor Hanson's word) be that those don't correlate at all. They "may" even be opposed. I'm not sure I'd bet my life that they will be alligned. I agree with Moldbug that there is no good way to test it - I have little confidence that "experimenting" in a small scale on inconsequential policy decisions won't produce the same outcomes that going big on major choices will. Wolf money will tend to follow, rather than counterballance, Bear money, and everyone else by definition will become Sheep and get shorn.
So in other words, no change.
Posted by: Porphyrogenitus | May 25, 2009 at 07:12 PM