Reply to Moldbug

A Mencius Moldbug has written a confused and rambling 7400 word critique of futarchy. But since Mencius seems to have passion and potential, let me try to communicate.  Most readers may prefer to skip this post; it will get tedious.

Prediction markets are a fine idea, whereas decision markets are... well... retarded. ... Almost every conceivable application of a decision market ... does not produce accurate predictions.

So if PM good, DM bad, your complaints should focus on features that distinguish decision and prediction markets, right?

For a market to produce accurate predictions, there must be genuine experts in the market, and they must be substantially better-funded than the morons.

OK, except that morons may largely cancel each other, in which case you don't need as many non-morons.  But this issue applies equally to prediction and decision markets, doesn't it?

If no one has ever seen the Emperor of China's nose, can a prediction market predict its length? ... The worst case is that in which nobody has any way of actually calculating the prediction, but no one in the market is sure that this is the case. Your market signal will look exactly like that of an accurate prediction market, but predict nothing at all.

You can ask for a full probability distribution.  If speculators know they don't know anything, then they will give you a broad distribution that expresses a lot of uncertainty.  This is them telling you they don't know much.  In your nose example, they may just give you the distribution over nose sizes for elite Chinese.  And how is this issue different for prediction vs. decision markets?

In an old and healthy prediction market we would indeed expect to see accurate predictions. But only in the case in which Darwinian forces have been operating for a substantial period of time.

Trader experience does seem to improve their collective accuracy.  But this hardly means accuracy was zero initially.  By now we have a great deal of experience in field and lab prediction markets where accuracy was valuable from day one.  And how does this issue distinguish prediction vs. decision markets?

 The adaptive explanation of market efficiency tells us why prediction markets with pretend money are a complete waste of time. These just aggregate the whims of the participants.

Play money markets do often fail badly, but many of them have done very well.  All else equal I prefer real money markets, but "complete waste of time" is way too strong.  And how does this distinguish prediction vs. decision markets?

A prediction market only makes valid predictions if it can resist feedback, where feedback is any effect of the market's result on the interests of the players. ... With feedback, players with a financial interest in the effect of the prediction have an interest which conflicts with the efficiency of the market.

There are several possible concerns you might have here.  First, traders might sabotage events in the real world to improve their forecast accuracy.  This can be a problem for any forecasting mechanism that rewards accurate contributions, and there are several ways to deal with it.  Also, traders might try to distort the market price because that price influences decisions.  Theory, lab data, and field data suggest this is not much of a problem.  And how do these issues distinguish prediction vs. decision markets?

A prediction market, like any other market, functions only in the general absence of asymmetrical information. It is with some pain that I absorb the realization that a member of the George Mason School is unable to correctly apply this concept. ... The rational approach to a market in which other players have more information than you is not to play. ... This is one of the many reasons why insider trading is illegal.

Er, in pretty much every speculative market that has ever existed, traders have had differing relevant info.  Identical info is a very rare situation.  Some traders had more total info, and the traders who had less would have been well advised to leave.  And how does this distinguish prediction vs. decision markets?

Futarchy is... retarded, ... because ... It must be trained. During the training period, it is an inaccurate prediction market. ... But who wants to run an inaccurate futarchy for a training period? ... Testing the accuracy of a prediction market ... is close to impossible.

After the fact it is quite easy to test for forecast accuracy.  And even if initial accuracy was poor, why not begin if we expect accuracy to greatly improve?  And how does this distinguish prediction vs. decision markets?

In order to make an accurate bet that some policy or other will increase GDP, as opposed to no policy at all, you first need an accurate estimate of GDP assuming no policy change at all. Whose error bars are shorter than the impact you expect from the policy change.  In other words, Professor Hanson is basically expecting the AM antenna on his ghetto blaster to pick up signals from Alpha Centauri.

We can sensibly compare small differences in conditional expectations even when conditional variances are high.  For example, even when you cannot predict the date of your death well, you can reasonable expect a longer lifespan if you exercise today than if you do not exercise today.  The difference in these two expectations can be tiny compared to the variance in your realized lifespan, and yet you can have reasonable beliefs about this difference, and act on them.  See also the presidential nomination decision markets which have been run for many years by IEM and Intrade as proof that decision markets are mechanically feasible, even in situations of great uncertainty.

Professors, Hanson and Abramowicz. Their primary angle seems to be in recommending decision markets to USG.

There is now a prediction market industry, where I do a lot of consulting, most of which is to private companies.  Most of this consulting is on prediction markets, but some are decision markets. 

Consider Professor Hanson's favorite statistic - GDP. If we translate GDP into Apple terms, it is gross revenue. GDP is simply the total sales of all American companies. So suppose our man, at Apple, had access to a perfect Hansonian decision market, which for any policy could calculate whether or not that policy would increase Apple's sales. ... Many bad decisions increase gross revenue. What about costs? What about quality? What about customer service and brand reputation? What about employee morale? Etc, etc, etc.

For firms I recommend forecasting profits or stock price, not revenue.  For nations I suggest a full national welfare; GDP is only a first step to such a measure.

Comments

[Moldbug]

Yvain,

All I can say is that if everyone approached prediction markets with the same skepticism as you, they would be truly useful tools. If they weren't basically illegal, that is.

There is no philosopher's stone. Good government is government by good people. In essence, an iterated prediction market is just a very clever way to employ good people, in a very unusual organizational structure which is unusually tolerant to the presence of bad people in the population. (Not to mention unusually inexpensive from the management perspective.) Treat the tool as what it is, and you can't go wrong. Treat it as a philosopher's stone...

[Robin Hanson]

Moldbug, you do not understand what you read. Our conversation is over.

[Moldbug]

rafal,

Note that if the decision is welfare-diminishing compared to the status quo, there will be a group of people losing more money than the bears gain. They would have a strong incentive to buy the decision back, wouldn't they?

I think that's the cleverest argument on this whole thread! Note, however, that this would create a kind of policy-control contest very familiar to, say, the citizens of the late Roman Republic. I'm not sure it passes Schneier's test. Well, okay, I'm sure it doesn't. On the other hand, that doesn't mean it wouldn't be an improvement on Washington's present policy process...

[Steve Johnson]

Note that if the decision is welfare-diminishing compared to the status quo, there will be a group of people losing more money than the bears gain. They would have a strong incentive to buy the decision back, wouldn't they?

They have the same incentive to buy the decision back as the consumers do to buy an open market in foreign sugar. Unfortunately, corn producers and sugar producers have stronger incentive to buy the market closed.

Concentrated benefits, diffuse costs.

[Moldbug]

Well! I feel it's only appropriate to give Professor Hanson the last word.

Conveniently, I finished the aforementioned paper - and discovered that it concludes with an admirably-motivated, and quite plausibly complete, list of all its spherical cows. (In case it isn't obvious, my objection above was to the last-named cow.) If this doesn't place Professor Hanson in the 99th percentile of academic honesty, I myself am a spherical cow:

Of course the fact that we have a particular model illustrating these results hardly implies that these results always hold in every context. Our model assumes risk-neutrality, normally distributed values and signal errors, interior choices of information quantity, only quantal-response-type irrationality, no meta-signals about the signals of other agents, no transaction costs of trading, no budget constraints, and a single rational manipulator with quadratic manipulation preferences and a commonly known strength of desire to manipulate. However convenient these assumptions may have been for solving the model, one can reasonably question the empirical relevance of models based on them.

The prosecution rests - and calls for leniency.

[Porphyrogenitus]

Whether they're "accurate" depends in large part on definitions of accurate, and who controls those definitions. They'll be different in decision markets (where accuracy depends on policy outcomes) than prediction markets (which simply predict, or not, what outcome will occur).

We face this today and I do not believe we have *truly* objective criterion for evaluating whether the implementation of a policy on any subject of import the least bit controvercial was accurately predicted. That is to say, we already have a lot of data on policy implementations and outcomes, and it's much harder to agree on whether their effects were accurately predicted in advance than with a market that simply predicts whether A or B will happen ("Will Barak Obama's Stimulus Package pass and be signed into law" is a much simpler question than "what will the effect of Stimulous Package A be, will it be better than Stimulous Package B, and will either be better than doing nothing, and over what time frame, and with or without other ripple effects" - CBO's predictions, for example, are considerabily different than the White House's).

In the abstract, people propose Policy A, others say Policy B will be better, some say the status quo is better than either. The advocates of Policy A say it will cost $100 Billion Woolongs and create 1.21 gigajobs. Critics say that it will cost four times that much in the end and produce half as many jobs.

Policy A is enacted, it ends up costing several multiples of what was predicted and net job growth is undetectable. Advocates say it was because it was underfunded at the beginning, and that other factors impacted it, and given how the economy performed, conditions would have been worse without it: It saved jobs. Critics say their criticisms were accurate and that the policy made things worse ultimately, particularly because of unintended consequences (Brawndo consumption went up 400%, causing crop failures and an increase in illiteracy and out of wedlock childbirth, and these things aren't included in the advocate's data regarding whether the policy outcome was good or bad). Adocate's rejoider is that well B, C, and D happened in the interval as well, and without Policy A in place, those things would have excacerbated the situation, so A really saved us from catastrophy.

This is much more open to manipulation than a prediction market in that sense alone.

Also, I think Hanson is well too dismissive of the ability of "Bears" to manipulate outcomes. In a world where George Soros & friends could cause a run on the Pound, it's not hard to foresee that a subset of interested parties who will benefit disproportionately from the implementation of a policy will have more interest in investing enough to make sure it happens than everyone else. Will "Wolves" follow that money, because it is more likely to occur (and thus pay out), or will they vote with the "sheep" (those whose direct interest in the policy choice is less strong, and indeed who may pay the costs and receve no benefit)?

"Smart Money" is more likely to go with the "Bears" on this, in my opinion. Their market behavior will be based on an evaluation of what will "win" rather than which will be the best policy to implement, and everyone else in the market will then be "sheep" (AKA the "marks").

For some reason a lot of the exchanges on "Wolves" vs. "Sheep" reminded me of Phil Helmuth going on rants about "idiots" who have offsuit Queen-Ten and raise. Phil's "idiots" win hands, get to final tables, even win tournaments, despite the fact that though over time - if your time horizon is long enough - the pros do much better than "dead money". When interest and passion is high, a lot of sheep enter the market who otherwise might not be there - and this will make the outcomes different than in the smaller-scale running experiments Hanson foresees as testing and building the case for Decision Markets.

But in this situation the way the sheep will be separated from their money by the wolves is not directly tied to which policy outcome was accurately predicted to be preferable. The decision market will be dominated by those "Wolves" who are most skilled in knowing which decision will dominate, not which one is good policy and which one(s) are worse, and by players with an interest in the outcome (Bears) regardless of it's overall impact (the "I'm All Right, Jack" set). The wolves will prosper in the market from investing correctly in which policy alternative will win, not which one will have the most benefitial impact.

It "may" (to use Professor Hanson's word) be that those don't correlate at all. They "may" even be opposed. I'm not sure I'd bet my life that they will be alligned. I agree with Moldbug that there is no good way to test it - I have little confidence that "experimenting" in a small scale on inconsequential policy decisions won't produce the same outcomes that going big on major choices will. Wolf money will tend to follow, rather than counterballance, Bear money, and everyone else by definition will become Sheep and get shorn.

So in other words, no change.