Overcoming Bias

Desert Errors

A story worth pondering:

In the summer of 1942 [Edward] Adolph, a physiologist at the University of Rochester in New York state, wanted to find out how people could live and work efficiently in the desert and how to get the best out of them. ...

Adolph was the first to test the presumptions most people still have about what to do if forced to make any sort of effort in extreme heat. Most, he discovered, were myths. Stripping to T-shirt and shorts, for instance, is not the best way to cope with dehydrating conditions. Long sleeves and long trousers may feel hotter, but they'll slow the loss of water. Nor is there any point in rationing water when supplies are low. Putting off drinking it merely makes you unhappier sooner. "It is better," wrote Adolph, "to have the water inside you than to carry it."

The most important of Adolph's findings was the simplest: drinking during exercise improves performance. Today, we take this for granted, but generations of coaches and distance runners were taught that drinking during exercise was for wimps. ...

Adolph tested the old assumptions by splitting his soldiers into two groups. Both marched through the desert for up to 8 hours during the time of year when the average afternoon high was 42°C. The soldiers in one group were allowed to drink as much water as they wanted and the others weren't allowed any. The results were clear: the drinkers outperformed the non-drinkers. ...

His findings stayed secret until 1947, when he was allowed to publish his pioneering Physiology of Man in the Desert. It went almost entirely unnoticed. In the late 1960s, marathon runners were still advised not to drink during races and until 1977, runners in international competitions were banned from taking water in the first 11 kilometres and after that were allowed water only every 5 kilometres.

So not only were authorities dead wrong, but they were so confidently wrong that, in the name of helping runners, they paternalistically forced runners to do the exact worst thing!  How could authorities be so wrong for so long on something that was so easy to personally test, and with such huge consequences?  And how could they remain wrong for three decades after careful study had proved them wrong?

Seek Superstar Slavery

The latest Review of Economic Studies has a great article (ungated here) by Marko Tervio.  I'll summarize.

CEOs, actors, directors, musicians, authors, and athletes make big bucks because:

1. Desired abilities are rare and lasting.
2. It is very expensive to try someone new.
3. Everyone can see which trials worked or not.
4. Winners are free to demand more money or walk.

Given these conditions, a few proven winners make big bucks, and few new folks get tried.  After all, a new trial who wins will soon demand as much as other winners.  Here winners avoid retirement to keep milking their gravy train, and small biases in weak signals on new guys to try can magnify into great social injustice. 

Condition 4 is crucial.  When long term deals are allowed, more folks are tried, because a few successes can pay for lots of failures.  Folks being tried get paid more, and there are more better winners who retire earlier and are paid less even when free to walk.  Distorted signals about who to try matter less.  Such long term deal gains were realized, for example, in the US movie studio system of the 1920-40s, the old US American baseball club system, and even now via exclusive long-term music album deals.

Over the last century, however, legislatures and courts have consistently moved to limit and prohibit such long term contracts, thereby increasing inequality and decreasing productivity.  France even forbids artists from selling the full value of their paintings.  The key tipping factor here seems to me to be a public displeased by seeing gains by admired musicians, actors, athletes, artists etc. going to less admired others.  The word "slavery" is often invoked. For example, music fans can be outraged to see their favorite musicians shackled to ungenerous album deals. 

So our vast wage inequality of superstar CEOs, artists, athletes, etc. is caused not by a lack of sensible regulation to limit random cruelties of unfettered markets, but by a public preferring its heroes unshackled, even if those heros had preferred otherwise. Now maybe insuring heroes against financial variations imposes a negative externality on wider admiring publics, one large enough to justify preventing long term deals.  But for now count me as skeptical; I'd rather allow CEO and other superhero "slavery," for their good and ours. 

Who Cheers The Referee?

Almost no one, that's who.  Oh folks may cheer a ruling favoring their side, but that is hardly the same.  On average referees mostly get complaints from all sides.   Who asks for their autograph, or wants to grow up to be one?

Similarly who cheers the officials who keep elections fair, or the teachers who grade fairly?  Inspiring stories are told of folks who win legal cases or music competitions, but what stories are told of fair neutral judges who make sure the right people win?  After all, competition stories are not nearly as inspiring with arbitrary or corrupt judges.  Oh judges are sometimes celebrated, but for supporting the "good" side, not for making a fair neutral evaluation.

Sure we give lip service to fairness, and we may sincerely believe that we care about it, but that mostly expresses itself as sincere outrage when our side is treated unfairly.  We usually can't be bothered to pay much attention to help settle disputes in which we have little stake.  So if you want to be celebrated and gain social support, take sides.  But if you want to instead do the most good for the world, consider pulling the rope sideways instead of joining the tug-o-war.  Consider being a neutral arbitrator, or better yet consider developing better systems of arbitration and evaluation.

Insincere Cheers

As my son is a senior marching band drum major, Friday I went to a local high school football game to film him for posterity.  And I noticed something obvious.

People like to cheer their teams on, but prefer to be encouraged by announcers, cheerleaders, bands, pep squads, and so on.  And while fans seem to care that these various leaders of cheer are impressive and loyal, fans don't seem to care nearly as much if these leaders sound sincere.  The announcer carefully controlled his voice inflections, the cheerleaders carefully synchronized their arm movements, and everything they said was consistently loyal, but you couldn't possibly have mistaken them for people who deeply and sincerely believed the words they spoke.

Sport cheers are often considered an analogy to political and ideological partisanship; we like to vote and declare our opinions similar to the way we like to cheer sport teams.  We prefer to support positions that have have loyal impressive cheerleaders.  It is nice if those cheerleaders are also sincere, but it is not especially important to us.  It is, however, important that our idea cheerleaders be impressive and loyal.  We might eagerly point out when leaders on the other side sound insincere, but that is mostly hypocrisy, since we don't care much about our leaders' sincerity.   

Just as we care more that our team wins than that they were actually the strongest team, we probably care more that our idea sides look good than that our ideas are true.

Poker Vs Chess

A computer has beaten top human poker players:

Humanity was dealt a decisive blow by a poker-playing artificial intelligence program called Polaris during the Man-Machine Poker Competition in Las Vegas.  Poker champs fought the AI system to a draw, then won in the first two of four rounds (each round had Polaris playing 500 hands against two humans, whose points were averaged.) But in the final two rounds of the match, Polaris beat both human teams, two wins out of four, with one loss and one draw. IBM's Deep Blue beat chess champion Gary Kasparov in 1997.

This has gained very little news coverage, in stark contrast to the Chess case, even though far more people play poker than chess.  To explain both facts, note that most people can more easily see that they lack chess than poker expertiSe.  They can watch a poker expert fold or raise and imagine that they would have done the same, but know they have no idea why a chess expert makes his moves.

This fools people into thinking they could be a poker champion, which is why so many more people try at poker.  And being less impressed by existing poker champions, people are less impressed by a computer who beats those champions.  The moral: beware of underestimating computers by underestimating the difficulty of the tasks at which they excel. 

More Referee Bias

Analyzing the neutrality of referees during 12 German premier league (1. Bundesliga) soccer seasons, this paper documents evidence that social forces influence agents' decisions. Referees, who are appointed to be impartial, tend to favor the home team by systematically awarding more stoppage time in close matches in which the home team is behind. They also favor the home team in decisions to award goals and penalty kicks. Crowd composition affects the size and the direction of the bias, and the crowd's proximity to the field is related to the quality of refereeing.

That is from Economic Inquiry. 

Don't Do Something

The October Journal of Economic Psychology says goalies do too much:

In soccer penalty kicks, goalkeepers choose their action before they can clearly observe the kick direction. An analysis of 286 penalty kicks in top leagues and championships worldwide shows that given the probability distribution of kick direction, the optimal strategy for goalkeepers is to stay in the goal's center. Goalkeepers, however, almost always jump right or left. ... The claim that jumping is the norm is supported by a second study, a survey conducted with 32 top professional goalkeepers. The seemingly biased decision making is particularly striking since the goalkeepers have huge incentives to make correct decisions, and it is a decision they encounter frequently. Finally, we discuss several implications of the action/omission bias for economics and management. ...

The action/omission bias might affect the decision of investors whether to change their portfolio (action) or not (inaction). It can affect the choice of managers whether to leave their company's strategy or investments unchanged (inaction), or to change them (action). The bias may also have implications for the decision of workers whether to stay in their job (inaction) or look for a better job (action), and one's decision whether to re-locate to another city or not. In the macro-economic level, the action/omission bias may also affect decisions made by governments and central banks whether to change various policy variables (interest rates, tax rates, various types of expenditures, etc.), or leave them unchanged.

Of course if you do nothing you risk being seen as cowardly or ignorant, so it might not pay you to correct this bias.   

Why I'm Betting on the Red Sox

One of the most pervasive beliefs among sports fans is a belief in "streaks".  I cannot tell you the number of times I have heard sports commentators this week tell us that the Rockies have won 21 of their last 22 games.  And this alone is the reason that I'm betting against the Rockies.

The "hot hand bias" was first documented in a fascinating paper by Tom Gilovich, Robert Vallone and Amos Tversky.  That original paper (available here) is a wonderful read, showing that the widespread belief among basketball fans of a strong "hot hand" is simply false.  That is, today's streak doesn't predict tomorrow's behavior.  I love teaching this paper to my MBA students, simply because they don't believe it.  The hot hand fallacy is a wake-up to how pervasive bias can be.  A nice example of how sports can yield very convincing teaching metaphors.

A subsequent literature has developed showing that many (most?) of the sports statistics that ESPN loves to share with us, are simply useless as inputs for forecasting the future.  It seems that our brains are a bit too willing to try to find order, even in a world where chaos reigns.  This leads me to believe that most baseball fans are a bit too optimistic that the Rockies' streak will persist.

Some will protest that subsequent research has found evidence of streakiness in specific sports.  I agree.  But this is beside the point: it is essentially an observation about sports.  What is more relevant here (and no-one has convincingly refuted) is that sports fans tend to believe that streakiness is even stronger. 

Believe it or not, there is now an entire blog devoted to the hot hand and streakiness in sports - read more here.  Or if you are interested in the performance of streaky baseball teams in the post-season, read this analysis at hardballtimes.com.

(OK, there is one more reason I'm betting on the Red Sox: I went to graduate school in Beantown, and learned to love baseball at Fenway.)

Belief as Attire

I have so far distinguished between belief as anticipation-controller, belief in belief, professing and cheering.  Of these, we might call anticipation-controlling beliefs "proper beliefs" and the other forms "improper belief".  A proper belief can be wrong or irrational, e.g., someone who genuinely anticipates that prayer will cure her sick baby, but the other forms are arguably "not belief at all".

Yet another form of improper belief is belief as group-identification - as a way of belonging.  Robin Hanson uses the excellent metaphor of wearing unusual clothing, a group uniform like a priest's vestments or a Jewish skullcap, and so I will call this "belief as attire".

In terms of humanly realistic psychology, the Muslims who flew planes into the World Trade Center undoubtedly saw themselves as heroes defending truth, justice, and the Islamic Way from hideous alien monsters a la the movie Independence Day.  Only a very inexperienced nerd, the sort of nerd who has no idea how non-nerds see the world, would say this out loud in an Alabama bar.  It is not an American thing to say.  The American thing to say is that the terrorists "hate our freedom" and that flying a plane into a building is a "cowardly act".  You cannot say the phrases "heroic self-sacrifice" and "suicide bomber" in the same sentence, even for the sake of accurately describing how the Enemy sees the world.   The very concept of the courage and altruism of a suicide bomber is Enemy attire - you can tell, because the Enemy talks about it.  The cowardice and sociopathy of a suicide bomber is American attire.  There are no quote marks you can use to talk about how the Enemy sees the world; it would be like dressing up as a Nazi for Halloween.

Calibration in chess

Daniel Kahneman posted the following on the Judgment and Decision Making site:

Have there been studies of the calibration of expert players in judgments of chess situations -- e.g., probability that white will win?

In terms of the amount and quality experience and feedback, chess players are at least as privileged as weather forecasters and racetrack bettors -- but they don't have the experience of expressing their judgments in probabilities. I [Kahneman] am guessing that the distinction between a game that is "certainly lost" and "probably lost" is one that very good players can make reliably, but I know of no evidence.

Despite knowing much less about decision making and (likely) less about chess than Kahneman, I have three conjectures: