« Wisdom of Rhode | Main | Less Wrong: Progress Report »

April 24, 2009


I conjecture that the audience's pre-debate opinion is an artifact of how the debate is marketed. In your case, the tagline for the debate was "liberty vs. efficiency". The former is an applause light; the latter is not.

Personally, I haven't watched the debate yet. After attempting to de-bias against the above, I'm still more favorable to what I currently understand as Bryan's position; I'll let you know how my opinion changes after seeing it.

If there were an obvious truth of a particular debate topic, then it's unlikely that there would be a debate. Also, hardly any real-world opinion is unidimensional, as if standing on a line diametrically opposite its competitor. Thus the space of likely adjusted viewpoints lies mostly closer between the opposing views. Think of it as regression to the mean, central tendency, law of large numbers, even a reflection of the principle of least action, all examples of entropy.

But why would you expect anything different, unless you hold a transparent belief in the efficacy of context-narrowing (as practiced in debate) for finding the truth? Is it the apparent crispness of the decision-making that appeals? Bear in mind that prediction markets exploit quite quite the opposite dynamic, harnessing an increased context selected for its winning, versus selecting from a reduced context in order to win.

To promote progress in the social domain, we don't need more context-narrowing debates, but rather more context-expanding explorations, where we discover, not "the truer position", but the more encompassing context containing the smaller "truths" of the competing positions.

I risk being perceived as less than scientific, verging on the postmodernist Far be it from that; I'm trying to convey that science has never dealt in Truth, but only increasing coherence over increasing context, and that is also the direction of increasing good.

Nudged by an online friend, I see I may have been unclear with regard to the distribution of likely adjusted opinions tending to be closer to the center of agreement. In case it wasn't obvious, this is because an opinion does not stand alone, but may be considered as the centroid of a high-dimensional cluster of beliefs, which more often than not, tend to be shared.

How did you get the fit in the top figure? The slope looks approximately right, but the y-axis seems shifted down.

Would it be mad to ask about the actual list of debates, or, say, the first five topics (along with who was in the minority)? I suspect a The Majority is Always Wrong-like effect, where, conditioning on the fact that both sides were willing to take part in the debate at all, the minority expected to win because of better logic, while the majoritarian party was overconfident because they were in the majority. Minorities will avoid most debates unless they think they can win, and their spokespeople are likely to better trained; the majority side spokesperson may just be going along with the majority. It's even possible that the majority of minority positions that are eager for debate, are superior to the corresponding majority positions - though obviously there will be exceptions to this rule even if it is true.

On a continuum from "Where should we eat dinner?" to "Gordian knot" your topics of debate tend toward the latter. On top of that, there is no real incentive to reach consensus in your forums.

In most "real" decisions, problems are simple and consensus is either necessary or preferred. In those cases I'd bet you see trends favoring the initial majority.

I would also take a look at the sample: is it representative of the public or is it representative of the type of people who read sites like OB? I think the second of those groups is way more likely to be composed of people who are able to abandon beliefs in light of new evidence. Given two beliefs with no known objectively correct answer and approximately the same amount of evidence for each, an initial minority should always expect to gain ground in debates among such people.

wait, if audiences move towards 50-50 then more debates isn't necessarily a good thing. You could engineer a series of debates that lead an audience further and further from the truth stepwise.

If Alex is correct, debates will not improve public opinion. It will move opinions toward a 50-50 split regardless of the relative merits of the positions.

I don't really understand the charts.

Shouldn't the two plots be the same data from different view points? But then there wouldn't be just one data point below the neutral line in the first and many in the second.

You speak about 27 debates but there are 26 points in the first and 25 in the second chart, if i did count right.

What do you mean by favourite? I thought " >50% percent for his position before", but most points in the second graph have a x-value < 50

And why don't the axis have the same scale?

What did I miss?

Consider how cheap is a binary choice, even with regard to a complex high-dimensional issue. Consider how easy it is to say "you're either with us or against us" when in the bigger picture it's clear we have much more in common than not. Consider the tendency toward bifurcation along political lines, right and wrong, good and evil, god and devil...Why do the branches of a tree bifurcate rather than something more complex?

Consider a simple electronic feedback loop, controlling for only a single variable such as voltage. As it becomes "locked", it will tend toward bipolar symmetrical variation about its set point.

A more complex system, dancing in many dimensions, may be considered not "locked" but adapted, such that its dynamical nature tends to mirror that of its environment. Is it surprising that studies of twins, in regard to traits of adaptation, often show variation correlated roughly 50% nature, 50% environment?

What does this imply for prediction of preferences in general, as we move from the common root of our evolved nature, to the tips of our leaves exploring their adjacent possible?

But also consider how cheap it is to dismiss choices as false dichotomies.

When a living thing kills another living thing, in some way it subsumes its useful material and leaves the rest. It's the same with sides in a debate: the victor is not categorically right, even if he gets a consensus victory. If he is smart he will adopt the better points of his opponents argument.

The implication is that anyone who commits to a belief is committed to being wrong. The truth-promoting strategy is "strong opinions, loosely held." The upshot is that preference is only an arrow in the direction of the more probably correct answer and to refine our answers we must gather new data and find new patterns in old data.

@Stephen: Yes, you've reinforced my point. And to say "an arrow in the direction of the more probably correct answer" rather than "...the more likely correct answer" were it intentional, is the distinction I've been trying to impress on our dear Archimedes.

Acinonys, there were probably a couple of people being neutral both before and after the debate, so the sum doesn't give 100% and favourite can have less than 50%.

Nevertheless, did the audience know whether they are on the favourite or contrarian side? Are there any data about strength of their beliefs before and after the debate? Has the audience consisted of people who are aware of cognitive biases or it has been a typical sample of general public?

Robin, are the results of the pre-debate votes know to the audience? Your explanation would predict an equalizing effect even if the audience did not know or were mistaken about which view was initially most popular, while Alex’s preferred explanation depends on the audience knowing which position is the “underdog.” (On Alex’s view, the audience should instead switch toward the favorite if they mislead about which is the favorite initially.)

Also, it looks like the source data includes percent undecided. Does your analysis change if you consider these percents? (If the only effect of the debate were to cause undecided people to commit randomly, then that would increase the percentage for the underdog position, but probably should not be interpreted as an increase in support.)

My guess is that the post-debate opinion needs to be measured at a later time.

I would expect a debate between two qualified opponents to present a mutually comparable number of arguments that the presenters have researched and understand, to an audience who does not. As a result, the real state of people's minds after a debate is not a changed opinion, it is confusion. When asked what opinion they now lean to, people will respond as if they were asked the question "which side's arguments do you recall the most right now". If the debaters were equally good, this is probably a linear function of how much time each debater got to speak.

Weighing the merits of substantive arguments is a bit like compiling code. It requires time, attention and precision, and cannot be done on the fly during a debate.

Measure people's opinions a week or a month after the debate, and you will know how much they were really affected.

I was confused by the charts at first, too, but I think I've figured it out.

Each debate reports the polling results before and after, as percent favoring, percent opposed, and percent undecided. The "contrarian" position is whichever has the smaller percentage supporting it initially, and the "favorite" position is the one with the larger percentage. Because the percent undecided changes before and after, contrarian and favorite positions do not move the same amount or even necessarily the same direction. So we see rather different effects in the two charts.

The red lines are not fits, they are the X=Y lines; if people's positions did not change at all, all of the points would be on a red line.

One thing I saw (not shown in the charts) is that undecided seems to (at least, almost) always decrease. People feel that they are becoming more certain in their views. That argues against interpretations where people get confused by being exposed to both sides.

It seems plausible that much of the change in position comes from undecided people adopting one of the two sides. That wouldn't explain all of it but it could explain a lot. Taborek argues that the before-undecided are lying, and were not really undecided. (Maybe they are following a social convention of adopting an unbiased stance before hearing the arguments.) Then you have to assume that these pseudo-undecideds have more moderate views than those who are willing to admit to having made up their minds beforehand.

Visually, the effect looks stronger with the contrarian than the favorite position. That would mean (in this interpretation) that people with the unpopular view are more likely to lie and claim to be undecided beforehand. That could make sense if people are used to hiding unpopular views.

Great video - shouldn't it have had its own thread, though?

...and where is *your* "economist's hat"? Please provide pictures! ;-)

Robin, I second Eliezer's desire to see a list of topics. If counterintuitive positions tend to be behind at first and ahead at the end, it may be a sign that debates are valuable.

About the idea that it's easier to steal votes from the other side if they have a large initial lead: shouldn't the arguments of a position with, for example, 90% support be good enough to convince 9 people for every 1 the other side convinces? If they're not, why does the position deserve to have 90% support?

Hal is right about the details, for anyone else who was confused.

Eliezer and Gary, follow the link for the titles.

blink, watch or listen to a debate to find out what folks were told.

Perhaps the solution is to simply debate to audiences that are overwhelmingly composed of the opposition. If the before verdict is unanimous, then conversion can only go one way.

Perhaps it's better to view debates as providing social evidence in both directions, rather than anything actually persuasive.

The comments to this entry are closed.

Less Wrong (sister site)

May 2009

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30