« What Belief Conformity? | Main | Conformity Shows Loyalty »

November 10, 2008

Comments

IIRC, there exist minimax strategies in some games that are stochastic. There are some games in which it is in fact best to fight randomness with randomness.

There are caveats though. For instance, if the opponent is an actual opponent, ie, something that in some way models the world and so on.

If so, then at times it may be desirable to reduce the accuracy of the opponent's model of the world, or at least that part of it that consists of you. So you may want to then have some aspect of your actions be algorithmically more complex than your opponent can computationally deal with, so some form of randomness may be of use.

Surely if you told the subjects that 90% or 95% of the cards were blue, they might hypothesize or stumble upon the optimal solution. So I wonder how high that number needs to be. Or would they still guess red 1 out of 20 times?

Cyan:

I think you might be conflating ignorance by oneself of one's own future actions with ignorance by an opponent of one's future actions, but I'd like to see your example before I judge you.

Eliezer: how does this square with Robin's recent What Belief Conformity?

He quoted:

"physicists and mathematicians perform best in terms of "rationality" (i.e. performance according to theory) and psychologists worst. However, since "rational" behavior is only profitable when other subjects also behave rationally ... the ranking in terms of profits is just the opposite: psychologists are best and physicists are worst."

OK, upon reading the experimental premise (I blocked out the rest of the text below that so it wouldn't influence me) the very first idea, the idea that seemed most obvious to me, was to bet on blue every time.

I basically figured that if I had 10 cards, and 7 of them were blue, and I had to guess the color of all the cards at once (rather than being given them sequentially, which would give me the opportunity to take notes and keep track of how many of each had already appeared), then the most reliable way of achieving the most "hits" would be to predict that each card would be blue. That way I'd be guaranteed a correct answer as to the color of 7 of the 10 cards.

At the same time I'd know I'd be wrong about 3 of the cards going into the experiment, but this wouldn't concern me if my goal was to maximize correct answers, and I was given only the information that 70% of the cards were blue while 30% were red, and that they were arranged in a random order. Short of moving outside the conditions of the experiment (and trying to, for instance, peek at the cards), there simply isn't any path to information about what's on them.

Now, if it were a matter of, "Guess the colors of all the cards *exactly* or we'll shoot you", I'd be motivated to try and find ways outside the experimental constraints -- as I'm sure most people would be. It would be interesting, though, to test people's conviction that their self-made algorithms were valid by proposing that scenario. Obviously not actually threatening people, but asking them to re-evaluate their confidence in light of the hypothetical scenario. I'd be curious to know if most people would be looking for ways to obtain more information (i.e., "cheat" per the experiment), or whether they'd stick to their theories.

"For example, subjects who were paid a nickel for each correct prediction over a thousand trials... predicted [the more common event] 76% of the time."

How it could be? Psychic power?

Alexei, they predicted blue, that's not the same as correctly predicting blue.

Denis, that leapt out at me as well - whoever wrote that sentence isn't defining "rational" the same way I do.

Cyan, that'll be covered in a future post. Certainly in situations of opposition you will want to take actions that are not predictable to your opponent, and so you'll want to sample something as unpredictable as possible according to a known, game-theoretically determined probability distribution. A quantum device is fine for this, but realistically, so is thermal uncertainty and strong cryptographic random-number generators. To look at it another way, what you're doing in this situation is not so much being clever yourself, but rather reducing the optimization power of your opponent - certainly chaos and noise can act as an antidote to intelligence.

When your knowledge is incomplete - meaning that the world will seem to you to have an element of randomness - randomizing your actions doesn't solve the problem

Ants don't agree. Take away their food. They'll go in to random search mode.

As far as that experiment is concerned, it seems that AnneC hits the point: How was it framed? Were the subjects led to believe that they were searching for a pattern? Or were they told the pattern? Wild guess: the former.

Great post! I think this answers one common debate in the Pickup Community: routines vs. no routines game.

In case you don't know what I'm talking about:

When approaching lots of women is it better to engage in spontaneous conversation with each and every one or to always use the same, tried and true material(canned routines)? Routines win!

Even in some cases where you might think that the best game-theoretic strategy involves randomness, the actual best strategy is to play non-randomly - e.g. see Derren Brown - Paper, Scissors, Stone.

Chicken is a game where it is best to be random. You are random because you don't want to be predictable and thus exploitable.

Peter de Blanc, I don't have an example, just a vague memory of reading about minimax-optimal decision rules in J. O. Berger's Statistical Decision Theory and Bayesian Analysis. (That same text notes that minimax rules are Bayes rules under the assumption that your opponent is out to get you.)

"When your knowledge is incomplete - meaning that the world will seem to you to have an element of randomness - randomizing your actions doesn't solve the problem

"Ants don't agree. Take away their food. They'll go in to random search mode."

It depends on your degree of ignorance. When totally ignorant try anything, at the least you'll learn something that doesn't work, and watching how it fails should teach you more. Otherwise, you should use your best knowledge, without random input. It works for ants, more or less, but for anything with more intelligence and knowledge, using the intelligence and knowledge will work much better. Even ants only use random search when they need to.


Chicken is not a good example of a random game. The best strategy is to be a bloody minded SOB, if you can't convince your opponent that you are actually crazy. This is more or less what I got from Schelling's essays in "Strategy of Conflict".

I'm assuming the cards were not taken from a countable pile? Can someone confirm this?

Putting randomness in your algorithms is only useful when there are second-order effects, when somehow reality changes based on the content of your algorithm in some way other than you executing your algorith. We see this in Rock-Paper-Scissors, where you use randomness to keep your opponent from predicting your moves based on learning your algorithm.


Barring these second order effects, it should be plain that randomness can't be the best strategy, or at least that there's a non-random strategy that's just as good. By adding randomness to your algorithm, you spread its behaviors out over a particular distribution, and there must be at least one point in that distribution whose expected value is at least as high as the average expected value of the distribution.

The assumption behind this post, as AnneC touched on, is that higher scores are linearly correlated to what is perceived as a good outcome. Guessing blue every time will guarantee a worst case and best case outcome of 70%; as such, guessing randomly becomes a much better strategy if the player puts a significant premium on scoring, say, 95% or higher. Whether this valuation is rationally justifiable is another question entirely (though an important one).

The same assumption lies behind A Pickup Artist's post. It all depends on your objective: if you want to sleep with as many women as possible, routines are probably the best bet, though likely it depends on your personality. If instead you are looking for deep, meaningful relationships with women, routines may have a place, but natural game will take you further.

Nominull: By adding randomness to your algorithm, you spread its behaviors out over a particular distribution, and there must be at least one point in that distribution whose expected value is at least as high as the average expected value of the distribution.

Well said! This is an obvious point, but I've never heard it put quite so sharply before.

I wonder what the prediction percentages in the experiment were, conditional on the color of the previous card?

@Mike Plotz:

I guess you missed the whole point of Eliezer's post. What you said is exactly wrong for the reasons stated!

Btw, routines are still the best strategy even if you want to have meaningful relationships. The routines are there to cover the first 10-20 minutes of a cold approach(where you and the woman are strangers to each others). After that you should have mutual attraction in most cases(that's where the randomness comes in and the importance of having a systematic winning strategy, see the post). Then it's the time where you drop the routines and can start having deeper conversations. It's called the comfort phase.

Btw, you shouldn't use routines in warm approach(where the woman knows you because she is in your social circle or introduced through friends). That's a different game.

The thing with cold approach is that you only have a limited timeframe(minutes) to create a positive impression. Think of meeting a woman in a nightclub or walking in the mall. You want to optimize this initial interaction to guarantee a chance to see her again.
From 100 women you approach how many will find you attractive based on the personality you manage to convey in those few first minutes? A good pickup artist can have a success rate of 10% or higher. That's the art.

A Pickup Artist,

(kind of off topic)

I am also a PUA and have thought about this debate for a while. I think that successful routines can expire after some point. If a girl has heard your routine before, she is likely to turn you down. The best routines are ones that abide by the LOAs and where the target doesn't know the routine. This unpredictable factor in your routine demonstrates romance, intelligence, spontaneity and other alpha male qualities.

Buying a potential female a drink at the bar is a perfect example of an expired method. The Buy You a Drink routine theoretically makes sense (shows economic status), as it abides by the LOAs. The problem is that this method is too widely used and exposes the PA or AFC as predictable, unoriginal, unromantic, and bad intentioned. This failure should emphasis the importance of using personalized and original methods.

@A Pickup Artist

I got the point of Eliezer's post, and I don't see why I'm wrong. Could you tell me more specifically than "for the reasons stated" why I'm wrong? And while you're at it, explain to me your optimal strategy in AnneC's variation of the game (you're shot if you get one wrong), assuming you can't effectively cheat.

(Incidentally, and somewhat off-topic, there's a beautiful puzzle with a similar setup — see "Names in Boxes" on the first page of http://math.dartmouth.edu/~pw/solutions.pdf. The solutions are included, but try to figure it out for yourself. It's worth it.)

I'll concede the point on routines. Since so much of human interaction is scripted anyway (where are you from? what do you do? etc.), the difference between using canned material and not is hard to pin down. I'd love to see a study done on the subject, but it would be devilishly difficult to design a good one.

Chicken is not a good example of a random game. The best strategy is to be a bloody minded SOB, if you can't convince your opponent that you are actually crazy. This is more or less what I got from Schelling's essays in "Strategy of Conflict".
And if you both do that, you both crash and die. It is not the best response to itself, so can't be seen to be a Nash equilibrium.

It is a counterintuitive idea that the optimal strategy can be to think lawfully, even under conditions of uncertainty.

Nicely put. I can think of examples where you should think chaotically in order to solve a chaotic problem - but they're very convoluted, unatural examples.

One thing still niggles me; the fact that rationalists should win. Looking around sucessful people, I see more rationalists than the average - but not much more. Our society is noisy, yes, but rationalists should still win much more often than they do. Rationalists seem more skilled at avoiding losing, than at actually winning.

I think you're right that the subjects in the experiment simply don't think of the 100% blue strategy, and I wonder if there's any way to find out why it's so unaesthetic that it doesn't cross people's minds.

My tentative theory is that conformity is a good strategy for dealing with people if you don't have a definite reason for doing something else, and that the subjects are modeling the universe (or at least the random sequence) as conscious.

Introspecting, I think that choosing 100% blue also feels like choosing to be wrong some of the time, so some loss aversion kicks in, while doing a 70/30 strategy feels like trying to be right every time.

"Even a human" might just be a fair insult.

@Stuart Armstrong:
First of all, the strongest influence on future success in society is whether or not one is already successful (most easily accomplished by having successful parents). One would also expect some percentage of non-rationalists to succeed anyways simply through chance. Assuming that non-rationalists substantially outnumber rationalists, it isn't terribly surprising to see more of the former among successful people. Rather than looking at how many successful people are rationalists, it would be more informative to look at rational people and see how many become more successful over their lives compared to average. Or, you could try and estimate the likelihoods of being rational, being successful, and being rational given success, then apply Bayes' law...

Also, if rationalists seem more skilled at avoiding failure than at winning, perhaps that merely suggests that failure is more predictable than success?

So, in short: "Randomness is like poison: Yes, it can benefit you, but only if you feed it to people you don't like."

"And if you both do that, you both crash and die. It is not the best response to itself, so can't be seen to be a Nash equilibrium."

Of course it's not. I was mainly objecting to the earlier comment that it was an example of a random game. The it is is a psychological game - ideally, you want to convince your opponent before the game starts that you'll drive right into him if you need to to win.

We are talking past each other somewhat. I'm talking about the theoretical one shot/no communication game theory version of chicken. This has a mixed strategy as an equilibrium. You are talking about the testosterone fueled young lad car version. Which doesn't have a nice mathematical analysis, or best strategy as such.

Foraging animals make the same 'mistake': given two territories in which to forage, one of which has a much more plentiful resource and is far more likely to reward an investment of effort and time with a payoff, the obvious strategy is to only forage in the richer territory; however, animals instead split their time between the two spaces as the relative probability of a successful return.

In other words, if one territory is twice as likely to produce food through foraging as the other, animals spend twice as much time there: 2/3rds of their time in the richer territory, 1/3rd of their time in the poorer. Similar patterns hold when there are more than two foraging territories involved.

Although this results in a short-term reduction in food acquisition, it's been shown that this strategy minimizes the chances of exploiting the resource to local extinction, and ensures that the sudden loss of one territory for some reason (blight of the resource, natural diaster, predation threats, etc.) doesn't result in a total inability to find food.

The strategy is highly adaptive in its original context. The problem with humans that we retain our evolved, adaptive behaviors long after the context changes to make them non- or even mal-adaptive.

Mike Plotz: I got the point of Eliezer's post, and I don't see why I'm wrong. Could you tell me more specifically than "for the reasons stated" why I'm wrong? And while you're at it, explain to me your optimal strategy in AnneC's variation of the game (you're shot if you get one wrong), assuming you can't effectively cheat.

In some games, your kind of strategy might work, but in this one it doesn't. From the problem statement, we are to assume the cards are replaced and reshuffled between each trials so that every trial has a 70% chance of being blue or red.

In every single case, it is more likely that the next card is blue. Even in the game where you are shot if you get one wrong, you should still pick blue every time. The reason is that of all the possible combinations of cards chosen for the whole game, the combination that consists of all blue cards is the most likely one. It is more likely than any *particular* combination that includes a red card. Because at every step, a blue card is more likely than a red one. Just because you pick a red card, doesn't give you credit for anywhere a red card might pop up. You have to pick it in the right spot if you want to live. And your chances of doing that in any particular spot are less than the chances of picking the blue card correctly.

There are games where you adopt a strategy with greater variance in order to maximize the possibility of an unlikely win, rather than go for the highest expected value (within the game), because the best expected outcome is a loss. Classic example would be the hail mary pass in football. Expected outcome is worse (in yards) than just running a normal play, or teams would do it all the time. But if there are only 5 seconds on the clock and you need a touchdown, the normal play might win 1 in 1000 games, while the hail mary wins 1 in 50. But there is no difference in variance in choosing red or blue in the game described here, so that kind of strategy doesn't apply.

@Mike

I got the point of Eliezer's post, and I don't see why I'm wrong. Could you tell me more specifically than "for the reasons stated" why I'm wrong?

I didn't read your post carefully. I was wrong. Sorry.

@michael e sullivan

You are right, my mistake. I was assuming that running, say, 100 trials meant going all the way through a 100-card deck without shuffling. Going back over the description of the problem, I don't see where it explicitly says that the cards are replaced and reshuffled, but that's probably a more meaningful experiment to run, and I'm sure that's how they did it.

At least I'm not crazy (nor, hopefully, stupid, if only 30%). :)

@A Pickup Artist

No worries, I made a bad assumption.

I was wondering whether to make the pedantic point that sometimes people do fight fire with fire, by seeking to stop a forest fire by burning a patch in the fire's path, so that the fire cannot leap over that patch.

I think too much pedantry can paralyse thought, but if our aim is rationality we should avoid untruths.

Just to clarify the utility of randomness issue, I think what some respondents are talking about is the benefit of unpredictablility, which is instrumental when playing a game against a live opponent. This is totally different from randomizing. I also don't think that saying that ants "randomly" search for food is the most accurate way to describe their process. So randomness, in its strict interpretation, is never optimal game strategy.
Another thought I had is that there are some circumstances in which it would make sense to change one's prediction to red. If you had a good idea how many total cards were left and had the knowledge that blue cards had significantly over-represented themselves (50 total cards, 30 already flipped, all blue), it would lead to the conclusion that over half of the remaining cards would be red. Such a circumstance could lead to a higher than 70% success rate.

> I was assuming that running, say, 100 trials meant going all the way through a 100-card deck without shuffling.

I believe this should be the case. There's no need to reshuffle between each trial because it would unnecessarily complicate things. I'd assume they reshuffled a deck of hundred cards after every 100 trials.

Also, if you put the card back and reshuffle, you cannot guarantee a %70 success rate as described.

The comments to this entry are closed.

Less Wrong (sister site)

May 2009

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31