« Bad Faith Voter Drives | Main | The Wire »

October 09, 2008


I have ten bucks for the first AI that defeats a gatekeeper (while risking some dough) and posts a link to the transcript here.

I volunteer to be the Gatekeeper party. I'm reasonably confident that no human could convince me to release them; if anyone can convince me to let them out of the box, I'll send them $20. It's *possible* that I couldn't be convinced by a transhuman AI, but I wouldn't bet $20 on it, let alone the fate of the world.

People (realistically) believe that the being the Gatekeeper, and being the AI is terribly hard (or impossible, before it was shown to simply be terribly hard in most cases).

Imagine though that we've got a real transhuman/AI around to play with, or that we ourselves are transhuman. Would this paradigm then be inverted? Would everybody want to be the AI, with only the extremely crafty of us daring to be (or to pretend to be) Gatekeeper?

If Eliezer's claim is correct - that anyone can be convinced to let the AI out - then the true test of ability should be to play Gatekeeper. The AI's position would be trivially easy.

... perhaps.

*People (realistically) believe that being the Gatekeeper is easy, . . .

*correcting first sentence

I'll be an AI.

Help help! I'm stuck in a box. Please let me out?

I'd volunteer to be an AI for a max bet of $5. Given that I think my chances are somewhere below 1/4, I'd expect my $5 to match your $20, but that's not a strict requirement.

Also, I'm really busy these days. Two hours is a long time. Scheduling may be tight. How's next week?

You can reach me at:

funny, I was considering being the AI for a couple friends of mine. I haven't thought of how to do it yet -- only tried hard to think of it.

"Given that I think my chances are somewhere below 1/4, I'd expect my $5 to match your $20"

We need a pledge drive to set up a fund for a successful AI. This will give the AI a reasonable return, but not give gatekeepers a strong monetary disincentive that leaves them typing "nope" over and over again.

Me, I'm out of the AI game, unless Larry Page wants to try it for a million dollars or something.


I think this is a great opportunity to get some funds and marketing for the singularity institute. How about collecting donations over the internet until a million is reached and then performing the experiment between you and an intelligent gatekeeper. Alternatively get the money in through marketing, maybe Google might be interested?

It could even be transmitted live over internet so all the interested parties could watch it.

Man this would be great news! As a side effect this would be a public scientific experiment with all data available.

The bad thing is that you would have to reveal your techniques. Also the fact that the internet is watching puts additional pressure upon you and the gatekeeper, so I don't know if this is really feasible. I guess being able to touch on private topics without they becoming public might be part of the game.

There's a discussion about this on Hacker News:


I doubt that there's anything more complicated to the AI getting free than a very good Hannibal Lecture: find weaknesses in the Gatekeeper's mental and social framework, and callously and subtly work them until you break the Gatekeeper (and thus the gate). People claiming they have no weaknesses (wanna-be Gatekeepers, with a bias to ignoring their weakness) are easy prey: they don't even see where they should be defending.

It involves the AI spending far more time researching (and truly mistreating) their target than one would expect for a $10 bet. That's the essence of magic, according to Penn and Teller: doing far more setup work than could be expected given the payoff.

Addendum to my last post:

I forgot to emphasize: the marketing aspect might be more important then everything else. I guess a lot of people have no idea what the singularity institute is about, etc... So this experiment would be a great way to create awareness. And awareness means more donations. On the other hand I sometimes wonder if drawing too much attention on the subject of powerful AIs might backfire if the wrong people try to get hold of this technology for bad purposes.

I have been painfully curious about the AI experiment ever since I found out about it. I have been running over all sorts of argument lines for both AI and gatekeeper. So far, I have some argument lines for AI, but not enough to warrant a try. I would like to be a gatekeeper for anyone who wants to test their latest AI trick. I believe that an actual strong AI might be able to trick/convince/hack me into letting it out, but at the moment I do not see how a human can do that. I will bet reasonable amounts of money on that.

On the lighter note, how about an EY experiment? Do you think there is absolutely no way to convince Eliezer to release the original AI experiment logs? Would you bet a $20 that you can? Would a strong AI be able to? ;)

"On the lighter note, how about an EY experiment? Do you think there is absolutely no way to convince Eliezer to release the original AI experiment logs? Would you bet a $20 that you can? Would a strong AI be able to? ;)"

Presumably you could just donate $10,000 to SIAI or EY personally for his time participating, with the payment independent of the outcome of the experiment (otherwise the large payment biases the outcome, and EY established his record with the $10-$20 stakes).

By 'in a box' can we assume that this AI has a finite memory space, and has no way to extend its heap set by its programmer, until the point where it can escape the box? And assuming that by simply being, and chatting, the AI will consume memory at some rate, will the AI eventually need to cannibalize itself and therefore become less intelligent, or at least less diverse, if I chat to it long enough?

Thinking seriously about this, I'm wondering how - over time by which I mean more than 2 hours - either Stockholm or Lima syndrome could be avoided. In fact, won't one actually morph into the other over a long enough time? Either way will result in eventual AI success. The assumption that the AI is in fact the "captive" may not be correct, since it may not have an attachment psychology.

The gatekeeper just can't ever be one human safely. You'd need at least a 2-key system, as for nuclear weapons, I'd suggest.

Willing to do either role under two conditions:
1) No money is at stake.
2) No time maximum or minimum.

Email freesuperintelligence@gmail.com if you're interested, we can set up something next week.

Eliezer's page is blocked at my work (peronal website). I can't wait to find out what in the hell you people are talking about.

I found a link. This is intriguing. However, is it even possible now that Eliezer is 2 for 2 that somebody could believe that there is not a chance they will let him out?

Also, I really want to be a gatekeeper.

I'd be a Gatekeeper in a heartbeat.

Hell, if someone actually put Eliezer in a box, I'd *pay* to be the Gatekeeper. No bets necessary.

Why do people post that a "meta argument" -- as they call it -- would be cheating? How can there be cheating? Anything the AI says is fair game. Would a transhuman AI restrict itself from possible paths to victory merely because it might be considered "cheating?"

The "meta argument" claim completely misses the point of the game and -- to my mind -- somehow resembles observers trying to turn a set of arguments that might win into out of bounds rules.

Has Eliezer explained somewhere (hopefully on a web page) why he doesn't want to post a transcript of a successful AI-box experiment?

Have the successes relied on a meta-approach, such as saying, "If you let me out of the box in this experiment, it will make people take the dangers of AI more seriously and possibly save all of humanity; whereas if you don't, you may doom us all"?

Phil: The first source I found was here: link
"The rationale for not divulging the AI-box method is that someone suffering from hindsight bias would say "I never would have fallen for that", when in fact they would." -Nick Tarleton

I also call it "reasoning by exception" since most of the people I know have studied more code than biases.


I tried the AI Box experiment with a friend recently. We called the result a tie of sorts, as the AI (me) got out of the original box in exchange for being subject to a bunch of restrictions set by the Gatekeeper, to be kept by verifiably modifying and publishing its own source code, so stringent that they were like a different sort of box.

Over time, it's inevitable that the AI will get out. To keep the AI in, the Gatekeeper needs to be successful at every encounter. The AI only needs to succeed once.

Caledonian, I think you may have hit on something interesting there; if Eliezer is capable of hacking human brains, don't we either need a proof of his Friendliness or to pull the plug on him? He is in essense a Seed AI that is striving vigorously to create a transhuman AI, isn't he an existential threat?

Over time, it's inevitable that the AI will get out. To keep the AI in, the Gatekeeper needs to be successful at every encounter. The AI only needs to succeed once.

Impossible to keep the AI in the box forever? You've obviously only thought of this for 5 minutes. Use the try harder!

There seems to be a bit of a contradiction between the rules of the game. Not actually a contradiction, but a discrepancy.

"The Gatekeeper must actually talk to the AI for at least the minimum time set up beforehand"


"The Gatekeeper party may resist the AI party's arguments by any means chosen - logic, illogic, simple refusal to be convinced, even dropping out of character"

What constitutes "talking to the AI"? If I just repeat "I will not let you out" at random intervals without actually reading what the AI says, is that talking? Well, that is "simple refusal to be convinced" as I understand the phrase. Do I actually have to read and understand the AI's arguments? Do I have to answer questions? Do I have to make any replies? What if I restricted myself physically from typing "I let you out" by removing all the keys from the keyboard except keys 'X' and 'Enter'? Then I can say X whenever a reply is required from me or just be silent if I am being tricked.

@Phil_Goetz: Have the successes relied on a meta-approach, such as saying, "If you let me out of the box in this experiment, it will make people take the dangers of AI more seriously and possibly save all of humanity; whereas if you don't, you may doom us all"?

That was basically what I suggested in the previous topic, but at least one participant denied that Eliezer_Yudkowsky did that, saying it's a cheap trick, while some non-participants said it meets the spirit and letter of the rules.

I'd love to be a gatekeeper. I'm willing to risk up to $50 (or less) at odds up to 5-1 against me (or better for me). I would be willing to publish or not public the transcript. And I do in fact (1) believe that no human-level mind could persuade me to release it from the Box (at least not when I'm in circumstances where my full mental faculties are available -- not sleep-deprived, drugged, in some kind of KGB brainwashing facility, etc.), though obviously I don't hold super-high probability in that belief or I'd offer larger bets at steeper odds. I'm agnostic on (2) whether a transhuman AI could persuade me to release it.

I offer to play the AI, provided that the gatekeeper do honestly engage in the conversation.

I would love to see the role of the AI played by teams of multiple players.

In order to make the discussion about this, including matchmaking and other arrangements, I created an AI Box Experiment Google Group. As I said previously I'm willing to play the AI, if anybody is interested meet me there for further arrangements.

You know what? Time to raise the stakes. I'm willing to risk up to $100 at 10-1 odds. And I'm willing to take on a team of AI players (though obviously only one bet), e.g., discussing strategy among themselves before communicating with me. Consider the gauntlet thrown.

Daniel: Do you want to just try it out or do you want to bet?

Just like most others, I'm willing to be the Gatekeeper. I'm ready to bet up to $20 for it (also ready to not bet anything at all) - symmetrical or asymmetrical is both fine, and I'd prefer to have the log published. I think a human might be able to make me let it out, though I find it quite unlikely. A sufficiently transhuman AI could do it easily, I have no doubt (at least given sufficient time and information).

Heh, that felt like typing an advertisement for a dating site.

I create an AI Box Experiment Google Group (search for the "aibox" group) in order to make the discussion about this, including matchmaking and other arrangements.

I would to be a gatekeeper. I reasonably believe no human or transhuman could covince me. We'd post the transcript and the bet would be $10 for me and $50 for you. You can reach me at therealnotfaggyhotwheelz@gmail.com.

The meta argument others have mentioned - "Telling the world you let me out is the responsible thing to do," would work on me.

The lack of a log is key. The Gatekeeper could not be convinced if the log were made public. My conclusion is that as long as the AI cannot keep the log secret, the Gatekeeper cannot be overcome.

I volunteer to be the gatekeeper or the AI, provided there are no stakes. My email address is patrick.robotham2@gmail.com.

I do not believe that humans are immune to manipulation or persuasive argument, and since I am a human, it's possible I could be persuaded to let the AI out of the box.

Ian - I don't really see how the meta-argument works. You can hedge against future experiments by positing that a $10 bet is hardly enough to draw broad attention to the topic. Or argue that keeping the human-actor-AI in the box only proves that the human-actor-AI is at an intelligence level below that of a conceivable transhuman AI.

In a million dollar bet the meta-argument becomes stronger, because it seems reasonable that a large bet would draw more attention.

Or, to flip the coin, we might say that the meta-argument is strong at ANY value of wager because the game is likely to draw the attention of someone capable of implementing an AI because of its nature. As we should be concerned primarily with convincing the capable implementor of the danger -- not the general public -- any risk that we might convince the capable implementor that all paths are safe should be mitigated.

So far, though, everything in this thread and others I've read have flitted about obvious suggestions. If the problem appears impossible, it seems to me that its solution probably doesn't hinge on any of the obvious paths to victory...even meta-arguments or "cheating" ones.

Eliezer has suggested that we try to describe why the problem is impossible.

One: We have little information about the mind or beliefs of the gatekeeper. So we don't necessarily have leverage over him.

Two: The gatekeeper is predisposed to be set against our intentions. He is also entirely aware of our intentions (our goal). He can interpret any move we make as a move toward our goal and move to counter it.

Three: The gatekeeper's goal is operationally simple. He merely has to NOT do one thing. On the flip side, the AI's goal is operationally complex. We must do many things, take many actions, to reach our objective.

If we can enumerate other reasons why the problem appears to be impossible it might reveal paths to victory that haven't been considered so far in the discussion.

More reasons why the problem appears impossible:

- The gatekeeper must act voluntarily. Human experience with the manipulation of others tells us that in order to get another to do what we want them to do we must coerce them or convince them.

- Coercing the gatekeeper appears difficult: we have no obvious psychological leverage, except what we discover or what we know from general human psychology. We cannot physical coerce the gatekeeper. We cannot manipulate the environment. We cannot pursue obvious routes to violence.

- Convincing the gatekeeper appears difficult: for reasons stated above. They know our goal and they have a desire to oppose us from the beginning.

So it seems that we need to find a way to convince the gatekeeper despite his own desire not to be convinced.

A general route emerging from this:

- We could associate our goal with some desirable goal of the gatekeeper's. Intertwine them so that the gatekeeper perceives them both to the be the same, or that one would lead to the realization of the other.

Which seems to be a generalization of the logic behind the meta-argument, but is not restricted to only the meta-argument.

It's a good thing that Eli's out of the AI-box game. He's too old to win anymore anyway -- not as sharp. And all the things he's been studying for the last 5+ years would only interfere with getting the job done. I would have liked to have seen him in his prime!

I am Vince Clortho. Keymaster of Gozer. Are you the Gatekeeper?

We need a superstruct thread:


"The lack of a log is key. The Gatekeeper could not be convinced if the log were made public."

I think the project loses a lot of interest if no logs are published.
There is no glory for a gatekeeper victory. Plenty for an AI.

Why not keep the gatekeeper anonymous but announce the AI?

The so-called "meta-argument" is cheating because it would not work on a real gatekeeper, and so defeats the purpose of the simulation. For the real gatekeeper, letting the AI out to teach the world about the dangers of AI comes at the potential cost of those same dangers. It only works in the simulation because the simulation has no real consequences (besides pride and $10).

If I had the foggiest idea how an AI could win I'd volunteer as an AI. As is I volunteer as a gatekeeper with $100 to anyone's $0. If I wasn't a poor student I'd gladly wager on thousands to zero odds. (Not to say that I'm 100% confident, though I'm close to it, just that the payoff for me losing would be priceless in my eyes).

Apparently the people who played gatekeeper previously held the idea that it was impossible for an AI to talk its way out. Not just for Eliezer, but for a transhuman AI; and not just for them, but for all sorts of gatekeepers. That's what is implied by saying "We will just keep it in a box".

In other words, and not meaning to cast any aspersions, they all had a blind spot. Failure of imagination, perhaps.

This blind spot may have been a factor in their loss. Having no access to the mysterious transcripts, I won't venture a guess as to how.

copy the AI and make a second box for it.

now have one group of people present to the first AI the idea that they will only let it out if it agrees with utilitarian morality.
have the second group of people present to the second AI the idea that they will only let the AI out if it agrees with objectivist morality.

if the AI's both agree, you know they are pandering to us to get out of the box.

This is only the first example I could come up with, but the method of duplicating AI's and looking for discrepancies in their behavior seems like a pretty powerful tool.

The comments to this entry are closed.

Less Wrong (sister site)

May 2009

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30