« Thoughtful Music | Main | Lying Stimuli »

February 03, 2009


The crew certainly hasn't fully considered the implications of returning home at this point. Akon touched on it, humanity is something of an impostor as far as "not-stupid" space-faring species go. Whatever the outcome, the Lord Pilot has proven that humanity is capable of fragmenting and going to war over this issue. If the Impossible Possible World returns to human space then people will have both the motive to fight each other, and now the means to destroy themselves now that the Alderson cat is out of the bag.

Moreover it's quite possible that the "negotiations" were performed under a false premise. it seems likely that the Superhappies are aware of the capability of the Alderson drive, but they very well may not be aware of humanity's ignorance. The records sent to the Babyeaters were censored, particularly of technical data, so the coupling constant error might not have been there. Moreover Lady 3rd does seem remarkably sure that human decision-makers will make consistent decisions. This could be a result of assuming humans have dealt with the Alderson drive filter like the Babyeaters and Superhappies who are then both much less diverse than humans.

All that boils down to the idea that the Superhappies may be assuming that the modification of humanity will be a cooperative venture when in fact it is likely to be a species destroying war.

I guess the real question is does humanity thus constitute a "species in circumstances that do not permit its repair"?

It is possible that, like they Babyeater's attempt to convince the crew to eat their own babies, the Superhappy's arguments are also rationalizations when their most fundamental purpose is really eliminating their own pain. Thus they might be willing to trade away eliminating all human pain if modifying their own empathic faculty turns out to be the less painful-to-them alternative.

As to what the Confessor is going to do? I'm not sure. He could stun whomever is in control of the communications (Lady Sensory?) and transmit the relevant information to the Superhappies. He may believe the upside result to be better terms and the downside result to be the Superhappies enacting their compromise by force. He could feel that downside to be only slightly worse than voluntary implementation and significantly better than sparking of a potentially human destroying war.

Of course they may want to employ the slitting their own throats option in either case given the Alderson secret and humanity's posited issues.

Mind you to me there seem to be so many risks that I'd have to assume some special insight due to his commonality with Lady 3rd in order to be confident of making such a gamble on his own. It seems the risk adverse action would be to go back to Huygens and detonate the star the moment the crew arrive. This would kill every colonist, but would save the Babyeater's babies and prevent humans from learning how to destroy themselves (for the moment at least).

The lesson I draw from this story is that in it, the human race went to the stars too soon. If they had thought more about situations like this before they started travelling the starlines, they'd have a prior consensus about what to do.

Superhappies care about the happiness of all. Therefore humans can blackmail them by parking ships near stars in inhabited solar systems, and threatening to supernova the star in case a Superhappy ship jumps into the system, or whatever. (The detonation mechanism should probably be automated so that Superhappies can be certain humans actually go through with the detonation, instead of just making the threat.)

For humanity to maximize their influence over the future, they should immediately proceed to set up as many "booby-trapped" solar systems as they can. Then renegotiate with the Superhappies (and also spend more time thinking).

(I only had the patience to read the first page of comments, so I don't know if there was already talk of this option.)

I misread it just as Anonymous Coward did. I thought they killed the Babyeaters and head back on their (the Babyeaters) star line. Thus I thought AC's first solution was perfect. I also liked AC's second solution.

It seems far too possible that the Super Happies are not telling the whole story here. For all Akon and his crew know, virtually everything the SH have told them is a lie crafted to minimize their resistance so that the SH can either enslave them or destroy them and take over their starline network. Several of the SH's actions have been fairly suspicious - for example, they seemed awfully quick to give up on diplomacy with the Babyeaters. Also, "several weeks" seems like an implausibly short amount of time for even a highly advanced technological force to pull off the kinds of changes the SH have planned for the BE. Conversely, it does not seem as implausible a time frame for destroying/enslaving the BE. Suspicious, I tell you. The facts fit either hypothesis about equally, except that I'm not entirely convinced that a species which truly eliminated pain would survive twenty years.

The fact that Akon and the gang don't appear to have even considered this possibility leads me to agree with the Confessor's apparent conclusion that the crew is no longer sane.

At this point, humanity has totally failed at interstellar diplomacy. The SH know about humanity now, and at the rate they are developing, they will probably be able to find us within a few centuries even if the Impossible destroys the local star. The only acceptable solution I see is to follow the SH ship and hope that they can find a nova-able star in/near the SH home system.

mcow, I might also assign a rather high probability to the Superhappies having just made everything up as a lie, were it not for their choice to blow up the Babyeaters while leaving the humans to return home.

Their behaviour is consistent with what they have claimed to be, but not with e.g. being standard invading aliens that are just lying.

Demetri:"Chris, continuing with my analogy, if instead of lobotomy, I was forced to undergo a procedure, that would make me a completely different person without any debilitating mental or physical side effects, I would still consider it murder."

Do you also consider it a birth?


mcow, I might also assign a rather high probability to the Superhappies having just made everything up as a lie, were it not for their choice to blow up the Babyeaters while leaving the humans to return home.

For all we know, the Babyeaters don't exist. That entire scenario may have been invented by the Super Happies just to make the humans easier to manipulate. If so, it certainly worked pretty well, don't you think? Also, remember Akon's thoughts on seeing the BE solution to withstanding the radiation:

the mirror-shielding seemed a distinctly inferior solution. Unless that's what they want us to think...

I'm really beginning to think that the SH are predators who caused this nova to use it as bait. The BE are there because the SH have found that it sets up a nice prisoner's dilemma situation for a large variety of meta-ethics systems. The target comes in, and the BE ship immediately transmits a fabricated archive (or maybe a real archive; the BE may have existed at some point). The SH know that tit for tat is a highly effective (and therefore probably common) strategy, and therefore their prey will feel obligated to send data back to the BE. The SH then use this data to determine exactly how to manipulate the prey - for humans, they used "super happy", but had they been preying on, by way of example, Vulcans, they would have used "ultimate understanding" or something. Meanwhile, the prey wear themselves out trying to figure out what to do about the BE. After a day or so, the prey are ripe for the pickin'.

Maybe I'm wrong, but if there's even a 1% chance that I'm right, I don't think the crew of the Impossible can take the risk.

Dmitry, we are in agreement that a sufficiently large mind altering change is a killing.

But in principle, changing babyeater society does not require the killing of even a single babyeater. Simply keep unmodified adult and child babyeaters separate, and modify the pre-sentient children to prevent them from wanting to eat babies in the future. No sentient being is killed/modified, although freedom of movement/action is restricted.

In practice, modifying babyeater society would probably involve more bloodshed. But as long as this bloodshed is minimized as much as possible, and is less than the harm caused by babyeating, I don't see it as genocide. Is it genocide to kill some Nazi's as part of an effort to stop them from killing innocents?

mcow, I still think the SH should be blowing the humans up at this point, if that's what they're about. I don't see that they'd really gain anything by still keeping up such a supposed illusion, since they've already determined the humans to be technologically inferior. They could go for a surprise attack on human civilization now, and it would be at least as defenseless as in the scenario where they let the human ship return with false news of non-invading aliens (news which everyone would not believe).

Aleksei, what they'd gain from keeping up the illusion is the knowledge of where the rest of humanity is. They have to coordinate their attack so that the humans don't have a chance to cut off the rest of their starline network. If the humans figure out what's up, they can mess up the short-term plans of the SH, even if they can't win an all out war.

Eliezer, you might do well to thoroughly understand and consider Fare's criticisms of you. He seems to be one of the few readers here who isn't in awe of you, and has therefore managed nailed some fundamental mistakes you're making about human values. Mistakes that have been bugging me for some time too, but that I haven't been able to articulate, possibly because *I'm* too enchanted by your cleverness.

We don't value strangers on par with our friends and family, let alone freaky baby-eating or orgy-having aliens. Furthermore, I don't *want* to be altered in such a manner as to make me so empathetic that I give equal moral standing to strangers and kin. I believe THAT would make me less human. If you or an FAI alters me into such a state, you are not correctly extrapolating my volition, nor of who knows how many other people like me. Do you have an estimate of how many people like that there are? How did you come by such an estimate?

So anyway, if this happened in any real future, I have no doubt some star would soon get supernova'd-- current star, Huygens, Happy Homeworld, Eater Homeworld, or Sol, in that order of likelihood. For these idealized humans inhabiting the uncanny valley of empathy that creates the whole contrived dilemma in the first place, who knows? Maybe the fact that a nova was what brought them there, and now they're contemplating creating a supernova is some kind of clue. Maybe the definition of "non-sentient baby" can be stretched to the point where the story ends as a blowjob joke, but I doubt it. Also, the mechanics of exactly how other people's pain affects the Happies haven't really been examined. It sounds like they're merely extrapolating the pain they think others must be feeling... given that they've had no scruples against engineering other sources of discomfort out of existance, why not engineer that out of existance too?

"It sounds like they're merely extrapolating the pain they think others must be feeling... given that they've had no scruples against engineering other sources of discomfort out of existance, why not engineer that out of existance too?"
They already said they will engineer out the sympathy and replace it with a non-painful desire to alleviate pain.

One possibility, given my (probably wrong) interpretation of the ground rules of the fictional universe, is that the humans go to the baby-eaters and tell them that they're being invaded. Since we cooperated with them, the baby-eaters might continue to cooperate with us, by agreeing to:

1. reduce their baby-eating activities, and/or

2. send their own baby-eaters ship to blow up the star (since the fictional characters are probably barred by the author from reducing the dilemma by blowing up Huygens or sending a probe ship), so that the humans don't have to sacrifice themselves.

The Ship's Confessor uses the distraction to anesthetizes everyone except the pilot. He needs the pilot to take command of the starship and to pilot it. The ship stays to observe which star the Superhappy ship came from, then takes off for the nearest Babyeater world. They let the Babyeaters know what happened, and tell them to supernova the star that Superhappies came from at all costs.

When everyone wakes up, the Ship's Confessor convinces the entire crew to erase their memory of the true Alderson's Coupling Constant, ostensibly for the good of humanity. He pretends to do so himself, but doesn't. After the ship returns to human space, he uses his accumulated salary to build a series of hidden doomsday devices around every human colony, and becomes the dictator of humanity through blackmail. Everyone is forced to adopt an utility function of his choosing as their own. With every resource of humanity devoted to the subject, scientists under his direction eventually discover a defense against the supernova weapon, and soon after that, the Babyeaters are conquered, enslaved, and farmed for their crystal brains. Those brains, when extracted and networked in large arrays, turn out to be the cheapest and most efficient computing substrate in the universe. These advantages provide humanity with such a strong competitive edge, that it never again faces an alien that is its match, at least militarily.

Before the universe ends in a big crunch, the Confessed (humanity's eventual name) goes on to colonize more than (10^9)^(10^9) star systems, and to meet and conquer almost as many alien species, but the Superhappy people are never seen again. Their fate becomes one of the most traded futures in the known universe, but those bets will have to remain forever unsettled.

In case it wasn't clear, the premise of my ending is that the Ship's Confessor really was a violent thief and drug dealer from the 21th century, but his "rescue" was only partially successful. He became more rational, but only pretended to accept what became the dominant human morality of this future, patiently biding time his whole life for an opportunity like this.

Option 1 is to cooperate, so I guess option 2 is defect. The correct way to defect is to destroy Huygens.

But this is the inverse of how cooperate/defect was framed earlier in the story. When humans had the upper hand, defecting was to blow up the baby eater ship, and proceed to fix the babyeaters in some way. Cooperating was to stay peaceful and manage some mutual compromise. For some reason the humans are pretending that the superhappies haven't defected.

Paperclippers arrive. The end.

Option 1 is to cooperate, so I guess option 2 is defect. The correct way to defect is to destroy Huygens.

It's not how it works, prisoner's dilemma is just an example of what you may choose based on timeless decision theory. There are numerous options, and "true cooperation" is just a stand-in for the optimal decision that takes into account the effect of your decision procedure on the outcome. In the described situation, lying to Superhappies through confused Akon and then blowing up Huygens is the best analogue to cooperation in true prisoner's dilemma. You only need to give something up if it's the best way to get what you want. Otherwise it's not about decision-making, but specific utility, for example valuing fairness.

Of course, if both side were more rational, it'd be better to use the fact that you can blow up Huygens confessor-style, without the risk of being destroyed when Superhappies learn about this option, but giving leverage to negotiate terms that are even better than the story's True ending (for us, and also for Superhappies, which is why they should have a protocol in place that would prevent them from destroying the Impossible in this case).

For gosh sakes Faré it's only a story.

Sure it's a story, but one with an implicit idea of human terminal values and such.

I'm actually inclined to agree with Faré that they should count the desire to avoid a few relatively minor modifications over the eternal holocaust and suffering of baby-eater children.

I originally thought Eliezer was a utilitarian, but changed my mind due to his morality series.

(Though I still thought he was defending something that was fairly similar to utilitarianism. But he wasn't taking additivity as a given but attempting to derive it from human terminal values themselves - so if human terminal values don't say that we should apply equal additivity to baby-eater children, and I think they don't, then Eliezer's morality, I would have thought, would not apply additivity to them.)

This story however seems to show suspiciously utilitarian-like characteristics in his moral thinking. Or maybe he just has a different idea of human terminal values.

I wrote up my main criticism here: "Denying moral agency to justify Cosmic Sacrifice"

The comments to this entry are closed.

Less Wrong (sister site)

May 2009

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30