« More Referee Bias | Main | The Second Law of Thermodynamics, and Engines of Cognition »

February 25, 2008

Comments

How many rationalists would retain their belief in reason, if they could accurately visualize that hypothetical world in which there was no rationality and they themselves have become irrational?

I enjoy the non-mathy posts. I believe Overcoming Bias is a worthy endeavor, and as a relatively new field of study, the math-oriented posts are important. They are often the most succinct and accurate way to convey concepts. With that said, I find that math posts to be dense with information, perhaps overly so. I find myself unconsciously starting to skim instead of read, and I find it difficult to force myself to pay attention.

The mathy posts appeal to people who are serious about moving this burgeoning field forward, and the non-mathy posts appeal to people who are more casually interested in the concepts, and allow you to have a wider audience. You will have a balance between the two no matter what you attempt, the only question is what your intended audience is, and the best way to reach those people.

I enjoy all posts here, but would love a post on what does it mean to be rational. Something introductory, something you can link to when you talk with people who think "if you can justify what someone did, no matter what the justification is, the action becomes rational".

then I am interested in hearing from you in the comments.

While I appreciate the mathy posts as well as I can, as someone without much training in mathematics I really enjoy these types of posts (I've got a large backlog of your more mathy posts bookmarked for me to work through, whereas your non-mathy posts I read as soon as they show up in my feed reader).

Let us have both!

The ability to endure cognitive dissonance long enough to find the resolution to the dissonance, rather than just short-circuiting to something that makes no sense but offers relief from the strain, is a necessary precondition for rational thought.

I don't think it can be cultivated, and I don't think there's a substitute. Either you pass through the gauntlet, or you don't.

I just want you to get to that "revelation" of yours already. I thought you were approaching it, if you're talking about neural nets and arithmetic coding. Where does it rank in your schedule? Or is this blog for human reasoning only?

I was expecting to read yet another mathy post tonight, but I was dissapointed. Less mathy stuff is ok, but shouldn't really come at cost of anything intresting.

I agree with Kriti - introductory essay, post, etc would be useful.

I prefer the less mathy.

I too prefer less mathy - well, to be precise I'll actually read the less mathy stuff in the first place.

More to the point, I've stopped listening to news reports about global warming - and this is harming my ability to think rationally about it. I'll change the channel instead of hear someone say "You know how we all thought we've got 50 years to live? Turns out it's only 30/25/20."

[Without having read the comments]

WTF? You say:
[...] I was actually advised to post something "fun", but I'd rather not [...]

I think it was fun!

BTW could we increase the probability of people being honest by basing reward not on individual choices, but on the log-likelihood over a sample of similar choices?
(For a given meaning of similar.)

As a mathematician I like your mathy posts, but this is also very welcome for a reason: it contains practical advice. Some posts are of little direct practical use but this one certainly is.

Keep on the good work!

"this is also very welcome" I'm refering to this post.

[having read the comments]

Kriti et al:
I'd recommend this and this to anybody who hasn't already read it. Otherwise I have not much idea for introductory texts right now.

I think you should go with the advice and post something fun. Especially so if you have "much important material" to cover in following months. No need for a big hurry to lose readers. ;)

I should however note that one of the last mathy posts (Mutual Information) struck a chord with me and caused an "Aha!" moment for which I am grateful.

Specifically, it was this:

I digress here to remark that the symmetry of the expression for the mutual information shows that Y must tell us as much about Z, on average, as Z tells us about Y. I leave it as an exercise to the reader to reconcile this with anything they were taught in logic class about how, if all ravens are black, being allowed to reason Raven(x)->Black(x) doesn't mean you're allowed to reason Black(x)->Raven(x). How different seem the symmetrical probability flows of the Bayesian, from the sharp lurches of logic - even though the latter is just a degenerate case of the former.

Insightful!

I agree with SnappyCrunch.

I like non-mathy posts. I particularly enjoyed this one, as it seems to have a clear practical application.

I liked this post, but then again, I like all your posts Eliezer! (I've just been hiding behind my feedreader, and so not commenting about it before.)

My opinion about mathy/non-mathy is that you should do what you think is most natural. Most days, you'll probably want to get on with the mathy exposition (and I am very much looking forward to the more advanced mathy posts), and then sprinkle in something lighter when the occasions to do so arise. For instance, I like that you based today's post on a recent discussion you had.

I believe this approach would be most conducive to interesting reading.

'Newly minted quadriplegics'? What's more fun than that?

Don't worry too much about who wants what when. Like you say, it's all important stuff, and at a post a day no-one's going to complain about the odd vignette. Just keep up the good work.

When I saw the title I thought you were responding to this:
http://www.overcomingbias.com/2008/02/more-moral-wigg.html

Thank GOD for non-mathy posts ;-)

There's a common literary technique used in most storytelling in which the author writes alternating "up" and "down" scenes -- it provides pacing and context; it also allows us time to digest the "up" scenes.

It seems to me that the technique is appropriate here -- it might be worth making a goal for yourself to write a mathy post, then to follow up with a post on the same topic but without any math in it at all, except maybe references to the previous post. That would be an interesting exercise for you, I think. It's supposed to accessible work -- how accessible can you make it? Can you write about these mathy topics without numbers?

I don't know, but if you never try to do impossible things...

There hasn't been much evidence of atheists forming groups that have the positive aspects that a church/synagogue/mosque holds in the social life of some humans. So you might forgive a theist pretending to be a rationalist, for not holding the probability of this happening very high, and that the world would lack said institutions and would be a worse place.

If rationalists truly wants to get rid of religions, without getting rid of humans, we would have to ask ourselves, "What do humans get out of being part of a religion?" And then provide that through organisations.

And please no strawmen of the comfort of ignorance, I am talking about reassurance of being with people who are trying to hold the same goal system.

Eliezer,

You know that you can't succeed without the math, and slowing down for posts like this is taking away 24 hours that might have been better used to save humanity. Not that this was a bad post, but I think you would be better off letting others write the fun posts unless you need to write a fun post to recover from teaching.

Eliezer, this was a welcome relief from the long series of mathy posts.

Eliezer, suppose it turned out to be the case that:

1) God exists.
2) At some time in the future, tomorrow, for example, God comes to Eliezer Yudkowsky in order announce His existence.
3) Not only does He announce His existence, but He is willing to have His existence and power tested, and passes every test.
4) He also asserts that according to Eliezer's CEV, although not according to his present knowledge, God's way of acting in the world is perfectly moral, even according to Eliezer's values.

How would you react to these events? Would you write a post about them on OB?

Thanks for feedback, all! The consensus appears to favor leavening mathy posts with less mathy ones. I'll bear that in mind, though I make no promises - I do have my own agenda here.

---

Unknown, can't say I've ever thought of that one. I've considered how to kill or rewrite a Judeo-Christian type God, but not that particular scenario you've just described.

I think I would simply reply to number 4, "I don't believe that without an explanation." After all, just because an entity displays great power doesn't mean it will always tell you the truth.

You can't necessarily force me to consider believing number 4 because it involves a moral question and those are not subject to forced visualization (by this rule) in the way that factual scenarios are.

You can invent all kinds of Gods and demand that I visualize the case of their existence, or of their telling me various things, but you can't necessarily force me to visualize the case where I accept their statement that killing babies is a good idea - not unless you can argue it well enough to create a real moral doubt in my mind.

If I myself am in actual doubt on a moral question, then I can visualize it both ways without confusing myself; and then you can demand that I visualize it. But when I am not in doubt, trying to visualize the contrary has the same quality as trying to concretely visualize 2 + 2 = 3, only more so.

I can visualize a mind constructed so as to possess a different morality, of course; but that is not the same as identifying myself with that mind.

---

This reminds me of an item from a list of "horrible job interview questions" we once devised for SIAI:

Would you kill babies if it was intrinsically the right thing to do? Yes/No

If you circled "no", explain under what circumstances you would not do the right thing to do:
___________________

If you circled "yes", how right would it have to be, for how many babies?
___________________

Alternatively, if you want something super scary, try 1), 2), and 3) without 4).

I've considered how to kill or rewrite a Judeo-Christian type God

Please make this your next "fun" post. (Speaking of which, I enjoy the digression.)

You can't necessarily force me to consider believing number 4 because it involves a moral question and those are not subject to forced visualization (by this rule) in the way that factual scenarios are.

But "my CEV judges killing babies as good" (unlike "killing babies is good") is a factual proposition. You know what your current moral judgments are, but you can't be certain what the idealized Eliezer would think. You might justifiably judge repugnant volition too unlikely to bother imagining it, but exempt?

This reminds me of an item from a list of "horrible job interview questions" we once devised for SIAI:

Would you kill babies if it was intrinsically the right thing to do? Yes/No

If you circled "no", explain under what circumstances you would not do the right thing to do:
___________________

If you circled "yes", how right would it have to be, for how many babies?
___________________

What a horrible horrible question. My answer is ... what do you mean when you say "intrinsically the right thing to do"? The "right thing" according to whom? If it was the right thing according to an authority figure but I disagreed, I probably would not do it. If the circumstances were so extreme that I truly believed it's the right thing(eg: not killing a baby results in the baby's death anyway + 1 million babies) then I would kill babies(assuming I could overcome my aversion to killing).

Actually I don't really know how I would react. This is how I wish I would act. Calmly theorising in front of the computer never having experienced circumstances remotely as awful is not the same as being in those circumstances when the fear and dread overtakes you. There would probably be a significant shift from what I consider and feel is "me" right now to the "me" I would become in that hypothetical situation.

"This reminds me of an item from a list of "horrible job interview questions" we once devised for SIAI:"

Could you post these?

"I've considered how to kill or rewrite a Judeo-Christian type God"

Okay, now I'm curious what you've concluded with regards to that. :)

Probably not worth doing more then just talking 'bout it in comments, if that, unless you feel like doing a post on that just for fun.

But as far as this post, I also liked it. Useful to have actual suggestions for mental practices to practice to help one debias oneself.

Why do the work of hypothesizing the world without God? It's not like Nietszche, Sartre, Camus, Marx, Shaw, Derrida, etc. haven't done a much better job of it than me, because they were better philosophers than me. However, I also consider Aquinas to be the better philosopher than the aforementioned. Is that so unreasonable?

Thanks for reminding me of The Art of War from your quote. You might be interested in this great translation - http://www.sonshi.com/huynh.html

"The mathy posts appeal to people who are serious about moving this burgeoning field forward, and the non-mathy posts appeal to people who are more casually interested in the concepts" - (Snappycrunch)

Beware of mistaking mathematical thinking for rational thinking; math is a tool like any other, to be used rationally or irrationally. Nassim Taleb demonstrates this very well in his book "Fooled by Randomness".

There's nothing casual about being interested in the concepts of rational thinking; even the mathematically minded (who will naturally be more interested in the mathy posts) need the concepts to understand what framework to put the math into.

I've considered how to kill or rewrite a Judeo-Christian type God
If God did not exist, it would be necessary to invent him. And if there really is a God it will be necessary to abolish him!

How does one go about visualizing a world without souls? Or, rather a world in which nobody believes in souls, and how would this visualization have any bearing to "reality"? It seems like the thought experiment is really: What would I do if everything were the same except I didn't have a soul?

slowing down for posts like this is taking away 24 hours that might have been better used to save humanity.

Sarcasm? Humour? Sincerity?

I've considered how to kill or rewrite a Judeo-Christian type God

Please make this your next "fun" post.

Seconded!

I've considered how to kill or rewrite a Judeo-Christian type God

Obligatory Pascal: Ah, but what if there's a tiny chance that He's reading along to figure out our tactics?

Steven: To kill or rewrite a Judeo-Christian God, obviously, the technique has to work even if the God can read your planning thoughts. It's a lot easier than dealing with an UFAI, though, because the Judeo-Christian God has anthropomorphic cognitive vulnerabilities and a considerable response time delay. ("You ate the apple?")

Naturally you prefer to rewrite the God if possible - shame to waste all that power.

Heh, so how do you know that it is not the case that this hypothetical JCG reads overcomingbias but not people's private thoughts?

(Of course as long as we're under these weird assumptions then not discussing tactics could be a fatal mistake too, etc etc)

I'm skeptical about the possibility of really carrying out this kind of visualization (or, more broadly, imaginary leap). Here's why.

I might be able to say that I can imagine the existence of a god, and what the world would be like if, say, it were the Christian one. But I can't imagine myself in that world -- in that world, I'm a different person. For in that world, either I hold the counterfactually true belief that there is such a god, or I don't. If I don't hold that belief, then my response to that world is the same as my response to this world. If I do hold it, well, how can I model that?

This point is related to a point that Eliezer made in the comments, that I think just absolutely nails the problem, for a narrower class of the true set of states for which the problem exists:

You can invent all kinds of Gods and demand that I visualize the case of their existence, or of their telling me various things, but you can't necessarily force me to visualize the case where I accept their statement that killing babies is a good idea - not unless you can argue it well enough to create a real moral doubt in my mind.

Exactly.

But I maintain that you can't model the existence of a God with the right properties (including omnipotence, omniscience, and omnibenevolence) without being able to model that acceptance.

And likewise, the woman who believed in the soul couldn't model her reaction to a world without a soul without being able to experience herself as a person who genuinely doesn't believe in a soul. But she can only have that experience by becoming such a person.

I think this is just a limitation of human psychology. Cf. Thomas Nagel's great article, What is it like to be a bat? The argument doesn't directly apply, but the intuition does.

This reminds me of an item from a list of "horrible job interview questions" we once devised for SIAI:

Would you kill babies if it was intrinsically the right thing to do? Yes/No

If you circled "no", explain under what circumstances you would not do the right thing to do:
I assume by "intrinsically right thing to do", you do not intend something straightforward like "here are five babies carrying a virus which, if left unchecked, will wipe out half the population of the planet. There is no means by which they can be quarantined, the virus can cross even the cold reaches of space. The only way to save us is to kill them". I assume rather, that you, Eliezer Yudkowsky, hand me a booklet, possibly hundreds of pages long. On page 0 are listed my most cherished moral truths, and on page N is written: "thus, it is right and decent to kill as many babies as possible, whenever the opportunity arises. Any man who walks past a mother pushing a stroller, and does not immediately throttle the infant where it lies, is nothing more than a moral coward." For all n between 1 and N inclusive, the statements on page n seem to me to follow naturally and self-evidently from my acceptance of the statements on page n-1. As I look up, astonishment etched on my face, I see you standing before me, grinning broadly. You hand me a long, curved blade, and tell me the staff of the SIAI are taking the afternoon off to raid the local nursery, and would I like to join?

Under these circumstances I would assign high probability to the idea that you are morally ill, and wish to murder infants for your own enjoyment. That somewhere in the proof you have given me is a logical error - the moral equivalent of dividing by zero. I would imagine, not that morality led me astray, but that my incomplete knowledge of morality led me not to spot this error. I would show the proof to as many moral philosophers as I could, ones whose intelligence and expertise in the field I respected, and held to be above my own, and who were initially as unenthusiastic as I am at the prospect of infanticide. I would ask them if they could point me to an error in the proof, and explain to me clearly and fully why this step, which had seemed so simple to me, is not a legal move in the dance at that point. If they could not explain this to me to my satisfaction, I would devote much of my time from then on to the study of morality so that I could better understand it, and until I could, would distrust any moral conclusions I came to on my own. If none of them could find an error, I would still assign high probability to the notion that somewhere in the proof is an error which we humans have not advanced sufficiently in the study of metamorality to discover. I would consider it one of the most important outstanding problems in the field, and would, again, distrust any major moral decisions which did not clearly add up to normality until it was solved.

Just as the mathematical "proof" that 2=1 would, if accepted, destroy the foundations of mathematics itself, and must therefore be doubted until we can discover its error, so your proof that killing babies is good, would, if accepted, destroy the foundations of my morality, and so I must doubt it until I can find an error.

I am well aware that a fundamentalist could take my previous paragraph, replace "killing babies" with "oral sex" and thus make his prudery unassailable by argument. So much the worse for him, I say. If he considers the prohibition of a mutually beneficial and joyful act to be at the foundation of his morality, then he is a miserable creature and all my rationality will not save him from himself.

I have tried indirectly to answer your question. To answer it directly I will have to resort to what seems a paradox. I would not do "the right thing to do" if I know, at bottom, that it simply is not the right thing to do.

If you circled "yes", how right would it have to be, for how many babies?
N/A

So, would I get the job?

I would show the proof to as many moral philosophers as I could

Boy, I sure wouldn't. Ever read Cherniak's "The Riddle of the Universe and Its Solution"?

I am well aware that a fundamentalist could take my previous paragraph, replace "killing babies" with "oral sex" and thus make his prudery unassailable by argument. So much the worse for him, I say.

I sympathize, but I don't think that really solves the dilemma.

Post what you want to post most. The advice that you should go against your own instincts and pander is bad, in my opinion. The only things you should force yourself to do are: (1) try to post something every day, and (2) try to edit and delete comments as little as possible. I believe the result will be an excellent and authentic blog with the types of readers you want most (and that are most useful to you).

Eliezer,

I think there is pretty overwhelming evidence that moral philosophers are almost never moved to do anything nearly so onerous and dangerous as killing babies by their moral views. See Unger, Singer, Parfit, etc.

The comments to this entry are closed.

Less Wrong (sister site)

May 2009

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31