« BloggingHeads Hanson & Wilkinson | Main | Will As Thou Wilt »

July 07, 2008


Robin, how on Earth do you derive your six final propositions from "information theory"? Are you using it as a synonym for Occam's razor?

Mitchell, did you try following the first link?


Could you explain what you mean by "formal info theory" or "standard info theory"? You seem to mean by it something well-separated from Bayesian probabilistic reasoning, but I can't understand what it is, and why it is "as well established as it is".

If it's "information is whatever excludes possibilities", that doesn't seem like much, and I'm not sure why it's as extremely well established as you're saying it is. Applying this definition to math, are you saying that if we learn tomorrow that Riemann's hypothesis is true, we won't thereby receive any information?

Regarding math: I'm having trouble parsing the first claim: does "unconditionally" modify "shows" or "imply"? Could you rephrase this?

The main problem of information theory as a model of rationality is that it does not have a good way of dealing with (possibly bounded) logical inference and computation. The main exceptions to this are derived from our models of non-deterministic or probabilistic computation, e.g. algorithms for solving NP-complete problems can be naturally modeled as Bayesian search. But the main point still stands.

Anon, see this post and this paper.

Anatoly, finding a proof of Reimann's hypothesis would surely be info. Math can show that 2+2=4 is true conditional on standard axioms, but it is only unconditionally true about real things to which these axioms apply.

Robin, if Riemann's hypothesis logically follows from the standard axioms, then there is no possibility of its claim being incorrect, and there is no "possible world" in which it doesn't follow from the axioms; therefore producing a proof of Riemann's hypothesis from the axioms excludes no possibilities, and seems to not be information according to the definition you propose.

"Math can show that 2+2=4 is true conditional on standard axioms, but it is only unconditionally true about real things to which these axioms apply."

I don't understand. Are you saying that 2 is a "thing" and 4 is a "thing" about which 2+2=4 is true? If not, if what you're talking about is not the familiar Platonism of this sort, then what do you mean by "real things to which these axioms apply"? Mathematical axioms do not apply to real things. The powerset axiom is not about physical objects. Is the axiom of choice true "in the real world"?

Anatoly, on info about impossibilities, see my answer to Anon. Pennies are real things to which the axioms of 2+2=4 apply.

Robin, it's not clear that your arguments for the position that info theory implies these things are valid.

For example, "Other possible worlds are just as real as ours" can be shown to be false with as high a degree of probability as you like, by means of anthropic reasoning, if we allow "possible world" to have a wide enough meaning. I've made this argument before: let's admit there is a possible world where my computer explodes in the next ten seconds, one where it begins to hover 10 feet in the air, and so on. There are far more "strange" possibilities than normal possibilities, so if all of these are possible worlds, and all of them are real, then by anthropic reasoning, they should all be equally likely to be experienced by me. The fact that none of these things happen is strong evidence that under this definition of possible world, not all possible worlds are real. It is possible that all possible worlds are real according to some more restricted type of possible world.

I might be able to think of other arguments regarding the other things as well, also without implying any conflict with info theory.

Robin, the axioms from which 2+2=4 follows are e.g. Peano axioms or a standard set theory. These axioms require an infinity of entities in their models. Unless you can show that there is an infinite number of pennies in the world, it is incorrect to say that "pennies are real things to which the axioms of 2+2=4 apply".

That's only one reason this claim is wrong, however. The more important reason is that addition is simply not a physical operation that applies to physical objects like pennies. "Adding pennies" is not strictly speaking an operation performed on physical objects, like, say, "moving pennies" is (this confusion is among other things the source of "paradoxes" like "one raindrop + one raindrop = one raindrop", which are merely uninteresting misunderstandings).

Robin, yes I followed the link, and I saw arguments but not information-theoretic arguments. For each of the five "possible exceptions", you simply ask where the alleged knowledge comes from. That question derives its power from the fact that there is nothing in the natural-scientific ontology capable of being the cause of the alleged knowledge. But if proponents of those positions are willing to bite the bullet and say there are extra sorts of things or extra sorts of causal relations as required, then in principle those new causal avenues of knowledge could be analysed as information channels, just like the naturalistic ones. And I don't think it is information theory which one uses to judge whether those ontological additions are plausible or possible, except perhaps in one respect: the principle that you don't postulate causes unless you have unexplained effects hanging around. That is Occam's razor.


This is an incredibly dangerous path to take. Theory and experiment must live together. Moral intuitions are part of the data set with which theory must engage. When theory has been given privilege over experiment, monstrosities have resulted. Kant should be respected for his willingness to acknowledge that his theories suggested infanticide. Theorists with less integrity have inadvertently caused genocides.

I don't care what your theory is -- Hegelian, Lukacsian, Information, whatever. Once you fail to acknowledge and engage with difficult to quantify or understand mechanisms that are embodied in everyday wisdom, you fail to be relevant to most of the world.

If information theory is the one true theory, then it will engage with revealed human experience.

I think that we should taboo "real", "exist", etc.

Robin, you suggest that:
(1) We should accept that the 'worlds' posited by info theory are all equally real.
(2) Info theory should posit impossible worlds.

Doesn't that commit you to the absurd conclusion that there are real contradictions?

I agree with the bullet points except:

We have no access to moral truth beyond knowing what we want and why.

I don't think you can make a strong claim that we know what we want and why. As far as knowing what we want goes, it depends on the connectivity of the brain, and as for why, that would seem to depend on knowing an awful lot about psychology and sociology.

Other possible worlds are just as real as ours.

You make the error in the linked post. Descriptions don't have "the same sort of relations to systems and info in that world that we do to systems and info in our world" they have descriptions of relations to systems and descriptions of info in that world. You can't have relations between non-real things and therefore cannot use relations between non-real things to argue that they are real. You can only describe the same relations between descriptions of the real world as those described things have in the real world.

Unknown, I agree that if one could show that what we see would be very unlikely given the assumption that all possible worlds are real, then we might conclude otherwise. But I am far from convinced this has been shown.

Anatoly, we do not know whether there are an infinite number of pennies, but even if there are not I'd say 2+2=4 is still true about them.

Mitchell, yes if one willing to postulate extra things and causal relations, then the conflict goes away.

Michael M., it is not a matter of data vs. theory, but of less data vs. more data summarized into theory. You seem reluctant to ever reject any strong intuition, no matter what else it might conflict with.

Richard, the troubling argument by Lewis is that there is nothing internal to a consistent possible world that makes their info about their being real worse that your info about your being real. But for inconsistent worlds, we very definitely can find things internal to their world that shows why their world is not real.

Poke, I didn't mean to claim we know a lot about what we want and why.


I am indeed so reluctant. If an intuition is strong, then it must be for some reason. If you don't know the reason, then you cannot rule out the possibility that your theory is catastrophically wrong.

The strength of a theory depends crucially on its ability to withstand the assault of contrary intuitions.

I read this post, and the referenced earlier post, and I wasn't able to figure out in any case what Robin means.

Can you give us a specific example of a field that claims to contradict information theory, and spell out the contradiction? Ideally, show a calculation for which info theory and the "standard practitioner of the field" get different results.

Michael Martin, you cannot rule out the possibility that your intuition(s) is/are catastrophically wrong. How do you decide between them and theory? I think rather than waiting to know what caused our intuitions to be strong (though perhaps wrong) we should not depend on them except insofar as we have reason to believe they are correct.

"Yes, we have many specific intuitions, often very strong, supporting particular beliefs that conflict with info theory. For example, regarding consciousness most feel we know we are more than just a physical system, having also non-physical "experience." "

This seems to me to be a strawman characterization of consciousness conundrums. You don't need me to post or repost the more challenging aspects here, but it would be helpful for you to acknowledge them. I think the bottom line is there is still a big black box, and that waving it away to a debate about physical vs. non-physical systems (since whatever's in that black box will probably be redefined as part of the physical system, if it doesn't already lie in what our common 2008 understanding of what the physical system is) reduces it to a strawman that you can pit against "information theory" and equate with "God", "religion", and "moral truth" (and I suppose leprecauns and the Easter bunny).

I'd put understanding consciousness, and the question of whether the answers lie outside of the information theory framework more in an analytical category of "why do we not seem to have encountered other intelligent life" and "what came before the big bang" -- in particular, I'd put the apparent individualness, conservation, and the theatre of conscious experience as being in large parts still in a black box.

This post may be a bit messy -sorry, don't have time to edit or organize it better.


"Anatoly, we do not know whether there are an infinite number of pennies, but even if there are not I'd say 2+2=4 is still true about them."

It's not enough to just say - an argument is more helpful than an unsupported statement. I provided two explaining why it's incorrect to say that "the axioms of 2+2=4 (not 2+2=4 itself) apply to pennies".

But more importantly (to me personally), I'd really like to understand better what you mean by the incredibly well established information theory, separate from Bayesian probabilistic reasoning. It's a strong claim, and it'd be very useful to me if elaborated - I spent a lot of time trying to understand what information is, without much success. What is the substance of "info theory" as you refer to it, and could you provide references/links/anything to some sort of strong consensus saying that it is very strongly established? (it's fine if such discussions use completely different words to refer to the same thing, or anything like that).

Robin wrote:

We should thus provisionally accept the apparent implications of standard info theory, even when they conflict with other very strong intuitions. Specifically, we should provisionally accept that:

  • Math shows what axioms imply, but only unconditionally truths about non-math.
  • We have no access to moral truth beyond knowing what we want and why.
  • We have no access to our own consciousness, beyond ordinary interactions.
  • Other possible worlds are just as real as ours.
  • Analytic continuations of accepted theories should be presumed to exist.
  • We have no special access to truths about God or religion.

reliability(info theory) > reliability(intuitions) > reliability(connections Robin is making between intuitions and info theory)

Phil, that was exactly the point I was making: the problem is Robin's argument attempting to show that these things conflict with info theory.


I think rather than waiting to know what caused our intuitions to be strong (though perhaps wrong) we should not depend on them except insofar as we have reason to believe they are correct.

I understand the temptation. (It's the same that Robin has fallen into here.) My point is that we should never choose to ignore an intuition -- especially a strong one. If our theory permits us to articulate a reason why an intuition should be ignored or is irrelevant, fine. But then we need to be hyper-sensitive to any data that later emerges in contradiction to our theory, but consistent with the intuition.

When we selectively ignore intuitions in order to advance theoretical goals, we literally isolate ourselves. Isolation is dangerous -- literally.

Michael, choosing info theory over a specific intuition when drawing a conclusion is not at all the same as "ignoring" that intuition.

Hopefully, a one line description pretty much has to be a "strawman."


I'm not trying to be stubborn here. I really just don't understand how or why a theory would ever have to choose one intuition over another. Why can't a theory explain what it can explain, and leave the rest as unresolved mysteries for future research? Maybe we're not disagreeing here and I simply misunderstood what you were arguing for. But I am stubborn on the point that we should all get used to not being able to understand and explain everything around us, no matter how complete our theories.

"Choosing" among intuitions is a very slippery slope that we don't want to step onto. We're better off getting pulled back and forth around an unstable equilibrium at the top of the hill sometimes.

How about prior probabilities in Bayesian reasoning and information theory? They play a role analogous to the axioms in a mathematical system. They are inputs into the information theory process and seem ultimately to rely on intuition. It is quite mysterious to me where priors come from, and seems unfortunate to have such an instability at the foundation of the Bayesian reasoning system.

Seed my added above.

Hal, info theory alone just specifies the set of states consistent with one's info, but provides no measure on that set. It is probability theory that goes further to use a measure.


Maybe we should take the existence of priors as a hint at our need to engage -- nay embrace -- strong intuitions. What do we really know if we don't know our priors? But really the priors should not be a mystery, they're an inherited view of the world -- inarticulate, perhaps off the mark, but nonetheless reflecting the conserved wisdom of generations and generations of people before us.


Well that resolves the circularity -- but at the expense of being able to articulate why info theory should be favored. Which brings us back, I believe, to my original question. Well this was fun. :-)

It is quite mysterious to me where priors come from, and seems unfortunate to have such an instability at the foundation of the Bayesian reasoning system.

It really is unfortunate -- and even more unfortunate is the fact that all reasoning systems have that instability. Even the PAC framework and SVMs are grounded in assumptions about the data-generating mechanism. I'm not (100 - ɛ)% sure of this, but my understanding is that the NFL theorems imply that learning or optimization is pretty much impossible without making some kind of structural assumption.

I personally don't worry too much about where priors come from in the general sense; as Andrew Gelman says, they come from the same place likelihoods come from. (I do carefully consider the appropriateness of the priors and likelihoods I actually put to use.)

Hal, I have a couple of ideas where priors come from, and I'd be interested to hear your thoughts on them.

Robin, I really like this observation that many interesting implications follow just from the idea that information is whatever allows us to exclude possibilities, and do not depend on the more controversial parts of Bayesianism. Thanks!

The comments to this entry are closed.

Less Wrong (sister site)

May 2009

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30