« Open Thread | Main | Generous Lust »

May 01, 2009

Comments

> Absence of evidence is not evidence of absence
Eh?

Absence of evidence is sometimes only weak evidence of absence, but it's legitimate evidence nevertheless.

Regarding: http://www.overcomingbias.com/2007/08/absence-of-evid.html

Neither slogan seems terribly accurate. However:

"Absence of evidence is evidence of absence *if* some relevant tests have been performed"

...is a bit of a mouthful.

This is a little off topic, but what is the point of reporting the overlap of confidence intervals? If you are testing equivalency of means, overlapping confidence intervals do not always correspond to significantly different means (unless the point estimate for one of the means is contained in the confidence interval of the other). Same goes for the multiple comparisons: a Bonferroni correction MUST be done, so why report the significance of tests at the uncorrected alpha level? That would be like saying "16+17 = 23 if we don't carry the one. If we do carry the one, 16+17=33."

Maybe there is a reason for this that I am missing?

Er, when the point estimate for one of the means is contained in the confidence interval of the other means they are NOT significantly different.

@Kieran: Better yet, use a hierarchical model, report the credible interval of the top level mean, and stop worrying about Type 1 error rates entirely.

@Tim Tyler:
"Absence of evidence is evidence of absence *if* some relevant tests have been performed"

...is a bit of a mouthful."

It's just not catchy enough, that's all. How about: "Absence of evidence is evidence of absence. Absence of looking for evidence is not."

Absence of [evidence of] anyone looking for evidence is itself evidence of absence, if we consider:

1. If nobody's looking, it suggests a consensus that there's nothing to find, which Aumann compels us to consider.

2. Publication bias means we tend not to hear about people looking unless they get positive results.

Simpleton wrote:

Absence of evidence is sometimes only weak evidence of absence, but it's legitimate evidence nevertheless.

Good point. All evidence needs to be considered. As I have argued, Karl Popper's writing led many to see evidence exclusively in terms of either falsifying a hypothesis or failing to do so. We see this in nonsensical language like, "There was no evidence of X (p=0.2)." Sure, it's shorthand for "There was no statistically significant evidence," but it's quite misleading.

It's important to note that confidence intervals (or credible intervals) allow for shades of gray, rather than just black or white.

It seems that for some readers I need to clarify that I meant that absence of evidence is not necessarily evidence of absence.

Robin is not being blunt.

Oops, I mean Robin is not necessarily being blunt.

It seems that for some readers I need to clarify...

So some readers think you mean what you actually write instead of the disclaimed version you obviously meant? Silly readers!

Robin,

Absence of evidence is necessarily evidence of absence. More specifically, if you could get some evidence E that would support a hypothesis H, but you get not-E instead, and you're a coherent Bayesian, then you must become at least somewhat less confident in H.

The true claim in the neighborhood of what you said is that absence of evidence isn't necessarily strong evidence of absence--absence of evidence for a hypothesis needn't make you very confident that the hypothesis is false.

CLOUD SEEDING WORKS TO FIGHT GLOBAL WARMING IMPACTS
---------------------------------------------------
Chinese proved that cloud seeding works in the hands of patriotic experts who plan operations as per standard procedures being followed in 40 to 50 countries since 50 years.see the following web sites to get scientific proof that cloud seeding works:
http://shivajirao.cloudseeding.googlepages.com/scienceofcloudseeding
http://jcsepa.mri-jma.go.jp/outreach/20070131/Presentations/P3_Yao.pdf
http://tshivajirao.blogspot.com/2008/02/cold-cloud-seeding-to-fight-water.html
http://gitam.edu/cos/env/English-Book.pdf
prof.T.Shivaji Rao.B.E.,M.S.[Rice,Texas,1962]Ph.D[Hony]
Director,centre for Environment,Gitam University,Visakhapatnam,India and
Expert,cloud seeding project of Govt.,of A.P.state

The comments to this entry are closed.

Less Wrong (sister site)

May 2009

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31