« New Tech Signals | Main | BSG is Detached Detail »

March 20, 2009


That is a very bold statement, which I'm sure will upset a large part of the sceptics community. Would you care to elaborate on how you came to this conclusion?

"modern medicine can only claim credit for a small fraction of our lifespan gains"

Do you mean "modern drugs".

Sanitation and Vaccines have contributed to large gains.

Marcus, we've discussed this issue many times on this blog.

Armchair, I don't just mean drugs.

Sanitation isn't medicine.

Neither is clean drinking water.

Someone needs to school them; they make the rest of us anti-mystics look bad!

Long-time readers know, and newcomers will learn, that Hanson approaches the subject of medicine with a combination of biases, most notably confirmation bias and intellectual attribution bias, the two he identifies as the most dangerous in others. Add the conflict-of-interest bias relating to his undergraduate teaching job, and his lack of any actual expertise in the field of medicine and you have the complete picture of loss of rationality. Sadly, much of what he points out about the shortcomings of American medicine is true. Unfortunately, his over-the-top crusader attitude devalues to the reader the truth in much of what he says, and makes him look as misguided (on this one particular topic) as those he criticizes.

In the early 1900's, eliminating trauma, the top ten causes for death in the US were infectious diseases or malnutrition. Today, HIV is the only ID in the top ten. I'd be interested in what brought about this change, in Hanson's view.

Two comments:

First, complaining about mainstream medicine gets you all sorts of approval from the alternative medicine freaks, who have extremely religious-like faith in their chosen sects. This is not pleasant.

Second, despite the paucity of support for most treatments, in fact some treatments are actually in defiance of the evidence, at least mainstream medicine pays lip-service to scientific methods. Most alternative medicine treatments are deeply stupid.

My advice is to stay well away from either, and if you criticize mainstream medicine, you preface your remarks by at least pointing out that you are not advocating alternative medicine.

Robin, a suggestion: it would be interesting if you were to debate with some of the participants over at Science-Based Medicine (http://www.sciencebasedmedicine.org/) on this topic.

The reduction in infectious disease in the last century came about partly from antibiotic treatments, notably for tuberculosis, and partly vaccinations, but more importantly for improved sanitation - and not always obvious improvements.
1. Cleaner water is obvious, this reduced cholera and typhoid.
2. Cheaper clothes, and especially the transition from woolens to cotton, made it easier to keep clothes clean which strongly affected the spread of typhus.
3. Cheap insecticides, especially DDT, which reduced incidence of insect borne diseases, again notably typhus, but also malaria and yellow fever.
4. Cheap window and door screening, again reducing malaria and yellow fever.
5. Better nutrition both reduced risk of infection and made it more likely the victim would survive.
6. More wealth which reduced the number of people crowded into small rooms, reducing the spread of tuberculosis, influenzas, and pneumonias.

@billswift. Weren't those all caused by "scientific advancements" like Minchin says in the poem?

Well "advances", if you want to get all can-I-actually-read-what's-in-the-post on me. Sorry about that.

"They don't realize that the vast majority of medical treatments have no better supporting 'scientific' evidence than the alternative medicine they deride."

I can't contradict you on the economics of health care, because you are an economist and I've yet to study the evidence on that issue with the attention it deserves.

But I just don't understand statements like this one. You are aware of the field of "evidence-based medicine", right? How most common interventions have multiple independent studies, in many cases random double-blind clinical trials, done on their effectiveness by different laboratories? That every respectable organization of medical practitioners publishes guidelines explaining to their members exactly which treatments do and don't have sufficient evidence of working, average number needed to treat, what the average life expectancy with and without the medicine is, et cetera? So that the average physician can say, with confidence, that if your kidneys aren't working you will on average die within a few weeks without dialysis, but live an average of five more years if you receive dialysis according to current best recommended guidelines (I'm not making those numbers up; as far as I know those are the real numbers)

I have sitting next to my computer a book called the BMJ Clinical Evidence Handbook. Opening it up to any given page, it lists a disease, the standard treatments on it, and separates them out into categories called "Beneficial", "Likely to be Beneficial", "Trade Off between Benefits and Harms", "Unknown Effectiveness", "Unlikely to be Beneficial", and "Likely to be Ineffective or Harmful". I know other clinical evidence handbooks that use letter grades from A to F and have a certain number of high-quality independent studies, preferably RCTs, required for each grade, but the evidence handbook I have in front of me happens to use these categories instead. I can look at a graph in the beginning of the book, and I find that 13% of the treatments it lists are in the "Beneficial" category, 23% are in the "likely beneficial category", 46% are in the "unknown" category, and so on. I've never met a doctor who doesn't have some sort of similar book, or more likely several such.

It may be that the average doctor does a bad job of reading and interpreting these studies. It may even be that the health care field is corrupt and doctors deliberately ignore evidence for their own personal gains. I don't know. If you say this is true I'll take your word for it unless I find evidence otherwise. But I am honestly baffled at how you can say "the vast majority of medical treatments have no better supporting 'scientific' evidence than the alternative medicine they deride". All I can think of is that you are taking a few admittedly disastrous failures of the system and then generalizing them to the entirety of medicine.

@Charlie Ullman. I was responding to a comment by retired urologist that has since disappeared, he was defending medicine specifically, and attacking Robin's rationality for Robin's criticisms of medicine.

Robin, I agree with the sentiment expressed here that "the vast majority of medical treatments have no better supporting 'scientific' evidence than the alternative medicine they deride." is an extraordinary claim, that should be accompanied with extraordinary evidence.

It has been discussed many times in the past on this blog, but there are many many posts about medicine, sifting through them all to find your supporting arguments is not easy. Compiling this for your readers seems like a much more effective strategy than deleting the comments of those that disagree with you.

"Or been proved, not to work."

The NIH has spent the last decade doing some fairly rigorous studies on alternative treatments. A few appear to "work" so far. The ultimate question is of course why any treatment appears to work, and it does seem likely more and more that it's at least partially, if not largely, due to social effects - having authority figures and your family pay more attention to you.

Such attention raises relative status, and it seems low-status people are sicker. So naturally a temporary status boost would make you "well." Relationships make you better.

The question of health is just appearing more complex than previously thought. People are not exactly machines - or rather they are evolved machines with social engines.

hege, this happens just about every time I post on the subject; people complain that all relevant evidence is not summarized in a single blog post. It just isn't possible.

Yvian, my claim is about a fraction, not about each and every case, and my claim is not controversial among specialists on this topic. Merely having a "study" does not rise to the standard of "scientific", especially given the severe publication biases. Treatments with net supporting random blind trials are a tiny fraction of the total.

There's no disagreement between Robin's views on medical treatment and a direct, literal reading of the quoted text. The quoted text makes no claims about the effectiveness of medicine in general; it only claims that treatments considered alternative medicine become accepted if their effects are proved (and that "if" is not to be understood as "if and only if"). It also credits "scientific advances", not medicine, with the expansion of life-span. But this is pure nitpick -- Robin is almost certainly correct about what the audience is intended to understand about medicine.

"people complain that all relevant evidence is not summarized in a single blog post"

Have you thought about writing a book -- something similar to Red State, Blue State?

@retired urologist

"In the early 1900's, eliminating trauma, the top ten causes for death in the US were infectious diseases or malnutrition. Today, HIV is the only ID in the top ten. I'd be interested in what brought about this change"

Lawd love a green-eyed duck, ru. Hanson doesn't deny the germ theory of disease.

Sometimes I feel like I'm the only person on this blog who has ever been anywhere. It's easy to go to ancient cities and see how even they - without obvious knowledge of germs - solved some of these problems.

Remember the Chinese were transporting natural gas to light ancient Beijing in clay and/or bamboo pipes by 400 BC; they also had separate pipes for waste and fresh water, altho' we couldn't call that "clean water" by today's standards.

Semiramis (Shammuramat), who ruled Babylon briefly as regent, had covered sewers, open aqueducts, street lighting, and free public schools for the upper classes. You can go see some of the ruins, and the rest is told in clay tablets that record the taxes and upkeep of these structures.

Even the "barbaric" ancient Aztecs had semi-decent sewers - the Spanish missionaries write with awe at the infrastructure and size of Tenochtitlan, which was larger, much larger, and more healthily laid out than Seville.

Having an endless supply of human labor and an ultra-rigid control structure really makes things possible, you know.

Compare ancient Beijing or Babylon with New York City in the late 19th century, with its open sewers in the streets, half-wild pigs wandering the streets to eat the garbage that was thrown out of windows to rot because there was no trash collection, rampant rats, the by-ways ankle-deep in horse manure, horse carcasses left to rot in the alleys, the hordes of disease-spreading biting flies that everyone complained about every summer (that lived in the horse manure), people living in dank tenements 10 to a room - all of this is clearly documented in the NY Times archives.

London was apparently worse - plus those famous choking pea-soup fogs, which killed a lot of people, and which were apparently created by coal particulates mixing with normal fogs and stuff blowing over from France - I mean, clearly Grudge is correct here.

Because we finally caught up to the Chinese millenia later isn't proof of scientific medicine.

Hanson may be correct in 2009, but watch what happens in 2012 when Glaxo puts out their resveratrol based pill.

the vast majority of medical treatments

Is that the vast majority of occurrences of someone receiving treatment, or the vast majority of different treatments that exist, or both?

Because we finally caught up to the Chinese millenia later isn't proof of scientific medicine.

According The Ghost Map the first major public health initiative of nineteenth century London was to build sewers to remove sewage from the city as quickly as possible. The quickest way was via the River Thames. Which supplied drinking water to half the city.

“The first defining act of a modern centralised public health authority was to poison an entire urban population.”

It has been estimated that half the children born in Africa died of malaria within a few years of their birth, quite apart from those who died of something else. The life expectancy of non-Africans who visited was a year or two. If you have ever wondered why history has seemed to pass Africa by, it is because no one wanted to go visit.

Similarly, 18th century Burma survived a Chinese invasion because the invaders were wiped out by malaria

No one bothered with mosquito nets because mosquitos were thought to be irritating, not dangerous.

So we owe a lot to the people who discovered the transmission mechanisms of the common fatal infectious diseases.

On the other hand, when you have eliminated the diseases that killed in childhood, spending a fortune on curing Alzheimer's will have a much smaller effect on life expectancy.

"They don't realize that the vast majority of medical treatments..." (Robin, original post)
"Yvian, my claim is about a fraction, not about each and every case" (Robin, reply)

Please reconcile these two statements and tell me which one you really believe.

Although bad studies are both possible and common, I still think it's an extraordinary claim to say the vast majority of medical studies are bad. I know epidemiologists and doctors. They're not stupid people. We OBers tend to have a high view of our own intelligence relative to everyone else, but these are people who often have doctorates in statistics and have spent their entire lives learning how to conduct studies properly. To just dismiss them by saying "all studies aren't necessarily scientific" is a gross injustice.

Nothing that you've said is news to the epidemiology field. They feel genuinely *guilty* when they can't conduct an RCT for ethical reasons, and they spend an inordinate amount of time trying to compensate for it. I don't deny that pharma companies often fund research, but this falls into the category of "possible confounder to watch for", not "we should throw all this research away and dismiss modern medicine as equivalent to homeopathy" which is literally what you said in your post.

Your focus on average life expectancy might be the problem. I think it's very plausible that medicine has only a minor (but still significant!) effect on average societal life expectancy compared to things like diet and exercise. Medicine is very much about disease. If you're healthy until you die of so-called "natural causes" at 80, medicine can't do much for you. If you have appendicitis, or AIDS, or kidney failure, or a heart attack, or a stroke, or many other acute diseases, medicine can do a lot for you. You may look in any beginning epidemiology textbook and find the exact number of years of your life that each medical intervention can save in each of these cases. I have never heard anyone seriously debate any of these numbers.

If you're saying that medicine is useless (or at least as useless as alternative medicine) because it can't help people who aren't sick, then needless to say I disagree with your definition of useless.

Richard, probably both, but more certainly the later.

Yvian, I don't see how you can be confused; I mean much more than half. Epidemiologists are not the same as docs. The vast majority of practiced medical treatments are not listed in the epidemiology textbook you seem to be looking at (it might help if you told us which one). Feeling guilty about not having scientific support is not at all the same as having such support. The data I rely most heavily on (the RAND experiment) looked primarily at non-mortality health indicators.

Except those researchers wouldn't exist but for having been taught by non-profits. At last a post where Robin admits his own irrelevance.

A colleague assures me most economic historians estimate we would be pretty much just as rich and healthy today had the only "scientists" been researchers funded directly by firms, with no government, charity, or student funding.

I think conclusions like this stem from a hindsight bias that dramatically underestimates the value of the most basic advances in understanding.

But there are deeper flaws in drawing conclusions from this. One is thinking the future is like the past. In the past, scientific understanding was localized and shallow, and significant progress could be made by part-time researchers (I don't know how scientists prior to 19th century were funded, but I would guess even it did in fact involve charity). To continue to advance science requires full-time professional attention, and for-profit firms only fund research with clear benefits to their operations.

A second flaw is thinking that the past arrangements of financing science were the most efficient. It is very possible that, had the modern research university developed 300 years ago and society invested in it as it invests now, that we would today be much wealthier for it. From another angle: given the incentive structure of competitive markets and the difficulty of sequestering knowledge and innovation in light of modern communications, one might argue funding public science is the most efficient allocation of resources toward creating wealth by scientific innovation.

A side point is that many consider the primary purpose of science not to create wealth or health, but to fulfill a central purpose of humanity -- to understand ourselves and our world -- created so as to give meaning to civilization.

Seems to me like you're just trying to sound controversial.

There are a set of common sense, well known observations that underpin the 'conservative care' approach to treatment: essentially that for many illnesses, modern medicine has little remedy, and the correct treatment is to simply allow the body to heal itself. This is widely practised by general practioners (family doctors) in Oz, and I suspect other parts of the world. Is this what you're on about - that a large portion of medical treatment is not necessary?

Presumably you don't doubt that vaccination prevents polio, that antibiotics can cure otherwise terminal infections, that trauma surgeons can prevent fatal blood loss or that dialysis allows people with failed kidneys to continue living? Clearly we're not going to run a double blind study where we only stich up half of all traffic accident victims - we can still reason based on experience that there is good evidence to suggest that sewing people up probably helps.

You seem miffed that you cop so much flak about this, but I find it hard to believe you're not aware of how petulant your arguments sound.

Robin --- your complete lack of basic medical knowledge makes me doubt that you are an "expert" on this topic. You often post links or cite one study without even bothering to do a cursory pubmed search or read a review on a topic for conflicting data. Your sole criteria for posting a link is that it agrees with your pre-established belief. I.e. confirmation bias.

Please CITE these other "experts" who agree with you. The people that come to my mind don't make anywhere near the outlandish statements you make.

BTW, your favorite study has SIGNIFICANT methodological flaws (LOSS OF FOLLOW UP!) and was done in an era BEFORE effective drug therapy for MANY chronic diseases.

And, I will end with the fact that some individual interventions in medicine are BETTER studied than interventions in ANY AREA of economics.This point you seem to conveniently REPEATEDLY ignore. Which is especially amusing given the CHAOS economists have produced currently.

Here's a post I wrote a few days ago on the subject - "How Robin Hanson increased my conviction that healthcare works". Robin probably won't care but other OBers might.

Robin writes an essay articulating some of the reasons for his view in this post here:


Robin - in one of your replies you asked David Cutler to identify which controls are missing in the aggregate regressions of health policy on spending. The obvious omitted variable is population health - more money is spent on medical care for less healthy populations. I'm sure many studies make some attempt to control for this - which in particular do you believe do so adequately? I'm not especially familiar with this literature, but the one study I know with a plausible IV is Doyle (2007) which estimated that emergency medicine spent about $50,000 for life year, a pretty good return. Admittedly this is confined to emergency medicine, and I don't know of other studies with a plausible IV.

I usually interpret fraction to mean "small fraction". No one says "only a fraction of these work" and then goes on to say they all work because 1/1 is an improper fraction. But if you use it to mean "anything less than 1", I'll accept that.

When the BMJ Clinical Evidence handbook says 46% of medical treatments listed are without sufficient clinical evidence, do you accept that number as plausible? Because my interpretation of the numbers is that 54% have evidence behind them (including the most commonly used, since there's more incentive to study a common than an uncommon technique). And even that doesn't mean that 46% are snake oil. They're more likely to be, for example, doctors applying an antibiotic known to work on one kind of bacterium to a very similar kind of bacterium. We can't be sure that it works until someone gets the time and funding for an RCT, but even if there's no clinical evidence there's a lot of Bayesian evidence if the treatment is based on valid theories of the antibiotic and bacterium involved. To say that the evidence is equivalent to alternative medicine is just silly. See also Randomized Control Trials of Parachutes, which was only written because there's such pressure in the medical field to get high levels of evidence for *everything*.

When I said "epidemiology textbook", I was using it figuratively, in the same way you might tell a protectionist that any beginning economics textbook could explain why free trade is a good idea. I didn't have a specific text in mind. But if you really don't believe that there are statistics on the average years of life saved by specific treatments, I will find a book for you next time I go to the library (probably Monday).

No, doctors are not epidemiologists. But many are required to study epidemiology in medical school, and almost all of them are aware that there is a large field working specifically to improve the effectiveness of medicine and that they should be listening to it.

Unless I am misunderstanding you or the RAND study, the presence of non-mortality indicators in there doesn't affect my point. RAND was on mostly healthy people who all received some level of care (ie, appendicitis patients were still going to get appendectomies). Of these mostly healthy people, some of them went to the doctor more for their non-problems, and others had to pay money and so only went when they had actual diseases. There was no difference between these two groups, and that makes sense, because you don't get any benefit from medicine when you're not really sick.

According to Wikipedia (which I admit is an imperfect source), "For those who were both poor and sick -- people who might be found among those covered by Medicaid or lacking insurance -- the reduction in use was harmful, on average" In other words, if there were actual sick people, who needed the insurance because they actually couldn't go to the doctor without it, then not going to the doctor did hurt them. This is your own favorite study, and it's saying very clearly that sick people do better with doctors than without them.

I think you're taking a basically okay point (that some people see doctors unnecessarily) and running way too far with it (to say that almost all modern medicine is a fraud).

there's more incentive to study a common than an uncommon technique

No, that's not true and that's a basic problem. It would be good for society if common techniques were studied, but society doesn't realize that and doesn't supply incentives to the researchers.

Medical research is largely done to get permission to do new treatments. No one is going to care if you demonstrate true something that everyone already believes. Incidentally, it is safer, malpractice-wise, to go with the crowd than to go with the evidence.

Mike, economic historians are well aware of your arguments, and still disagree with you.

ac, vaccination, antibiotics, trauma blood-loss surgery, and kidney dialysis are only a tiny fraction of common medical treatments.

diogenes, as I've said before, every study has "flaws."

Jason, emergency room care is one of the most likely useful class of medical treatments. Many aggregate regressions seem to have adequate health controls, e.g.: this one.

Tomasz, you seem to reason that since you don't believe my conclusions I must be wrong.

Yvian, in the RAND experiment those with cheaper med who took more med had conditions with just as severe symptoms and just as appropriate treatment. This clearly contradicts your "non-problem" vs. "actual disease" theory. Wikipedia is a poor source on this. I and many researchers in this area do not accept that everything not listed in the BMJ handbook as "without sufficient clinical evidence" therefore does in fact have sufficient clinical evidence. We don't get to just assume that their reasons must be "Bayesian" and therefore reliable.

In case people missed the Harvard researcher interviews on 60 Minutes, the pill coming around 2012 supposedly cures type II diabetes and likely significantly reduces the risk of strokes, heart attacks and possibly many cancers. All for about $3 a day.

So Hanson's comment, "... nor that modern medicine can only claim credit for a small fraction of our lifespan gains."

Is about to be turned on its head.

Robin Hanson: My post was more against Aumann's agreement principle than about what you're saying (weakness in your arguments have been pointed too many times by too many people already) - I find your arguments completely unconvincing, what I must take either as Bayesian evidence against your ability to persuade, or as evidence for health care working and you not having facts on your side no matter what are your persuasion skills. Based on your non-healthcare-related posts, and all the pro-healthcare evidence I have, I find the latter option a lot more likely, the facs are simply not there.

Hanson tends to express a lot of religious conviction toward "I'm better than the masses," counter-intuitive, stir-the pot type of declarations. I don't know what proportion of drugs, surgeries, or other interventions are useful, or if most doctors are using only the beneficial ones. All I know is that, on an absolute level, there are thousands of useful drugs and interventions available to me, if am I interested on focusing on those with evidence-based efficacy. Cancer used to be an automatic death sentence, and now Gleevec and Trentoin have led to a cure rate of nearly 90% for the thousands of leukemias that feature their specifically targeted mutations. The fact that an appendectomy or cholecystectomy are now simple laparoscopic procedures, is amazing, considering that appendicitis used to just mean rupture and death. The fact that "questionable" interventions, such as certain types of spinal surgery or prescriptions of statins, tend to bring the "overall efficacy" numbers down, simply means that we should get rid of the myriad specific treatments that work. We should also stop spending the majority of our health care dollars on numerous expensive interventions on end of life meddling, in instances of reasonably clear futility, and we should just allow certain people to die with dignity. Additionally, we should find incentives to rid of the some of the tests that doctors do only to ensure that they don't get sued if they "one day" miss something. I'm sure that, once you remove all of the "clutter" of things that just don't work, (and when the medicine skeptics start to develop better studies that account for the benefits of alleviation of pain and symptoms and more complete understanding of prognosis, rather than just focusing on things that prolong life), then we'd be left with thousands of interventions that are very useful to many people.
Most people are not as stupid as you may think. The public is not being hood-winked into using healthcare, but are often responding to real incentives to better their lives. There is danger in assuming that everything some doctors say is true, but there is also danger in overstating the "anti" case, by pretending that the bit of analytical critique that we've made is as strong or as broad as we think it is.

Robin, I don't like to use Wikipedia but I can't find the full study. Do you know anywhere it's freely available online?

You seem to be claiming that you are better qualified to say what interventions are or aren't clinically supported than the people who write the BMJ handbook, who are basically the people in the medical field best trained and most qualified to analyze this sort of data. If for some reason you distrust the BMJ team, I could find you any of fifty similar handbooks that probably say more or less the same thing. You seem to think that you are better able to judge medical evidence than all the world's top doctors and epidemiologists combined. So far, you haven't presented a lot of evidence for this except that you've read the RAND study, which most of the doctors I've talked to have also read.

Even in the case of the "unknown effectiveness" studies, I'm not just "assuming" the doctors have some evidence for them. You seem to be assuming they don't. But I assure you doctors don't just think "Well, I have no reason to believe this would work, but why don't I pour this chemical down my patient's throat?" Fields like microbiology, pharmacology, physiology, and organic chemistry exist for a reason. We don't have to believe they're always right, and I certainly don't, but to dismiss their inventions as always wrong, or just as likely to be wrong as homeopathy, is perversely anti-scientific.

I don't understand why you think the people with cheaper care having "just as severe symptoms" contradicts my theory. I'm not saying one group had more symptoms than the other. I'm saying both groups were on average healthy, and on average would pay for the *really* important treatments even without insurance. (I did claim they did a separate analysis later limited to low-income, generally unhealthy people, though. If you're saying that's false, please clarify and I'll try to find more information).

Let me give an example. Consider a study of 2000 people: 1000 with great health insurance and 1000 with poor health insurance, watched for ten years. In those ten years, 1000 out of 2000 people (evenly distributed across both groups) get minor diseases or aches that wouldn't have hurt them anyway. The ones with great insurance go to their doctors and get minimal benefit. In those same ten years, 100 out of 2000 (again evenly distributed) get serious diseases that could kill them. All of these people go to their doctors and get appropriate treatment, because even a low-coverage health-insurance will pay for a vital life-saving treatment, and even if the health insurance didn't the person would pay out of pocket anyway.

Therefore, in the case where all medicine works, we would still expect to see no difference between high-coverage and low-coverage groups. The only case where we would expect to see a difference would be people who having serious disease but can't afford out of pocket treatment (ie the rare people who won't get necessary treatment in control group but will in experimental group), and the study did show a difference there (at least if I trust Wikipedia, which is rarely perfect but also rarely lies outright).

I interpret the RAND study as successfully showing lots of things of interest to economists and policymakers, and of showing rather less of interest to doctors: possibly that "preventative medicine" - treating currently healthy people to prevent future disease - is flawed.

You're taking a single study intended to look at something else, with the completely wrong design to study what you think it's studying, which actually ends up disproving your position anyway, drawing an extreme conclusion contrary to that of everyone actually in (or out of) the field, and in the process you're contradicting literally millions of much more relevant studies with the opposite result, which you have never read but are certain must be flawed anyway...

...and it's why you're Robin Hanson, and why we love you, and quite often you're even right. But in this particular case I think you may be going too far.

Yvian, I've posted many times before on the RAND experiment, giving cites and summarizing the details. Why not read those before asking all the same questions again? Why should I have to repeat all my arguments every time I even mention the subject? I'm not arbitrary dismissing docs judgements. There is a serious contradiction between docs local claim about the effectiveness of each treatment and our data on aggregate effectiveness, and the aggregate results must win I think.

Yvian, why don't you compose a post where you lay out your critique/argument, so I can respond to that?

The RAND study is interesting, and I'm somewhat distressed not to have come accross it before.. Another bit of evidence for RH's argument is the repeated observation (in Canada, Scandenavia, and Israel) that when physicians go on protracted strikes, the mortality rate drops (PDQ Epidemiology, 1998, pg. 10; unfortunately the text doesn't cite the primary references).

While I think RH might overestimate the uselessness of medicine, the observation that more access to healthcare doesn't necessarily result in a healthier population is a usefull one.

... "the vast majority of medical treatments have no better supporting "scientific" evidence than the alternative medicine they deride"

Errr, sure, many conventional medical interventions have weak evidence bases (here's a great example ), but as Yvain points out, EBM is huge and we're generally aware of the problem. Contrast that with alternative medicine circles, where there is often outright hostility to criticism and disdain for the very pursuit of evidence. Equating the scope of the problem in mainstream vs. alternative medicine is inaccurate.

... "nor that modern medicine can only claim credit for a small fraction of our lifespan gains."

That, surprisingly, is true. I take it you're not considering hygene/vaccination/nutrition to be medicine (which I get) but you could reasonably chalk them up to non-mystical science, which I think is Minchin's point.

Robin - thanks for the link to the Fisher et. al. study. After partialling out the controls for disease burden, where does the residual variation in the number of hospital beds come from? Why do some Hospital Referral Regions have more hospital beds than others? It seems there are many possible answers, but in most of these, hospital beds would be endogenous. For example, there may be more per capita hospital beds in regions with higher demand due to any disease characteristic not controlled for! This seems to be a natural explanation for why some regions would have more hospital beds than others after controlling for demographic characteristics, and if it's correct, then the regression results would be seriously biased. Is there some factor I'm missing which would could plausibly drive most of the variation in the number of hospital beds which is otherwise unrelated to mortality?

Since this discussion is winding down (or moving to another venue), I thought I would link to Hanson's early post on his views on medical care for posterity:


Robin, as has been mentioned a hundred times, and like always you ignore religiously because it would actually require an ounce of knowledge of medicine. The MAJOR flaw with RAND is that preventative medicine -- which is basically what it was looking at -- was non-existent at the time the study was conducted. I've TOLD you OVER, and OVER, and OVER again, that the drugs used for prevention (or the trials documenting how to use them) were not available during the study period. This is called "generalizability" in the epidemiology literature -- which again you seem to be completely unaware of. I've even listed the interventions commonly used in preventive medicine for you so you can check the dates yourself. I've even told you how you could easily run a simulation using randomized trial data to SEE that I'm basically right and you are WRONG. OF course any verification of your religious beliefs apparently seems too much for you to bear.

I've also repeatedly asked for you to cite other people who make NEARLY as outrageous claims as you do. WHERE are these other "rationalists". Apparently other people on this blog, think you are delusional and this is a case study in bias. If we used your favorite tool -- decision markets -- restricted to the overcoming bias audience, it seems like the majority would bet on you being delusional.

Alex, vaccination counts as med, but not food or washing, neither of which is plausibly "science." Distain for study does not count as negative evidence, nor does being aware of a lack of evidence count as positive evidence.

Jason, there is a huge and well-documented "practice variation" due to differing local medical cultures. This easily and plausibly explains med scale variance.

Robin: I should make a post on it someday, but before I do I should understand your position in much greater detail. Can you recommend a book for someone who's not a trained economist, available at most good libraries, that states your views in a way you'd agree with? (also okay: something free on the Internet. I've read your OB posts but don't find those sufficient.)

Robin, I guess I should put my question differently: why would we think that variation in the number of hospital beds across regions is driven by differing medical cultures rather than differing demand for hospital beds? If we were to regress the number of hospital beds on some measure of "local medical cultures" what would be the R^2? Even if just a third of the variation were driven by demand side factors, this would lead to seriously biased estimates because those factors may directly impact mortality. Fisher et. al. provide no evidence on this point which is central to their identification strategy. If they were to observe some direct measure of "local medical cultures" and use that as an instrument for resource expenditures, their finding would be much more believable.

Let me play devil's advocate here and state the case against your view as strongly as I can (and more strongly than is warranted given my own familiarity with the literature):

1) The RAND experiment is 30 years old and so may severely underestimate the benefits of health care today

2) These aggregate regressions don't meet the standards of evidence required for publication in top economics journals (can you link to any from the AER, JPE, QJE or ECTA?). They are systematically biased towards finding no effect of medical treatment or a negative effect because they fail to fully control for the fact that medical spending varies in response to the demand for medicine. It is not sufficient to point to the quantity of these studies because we know that that whole literatures can give systematically wrong answers if they fail to adequately grapple with identification (e.g. the class size literature prior to the Tennessee Star Experiment).

3) Even if we take them at face value, these aggregate regressions might not tell us the impact that increasing the price consumers face for medical treatment would have on which treatments they select. If your contention above is correct, the variation in current medical expenditures across regions is driven by supply-side factors ("local medical cultures"). If we increase the price of medical spending for consumers, that variation would still exist, and it is a separate empirical question what treatments consumers would forgo. As Cutler and Garber point out, there is some evidence that consumers are not good at forgoing only the least effective treatments. In fact, the marginal treatment induced by variation in "local medical cultures" is a treatment regarded as marginal by experts (some doctors choose to practice it and others don't), so we might be especially worried that this treatment would be less valuable than the treatment consumers would forgo if facing higher prices.

4) We just don't have good evidence on the question of whether large cuts in medical spending would lead to welfare gains

5) Repeating the RAND experiment could have potentially HUGE returns

6) We should attempt to reduce the number of dollars spent on many procedures which are known to be inefficient

To the extent that we disagree, presumably it is on points 2)-4).

Robin, I take it that you believe yourself to be an anti-mystic. As such, I assume your argument is not that medicine based on good science has the same effectiveness as medicine that is not. If that is your argument, I would love a link to a piece that makes it in more detail. If it is as I assume, them maybe it is the wording of the post that causes such dissent.

A hypothetical: You are afflicted with a disease. You have a simple choice (I recognize this is unlikely). You could see an MD who you know to be honest, intelligent, empathetic, and competent. You know this person tries hard only to prescribe medication and treatment that is backed by unbiased study. You could also see a practitioner of "alternative medicine" who has the same personal qualities as the MD and is considered very highly in his/her profession by his/her peers. Should you see one or the other? Should you just stay home?

I suppose what I'm wondering is: do you think that medicine is failing to be empirical because of various reasons (biased studies, market factors, etc), and as a result it fails to be significantly more effective than alternatives? Or, that even striving for science-based medicine is folly? Or, something else completely?

You won't get as much argument from the first premise as from the second, is my thought. Forgive me if this has already been made clear elsewhere.

"...but not food or washing, neither of which is plausibly "science." "

Decidedly non-magical benefits of our better understanding of the world though, no? I don't think we really disagree here.

"Distain for study does not count as negative evidence, nor does being aware of a lack of evidence count as positive evidence."

That's not my claim. I merely think that the evidence problem in medicine vs. CAM is different in character, thus equating the two is misleading.

Posted by Robin: A colleague assures me most economic historians estimate we would be pretty much just as rich and healthy today had the only "scientists" been researchers funded directly by firms, with no government, charity, or student funding.

Mike, economic historians are well aware of your arguments, and still disagree with you.

This claim is very interesting. What are the arguments, and where is the evidence?


" If we used your favorite tool -- decision markets -- restricted to the overcoming bias audience, it seems like the majority would bet on you being delusional."

Of course you could toss a market up in a relatively short time, I know. I think you should; it would be a great idea. But my strong impression is that you are Quite Wrong. The majority of the steady OB readership in fact probably doesn't disagree with Hanson here, to the best of my current knowledge.

I'll give that 80%. Delusional seems ad hom. at best; do you have a real argument, one that's concise, with strong evidence? I don't mean to seem partisan, but you're not making it here that I can see.

Regular readers really don't seem to be participating in this discussion - you non-regulars who don't seem as familiar with Hanson's thinking or past posts and the links he has provided in them appear to be re-hashing your something that is looking more and more like a personal drama endlessly.

Perhaps the discussion would be more fruitful if you would do the work of reading what's available on the subject and then make a true case? I would be interested in reading that, thanks!

Jason, every study has flaws. If you decide you can dismiss all aggregate med studies as meaningless, well then sure the default would be to accept the usual local med studies and claims at face value. I did post saying we should repeat the RAND experiment.

Walter, once I knew a treatment was "backed by unbiased study" that would be enough, regardless of if that treatment were "alternative" or not. Yes, it is the biases that make most med untrustworthy.

I've read the majority of Robin's posts on health care, and have found very few of them convincing. Basically, I think the ad homs are an appropriate response to the quality of Robin's arguments. I've asked Robin to cite someone else who agrees with his views -- but he can't seem to find anyone. Their is obviously a career advantage to taking extreme views -- I'm not sure what form of "bias" that would be called.

Robin rarely has read the medical literature on an intervention he has posted. The most recent example comes to mind, when he cited some alternative-diet quack who was railing against statins. Robin found his "argument very convincing". And he is right -- the argument is convincing if you don't bother to do a pubmed search to find contradictory evidence. Alan Garber, who is director of health care policy at NBER recently wrotestatins are so effective, that they are killing research for other cardiovascular drugs

Lastly, Robin's favorite study has big holes in it, that he dismisses as "every study has flaws". Now maybe thats the standard of evidence for Economics -- but in medicine, a study with as many flaws as RAND would not be accepted until it had good replication.

So to sumarize -- Robin has a career advantage to take an extreme view, doesn't bother to look at the relevant medical literature (EBM or clinical trials), and ignores the gigantic holes in the studies supporting his views. My sense is that this justifies an ad hom attack.

BTW, the problem with the decision market on Robin being delusional, is coming up with objective decidable criterion that Robin would agree to as well.

The comments to this entry are closed.

Less Wrong (sister site)

May 2009

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30