« ...Recursion, Magic | Main | Physicists Held To Lower Standard »

November 25, 2008

Comments

It's good to know. I've always suspected that changing answers wasn't as bad as many people think. In fact, I've always been an answer changer. It just seems logical that if one has thought a second time about something, the probability of getting it right rises.

Of course, those who prefer avoiding "if only" self-recriminations, rather than raise their scores, should stick to the common wisdom and to their first intuitions.

It's not especially clear how to reconcile this with the observation that "second guesses were less accurate than first guesses":

http://www.overcomingbias.com/2008/06/average-your-gu.html

Spin, forcing people to make a second guess tends to get a worst guess. But in the situations where people naturally feel inclined to change their guess their second guess gets to get a better guess.

Spin, also, one might think there could be a difference between choosing from a list of supplied choices and guessing without guidance.

When I was in school, I was actually told the opposite on multiple occasions: That most answer-changes were from correct to incorrect.

I wonder if anyone has done studies regarding answer patterns (ABBCDACBAA....etc) in tests. I know that I used to fill in the blanks (the ones that I didn't have any clue on) by searching for unused letters. Worked for the SAT...

I think the "go with your first answer" is part of an anti-rational worldview, that teaches that gut intuitions are better than reason.

If anyone is interested in a single informal anecdote that contradicts a peer-reviewed scientific study:

Back when I was eleven years old and taking some practice tests for the SAT, I found that my SAT Math score was going down each time. Checking the answer sheet, I found that I was changing right answers to wrong answers. So I sat down and said to myself: All right, this time I'm going to use the Force and act on instinct. And my score shot back up, and stayed there through the actual SAT.

Could be an individual difference.

If I may deconstruct the argument:

1) Assume the relationship between the T (threshold-for-answer-changing) and C (chance-that-a-change-improves-test-score) isn't sharply discontinuous;
2) We know that at current levels of T, C is > 50%
therefore:
3) On average, people should decrease their levels of T.


Taken literally this reasoning is fallacious. What we want to know is the marginal gain from decreasing our threshold, not whether the average C given the new threshold is >50%.

It would be fair to say that this gives us some evidence that it might not be harmful to further decrease our answer-changing threshold, T.

Of course, it is possible that some of the cited studies have attempted to plot the relationship between T and C (for example, by instructing different students to use different heuristics for when to change one's answer, and plotting the relationship). In which case the quoted passage is fallacious at face value but only because it oversimplifies the point.

If there's an accepted bias toward not changing your answer it stands to reason that most changed answers will be from incorrect to correct. Only a clearly wrong original choice would be changed. If you adopt a strategy of changing your answers, you might find yourself changing from correct to incorrect more often.

The comments to this entry are closed.

Less Wrong (sister site)

May 2009

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31