grain of salt

Since the staff of the Commercial Appeal banned my commenting account there without warning, justification, or notification (this paper is as much a champion of the First Amendment as it is the Second – in a word, “not”), I have to respond to one of its commenters here. Oh well. I guess it helps my Google Rankings for things related to the Commercial Appeal.
At any rate, one of the frequent commenters at the Commercial Appeal is absolutely infatuated with repeating, to no end, the “43 times” fallacy, as penned by Arther Kellerman. For long-time rights advocates, I probably do not need to say any more than that, but for newcomers to the field, this study was published in the 1986 New England Journal of Medicine, and purported to find that a firearm kept in the home is “43 times more likely to kill a family member than an intruder”. Simply put, this entire study is patently and absolutely false.
Unfortunately, though, it carries the weight of “peer review”, which only serves to increase its adherents’ vociferousness that it must be accurate. As mentioned previously on this weblog, peer review helps decrease the probability of errors in published papers, but it does not remove the possibility in its entirety, as even admitted by editors of major medical journals. Unfortunately, blind faith comes in many sizes, shapes, and colors, and those who blindly accept Kellerman’s “study” are quite defensive of it, and their mindless, erroneous devotion.
Thankfully, someone with a little “M.D.” after their name went ahead and demonstrated just how very wrong Kellerman and his study are:

To suggest that science has proven that defending oneself or one’s family with a gun is dangerous, gun prohibitionists often claim: “a gun owner is 43 times more likely to kill a family member than an intruder.” This is Kellermann and Reay’s flawed risk-benefit ratio for gun ownership, [25] heavily criticized for its deceptive approach and its non-sequitur logic. [10] [26] [27] Clouding the public debate, this fallacy is one of the most misused slogans of the anti-self-defense lobby.

The true measure of the protective benefits of guns are the lives saved, the injuries prevented, the medical costs saved, and the property protected — not the burglar or rapist body count. Since only 0.1% to 0.2% of defensive gun usage involves the death of the criminal, [10] any study, such as this, that counts criminal deaths as the only measure of the protective benefits of guns will expectedly underestimate the benefits of firearms by a factor of 500 to 1,000.

Interestingly, the authors themselves described, but did not use, the correct methodology. They acknowledged that a true risk-benefit consideration of guns in the home should (but did not in their “calculations”) include “cases in which burglars or intruders are wounded or frightened away by the use or display of a firearm [and] cases in which would-be intruders may have purposely avoided a house known to be armed….” [25]

Kellermann and Reay had repeated the harshly criticized folly of Rushforth [28] from a decade earlier. In 1976 Bruce-Biggs criticized Rushforth noting that the protective benefits of guns are the lives saved and the property protected, not the burglar body count. [29] Kellermann and Reay would have done well to heed that simple caveat. Objective analysis, even by their own standards, shows the “more likely to kill a family member than intruder” comparison to be deceptively appealing, though only a specious contrivance.

Caveats about earlier estimates of 1 million protective uses of guns each year [10] have led Kleck to perform the largest scale, national, and methodologically sound study of the protective uses of guns suggesting between 800,000 and 2.4 million protective uses of guns each year [30] — not quite as “intangible” as Kassirer claimed [31] — as many as 75 lives protected by a gun for every life lost to a gun, as many as 5 lives protected per minute. Guns not only repel crime, guns deter crime as is shown by repeated National Institute of Justice surveys of criminals. [9] These are the benefits of guns overlooked by scientists whose politics overshadow their objectivity.

At his presentation to the October 17, 1993 Handgun Epidemic Lowering Program conference, Kellermann emotionally admitted his anti-gun bias, a bias evident in the pattern of Kellermann’s “research.”

Doctor Edgar Sutter‘s article goes on to disprove another “study” Kellerman published in 1993, again in the New England Journal of Medicine (seeing a trend?), wherein Kellerman claims that, instead of “43 times”, the answer was really “2.7 times”.
Coincidentally, this disputation of Kellerman and his works comes as part and parcel of a larger package, documenting the flaws and problems inherent with the concept of “peer review”, especially in relation to medical articles concerning firearms.
The moral of the story is simple – studies can show basically whatever you want them to. Statistics is not nearly as cut-and-dried as some people believe, and others would like you to believe, and how you handle a data set directly determines the conclusions one can draw from it. Furthermore, “peer review” often guarantees nothing more than that a select group of your peers agrees with you and your conclusion, not necessarily that your method of arriving at that conclusion, or even the conclusion itself, is factually and logically correct. As the saying goes, “Statistics are like bikinis – what they show is interesting, but what they hide are the really important parts.” Or, more simply, people lie.