I had occasion to read an article about six sigma and reliability – it is here.

The essence of this article is that by measuring a long term sigma value, one will know the reliability of results, where reliability is equated to the number of defects. Defects are defined as results that are outside of the performance goals.

To recall: six sigma = (allowable total error – bias)/CV. High six sigma numbers are good, low numbers not so good.

The problem with this article is that it bases everything on Normal distribution statistics. Now this may make sense if you are measuring rulers sold by Home Depot but it doesn’t work for blood tests.

Consider a glucose meter. Unlike the Home Depot ruler, there’s a lot more going on in a drop of blood. There are thousands of compounds that can interfere. Say that one does, and that the meter reads 340 and truth is 40 mg/dL. Assume that the CV at 40 is 3%. The value at 340 is 250 standard deviations away! I challenge anyone to try to calculate the probability of such an event. There’s not enough zeros on the planet. Thus, the 340 value, which can happen, is not part of the measuring error of the usual process.

So any attempt to judge the number of defects by a six sigma calculation will miss the really big errors. And these are the errors that cause harm to patients.

An additional problem is attaching significance to the numerical six sigma results. Now this may sound like heresy but here’s an example.

Say you were comparing the Roche glucose meter a few years ago (yes the one with the maltose interference problem) with some other meters. The Roche meter would have probably had a high six sigma value and thus looked good. Obviously, this would have been a bad choice.

But what about in general? Consider what a lower six sigma number means. Yes there will be more values beyond the performance limits but these values will be a few standard deviations away and close to the performance limits. Unfortunately, six sigma values provide no information about large errors.

Sorry, but to evaluate the possibility of large errors caused by interferences requires extensive interferences studies or alternatively huge patient correlations (the kind that no one does).