CLSI C51 – measurement uncertainty – or the classic comic version of GUM

February 29, 2012

GUM (Guide to the Expression of Uncertainty in Measurement) for laboratories (and manufacturers) is what CLSI C51 is all about.  (GUM was originally used to provide information about reference materials). I have previously commented that I didn’t think that GUM was a good idea for laboratories (1). I was also initially on the C51 subcommittee but since I couldn’t convince anyone about my point of view, I bailed.

To recall some of the problems with GUM …

  1. bias is not allowed – it must be corrected. But you could ignore big, rare biases (outliers) as well as real small biases.
  2. To obtain the standard deviations or bias corrections applied by manufacturers was impractical if not impossible for laboratories as in … Let’s set up a fixture and measure the variability of 10 pumps we just bought for this experiment.
  3. The math required to put together an estimate will make most people’s head spin.

In the C51 version of GUM, there is only 1 example – that of measuring a bunch of controls. This is not GUM! and will not provide an uncertainty estimate for patient samples since controls do not estimate the non specificity assay errors in patient samples.

Reference

  1. Krouwer JS A Critique of the GUM Method of Estimating and Reporting Uncertainty in Diagnostic Assays Clin Chem 2003;49:1818-1821.

Matrix Effects (and CLSI EP14-A2)

February 8, 2012

Here’s my problem with matrix effects using an oversimplified and somewhat not real example.

Take a glucose assay. If hemoglobin in a patient sample causes a signal and provides a biased glucose result, the hemoglobin is called an interference.

But if a control were to cause a biased glucose result due to the same amount of hemoglobin in the control, the control would be said to have a matrix effect. (I know that hemoglobin doesn’t interfere that much but I wanted to avoid using compound X.)

Note that the definition of a matrix effect, given in EP14-A2 is synonymous with the definition of an interfering substance. (There is no definition of an interfering substance or its strange ISO cousin “influence quantity” in EP14-A2.) But in the text of EP14-A2, a matrix effect is defined as a material which does not behave as fresh human specimens.

Yet, in the above case, the control is acting identically to a human specimen.

Now I know that what people really mean by matrix effects in controls is that either the processing of the control or something added to the control (such as a preservative) that does not occur in fresh human specimens, and causes interferences.

But the experiment in EP14-A2 does not distinguish between artificial and natural interferences. So it is possible that differences in results between controls and patient specimens are due to substances that occur in human specimens, substances that don’t occur in human specimens or a combination.

Unless a method can distinguish between artificial and natural interferences, the assay is as suspect as the control.


The “Duh” Table in Risk Management

February 1, 2012

In ISO 14197 and other places, there is a table to help people decide about acceptable risk. A reproduction of this table follows. Here’s why this table is silly.

  negligible minor major critical catastrophic
frequent Unaccept. Unaccept. Unaccept. Unaccept. Unaccept.
probable Acceptable Unaccept. Unaccept. Unaccept. Unaccept.
occasional Acceptable Acceptable Acceptable Unaccept. Unaccept.
remote Acceptable Acceptable Acceptable Unaccept. Unaccept.
improbable Acceptable Acceptable Acceptable Acceptable Acceptable

This table says a frequent catastrophic error is unacceptable – duh! For example, half of the surgeries in a hospital are performed at the wrong site or patient. One doesn’t need a risk management standard to tell us this is unacceptable.

Perhaps the most important function of risk management is to prevent catastrophic errors. For any reasonable process in healthcare, the above table really looks like the one below.

  negligible minor major critical catastrophic
frequent          
probable          
occasional          
remote         Unacceptable?
improbable         Acceptable?

The use of question marks suggests that it is difficult to really know when the risk is acceptable, but there always is a point where the remaining risk has to be acceptable because risk is never zero and funds are limited. But most of the effort should go into reducing risk of catastrophic errors.

To deal with all errors, the following table is suggested.

  negligible minor major critical catastrophic
frequent Unaccept.        
probable Acceptable Unaccept.      
occasional Acceptable Acceptable      
remote Acceptable Acceptable Unaccept. Unaccept. Unaccept?
improbable Acceptable Acceptable Acceptable Acceptable Acceptable?