Bias advice for GUM – It’s uncertain!

December 17, 2004

I have critiqued GUM (1) and a recent Letter reminds me of another concern with GUM (2). This Letter contains advice about bias which is taken from GUM: “if a bias is considered small compared with the overall uncertainty, it simply may be neglected”.

The spirit of this advice is straightforward – if a bias would affect values after the seventh decimal point, ignore it. But life is not always that simple. Take linear drift for example, which is a “protocol dependent” bias (3) and depends on at least two quantities:

1. an incorrect calibration algorithm
2. the time since the last calibration

Assume that the calibration algorithm is incorrect because it leaves out temperature in the calculation of the response. Temperature is left out because it is supposed to be constant with respect to the temperature when the calibrator was assayed but assume that the temperature is drifting. The bias that results is a function of when the sample is assayed, and the rate of temperature drift. This brings up another concept which is the probability of a large enough error due to drift. That is, a large temperature drift and a sample assayed at the end of a calibration cycle (both of which would cause an error that is not negligible) might occur about 5% of the time, right at the border of many uncertainty intervals. Thus, can one ignore rare (e.g., those that occur much less often than 5%) large biases?

So far, all of this is for one bias. What happens if one has more than one bias, whereby each bias considered by itself is “negligible”?

So I find some “uncertainty” in the “Guide to the Expression of Uncertainty in Measurement”. I note also in passing that the Guide is really two guides: the Guide (pp 1-28) and the appendices – annexes in the ISO world (pp 29-90).

So is there an alternative to GUM? Yes, calculate empirically based uncertainty intervals (4-5).

References

1. Krouwer JS A Critique of the GUM Method of Estimating and Reporting Uncertainty in Diagnostic Assays Clin. Chem 2003;49:1818-1821.
2. Stöckl D, Van Uytfanghe K, Rodríguez Cabaleiro D, Thienpont LM, Patriarca M, Castelli M, Corsetti F, and Menditto A Calculation of Measurement Uncertainty in Clinical Chemistry Clin Chem 2005 51: 276-277
3. Krouwer JS Multi-Factor Designs IV. How Multi-factor Designs Improve the Estimate of Total Error by Accounting for Protocol Specific Bias. Clin Chem 1991;37:26-29.
4. Estimation of Total Analytical Error for Clinical Laboratory Methods; Approved Guideline NCCLS EP21A, NCCLS, 771 E. Lancaster Ave. Villanova, PA., 2003.
5. Krouwer JS and Monti KL A Simple Graphical Method to Evaluate Laboratory Assays, Eur J Clin Chem and Clin Biochem 1995;33:525-527.

Why progress in quality is slow

December 13, 2004

In many organizations, quality activities are carried out at a slower pace than other activities. Here are some reasons why.

Quality activities are under-funded because other investments are more attractive – There are typically three types of spending (1):

1. (High risk) projects that if successful will be financial blockbusters. Example, research for a new tumor marker.
2. (Bread and butter) projects that have a known return. Example, a new MRI machine.
3. Quality and maintenance projects. Example, performing an FMEA to prevent medical errors.

Given the financial challenge facing hospitals, a quality type project is not as attractive financially as a # 1 or # 2 project. One cannot advertise the absence of medical errors – customers expect this. The rationale to spend any money is both to comply with regulations and to ensure that enough quality is in place to prevent errors and the resulting financial losses that would follow.

It’s not what most people signed up for – In medical diagnostic companies, engineers and scientists enjoy designing and building things. Focusing on quality activities is perceived as a diversion from design. In hospitals, clinicians sign up to treat patients and or to do research.

It’s often perceived to be adversarial – In diagnostic companies and hospitals, people don’t like to be audited as it is often perceived as an adversarial task.

It can require excessive documentation – Quality activities are at times driven by having to prove to inspectors that certain tasks has been performed. Often the inspectors rely on the site to produce documentation, which by itself doesn’t guarantee quality (2).

It can be confused with certification or accreditation – Passing certification or accreditation is necessary for many organizations to operate, but certification and accreditation don’t guarantee freedom from error. Other quality activities may be neglected because limited resources are devoted to certification and accreditation.

Management commitment is lacking – When management commitment to quality is lacking (e.g., management is faking it), people realize this quickly and pay less attention to quality activities. For example, a high level quality position is assigned to a second rate manager, or pressure is applied to release a product which has not met quality targets.

Are these problems present at your institution? – They may be if any of the following situations arise when trying to implement a quality project (3):

• Great idea – let’s do it on the next project
• But our situation is different
• It will take too long / is too expensive / is too complicated

The solution – There is no easy solution; however, by recognizing the above situations and pointing them out whenever possible, improvements in the climate that facilitate more focus on quality are possible.

References

1. Krouwer JS Assay Development and Evaluation: A Manufacturer’s Perspective., AACC Press, Washington DC, 2002, pp 29-30.
2. Krouwer JS ISO 9001 has had no effect on quality in the in-vitro medical diagnostics industry. Accred. Qual. Assur. 2004;9:39-43.
3. Krouwer JS Assay Development and Evaluation: A Manufacturer’s Perspective., AACC Press, Washington DC, 2002, pp 7-15.