The real story about acceptable risk for diagnostic assays

Both manufacturers and clinical laboratories use risk management techniques such as FMEA (Failure Mode Effects Analysis) which involve:

• enumerating potential failure mode events that could cause patient harm
• classifying the events with respect to severity and probability of occurrence
• preparing a Pareto chart or table of events with the highest severity X probability
• implementing mitigation for the most important events

One is done when the mitigations have reduced the residual risk to an acceptable level. Unfortunately, there is no guidance as to what is an acceptable level and this is the subject of this blog entry.

Manufacturers could consult the ISO standard on risk management 14971. This suggests performing a risk benefit analysis. While this might work for some medical devices, it is not useful for diagnostic assays. Consider glucose meters, as an example. Each year, a small number of patients are seriously harmed by incorrect glucose meter results. Yet, if all glucose meters were removed from the market, the harm from the lack of information would be much worse.

For clinical laboratories, the decision of acceptable risk is up to the laboratory director. Do laboratories directors have some special power to know when residual risk is acceptable? The answer is no, and to explore how that decision is made – which applies to manufacturers as well – consider a blood gas analyzer. The results from these instruments are time critical. If a result is unavailable due to the instrument going down, serious harm could occur. Therefore, laboratories use the mitigation principle of redundancy and have multiple instruments. If one knows the MTBF (Mean Time Between Failure) and MTTR (Mean Time To Repair), one could calculate the probability (e.g., risk) that a result would be unavailable due to all instruments having simultaneously failed, as a function of the number of instruments. Whereas, one could lower this risk to an infinitesimal amount, one is financially constrained and it is this constraint that plays a major role in determining the level of acceptable risk. And the basis of the financial constraint is the socioeconomic climate of the prevailing country.

One would like to think that risk is always low enough so that if serious harm does occur, it is due to some unanticipated event. (Of course, this brings up another issue, which is how much effort should be devoted to studying things to reduce the risk of unanticipated events). But there are known events which can cause serious harm. I once heard from some laboratory directors that in spite of the fact that they know that HAMA interference can cause wrong results and has caused serious harm, they do not put in place procedures to test for HAMA interference in results, due to economic constraints.