|Long a part of other disciplines, risk management has become very visible in healthcare in the last few years. Perhaps the biggest reason is that since 2002, the JCAHO (Joint Commission on Accreditation of Healthcare Organizations) requires healthcare organizations to conduct at least 1 FMEA (Failure Mode Effects Analysis) each year. Since then, more attention is being paid to risk management in healthcare. For example, the organization CLSI (Clinical and Laboratory Standards Institute) devoted its 2006 annual meeting to this topic. For diagnostic assay manufacturers, risk management was on the map earlier since starting in 1996, FDA has required manufacturers to perform risk management activities through QSRs (Quality System Regulation).
This essay covers various dimensions of risk analysis as shown by the following figure.
Risk management standards – horizontal vs. vertical
Standards are a good place to start since standards should contain guidance about risk management.
There is an ISO standard (14971) about risk management for medical device manufacturers. (There is an appendix – it’s called an annex in ISO speak – for manufacturers of diagnostic assays). This is a so called “horizontal standard” meaning that principles are explained but unlike a “vertical standard” no detailed specific procedures to follow are recommended such as those – for example – in the CLSI standard EP9 (how to perform a method comparison to estimate bias). Horizontal standards are often called “flexible” with the appeal that one can pretty much do anything and say that one is conforming to the standard. So vertical standards would be useful for risk management.
There is a CLSI standard EP18A concerned with FMEA for unit use devices. This is also pretty much a horizontal standard.
Risk management goals – qualitative vs. quantitative
Goals have been hard to come by in lab medicine. Often, lab goals are more qualitative than quantitative as in “we don’t want to release any problem results.” One possible set of quantitative goals might be answers to the questions:
1. what difference from reference constitutes an unacceptable error and
2. how many times is that difference allowed to occur.
Although much have been written about the topic, there are few standards. In fact, two groups gave up on this task (CLSI EP20 and ISO 15196). There is an ISO standard (15197) for home use glucose assays, which answers both questions. That is, medical acceptability limits are given but the rate required for acceptable results is only 95%. This means that up to 50,000 medically unacceptable results per million results would meet the ISO goal. Clearly, something is wrong which was described in an article about goals (1) which suggested that along with limits given by the glucose standard, (allowable total error), there needs to be another wider set of limits for which no or few results are permitted. Note that the new proposed waiver guidance from the FDA (see related essay) embraces this concept.
Levels of risk management
Anyone can write software, yet in the software industry, the level of expertise of an organization to write software has been formalized through the Capability Maturity Model Integration (2) from Carnegie Mellon. So here is an attempt to categorize risk management practices in a similar fashion.
From what I see, most of the diagnostic assay industry is at level 2 and sometimes approaching level 3. Labs are transitioning from level 1 to level 2. Nuclear power is an example of an industry at level 4. A recent article (3) about risk management for medical devices describes fault trees in terns of level 4, although it is doubtful that this level of fault trees are carried out.
The issue is that saying one is conducting risk management can mean anything from level 1 to level 4.
Knowledge of risk management
In my experience in clinical chemistry, I often see people with a limited knowledge of risk management involved in risk management activities. To assess your interest/knowledge, take the risk management quiz
Potential vs. observed errors
Any process has both:
Risk management can be thought of as addressing both types of errors. FMEA addresses potential errors and FRACAS (Failure Reporting And Corrective Action System) addresses observed errors. FRACAS is not mentioned as a technique in ISO 14971 but consider the case where errors have been observed. If one puts in place effective corrective actions, one has reduced the risk that these errors will recur. This is an important case for manufacturers, who often exercise their instrument system during development to expose and correct for errors (4).
Risk management within the culture of an organization
I give a training session on risk management which has a section which starts with “I have never met an engineer who wanted to be in FMEA meeting”. There are many reasons why this is so including:
Risk management activities are at times conducted during lunch, with lunch provided as an attendance enticement, but also a signal that the risk management activity shouldn’t interfere with normal work activities. Moreover, risk management activities are often conducted to meet regulatory requirements and are conducted once. The goal of such programs is to pass an inspection or audit.
When a department initiates its own risk management program, it usually has quantitative, measurable goals and is often carried out over a longer time period for the product or process. Success is defined as meeting these measurable goals. To summarize:
Risk management programs
Risk management possibilities
Risk management has the potential to improve quality and reduce error rates. Yet risk management practices can range from almost no activity to extensive programs. Benefits from risk management will be proportional to the effort that is expended.