Westgard Quality Control Workshop – Part 3

I just returned from the Westgard quality Control Workshop, where I was a speaker and have a few blogs worth of comments – this is the third.

EQC – Equivalent Quality Control

This is the CMS proposal (1) to allow clinical laboratories to reduce the frequency of quality control from twice per day to once a month given that 10 days of running QC shows no values that are out (and given some other conditions).

Let’s try to construct a hypothesis to base such a recommendation. For example:

given any possible error condition that could be detected by external quality control, internal quality control would detect the same error 100% of the time.

This is about the best I can think of, which would result in the recommendation:

Stop running external quality control.

What does running 10 days of external QC with no out of control results show? The answer is nothing. This is because one can assume that during these 10 days, there were either no errors or if there were errors, external QC was not able to detect them. (It is possible that internal QC detected errors during these 10 days). In fact, this experiment is guaranteed to be meaningless. To see this, one must realize that internal QC is always “on” and precedes external QC. So to see if external QC is redundant to internal QC for an error, would mean that internal QC would detect the error and either shut down the system or prevent the result – this being the external QC sample – from being reported. However, one can get different information by running external QC for a longer period because if internal QC misses an error but external QC detects the error, then one has proved that external QC is not redundant to internal QC. This was shown to me (2) as out of control results for a range of assays ranging from 1 to 10 per year, where these were real problems. Since controls are run twice per day, the number of affected patients samples is larger.

So a lab that reduces external QC to once a month is risking an even larger number of patient samples which is made worse since the clinician has probably acted on the erroneous results.

Rather than do the experiment suggested by CMS, a lab can simply examine its external QC records for a sufficient length of time.

References

1.       To review, see: See http://www.aacc.org/events/expert_access/2005/eqc/Pages/default.aspx

2.       Personal communication from Greg Miller of Virginia Commonwealth University

Westgard Quality Control Workshop – Part 2

I just returned from the Westgard quality Control Workshop, where I was a speaker and have a few blogs worth of comments – this is the second.

How does one determine acceptable risk

This was one of the questions asked by a participant – are there any guidelines? I also commented recently, that in spite of all of talk about risk management and putting in place control measures until one has acceptable risk, no one knows what acceptable risk means. Here’s some more thoughts on this.

There are different risks (1). These can be enumerated. These include:

perception – complaints from either hospital or non hospital staff

performance – traditional quality, including errors that can affect patient safety

financial – errors that threaten the financial health of the service including lawsuits

regulatory – errors that threaten the accreditation status of the service

So first, one must say which risk one has in mind. One can envision an acceptable regulatory risk (we always pass inspections) but an unacceptable patient safety risk.  Note also, that the risks are not necessarily unique. One can have a patient safety failure with or without a lawsuit.

Assume the risk in question is the performance risk and specifically about patient safety. The Cadillac version of assessing risk would be to perform a quantitative fault tree and arrive at a numerical probability of patient risk. This is unlikely and one would probably have a qualitative assessment. Whether the assessment is quantitative or qualitative, this still hasn’t answered the acceptability question.

The problem is there is no easy answer to this question. If one had unlimited funds, one could lower the risk to whatever level was desired but funds are limited by the economic healthcare policy of the laboratory’s country (2). So one answer of acceptable risk is how this economic policy is translated into regulations. (e.g., one follows existing regulations and passes inspections). Yet, this is only a quasi legal way of stating acceptable risk.

Recommendation

I suggest that risk be assessed by traditional means (FMEA, fault tree) which includes a Pareto chart or table to rank the risks. Then, if one optimizes the money that one has in implementing control measures (mitigations) by a portfolio type means, then one has an acceptable risk under the imposed financial constraints.

References

1.       Managing risk in hospitals using integrated Fault Trees / FMECAs. Jan S. Krouwer, AACC Press, Washington DC, 2004.

2.       See http://covertrationingblog.com/

Westgard Quality Control Workshop – Part 1

 

I just returned from the Westgard quality Control Workshop, where I was a speaker and have a few blogs worth of comments – this is the first.

What’s Missing from Clinical Laboratory Inspections

At the Westgard Workshop, most of the participants were from clinical laboratories and I was impressed with how smart these people are. I also got a sense of a tremendous regulatory burden. From the CAP CD, I obtained at the Workshop:

      The mission statement of the CAP Laboratory Accreditation Program is:

“The CAP Laboratory Accreditation Program improves patient safety by advancing the quality of pathology and laboratory services through education and standard setting, and ensuring laboratories meet or exceed regulatory requirements.”

I have had mixed feelings about inspections that certify quality and have previously reported my experience with an industry quality program – ISO 9001 (1).

Here’s my assessment of clinical laboratory inspections to certify laboratories. It would seem that the premise of these inspections is to ensure that specific policies and procedures are in place and executed as proven largely by documentation, which guarantees high quality. So what’s missing? As far as I can tell – and it is with great difficulty to read through these materials – that there is no measurement of error rates. Without such measurements, quality is unknown.

Recommendation

The regulatory bodies would describe a list of errors and their associated severities. The severities would be given numerical values such as the VA hospital system which uses 1-4. Every clinical laboratory would record each error (failure mode) that occurs in their laboratory, its severity, and its frequency (default frequency is of course 1).  They would multiply frequency x severity for each unique error (failure mode), add this up and get a rate by dividing by the number of tests reported per year.

Failing to count errors would be a serious violation.

This would be the start of a new premise for the regulatory bodies. Measure quality – if it’s unacceptable, the clinical laboratory would suggest and implement process changes. It’s a simple closed loop process. With emphasis on measurement, reliance on documentation should decrease and inspections should be less burdensome.

References

1.       Krouwer JS. ISO 9001 has had no effect on quality in the in-vitro medical diagnostics industry. Accred. Qual. Assur. 2004;9:39-43