I attended the 2007 CLSI forum, which took place on April 20th in Baltimore. I got to speak about Evaluation Protocols. There were several highlights of the conference, one of which was to hear Neill Carey speak about Evaluation Protocols. His presentation is so clear that it is easy to see why his workshop at the AACC annual meeting is so popular. Another highlight was to hear Marina Kondratovich and Kristen Meier, both from the FDA, describe some of the statistical inadequacies that they encounter when reviewing FDA submissions.
I was also struck by the presentation of Judy Yost from CMS. She gave an update about equivalent (or alternative) quality control (EQC), which is a program which allows clinical laboratories under certain circumstances to reduce the frequency of external quality control to once a month. I presented an AACC session to show that this policy is not supported scientifically.
Ms. Yost ignored anything that I and others have said regarding EQC and went on to say that EQC has been a success story for clinical laboratories that have been using it, meaning that the inspection process for these laboratories has not uncovered any problems related to the reduced frequency of external quality control. This makes me think of an analogy. If someone took out the airbags in their car and stopped wearing seat belts and didn’t get into an accident, they might claim that one doesn’t need airbags or seatbelts because they have had no injuries without their use.
Changing the frequency of external quality control changes the risk of adverse events. Ms. Yost’s assertion that EQC is working in clinical laboratories that are using it because of successful inspections does not inform about the change in risk. Jim Westgard got up and questioned Ms. Yost about the lack of scientific basis of EQC. Okay, I have some differences with some of Jim Westgard’s writings, but not only am I in agreement with him on this issue, I applaud him for getting up and asking these questions. It’s the right thing to do and demonstrates leadership.
Two years ago, a new CLSI subcommittee was formed to provide a scientific basis for manufacturer’s recommendations for external quality control. Whereas the work of this subcommittee is still in progress, the scope of the subcommittee has been changed – it will not provide guidance for manufacturer’s recommendations for external quality control. This means that there will continue to be no scientific basis for EQC.
CLSI has many valuable Evaluation Protocol standards about analytical performance. They have an opportunity to develop and promote standards in risk management. These are sorely needed.