Westgard bemoans the current process of establishing performance claims for assays. He states that
“There is one major fault with this approach [precision, accuracy, linear range, reference range(s), etc.]. Manufacturers do not make any claim that a method or test system provides the quality needed for medical application of the test results, i.e., FDA clearance does not require a claim for quality! To do so, a manufacturer would need to state a quality specification, e.g., the allowable total error, the maximum allowable SD, or the maximum allowable bias, then demonstrate that the new method or test system has less error than specified by those allowable limits of error.”
You’re either part of the problem or part of the solution. In this case, Westgard is part of the problem. His suggestion of allowable total error as stated above sounds good, but as I have pointed out many times,
- Westgard’s maximum allowable total error is for a specified percentage of results – often 95% – which allows for too many results to fail to meet clinical needs
- Westgard’s suggested testing procedures as described by his quality control rules fail to include all contributions to total error
Thus, 5% of a million results means that there could be 50,000 medically unacceptable results – that’s not quality. When one tests with control samples, one cannot detect interferences, which is often a source of important clinical laboratory errors so all of Westgard’s control quality algorithms for total error are meaningless – they inform about a subset of total error.
Things are improving. In the FDA draft guidance for CLIA waiver applications, FDA requires use of error grids (such as those in use for glucose) and demonstration of lack of erroneous results as defined in those error grids in addition to total allowable error. Many of my essays stress the need to go beyond total allowable error – as used by Westgard – and to put in place procedures to estimate erroneous results (1).
- Jan S. Krouwer: Recommendation to treat continuous variable errors like attribute errors. Clin Chem Lab Med 2006;44(7):797–798.