I’ve been critiquing assay specifications for some time, including:
| Assay | Standard Organization | Reference |
| cholesterol | NCEP | 1 |
| glucose | ISO 15197 | 2 |
| creatinine | NKDEP | 3 |
| hemoglobin A1c | NGSP | 4 |
For more colorful critiques, go to http://krouwerconsulting.com/ and enter a search term or click on publications. Some of my critiques go back almost 20 years.
These standards have one or more of the following problems:
- Limits are given for only 95% of the data, so 5% of the data are unspecified
- The wrong model is used (often total error = bias ± 1.96 X imprecision)
- Outliers are discarded
- User error is excluded
The ideal specification should have:
- Limits for 100% of the data, as exemplified by an error grid
- A protocol for collecting method comparison data. The protocol should not exclude user error
- An analysis method, whereby no data is thrown out. The analysis could be as simple as tallying the percentage of data in each error grid zone
- FMEA and fault tree analysis to evaluate the risk of rare errors
References
- Krouwer JS. Problems with the NCEP (National Cholesterol Education Program) Recommendations for Cholesterol Analytical Performance. Arch Pathol Lab Med 2003;127: 1249 (2003).
- Krouwer JS and Cembrowski GS. A review of standards and statistics used to describe blood glucose monitor performance. Journal of Diabetes Science and Technology 2010;4:75-83.
- Jan S. Krouwer: A recommended improvement for specifying and estimating serum creatinine performance. Clin Chem 2007;53:1715-1716.
- See: https://jkrouwer.wordpress.com/2009/12/03/wrong-thinking-about-hemoglobin-a1c-standards/
Appendix – Disagreeing with so many experts
Each of the standard organizations comprises a group of experts and four groups equals a lot of experts! I know people in these groups and respect their expertise. These experts are much more knowledgeable than I am in the clinical chemistry of each analyte. However, another domain of interest is how to specify and measure the quality of these assays. I suspect that these groups are underrepresented in this area.
