There will be an article and editorial (subscription required) about 6 of 8 assays that fail the NGSP hemoglobin A1c standard, which is here. As an aside, the NGSP could use a little revision control so that one can understand what is new.
There are problems with this standard. Here’s why. The standard states:
“In order for a commercial method to be considered traceable to the CPRL, the 95% CI of the differences between methods (test method and SRL method) must fall within the clinically significant limits of ±0.85% GHB.”
The problem is this is a measure of the average difference. While it is true that the 95% CI (confidence interval) will fail if there is too much scatter in the differences, reading further suggests a another problem.
“All data analysis will be performed by the NETCORE following Bland and Altman Assessment of Agreement. Outliers will be analyzed for informational purposes only; an outlier is defined as > mean + 3SD of the absolute differences between pairs. All outliers will be investigated by the NETCORE to determine if the discrepancy could be due to characteristics of the specimen rather than the assay method. If results show that a discrepancy could be due to characteristics of the specimen, then the manufacturer will be asked to submit a new specimen and the data will be reanalyzed.”
This doesn’t make too much sense to me. An evaluation should try to estimate performance that will be observed under routine conditions.
1) Routine conditions don’t include a reference assay with which one can calculate differences.
2) Eliminating data will provide a biased and too favorable performance estimate
3) Why should one throw out a result “due to characteristics of the specimen rather than the assay method.” The assay method performance is a summation of many things including how characteristics of the specimen are handled by the assay.
The Bland-Altman approach requires normal (distribution) data. If the data is not normal, it must be transformed.
A simpler specification would be use an error grid, which accounts for 100% of the data.