November 21, 2014
At a recent AACC dinner meeting, I heard an interesting talk by Nader Rifai, the editor of Clinical Chemistry. About halfway through his talk, I remembered an event that took place a couple of years ago, so I asked him a question after his talk ended. My question and Nader’s responses went something like this:
Me: “A while ago, I read a commentary article that I didn’t agree with and submitted a Letter to the editor about it. The response from the journal was…”
Rifai: “It wouldn’t be reviewed because it wasn’t about an original article, right?”
Me: “Yes, that’s right, then I looked at a few issues and saw that the percentage of original articles is only about 50% of the journal. This means that one can’t comment about a large portion of the journal.”
Rifai: “Well, we were seeing Letters to the editor about other Letters to the editor and with commentary articles it is common that many people won’t have the same opinion as the author, so we don’t want to fill up the journal with such stuff.”
This is sort of what I remembered, not verbatim but that is the gist of it.
So basically, Rifai is putting Letters to the editor into a generic category similar to junk mail or the endless comments associated with Twitter or a blog and at the same time giving immunity to authors – other than those who write original articles – from any kind of comment.
But the problem is that commentary articles in Clinical Chemistry are about science and if the authors get the science wrong, it is a mistake to prevent people from pointing that out. That is unscientific / elitist. Perhaps contributing to this elitism was that Rifai mentioned that articles in Clinical Chemistry are of high quality due to the extensive review process. But this doesn’t guarantee correctness.
And Clinical Chemistry has changed its policy. I commented briefly on this topic before in this blog. My 2010 Letter to the editor about a “Question and Answer” type article was published. Moreover, I think my 2010 Letter had a role in shaping glucose meter standards but these days the Letter would not have been considered.
So now I have less interest in reading Clinical Chemistry.
November 20, 2014
A recent article in Clin Chem – available without subscription – purports to show the result reliability of different A1c assays (1).
The basic premise of this paper is that given:
- total error goals
- a QC program
- a study to estimate imprecision (CLSI EP5) and average bias (CLSI EP9)
one can determine the risk of reporting unreliable results.
This is simply not true. I have shown before – see ref 2 for the most recent – that the Westgard model of total error = (a multiple of imprecision + average bias) is incomplete and typically underestimates the true amount of error.
Thus, the authors’ risk of reporting unreliable results is itself unreliable and probably underestimates things because:
- There is no information about interfering substances, not even a list of the standard error of estimates from the regessions which would provide some information about this error source.
- One can assume that one reagent was used. Yet lot-to-lot reagent error is usually the largest component of error in an assay. Hence, this error source is inadequately measured
- One does not know if the people that ran the study are representative of people who routinely run the assay – important since user error is often a significant source of error.
And finally, the use of one set of total error goals is questionable. If some results fail the total error goal, one wants to know if they just fail or if they are way out because just as error can be small or large, so can the resulting patient harm. Studies of the type in the paper can’t really help here because they use one Normal distribution. But in the real world, errors tend to come from different sources (distributions) so the risk of large errors is completely unknown.
What should one do to get a better prediction of risk?
- Conduct risk analysis by performing a fault tree and FMEA (Failure Mode Effects Analysis) that includes
- The correct model (see reference 2)
- Account for the error sources missed in the paper (part of the fault tree / FMEA)
- Woodworth A Korpi-Steiner N, Miller JJ, et.al. Utilization of Assay Performance Characteristics to Estimate Hemoglobin A1c Result Reliability Clin Chem 2014 60 1073–1079 (2014).
- Krouwer JS The danger of using total error models to compare glucose meter performance. Journal of Diabetes Science and Technology, 2014;8:419-421.
October 24, 2014
Recommended reading – CAP interview of Jim Westgard regarding lab QC over the last 30 years including the current focus on risk management: http://www.captodayonline.com/lab-qc-much-room-improvement/
October 15, 2014
This is the full paper http://dst.sagepub.com/content/early/2014/10/10/1932296814554415.abstract – a Letter to the Editor was already mentioned.
The paper suggests that a different glucose error grid is needed for diabetes complications such as diabetic retinopathy rather than the traditional glucose error grid which deals with acute injuries. This is because slightly or even moderately elevated glucose would fall in the no treatment needed zone of a traditional glucose error grid but would be harmful for diabetic complications. Thus, a glucose meter could look good in terms of a traditional glucose error grid but have a bias that would allow elevated glucose to occur for up to 6 months – until the patient’s next A1c determination. Patients and providers would be better informed if they knew which glucose meters were free from these long-term biases.
September 10, 2014
This entry adds to but does not contradict the Westgard Web site’s: http://www.westgard.com/badqc-goodsoftware.htm
Clinical limits provide a range of results that do not cause patient harm.
A problem with traditional clinical limits is that going from just under the limit to just over the limit changes the outcome from no harm to harm. This is difficult to understand for two results that are almost identical. That is why error grids were developed. They separate harm into categories so when one goes from just under the limit to just over the limit the outcome changes from no harm to minor harm.
QC limits are set to control the process
Thus, QC and clinical limits are different.
For an assay with a high process capability (similar to a high six sigma) clinical limits will always be much wider than QC limits. And for an assay that is out of control (values that exceed QC limits), results may still be within clinical limits. But it is nevertheless important to detect an out of control process because the process may become so out of control so that results fail clinical limits.
For a process that is in control (all values that within QC limits), can results be outside of clinical limits (potentially cause patient harm?) YES!
Patient spills coke on himself. Provider comes in to use glucose meter and fails to wash and dry the site from which the capillary sample is taken. The result is 200 mg/dL too high. There is nothing wrong with the glucose meter or QC.
Patient has an interference. Result is way off. QC is ok.
There is a one hour shift in results – they are way off. QC – once every eight hours doesn’t detect the shift.
Thus, QC detects long-term bias which is important but there can still be other errors that can harm patients.
August 29, 2014
Looking at the table of contents of Clinical Chemistry for September, there is a list of the most downloaded point / counterpoint articles and I am number one on this list for my discussion of GUM (The guide to the expression of uncertainty in measurement): http://www.clinchem.org/content/60/9/1245.full
August 24, 2014
For those who have had some papers rejected over the years (like me), this post is worth reading … http://majesticforest.wordpress.com/2014/08/15/papers-that-triumphed-over-their-rejections/