March 25, 2018
What I mean by a clinical chemist is anyone associated with clinical chemistry which includes people who work in hospitals and anybody who works for a manufacturer.
A recent example is about blood lead, a product for which I consulted. As reported recently, the electrochemical method was at times giving the wrong answers. It was finally determined that a compound in the rubber stoppers of blood collection tubes was dissolving in blood and absorbing lead. Thus, nothing can be assumed – anything including the blood collections tubes can cause problems.
March 23, 2018
Consider the following recent weather event in late March 2018 in the Boston area:
|Day forecast made
Clearly, the forecasts were not very good. With huge amounts of data, there is only one thing that can be blamed – the model(s). This might seem surprising with all of the scientists involved and the availability of superfast computers. Of course, with weather one can always compare the forecast with actual. (In the three previous recent nor’easters in Boston, the forecasts were quite close to actual).
The difference in models used to evaluate assays in clinical chemistry, from models used to forecast weather is that in clinical chemistry, one takes a sample of actual data. The issues are:
- whether the sample will be valid for future data
- whether the model is correct
I have complained against the use of models in clinical chemistry since the models are incomplete (wrong) and one can simply directly estimate differences between two methods without using models. This is akin to “forecasting” weather by looking out the window although one still has to worry about whether the sample will be valid for future data.
March 22, 2018
For most of my career, I’ve been either an internal or external consultant. Consultants are always trying to get work by convincing someone that the consultant can solve a problem. Whereas this might seem to be a win / win situation, to some clients, using a consultant can be frightening as the client fears loss of control. Here’s a real example.
At Ciba Corning in the 90s, our instrument reliability was not very good. Our group thought that we could help using data analysis methods. A key success factor was someone we hired – without him the project would not have been successful.
But an additional problem was that engineering management didn’t want us to help. From their perspective, it’s easy to see why. We proposed using some of their staff to collect data, the engineers would have to attend meetings that we ran, we would report to management on instrument reliability, direct which problems to work on, and advise management as to when the instrument reliability goal would be met… All of this amounts to a partial loss of control for the engineering manager.
Of course, loosing control was never discussed. Their objections came in the form of resistance. “That’s a great idea, let’s try it on the next project.” Or, “yes let’s do it.” But the first meeting could never be scheduled (e.g., when yes means no). And so on.
The project was allowed to proceed because the engineering manager and I had the same boss, who overruled the engineering manager’s attempts to decline participating.
The project was a big success and the engineering manager’s response was to take back control. Thus, with things now in place he could now run the new programs. The only thing that irked me was not only did he never credit us for the work but it was suggested that the success occurred in spite of our group. But this is simply an example of the successful consulting cycle.
March 18, 2018
Reading the series of articles and editorial in March 2018 Clinical Chemistry about commutability reminds me of my job that started almost 40 years ago at Technicon Instruments. My group, under the leadership of Dr. Stan Bauer, was responsible for putting the right values on calibrators for all of our assays. Back then, when customers complained that they weren’t getting the right result, the calibrator value was often blamed. I seem to recall that the customer even had the ability to choose a different value for the calibrator (we called the calibrator values “set points”).
In any case, what we did was as follows. We occupied space at the hospital of New York Medical College in nearby Valhalla (Technicon was in Tarrytown). We acquired patient samples that were no longer needed by the hospital and ran them both on our instruments and reference methods. Then, through data analysis, we assigned a calibrator value to the master lot of calibrator that would make the patient samples in the Technicon method equal what was obtained for the reference method. For some assays such as bilirubin if I remember correctly, the calibrator contained a dye and thus no analyte at all! Suffice it to say that whereas commutability of our calibrators didn’t exist, the patient samples nevertheless came out right (same as reference method).
It was this data analysis work that turned me into a statistician. I enjoyed the work and was finding out properties of our Technicon assays that the biostatisticians had missed and some of these properties were critical in calibrator value assignment.
On another note, I was at a small company a few years ago on a sales call. As I was describing my background including Technicon, I asked the small group – anyone hear of Technicon? No one raised their hand.
March 16, 2018
Ever notice how in Clinical Chemistry (and other journals), an editorial accompanies an article (or series of articles) in the same issue. The editorial is saying – hey! listen up people, these articles are really important. And then the editorial goes on to explain what the article is about and why it’s important. It’s the book explaining the book.
March 15, 2018
I have previous posted about Theranos (here and here) and now Theranos is in the news again in a bad way (thanks to the AACC artery for spotting this).
Thus, Elizabeth Holmes was charged with massive fraud by the SEC. Makes me wonder if the AACC past presidents are happy that they accepted positions on the Theranos board. And also if AACC regrets it decision to feature Elizabeth Holmes at the 2016 AACC national meeting.
March 13, 2018
Commutability is a hot topic these days and it should be. One would like to think that someone tested on one system will get the same result if they are tested on another system.
In reading the second paper (1) in a three series set of articles, I note that a term for interferences is present (in addition to average bias and imprecision) to estimate error. Almost forty years ago, this was suggested (see reference 2).
Although reference 2 was not cited in the Clinical Chemistry paper, at least a model accounting for interferences is being used.
- Clinical Chemistry 64:3 455–464 (2018)
- Lawton WH, Sylvester EA, Young-Ferraro BJ. Statistical comparison of multiple analytic procedures: application to clinical chemistry. Technometrics. 1979;21:397-409.
March 11, 2018
Readers of this blog know that I’m in favor of specifications that account for 100% of the results. The danger of specifications that are for 95% or 99% of the results is that errors can occur that cause serious patient harm for assays that meet specifications! Large and harmful errors are rare and certainly less than 1%. But hospitals might not want specifications that account for 100% of results (and remember that hospital clinical chemists populate standards committees). A potential reason is that if a large error occurs, the 95% or 99% specification can be an advantage for a hospital if there is a lawsuit.
I’m thinking of an example where I was an expert witness. Of course, I can’t go into the details but this was a case where there was a large error, the patient was harmed, and the hospital lab was clearly at fault. (In this case it was a user error). The hospital lab’s defense was that they followed all procedures and met all standards, e.g., sorry but stuff happens.
As for irrelevant statistics, I’ve heard two well-known people in the area of diabetes (Dr. David B Sachs and Dr. Andreas Pfützner) say in public meetings that one should not specify glucose meter performance for 100% of the results because one can never prove that the number of large errors is zero.
That one can never prove that the number of large errors is zero is true but this does not mean one should abandon a specification for 100% of the results.
Here, I’m reminded of blood gas. For blood gas, obtaining a result is critical. Hospital labs realize that blood gas instruments can break down and fail to produce a result. Since this is unacceptable, one can calculate the failure rate and reduce the risk of no result with redundancy (meaning using multiple instruments). No matter how many instruments are used, the possibility that all instruments will fail at the same time is not zero!
A final problem with not specifying 100% of the results is that it may cause labs to not put that much thought into procedures to minimize the risk of large errors.
And in industry (at least at Ciba-Corning) we always had specifications for 100% of the results, as did the original version of the CLSI total error document, EP21-A (this was dropped in the A2 version).