Reply to Letter Published

January 30, 2015

KMVYsb

[The photo is Martha’s Vineyard snow removal operations taken during a flight two days after two feet of snow fell in the Boston area.]

I had mentioned that a Letter to the editor to Clinical Chemistry had been accepted. It is now online with a reply to my Letter (subscription required).

I had previously mentioned in this blog how the editor of Clinical Chemistry is not fond of letters and replies so any thought of me replying to the reply would be a lost cause. Not that I would anyway. The authors who replied were kind in their comments and I have only one comment which I make at the end of this entry.

One cynical comment about these glucose meter models that relate precision and bias to total error is that you can make beautiful contour graphs because there are three variables. If you add interferences, no more simple contour graphs.

But what does it take to add interferences to the glucose meter (simulation) model. First one needs to list all candidate interfering substances and test them. Manufacturers have already done this but unfortunately, don’t try to use the information in the package insert. You can thank CLSI EP7 for this which allows a manufacturer to say compound XYZ does not interfere – if the manufacturer finds that the interference is less than 10% and the goal was 10%. So there could be a bunch of compounds that interfere but at levels less than 10%. This means that unless one can access the original manufacturing data, one would have to do over all of the interference studies. Then one needs the patient distribution of the concentration of each interfering substance. With this information one can randomly select a concentration of each interfering substance and apply the appropriate equation to generate a bias.

Thus, simulations, while still models and subject to the possibility of being incorrect, can require a significant amount of work.

My comment to the authors who replied to my Letter deals with their statement: “This is exactly the reason we advised in our work to adopt accuracy requirements more stringent than those resulting from simulations.” A similar statement was made by Boyd and Bruns back when I similarly critiqued their model. Now for sure, if the required bias is reduced and interferences are small, this will work because the total error will meet goals. The problem is, one has no knowledge of the bias contributed by interferences. And perhaps more importantly, this strategy will not work to prevent errors in the D zone of an error grid. I mention in my last post that with a bias of zero and a CV of 5%, one could get a D zone error if the observation is 80 standard deviations away. This will not happen anytime soon, but a gross interference is possible.


Published and Bad Model

January 2, 2015

debate

I complained about two glucose modeling papers and an accompanying editorial in the December issue of Clinical Chemistry. My Letter to the editor about one on the papers has been accepted in Clinical Chemistry.

Although not in the Letter, here’s another example of why modeling glucose meter error using average bias plus multiples of the standard deviation (e.g., sampling from a Gaussian distribution) can be misleading. Say truth is 50 mg/dL, which is hypoglycemic and the meter reads 200mg/dL, which is hyperglycemic. This would be a serious error because the provider (or patient if self-monitoring) would administer insulin, when in fact sugar is needed.

But in terms of modeling, say the bias is zero and the glucose CV is 5%. This means the sd at 50 is 2.5 mg/dL. Now to get a value of 200 due to imprecision requires 80 standard deviations! Using a spreadsheet, I can’t get this probability – however, for 30 standard deviations, the probability has 200 zeros to the right of the decimal point followed by a one. In other words not going to happen.

But such errors do occur – albeit rarely – but much more frequently than an 80 standard deviation error.