Clinical Chemistry Training Council

January 30, 2016

KVSFOK, I admit it I’m a little slow and behind the times. By looking at the email for the Feb. table of contents for Clinical Chemistry, I discovered online content that I never noticed – a section about “Pearls of Laboratory Medicine.”

After looking at a few of the ones supplied, I realized there was a lot more available by joining the Clinical Chemistry Training Council site.

Wow, I was impressed by the amount of available material. I may not agree with everything there – as always – but this looks like a valuable resource. And I like the fact that material is presented by slides and webcasts as well as more traditional means.

 

 


How loss of market share affects standards

January 19, 2016

DSC00426edpTWhen the 2003 ISO standard for glucose meter performance was prepared, the regulatory affairs people of industry controlled the standard. The standard called for 95% of results above 75 mg/dL to be within a total error of ± 20%. The standard was said to be based on medical requirements – clearly it was not based on state of the art since glucose meters perform better.

A problem probably unforeseen by these regulatory people was that a bunch of new players entered the glucose meter market and of course had no trouble getting FDA approval – the FDA used the ISO standard in its approval process. The number of meter brands on the market grew from 32 in 2005 to 87 in 2014. And some of the new meters sold their strips at a much lower price than the major manufacturers. This caused the four major companies to lose some market share.

Industry still plays a dominant role in glucose meter standards, but it seems that the original regulatory affairs people are out. Now, industry is working with the Diabetes Technology Society to certify glucose meters under new performance standards. Thus, meters that have FDA approval will be tested according to the tighter 2013 ISO standard and only meters that pass will receive a seal of approval from the Diabetes Technology Society.

Klonoff DC, Lias C, Beck S Development of the Diabetes Technology Society Blood Glucose Monitor System Surveillance Protocol. J Diabetes Sci Technol, in press. available at http://dst.sagepub.com/content/early/2015/12/10/1932296815614587.full.pdf+html


Improving the Glucose Meter Error Grid with the Taguchi Loss Function

January 2, 2016

 

DSC00228edpT

A couple of entries ago, I mentioned an upcoming publication. It has now appeared in the J Diab Sci Techol ahead of print (subscription required).

Basically, each glucose meter result in the A zone of an error grid has its difference from reference squared and that value scaled ranging from 0 (result=reference) to 1 (result is on the A zone boundary). If the value is outside of the A zone no A zone Taguchi loss value is calculated. These “Taguchi loss” values are then averaged to give the average Taguchi loss (ATL). One can calculate the ATL for all zones, although I did not do this for the article.

The ATL represents a way to distinguish performance among different glucose meters with similar performance (most values in the A zone and no values beyond the B zone). I believe it is an improvement over the MARD statistic (mean absolute relative deviation).


Normalization of deviance

December 29, 2015

DSC00303edpRecently, I became aware of an analysis of the Gulfstream IV crash last year in Bedford, MA (KBED), an airport where I train.

To recall, the crew attempted to takeoff with the gust lock* engaged – the plane was never airborne and crashed after overrunning the runway. All aboard died.

*gust lock is a device which prevents the control surfaces from moving to protect them on the ground from wind gusts. It is supposed to be disengaged before flight.

Besides not disengaging the gust lock, the pilots failed to perform a flight control check (that verifies that controls move in all directions that they should). This check is standard for any plane and if performed would have alerted the crew to the problem.

But the astounding revelation in the NTSB report is that the flight crew almost never performed checklists: “A review of data from the airplane’s quick access recorder revealed that the pilots had neglected to perform complete flight control checks before 98% of their previous 175 takeoffs in the airplane, indicating that this oversight was habitual and not an anomaly.”

This has been referred to as normalization of deviance and is explained here. That is, deviant behavior is so commonplace that it is no longer considered deviant. And yes, it happens in healthcare too.

For those who want more details, the NTSB report is here.

 

 


A way to improve glucose meter error grids using Taguchi principles

December 5, 2015

KEWB2So I was reading an article about glucose meter performance and I came across the MARD (mean absolute relative difference) statistic. I have seen this before – it is used for glucose meters and almost nowhere else. What bothered me was that the paper used MARD as a summary statement about the performance for different meters – the problem is MARD has so many problems I wrote a paper critiquing MARD and submitted it. As soon as I clicked the submit button, I realized I had left out an important element; namely why were people using MARD?

In any case, I got mixed reviews about my paper and the editor said I could try to submit a revision. But the more I thought about it, I realized that my paper was not that good. I was going to drop it when it occurred to me, rather than complain about MARD, I might be able to come up with a better statistic. After all, people use MARD because they want to differentiate meters that appear to have similar performance when analyzed with error grids.

So I wrote a new paper which provides an alternative to MARD. It has been accepted and will appear shortly.


Decision analysis? – where are the details?

November 15, 2015

DSC00285edp

In the Milan conference (1st EFLM Strategic Conference Defining analytical performance goals) one of the papers (1) suggests that analytical performance specifications should be prepared from indirect outcome studies using decision analysis. The only example presented is a simulation, which is not decision analysis. Decision analysis is also discussed in this section but on an abstract level.

I have performed decision analysis and discuss it in my book (2). Decision analysis requires a quantitative variable that is either maximized or minimized. In my case, we performed financial decision analysis and the parameter to be maximized was net present value (NPV) of future cash flows. The Milan paper never identifies a quantitative parameter to be optimized.

I don’t understand how decision analysis can be recommended without any known examples or details about how one would go about it.

References

  1. Horvath AR, Bossuyt PMM, Sandberg S, et.al. Setting analytical performance specifications based on outcome studies – is it possible? Clin Chem Lab Med 2015; 53(6): 841–848.
  2. Assay Development and Evaluation: A Manufacturer’s Perspective. Jan S. Krouwer, AACC Press, Washington DC, 2002, see Chapter 3.

How to insult clinicians

November 5, 2015

DSC00264edp

In my blog post Total Error and Milan, I mentioned how clinician surveys were in the draft consensus statement but dropped from the final and published consensus statement. (The draft consensus statement is no longer available on the EFLM site).

I had occasion to read a paper from the Milan conference (1) where it is clear why clinician surveys were dropped.

“RCVs from vignettes should probably not be used on their own as a basis for setting analytical performance specifications, since clinicians seem “uninformed” regarding important principles.”

RCV = reference change values.

For an example of how clinician surveys were used to set analytical performance specifications, see reference 2.

References

  1. Thue G and Sandberg S: Analytical performance specifications based on how clinicians use laboratory tests Clin Chem Lab Med 2015; 53(6): 857–862.
  2. Klonoff DC, Lias C, Vigersky R, et al The surveillance error grid. J Diabetes Sci Technol. 2014;8:658-672.

Follow

Get every new post delivered to your Inbox.