Simulations need to account for more than 95% of results

In 2001, Boyd and Bruns published a paper about the effects of glucose meter error on insulin dose errors (1). I commented that their simulation model was incomplete (2) and they agreed (3). Now, one of the authors has with others published a new paper (4), which is similar to the original simulation so once again I have sent a Letter to the editor in response.

Since the first paper, I and my colleague published a review on glucose meter performance statistics (5) and I published a Letter (6) critiquing experts’ views (7-8) on glucose meter performance standards. References 5 and 6 point out additional problems beyond those mentioned in reference 1.

The authors of reference 4 have chosen to ignore all of this previous work, and although there are a bunch of problems with their simulation, here’s the biggest problem – the authors suggest constructing limits based on 95% of the results that are within limits. They then go on to say that such and such insulin dose errors will occur for various limits. But by their definition, 5% of results are beyond limits, so that if one accounts for 100% of the population, one always has to add 5% to the bad results. Thus, if they came up with limits whereby according to them no serious insulin dose errors were made, in reality, 5% serious dose errors would be made.

It is equivalent to specifying that 95% of surgeries should be surgeries that involve the correct site. But no one would accept a 5% wrong site surgery error rate.

In glucose meters, there will be insulin dose errors (including serious ones) because with 7.2 million diabetics in the US who inject insulin every day, even for one test per day, this gives 2.6 billion glucose meter assays per year, so one cannot expect all meter errors to be below a limit such that no serious insulin doses occur. But 5%, which turns out to be 131 million serious insulin dose errors as a result of glucose meter errors are too many for a glucose meter performance specification.

References

  1. Boyd JC and Bruns DE Quality Specifications for Glucose Meters: Assessment by Simulation Modeling of Errors in Insulin Dose Clin Chem 2001;47:209-214.
  2. Krouwer JS. How to Improve Total Error Modeling by Accounting for Error Sources Beyond Imprecision and Bias Clin Chem  2001;47:1329-30.
  3. Boyd JC and Bruns DE Response to How to Improve Total Error Modeling by Accounting for Error Sources Beyond Imprecision and Bias Clin Chem  2001;47:1330-31.
  4. Karon BS, Boyd JC, and Klee GG. Glucose Meter Performance Criteria for Tight Glycemic Control Estimated by Simulation Modeling http://www.clinchem.org/cgi/doi/10.1373/clinchem.2010.145367
  5. Krouwer JS and Cembrowski GS. A review of standards and statistics used to describe blood glucose monitor performance. Journal of Diabetes Science and Technology, 2010;4:75-83.
  6. Krouwer JS. Wrong Thinking about Glucose Standards Clin Chem 2010;56: 874 – 875.
  7. Sacks DB. Tight glucose control in critically ill patients: Should glucose meters be used?. Clin Chem 2009;55:1580-1583.
  8. AACC. September 2009 clinical laboratory news: higher standards on the way for glucose meters? http://www.aacc.org/publications/cln/2009/september/Pages/inside0909.aspx (Accessed November 2009).
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: