The value of error grids

March 29, 2019

My colleague and I sang the praises of error grids as a way to specify performance – for any assay. To recall, here are some of the benefits:

  1. Unlike most specifications, the limits can change with concentration
  2. Unlike most specifications, the limits need not be symmetrical
  3. Most specifications have one set of limits, implying that results within limits cause no harm and results outside of limits cause harm. Error grid have multiple sets of limits – called zones – whereby harm can be none, minor, or major.
  4. Error grid zones account of 100% of the results – they cover the XY space of candidate assay vs reference assay. Most specifications cover 95% or 99% of results, leaving the balance unspecified.

Krouwer JS and Cembrowski GS Towards more complete specifications for acceptable analytical performance – a plea for error grid analysis. Clinical Chemistry and Laboratory Medicine, 2011;49:1127-1130.

Summary of what’s wrong with the ISO 15197 2013 glucose meter standard

March 24, 2019

  1. Minimum system accuracy performance criteria (6.3.3) – I previously commented that the word “minimum” is silly. One either meets or does not meet the requirements. But the big problem is Notes 1 and 2 in this section that says that the test is not to be carried out by actual users. Thus, the protocol is biased by excluding user error. In the section where users are included, the acceptance criteria (8.2) drop the requirement for 99% of the results to be within the A and B zones of an error grid. The requirement for 95% of the results to be within ± 15 mg/dL below 100 and within ± 15% above 100 remain. Thus 5% of the results are unspecified, same as the 2003 version. This means that for people who test 3 times daily, they could have a dangerous error for their meter once a week in spite of their meter meeting the ISO 15197 standard.
  2. Safety and Reliability Testing (Sections 5) – A hallmark of reliability testing is the frequency of failures to obtain a result. There is nothing in this section (or elsewhere in the standard) to tally the frequency of failed results or specified limits for percent failures. This makes no sense for a standard about a POC test that is needed emergently. Failure to obtain a result is a frequent event in the FDA adverse event database for glucose meters.
  3. If you want to see who wrote the standard, you can’t. As with all ISO standards, there is no list of authors or members who served on the committee.

Who influences CMS and CDC?

March 23, 2019

A recent editorial disagrees with the proposed CLIA limits for HbA1c provided by CMS and CDC (The Need for Accuracy in Hemoglobin A1c Proficiency Testing: Why the Proposed CLIA Rule of 2019 Is a Step Backward) online in J Diabetes Science and Technology. The proposed CLIA limits are ± 10% – the NGSP limits are 5%, and the CAP limits 6%. Reading the Federal Register, I don’t understand the basis of the 10%.

This reminds me of another CMS decree in the early 2000s – Equivalent Quality Control. Under this program, a lab director could run quality control for 10 days as well as the automated internal quality checks and decide whether the two were equivalent. If the answer was yes, the frequency of quality control could be reduced to once a month. This made no sense!

More Theranos

March 1, 2019

Another article about Theranos deplores the lack of laboratory medicine input. It does review articles which doubted Theranos. But nowhere, including the above article, do I see anyone questioning that four previous AACC presidents joined the Theranos board. This is laboratory medicine input and IMHO in a bad way.