New FDA Glucose meter draft guidelines (November 2018)

January 31, 2019

The FDA continues to dis the ISO 15197 standard in both their POC and lay user (over the counter) proposed guidelines:

POC“Although many manufacturers design their BGMS validation studies based on the International Standards Organizations document 15197: In vitro diagnostic test systems—Requirements for blood glucose monitoring systems for self-testing in managing diabetes mellitus, FDA believes that the criteria set forth in the ISO 15197 standard do not adequately protect patients using BGMSs in professional settings, and does not recommend using the criteria in ISO 15197 for BGMSs.”

The POC accuracy criteria are:

95% within +/- 12 <75 mg/dL and within +/- 12% >75 mg/dL
98% within +/- 15 <75 mg/dL and within +/- 15% >75 mg/dL

Over the counter“FDA believes that the criteria set forth in the ISO 15197 standard are not sufficient to adequately protect lay-users using SMBGs; therefore, FDA recommends performing studies to support 510(k) clearance of a SMBG according to the recommendations below.”

The over the counter accuracy criteria are:

95% within +/- 15% over the entire claimed range
99% within +/- 20% over the entire claimed range

To recall, ISO 15197 2013 accuracy criteria are:

95% within ± 15 mg/dl <100 mg/dL

95% within ± 15% >100 mg/dL
99% within A and B zones of a glucose meter error grid

Advertisements

Establishing QC mean on multiple instruments

January 24, 2019

I sent my first post to the AACC artery on the title’s topic. My post concerns some of the comments I saw, such as: “QC limits were established based on total allowable error for each analyte.”

Two important questions for a lab are:

  1. Is the process in control?
  2. Are the patient results medically acceptable?

QC can answer the first question, it cannot answer the second question.

Results can be viewed in a 2×2 table

  Patient results are
medically acceptable
Patient results are not
medically acceptable
The process is in control

1

2

The process is not in control

3

4

An example of case 2 is the first generation of troponin assays in the 90s. The American and European cardiology societies specified performance for troponin assays and no one met the performance. Hence these were assays in control but failed to provide medically acceptable results.

Changing QC limits does nothing.


Review of why FMEA is so difficult

January 20, 2019

FMEA stands for Failure Mode Effects Analysis
FRACAS stands for Failure Reporting and Corrective Action System

Definitions – FMEA is a process to reduce the risk of undesirable events. FRACAS is a process to reduce the rate of undesirable events. Hence FMEA failures haven’t happened while FRACAS failures have. FMEA failures are often much more severe than FRACAS failures.

Not a FMEA – When someone says they have reduced the rate of failures for a process, they have performed FRACAS not FMEA.

Lack of management support – Companies make money by selling products and services and also by reducing costs. FMEA does neither. It consumes resources while reducing the risk of costs (caused when failures occur). A nonprofit has to worry about the same issues. If they lose money, they will have to reduce services.

The following is not a compelling appeal to management for resources – “There are some catastrophic failures that have never happened. We can spend some money to make these events even less likely”.

I participated in a company FMEA that was always held during lunch. The reason given by management was that they have more important tasks to do during regular working hours.

Lack of a facilitator – The purpose of a FMEA is to question the design of a product or process. Often, the designer is present. A facilitator can prevent an adversarial confrontation.

Lack of interest – People who design medical instruments like to design. It’s a challenge to motivate them to perform tasks other than design.

Insufficient detail – FMEA is a bottoms up approach and requires listing all process steps. (Fault trees are a top down approach). Mapping out a process requires inclusion of all relevant events. Providing insufficient detail is a problem. Example, a technician examines a clinical sample before it is analyzed. An additional branch could be how is this person hired, trained, etc.

Stopping at the status quo – As one conducts a FMEA, one lists the possible failure modes for each step, the effect of each failure, and the mitigation for each failure. Example: a process step is to test whether a sample is hemolyzed, the effect (of a failure) is potassium is artificially elevated, and the existing mitigation is to rely on the instrument’s automatic flagging of hemolyzed samples. One might conclude that’s it – no need to do anything further. But all that’s been done is to describe the existing process. This is not FMEA. One must ask questions such as how can the instrument’s flagging system fail.

Acceptable risk is hard to quantify – One can never have a zero failure risk. For example, for blood gas, which is an emergent assay, one remedy to mitigate against an instrument failure is to use two instruments. One can estimate the probability of both instruments failing at the same time. One can add a third machine and so on, always lowering the risk, but it is never zero. Mitigations cost money so one must make a tradeoff between cost and risk.