Let’s view the glucose meter situation as a FRACAS (Failure Reporting And Corrective Action System). A FRACAS is a process to reduce an unacceptable error rate. As an example of a successful FRACAS, Dr. Peter Pronovost reduced the infection rate for placing central lines (described here). The steps he used are:
- Measure the error rate
- Observe events (e.g. errors)
- Classify events
- Propose and initiate corrective actions
- Re-measure the error rate
In this case, the initial error rate was 11% and the re-measured rate was 0.
So how does this relate to glucose meters?
Step 1 – FDA has an estimate of an error rate of about 0.1% for the 7.2 million insulin using diabetics.
Step 2 – which for the FDA is the same as step 1, is the MAUDE adverse event database. However, if one looks at a MAUDE adverse event report, one sees problems. The advice on how to report an event suggests a user should email the problem with as much information as possible as listed on this web page. How many users actually find this web page? What should be used is a web form using dropdowns lists where appropriate, which guides the user in filling in information. The web form address should be publicized (listed on every glucose meter?) This would reduce wrong spellings and duplicate categories. The currently suggested items to fill in are meter centric – they do not follow the process of obtaining a glucose result. In the central line FRACAS, each process step was observed and it was determined whether it was followed or not. In glucose testing, for example, it is important to wash and dry one’s hands. This should be listed on the web form:
Wash hands? Y or N
Dry hands? Y or N
Steps 3-5 – Now as one collects statistics about the frequency of hand washing, one cannot say that lack of hand washing causes adverse events; however, if it is important to wash hands, then the procedure should be followed and one could postulate that a corrective action – to increase the frequency of hand washing – might lower the adverse event rate. So a program would need to be put in place to increase the frequency of following the process and then seeing what effect the action has on the adverse event rate. In the central line infection rate case, it was observed that doctors omitted one of the five steps in placing a central line 30% of the time. When the compliance rate was raised to 100%, the infection rate dropped from 11% to 0%.
On another note, a lot of effort is spent on evaluating glucose meters to qualify them before they are sold. To evaluate glucose meters, a method comparison is performed. There are various ways to analyze the data from this experiment but even if an error grid is used, the results are most likely not relevant to the error rate.
This is because – as specified in ISO 15197–and as supported by FDA comments, the method comparison study excludes the possibility of many use errors. So the analytical and usability performance of the meter are assessed separately without an attempt to combine them to arrive at the only meaningful performance for clinicians and patients – how the meter performs in routine use. This needs to be changed such that the potential for use error is included in glucose evaluations. Even with the inclusion of use error, large errors will probably not show up due to the small sample size (e.g., you need 10 evaluations of a sample size of 100 for one large error, assuming that an adverse event rate of 0.1% is triggered by a large error). But including use error will give a much better estimate of the width of data around the “A” zone in an error grid, which is the subject of much interest.