Our paper on error grids has been published online. As is often the case, there is another opinion paper that comments on our paper. One of the comments disagrees with us that error grid limits should be based on clinician opinion. The commentators favor limits “defined retrospectively, after having accumulated, analyzed and troubleshot a large number of clinical adverse events strictly related to laboratory errors which have arisen throughout the total testing process.” One rationale for this comment is that it is an evidence based approach.
Now I’m all for this approach when it is practical. For example, to assess the best treatment for prostate cancer, rather than performing a series of randomized clinical trials, one could follow up for several years with a questionnaire, the 200,000 patients diagnosed each year (USA) with prostate cancer. If this had been done 20 years ago, we would have 2 million records for patients diagnosed at least 10 years ago.
But I would challenge the commentators to provide evidence that there would be sufficient data for their evidence based approach. How many times does one know that laboratory error was responsible for an adverse event and just as important, the magnitude of the laboratory error? Moreover, the only errors that would help one decide on limits are errors that are close to the proposed limits. For example, we do not need an actual case of a glucose meter reading 300 mg/dL when truth is 30 mg/dL to know that this error is harmful. We would need to see actual adverse events for smaller errors with the magnitude of the error known precisely. Sorry, but I don’t see this happening anytime soon.
But there’s another important reason for not favoring the commentators’ suggestion. Going back to the prostate cancer example, each year, unless there’s a cure for prostate cancer there will be 200,000 records that can be used to assess side effects by treatment, recurrence by treatment, and many other things of interest to patients. The data is there, all we have to do is collect it. But waiting for adverse events due to laboratory error in order to set limits is akin to waiting for planes to crash in order to improve safety. Of course when planes do crash, the information is used but design and regulation are preventive measures. Airplane safety is a good comparison to laboratory error because about 85% of airplane accidents are due to pilot error and the rest are due to aircraft problems.