EP21 is a CLSI standard about estimating total error for diagnostic assays. It was first published in 2002 and has been a core part of the FDA’s guidance for CLIA waived assays. I published a paper in 1992, which among other papers supports the standard (Krouwer JS. Estimating Total Analytical Error and Its Sources: Techniques to Improve Method Evaluation. Arch Pathol Lab Med 1992;116:726-731).
I led a group to revise EP21A, as required by CLSI procedure and this was completed about a year ago. But although the revision to EP21A has been minor, EP21A2 has been stuck for a year!
So here are some of the objections to EP21A2.
One reject vote had a lengthy harangue which for the sake of simplicity could be summarized that EP21A2 was too complicated. This comment included:
Finally, this document not only is a departure but an abandonment of the traditional formula for estimating total error as a quantity [TE= (z factor)(CV%) +Bias%) widely publicized by Westgard and others.
I pointed out that the owner of this comment is listed as an author of the original EP21A standard and never complained about the original standard being too complicated – the revision to the original standard is minor. But since the original standard was issued, the commentator’s company entered into a financial arrangement with Westgard. I should say that I tried to point out this conflict of interest but this part of my response to the comment was always deleted by CLSI.
Another comment objected to the inclusion of pre- and post-analytical error to EP21A2. I have since modified the text of the document but the basic idea remains – it is intended to estimate all (e.g. total) error in the protocol that would be expected to occur in routine use. So for example if a finger stick sample were used routinely, then the protocol should be conducted with finger stick rather than venous samples. Putting things another way, if venous blood were used, then the potential error source from finger sticks would have been excluded from the protocol, which is a source of bias.
The objection to this was:
Pre-and post analytic error is not measurable in a reproducible manner. It will vary by location (ED vs. clinic, ICU private office etc.), personnel obtaining samples, reporting results. … We in the lab and manufacturers have very little control over this. We can however measure bias and imprecision quite well and then estimate total analytic error.
This is a misconception – even if it weren’t – it wouldn’t matter. Whether one has control over an error is not the issue, the task is to measure the error that will be observed in routine use.
As an example, say there are nurses performing finger sticks on assays that are under control of the laboratory. One could perform an experiment to measure error from this source as an imprecision component. It would be as reproducible as any other imprecision component (e.g., calibration error). And as for control, reagent bias is an analytical error. Why does the commentator think that he has control over the amount of reagent bias in an assay.
The problem is that laboratorians have been comfortable in using the wrong model (e.g., Westgard) to measure total error and also in excluding pre-analytical error from evaluation experiments that could easily be included (an error that occurs in some standards such as the glucose meter standard ISO 15197).
EP21A2 attempts to make things right and provide more faithful estimates of the error that will be routinely observed.