CLSI Evaluation Protocol Area Committee – Tough crowd


Although I responded about a written comment about EP21 (Total Error), this comment was also presented verbally at the AACC meeting in Chicago. There were some bizarre additional rants by this commentator which I will ignore but the basic comment was that EP21 should not attempt to include pre-analytical error but focus only on analytical error.

So here is my response.

EP21 simply looks at differences between a candidate and comparative method. These differences are typically generated in a method comparison experiment whereby one sequentially assays patient samples by both methods. There is no way to know the source of these differences. Because most of the time, the same sample will be split in two and analyzed on two instrument systems, pre-analytical error will not be observed but there is no guarantee that the same sample will (or in some cases should) be used.

Consider a manufacturer who compares a blood gas pO2 value (syringe drawn from a tonometer) to its reference value (gas tank). Here, the difference is not just analytical error but also any pre-analytical error from room air mixing with the blood in the syringe. This pre-analytical error is dependent on the technique of the operator.

Another example is comparing a Point-of-Care assay that relies on a fingerstick (glucose, PT) to a laboratory comparative method that uses a (venous) serum sample. In this case, if one split the serum sample and used it for the Point-of-Care assay, one would be excluding the possibility of error that might occur routinely (e.g. from an improper fingerstick).

So in one sense, the revised version of EP21 is simply correcting a mistake in the original version, because there is nothing new to either the protocol or the analysis method.

Putting things another way, the “total” in total error will capture all errors that are possible in the experiment that has been performed. If pre-analytical error is possible in the experiment, then it may appear in the differences. In all cases, not all error sources will be sampled so the “total” only applies to that experiment. For example, for analytical error, (usually) only one reagent lot is being sampled, most pre-analytical error is not captured when a split sample is used.

Finally, there was another commentator with a reject vote and rather hostile I might add (thanks to the several people who came to my defense). One of his comments was about the LDL cholesterol example in EP21. He said it was a poor example because there were three gross outliers which should lead to troubleshooting and further analysis would be a waste of time.

Now most Evaluation Protocol documents use simulated examples. The LDL example is a real example from a clinical laboratory. The commentator has missed the purpose of the experiment and analysis. Data is not deleted in EP21 (e.g. outliers). The data is what it is – a snapshot in time of what the laboratory will see routinely. It is naive to expect that the laboratory has the resources to uncover the sources of the differences. That one can expect some large differences to occur in routine use is valuable information.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: