The myth of the sample size for evaluations


Virtually any evaluation protocol has a recommended sample size, or at least a procedure for calculating a sample size. This entry explores some problems with sample sizes.

Assume that it is desired to calculate a sample size for an error grid evaluation (1). See reference one for details, but in an error grid evaluation, one performs a method comparison, plots the results in the grid and calculates the percentage of results in each error grid zone. In an error grid, there are least two areas of interest – the innermost zone (called “A” here) which contains most of the differences and an outer zone (called “C” here) which should contain no results as these differences have a high potential for serious patient harm.

I will skip the discussion of calculating the sample size for zone A. To calculate the sample size for zone C, one needs a goal – assume that one wishes less than one result per million in zone C. It can be shown (2) that the required sample size to prove with 95% confidence that less than one result per million is in zone C is to run 371,000,000 samples and observe no results in zone C. Additionally, the candidate assay has to be run in a representative way (with respect to routine use) in the method comparison. Since this number of samples is a bit much, how can one be confident that the goal for zone C will be achieved? The answer is using risk management techniques. The clinical laboratory has to perform FMEA/fault tree analysis (3) to ensure that user errors don’t cause zone C results and the manufacturer has to perform FMEA/fault tree analysis to ensure that the system itself doesn’t cause zone C results.


  1. CLSI/NCCLS. How to Construct and Interpret an Error Grid for Diagnostic Assays EP27 Proposed Guideline. CLSI/NCCLS document EP27-P. Wayne, PA: NCCLS; 2009.
  2. Hahn GJ and Meeker WQ. Statistical intervals. A guide for practitioners. Wiley: New York, 1991, pp 103-105.
  3. CLSI/NCCLS Risk Management Techniques to Identify and Control Laboratory Error Sources. Proposed Guideline –Third Edition CLSI/NCCLS document EP18-P3 Wayne, PA: NCCLS; 2009.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: