Cuthbert Daniel, was one of my statistical mentors. From his book (Daniel, C. Application of statistics to industrial experimentation, Wiley, New York, 1976. p5) is a valuable quote:
“the observations must be a fair (representative, random) sample of the population about which inferences are desired”
Simple enough, but when one views most assay evaluations, it is clear that the data does not meet the above quote. For example, glucose meter evaluations often have highly trained, rather than routine users and exclude samples that wouldn’t be routinely excluded. The logic for this is that the evaluations are following the ISO guideline. But bias – even though it is part of the ISO guideline – is still bias and the results will not reflect the actual errors that will be observed.