I was once again interested in a title I saw in Clinical Chemistry – this one was: An unbiased view of bias. Ok, I get it – this is a humorous piece. But bias is an important topic and Clinical Chemistry is not a very friendly journal to statistical issues.
So here, I describe bias in method evaluations, based on my article that was recently published.
First, some comments about bias.
Bias when present does not guarantee that the results will be different than the case with no bias. The problem is you just don’t know. For example, if you preselect samples for a method comparison instead of randomly selecting samples, the differences observed may not be affected due to the preselection.
Bias is not necessarily an evil strategy concocted by a manufacturer – many biases are unavoidable.
In my paper, I cover several sources of bias – one will be mentioned here.
Reagent bias – In many regulatory evaluations, several (usually 3) lots of reagents are used to demonstrate that lot-to-lot reagent bias is small. An unbiased way of selecting the lots would be to randomly select lots from the population of lots used during the lifetime of the assay. Of course, this is impossible since future lots don’t exist. Hence, the selection of the lots is biased. There are several reasons that variation due to reagent lots is likely to be underestimated.
- The lots selected – usually the only ones that exist – are similar to each other and potentially different from future lots because future lots could have components from different lots, and those components could be qualified using different procedures.
- The procedure to qualify reagents may be different in the future.
- Usually during a regulatory method comparison using different reagent lots, the observed bias is centered by adjusting the instrument algorithm. For example, if the bias with 3 lots is 0, 1, and 2%, the instrument algorithm will be changed so that the bias reported is -1, 0, and 1%. (Note that this centering does not affect the spread of the reagents.)
- This centering is not used in future lots.
- The lots used in the evaluation are under more scrutiny than future lots, which is another case of bias. For example, the lots, besides being qualified through the manufacturing process, are also examined by a method comparison using patent samples compared to a reference assay. Future lots don’t have this scrutiny. They are usually qualified without a method comparison.
A possible remedy to try to better estimate the reagent bias would be to prepare a set of reagents according to factorial principles using components concentrations at the range of their manufacturing tolerances.
Biases in these studies other than reagent bias include: conflict of interest, protocol, patient selection, and user.