Comments about bias

June 5, 2016

DSC_1458edp

I was once again interested in a title I saw in Clinical Chemistry – this one was: An unbiased view of bias. Ok, I get it – this is a humorous piece. But bias is an important topic and Clinical Chemistry is not a very friendly journal to statistical issues.

So here, I describe bias in method evaluations, based on my article that was recently published.

First, some comments about bias.

Bias when present does not guarantee that the results will be different than the case with no bias. The problem is you just don’t know. For example, if you preselect samples for a method comparison instead of randomly selecting samples, the differences observed may not be affected due to the preselection.

Bias is not necessarily an evil strategy concocted by a manufacturer – many biases are unavoidable.

In my paper, I cover several sources of bias – one will be mentioned here.

Reagent bias – In many regulatory evaluations, several (usually 3) lots of reagents are used to demonstrate that lot-to-lot reagent bias is small. An unbiased way of selecting the lots would be to randomly select lots from the population of lots used during the lifetime of the assay. Of course, this is impossible since future lots don’t exist. Hence, the selection of the lots is biased. There are several reasons that variation due to reagent lots is likely to be underestimated.

  1. The lots selected – usually the only ones that exist – are similar to each other and potentially different from future lots because future lots could have components from different lots, and those components could be qualified using different procedures.
  2. The procedure to qualify reagents may be different in the future.
  3. Usually during a regulatory method comparison using different reagent lots, the observed bias is centered by adjusting the instrument algorithm. For example, if the bias with 3 lots is 0, 1, and 2%, the instrument algorithm will be changed so that the bias reported is -1, 0, and 1%. (Note that this centering does not affect the spread of the reagents.)
  4. This centering is not used in future lots.
  5. The lots used in the evaluation are under more scrutiny than future lots, which is another case of bias. For example, the lots, besides being qualified through the manufacturing process, are also examined by a method comparison using patent samples compared to a reference assay. Future lots don’t have this scrutiny. They are usually qualified without a method comparison.

A possible remedy to try to better estimate the reagent bias would be to prepare a set of reagents according to factorial principles using components concentrations at the range of their manufacturing tolerances.

Biases in these studies other than reagent bias include: conflict of interest, protocol, patient selection, and user.


Unwarranted Conclusions

June 2, 2016

DSC_1509edp

Looking at a paper about QC procedures (subscription required), I admit I was intrigued by the title: “Selecting Statistical Procedures for Quality Control Planning Based on Risk Management.”

Just reading the abstract and the first few lines informs me that the conclusions are unwarranted because the authors claim, they can estimate the probability of patient harm based on which QC procedure is chosen.

A QC procedure helps to detect problems with the assay process. Patient harm can be caused by an assay process gone astray but it can also be caused by things with an assay process that has not gone astray. For example, a patient interference can cause patient harm and will not be detected by QC. Moreover, the authors assume that an out of control condition will occur in a constant fashion until it is detected by the next QC sample, but a shift in results that occurs for a limited number of samples can occur and is eliminated from consideration. So even QC considerations don’t include all possible errors.

Ok, I admit that I have stopped reading but it is clear that whatever the authors estimate (assuming their logic is correct) is an underestimate of the probability of patient harm.

That also makes me wonder, of all cases of patient harm caused by wrong medical decisions caused by assay error, what percentage are due to the assay process gone bad vs. other causes (e.g., interferences). For example searching for the word “interference” in the title of Clinical Chemistry over the last 10 years yielded 912 results.