Picture is aerial view from a Cirrus of Foxwoods casino in CT
MU=measurement uncertainty TE=total error EG=error grid
Having looked at a blog entry by the Westgards, which is always interesting, here are my thoughts.
To recall, MU is a “bottoms-up” way to model error in a clinical chemistry assay (TE uses a “top down” model) and EG has no model at all.
MU is a bad idea for clinical chemistry – Here are the problems with MU:
- Unless things have changed, MU doesn’t allow for bias in it modeling process. If a bias is found, it must be eliminated. Yet in the real world, there are many uncorrected biases in assays (calibration bias, interferences).
- The modeling required by MU is not practical for a typical clinical chemistry lab. One can view the modeling as having two major components: the biological equations that govern the assay (e.g., Michaelis Menten kinetics) and the instrumentation (e.g., the properties of the syringe that picks up the sample). Whereas clinical chemists may know the biological equations, they won’t have access to the manufacturer’s instrumentation data.
- The math required to perform the analysis is extremely complicated.
- Some of the errors that occur cannot be modeled (e.g., user errors, manufacturing mistakes, software errors).
- The MU result is typically reported as the location of 95% of the results. But one needs to account for 100% of the results.
- So some people get the SD for a bunch of controls and call this MU – a joke.
TE has been much more useful than MU, but still has problems:
- The Westgard model for TE doesn’t account for some important errors, such as patient interferences.
- Other errors that occur (e.g., user errors, manufacturing mistakes, software errors) may be captured by TE but the potential for these errors are often excluded from experiments (e.g., users in these experiments are often more highly trained than typical users).
- Although both MU and TE rely on experimental data, TE relies solely on an experiment (method comparison or quality control). There are likely to be biases in the experiment which will cause TE to be underestimated. (See #2).
- The TE result is typically reported as the location of 95% of the results. But one needs to account for 100% of the results.
- TE is often overstated e.g., the sigma value is said to provide a specific (numeric) quality for patient results. But this is untrue since TE underestimates the true total error.
- TE fails to account for the importance of bias. That is, one can have results that are within TE goals but can still cause harm due to bias. Klee has shown this as well as me. For example, bias for a glucose meter can cause diabetic complications but still be within TE goals.
I favor error grids.
- Error grids still have the problem that they rely on experimental data and hence there may be bias in the studies.
- But 100% of the results are accounted for.
- There is the notion of increasing patient harm in EG. With either MU or TE, there is only the concept of harm vs no harm. This is not the real world. A glucose meter result of 95 mg/dL (truth=160 mg/dL) has much less harm than a glucose meter result of 350 mg/dl (truth=45 mg/dL).
- EG simply plots test vs. reference. There are no models (but there is no way to tell the origin of the error source).