I have started the process of transferring older posts, which were in my now defunct website, to this blog. It will take a little while and additionally it will take some time for the search engines to catch up.
As of the end of 2016, I stopped consulting. I still follow lab medicine topics that interest me, and will still publish both papers in journals and entries on my blog.
I ended my website KrouwerConsulting.com – it doesn’t work anymore – and I realize that some of my earlier blog entries can’t be accessed because they were on KrouwerConsulting.com (written before I started my current blog). I will fix this and transfer all of these entries to my current blog, as soon as I figure it out!
I want to thank all of my clients. Their problems kept me thinking and I enjoyed both the technical tasks and meeting people to discuss a variety of issues.
Most people have seen the video of a passenger being forcibly pulled off a United flight. After some missteps, one would think that the CEO would say the right thing. But he said the following (as seen on CBS news on 4/27).
“Our policies were placed ahead of our shared values and procedures got in the way of what we know is right.”
To people like me, who have worked in industry, “shared values” sounds like a mission or vision statement. The problem is that a mission statement is supposed to dictate policies and procedures and of course, it is management that defines policies and procedures. Hence, the quote sounds hollow.
What a company does can be considered its mission. If it conflicts with its mission statement, then the mission statement is out of whack.
A manufacturer designs a diagnostic assay system and after considerable testing, the assay is approved by regulators and sold. Subsequently, there are some patient harm incidents traced back to incorrect results from this assay. Upon analysis, the manufacturer maintains that the incorrect results were caused by customer misuse.
The spectrum of customer misuse
When I worked for a diagnostic company, I remember a product that had poor reliability. In meetings devoted to solving the reliability issues, the head of engineering claimed for many of the problems that he could do nothing, because the problem was caused by customer misuse. One of these problems stuck in my mind because the customer was required on a regular frequency to disassemble a valve and clean it. To frequently rebuild the valve seemed excessive and the next generation product employed a new design which obviated this maintenance.
An example on another end of the spectrum is that some instrument systems allow the user to delay a required calibration. If the user continues to delay the calibration and to either not run quality control or to ignore failed quality control, incorrect results could easily be generated.
Although in the second case, one could argue that the user had violated the policy set up by the laboratory, the same might be true for the first case.
Of course there will be customer misuse issues which are less black and white (if in fact the above examples are).
These are hypothetical examples. A real example was reported for a home glucose analyzer (1) when users did not completely insert the reagent strip and got incorrect results leading to some hospitalizations. Not completely inserting the strip did not cause any error message and also represented an example of not following the instructions (e.g., customer misuse). In this case, the government successfully brought legal action against the manufacturer. In another example, a clinical laboratory (Maryland General Hospital) grossly violated their own policies (2).
Regarding customer misuse and blame, the taxonomy for errors described by Marx (3) is of interest.
The FMEA (fault tree) approach to customer misuse
If one sets up a FMEA or fault tree, the effect “incorrect results” can be caused by a variety of events, and the cause of these events can be the customer doing something incorrectly (customer misuse). Actually, some companies divide FMEAS into separate categories, with one FMEA devoted to customer use (e.g., misuse).
There are several questions to be addressed in these customer use FMEAs (as with all FMEAs).
- what is the severity of the event caused by misuse– e.g., will it lead to potential patient harm such as potential causing incorrect results
- what is the estimated probability of occurrence
- what is the best control or mitigation to prevent this misuse error from occurring
- what is the best way to detect this misuse error and recover from it
The spectrum of solutions for preventing customer misuse
Just as there is a spectrum of customer misuse, there is a spectrum for the mitigations to prevent customer misuse. The default mitigation for customer misuse is virtually always customer training ranging from the instruction manual (and offshoots such as videos) to onsite training. One must understand that since there is always an instruction manual (or package insert), the mitigation is to improve the instruction manual. The mechanism to do this involves usability testing. The other end of the spectrum is redesign. As the reliability consultant Ralph Evans suggested “make it easy to do the right thing and hard to do the wrong thing.” A previous essay on a medical error also illustrates this spectrum.
Some of the mitigations are also not that black and white. Consider a manufacturer that has conducted extensive interference testing for an assay and has reported in the product insert that 7 drugs interfere with the assay and when any of these substances are present above the concentrations listed, the manufacturer’s assay should not be used. If the clinical laboratory is wired into the hospital’s EMR (Electronic Medical Record) assuming that an EMR exists, one could suggest that rules could be built into the LIS (Laboratory Information System) to follow the manufacturer’s recommendation. Without these computerized systems, one would have to manually inspect (potentially) each patient’s medical record, which is a daunting task.
The current environment
Ever since the Auto Analyzer was invented, the trend has been towards instruments that are easier to use. Since clinical laboratory staff are less trained today, manufacturers design their products to be easier to use to gain competitive advantage and advertise this feature. Regulators such as the FDA also recognize the value of ease of use and require hazard analysis.
Ease of use is thus a key product attribute for many systems and fulfills Ralph Evans suggestion. But there will nevertheless always be customer misuse issues and each one must be considered with the result that some will be shown to be the responsibility of the manufacturer, some the responsibility of the clinical laboratory and for some agreement of responsibility will never be reached.
“The following list presents 10 persons who have made a significant impact on the IVD industry.” This is how the magazine IVD Technology begins and then gives a short description of each of the 10 people (1). Two of the people listed in the top 10 happen to be on the editorial advisory board of IVD Technology (2). Hmmm…..
About half of the editorial advisory board are in regulatory affairs and four of the top 10 are also in regulatory affairs (including the two above). In case you’re wondering, Leonard Skeggs, the inventor of the auto-analyzer didn’t make the list! OK, to be fair, the text also says “Efforts were made to ensure that this list reflects contributions in both the regulatory and scientific areas.”, but the title and first sentence are misleading.
- Top 10 Persons in the IVD Industry IVD Technology April 2005, see http://www.devicelink.com/ivdt/archive/05/04/002.html
- See, http://www.devicelink.com/ivdt/eab.html
A Wall Street Journal article discusses the role of the New England Journal of Medicine in the Vioxx affair (1). An aspect of the article that caught my attention was the attempt by a pharmacist, Dr. Jennifer Hrachovec, to make known the dangers of Vioxx.
She first tried to do this during a radio call in show which had as one of its guests, Jeffrey Drazen, the editor of the New England Journal of Medicine. He blew off her comments.
She next submitted a Letter to the editor to the New England Journal of Medicine. It was rejected.
Finally, she was able to get a Letter published in JAMA, the Journal of the American Medical Association.
I can relate to this sequence of events and suggest that part of the problem is that however relevant and correct a person is on an issue, the person’s issue may not be taken seriously if that person is not “a member of the club”. Journals such as the New England Journal of Medicine have so many submissions that they are always looking for ways to reject papers. I suspect that one criteria used is simply the status of the person submitting the paper. Fortunately, Dr. Hrachovec persisted. For me, when someone blows off my comments, it is a source of motivation, and I have had my share of rejected Letters.
- Bitter Pill How the New England Journal Missed Warning Signs on Vioxx. David Armstrong Wall Street Journal, May 15, 2006, page A1.
Facilitators play an important role in quality activities. For example, they often lead training and brainstorming sessions. Brainstorming is a key part of FMEA (Failure Mode Effects Analysis) and fault trees. Whereas this blog entry is not meant to be a summary of what makes a good facilitator, I was recently reminded of a problem with some facilitators; namely that of the filter.
Filters are people who while serving as facilitators, feel compelled to have all information go through them. The filter then re-releases the information but in a changed form. That is, whatever was originally submitted to the filter becomes changed into a form that the facilitator understands (which may or may not be the same as the person who had the original idea). In some cases the facilitator changes the way the information is presented by rewriting it or restating it (e.g., out loud). The latter must be familiar as one often hears, “now let me make sure I understand what you’ve said, you mean that, …., “.
There is nothing wrong with the concept of a filter, since in principle, a filter could make ideas more clear and if nothing else ensures that an idea is understood as intended. Whereas this is often useful – sometimes essential – between two people, the danger is when the filter is used in a group setting and the filter makes ideas less clear, changes ideas, or omits ideas.
I recall a CLSI (formerly NCCLS) strategy session a few years ago. I had prepared a list of issues which the facilitator rewrote. My list had already been read by the head of the organization with a few minor changes so this rewrite by the facilitator seemed completely unnecessary and more importantly, it failed to capture the issues as clearly as I had and at the same time dropped some issues. So the strategy session took place without the right list of issues and during the strategy session, all material went through the facilitator, as in “now let me make sure I understand what you’ve said, you mean that, …., “. The facilitator of course also wrote up the results of the meeting. In all, this was a lost opportunity, largely driven by a filter.
So a better way is for the facilitator to assemble all ideas through a consensus process. The final product may have some editing for readability but without the effects of a filter.