Do it right the first time – not always the best strategy

December 14, 2017

Watching a remarkable video about wing suit flyers jumping into an open door of descending plane, it appears that they had tried to accomplish this feat 100 times before having success.

On page four of a document that summarizes the quality gurus: Crosby, Deming and Juran, Crosby’s “Do it right the first time” appears. Clearly, this would have been a problem for the wing suit flyers. Crosby’s suggestion is appropriate if the state of knowledge is high. For the wing suit flyers, there were many unknowns, hence the state of knowledge was low. When the state of knowledge is meager, as it was at Ciba Corning when we were designing in vivo diagnostic instruments, we used the test analyze and fix strategy (TAAF) as part of reliability growth management and FRACAS. This sounds like the opposite of a sane quality strategy but in fact was the fastest way to achieve reliability goals for our instruments.

Advertisements

Risk based SQC – What does it really mean

December 4, 2017

Having just read a paper on risk based SQC, here are my thoughts…

CLSI has recently adopted a risk management theme for some of their standards. The fact that Westgard has jumped on the risk management bandwagon is as we say in Boston, wicked smaaht.

But what does this really mean and is it useful?

SQC as described in the Westgard paper is performed to prevent patient results from exceeding an allowable total error (TEa). To recall, TEa = |bias|/SD*1.65. I have previously commented that this model does not account for all error sources, especially for QC samples. But for the moment, let’s assume that the only error sources are average bias and imprecision. The remaining problem with TEa is that it is always given as a percentage of results, usually 95%. So if some SQC procedure were to just meet their quality requirement, up to 5% of patient results could exceed their TEa and potentially cause medical errors. This is 1 in every 20 results! I don’t see how this is a good thing even if one were to use a 99% TEa.

The problem is one of “spin.” SQC, while valuable, does not guarantee the quality of patient results. The laboratory testing process is like a factory process and with any such process, to be useful it must be in control (meaning in statistical quality control). Thus, SQC helps to guard against an out of control process. To be fair, if the process were out of control, patient sample results might exceed TEa.

The actual risk of medical errors due to lab error is a function not only of an out of control process but also due to all other error sources not accounted for by QC, such as user errors with patient samples (as opposed to QC samples), patient interferences, and so on. Hence, to say that risk based SQC can address the quality of patient results is “spin.” SQC is a process control tool – nothing more and nothing less.

And the best way of running SQC would be for a manufacturer to assess results from all laboratories.

Now some people might think, this is a nit-piking post but here is an additional point. One might be lulled into thinking that with this risk based SQC that labs don’t have to worry about bad results. But interferences can cause large errors that can cause medical errors. For example, in the maltose problem for glucose meters, 6 of 13 deaths occurred after an FDA warning. And recently, there have been concerns about biotin interference in immunoassays. So it’s not good to oversell SQC, since people might loose focus on other, important issues.


More commitment needed from authors

November 5, 2017

I just read an interesting paper about irreproducibility in science. The authors suggest a remedy: namely; that “authors of such papers should be invited to provide a 5-year (and perhaps a 10-year) reflection on their papers”.

I suggested to Clinical Chemistry a few years ago that every paper should have a “recommendations” section. To recall, most papers have some or all of: an introduction, methods, results, discussion, and conclusion sections. But rarely if ever is there a recommendations section, although sometimes there is a recommendation in the conclusions section.

In my company, I established a reporting format that required a recommendations section. The recommendations required action words (e.g., verbs).

So a study to evaluate an assay might have as a conclusion: “Assay XYZ has met its performance specifications.” The corresponding recommendation might be: “Release assay XYZ for sale.”

Although the recommendation might seem to be a logical consequence of the conclusion, psychologically, the recommendation requires more commitment. Were there outliers? Did the study have enough samples? Was there possible bias?

In any case, Clinical Chemistry declined to accept my suggestion.

 


A measure of influence

October 30, 2017

When I worked at Corning Medical (later Ciba Corning), we worked in cubicles and everyone had a whiteboard. These whiteboards were used often and sometimes a section of the whiteboard would be marked so that the contents of that section would remain as in DO NOT ERASE! I was pleased to note that on a few whiteboards, something that I had said was in the do not erase section.

But there was one person who dominated the do not erase section, even after he had long left the company. This was David Simmons, who acted as in in-house psychologist. This was one measure of his influence.


The difference between reviewing an article and writing a commentary about it

August 13, 2017

Recently, I was asked to review an article, which I did. I thought the article was impressive but as usual I still recommended some ways to improve it. Upon resubmission, I reviewed it again – my recommendations were implemented – and the article was published (online first). But that’s not the end of the story. A while later I was asked to write a commentary about the article, which would be published along with the article.

In a sense, I had to review it again and this time was more critical. It was (and is) an impressive article but when my commentary is published, I have to be sure that I have written about all of the positive parts of the article and any remaining deficiencies. Hence I found new deficiencies!

It reminds me when I managed a group at Ciba Corning that I always insisted on a written rather than a verbal report. A verbal report is ephemeral but when you put your name on something you think about it much deeper.

 


Limited release for sale

July 12, 2017

When I was in industry, we would have a “release for sale” meeting in order to determine whether the product should be released or not. The person from regulatory affairs was always irksome because he always insisted the product was not ready for release. Yet, two hours or so later he signed the release for sale form. I think he behaved this way to show that 1) he was not some guy who would rubber stamp anything that was put in front of him and 2) if there were product problems he could say, he never wanted to release the product. But he nevertheless always looked bad because basically no new data had been put in front of him, yet he signed the release for sale form without someone putting a gun to his head. If he truly believed the product was not ready, he should have stuck to his guns.

But back to the title…, as a “concession” the marketing guy would say, ok let’s have a limited release for sale implying that fewer units would be delivered than possible. Of course, this was fiction – what the limited release really meant was marketing was limited to selling units as fast as they could, and because of startup issues, product sales usually took some time to get going.

On a more positive note, our release for sale meetings – as I recall – were always unanimous, and not every time was the decision to release – sometimes it was decided to not release the product.


A good vision / mission statement

July 11, 2017

I heard an advertisement from Alere which ended with the statement: knowing now matters. Unlike the vision and mission statements that I suffered through (and no longer remember) while I was in industry, the statement knowing now matters seems to be a perfect summation for a point of care company.