I don’t brood over things, but recently I had occasion to revisit something which still bothers me. The CLSI (formerly NCCLS) document EP11 which is about uniformity of performance claims was a controversial document, which I previously discussed. There was pressure from CLSI management to cancel this document, which resulted in a CLSI strategy conference for Evaluation Protocols in 2003.
This conference was facilitated by Ed Kimmelman, who in my opinion did a poor job. Everything had to go through him which was then reissued by him, often in a garbled form. I called this a filter (scroll down to: 5/14/06 – Beware of the filter) not a facilitator. This was a lost opportunity but what made me mad was a report circulated to the attendees – 30 or so influential people in the field – in which Kimmelman made the following statement on page 1 under “Background”:
“NCCLS management has the belief that the process of developing evaluation protocols within the NCCLS consensus process can be improved.
In recent years it has become apparent that there has been difficulty moving certain evaluation protocols through the consensus process due to a number of reasons, including
– dissatisfaction with the content of such protocols, – dissatisfaction with the constituency representation on the Evaluation Protocols Area Committee, and perceived failure of NCCLS and area committee management to meet their responsibilities under the NCCLS Administrative Procedures”
I have highlighted the item in yellow that most irritated me, although the rest of the statements are also not true. So basically, my management – I was the area committee chairholder – was being questioned. Therefore, I put together some facts about – and have now updated – document status when I was chairholder (1999-2004), before and after.
Evaluation Protocol Project Activity: 1999-2004
|Document – Last pre 1999 Action Formal title||Status 1999||Status 2004||After 2004||2002 Core: Ave. Sales / year|
|EP5T – 1992 Evaluation of Precision Performance of Clinical Chemistry Devices||EP5A published||EP5A2 published 2004||1:266|
|EP6P – 1986 – Evaluation of the Linearity of Quantitative Analytical Methods||No action for 13 years!||EP6A published 2003||2:96|
|EP7P – 1986 Interference Testing in Clinical Chemistry||No action for 13 years!||EP7A published 2002||1:219|
|EP9A – 1995 Method Comparison and Bias Estimation Using Patient Samples||No action for 7 years||EP9A2 published 2002||2:283|
|EP10A – 1998 Preliminary Evaluation of Quantitative Clinical Laboratory Methods||EP10 A2 published 2002||EP10 A3 published 2006||2:237|
|EP11P – 1995 Uniform Description of Claims for in Vitro Diagnostic Tests||No action for 8 years||EP11 cancelled by Board 2003||2:45|
|EP12 Project approved – 1986 User Protocol for Evaluation of Qualitative Test Performance||No action for 13 years!||EP12A published 2002||2:27|
|EP13R – 1995 Laboratory Statistics—Standard Deviation||————||————||2:45|
|EP14 Project approved – ? Evaluation of Matrix Effects||EP14P published||EP14A published 2001||EP14A2 published 2005||2:51|
|EP15P – 1998 User Demonstration of Performance for Precision and Accuracy||EP15A published 2001||EP15A2 published 2006||2:48|
|EP17 Project (see note) Protocols for Determination of Limits of Quantitation||EP17A published 2004||—–|
|EP18 Project approved – ? Quality Management for Unit-Use Testing||EP18P published||EP18A published 2002||2:23|
|EP19 Project approved – 1997 A Framework for NCCLS Evaluation Protocols||EP19R published 2002||2:25|
|EP20 Project approved – 1998 Quality Goals for Acceptable Performance and Threshold Criteria for Outliers||EP20 cancelled 2003||—–|
|EP21 Project approved – 1998 Total Error for Clinical Laboratory Methods||EP21A published 2003||No Data|
EP10 – I was the chairholder of EP10A, EP10A2 and EP10A3. EP17 – an earlier version had been cancelled in 2000, EP17 was re-approved 2001 EP19 – This was published as a “P” document in 2000, then as a “R” (Report) EP20 – The chairholder resigned in 2001, a new subcommittee was started 2002, the project was cancelled by the area committee. EP21 – I proposed and led EP21. The last column refers to a financial tracking system, I put in place for all CLSI documents. Core refers to sales – 1: most sales, 2: moderate sales, 3: least sales.
Of the 14 projects for which action was expected – a chairholder serves for six years – every document was either advanced or cancelled during my term. This included three documents for which no action had taken place each for 13 years! (e.g., more than two chairholder terms).
I called the head of NCCLS and requested an apology in writing to be distributed to correct the Kimmelman statement. After a heated exchange, I was later told that I might not serve my final year as chairholder and the next area committee meeting might be cancelled. Well, neither of those things happened. The area committee took place with the president and president-elect in attendance, who tried to get me again to cancel EP11. I didn’t – EP11 was advanced and the Board cancelled it.
Dan Tholen as the vice-chairholder and who supported me often in writing, was slated to replace me but was not invited by NCCLS to be chairholder. I think that NCCLS reasoned that they could put up with me for one more year to finish my term, but putting up with a new chairholder with similar views for an additional six years was to be avoided.
So in all, there are battles within organizations to get one’s way – that’s not the issue. What bothered me was that I was attacked publically for poor performance and this was not true. I never got my apology – the other people all won awards.
These days, I still contribute to CLSI. I gave a recent talk at their annual forum, am the chairholder of an existing project (EP18) and have just proposed a new project.