Publishing in peer reviewed journals vs. “publishing” on a commercial web site

November 19, 2005

I have published in peer reviewed journals for many years (1). The impartiality of peer review, while not perfect, works reasonably well. Yet, there are drawbacks, especially for people like me who some of the time write articles that are more opinions and editorials than research. So here are some opinions about publishing in peer reviewed journals vs. “publishing” on a commercial web site.

Publishing in peer reviewed journals

The peer review process – works well for research articles. It is sometimes questionable for unsolicited, opinion type articles. For an opinion article that I published recently (2), the total time from initial submission to acceptance was about six months. The journal in which the article was published was the second journal to which the article was submitted. The article critiqued GUM (Guide to the expression of uncertainty of measurement). It was perhaps natural that the editors sent this manuscript to experts in GUM, who do not look kindly on articles that take an opposing view (e.g., I stated that GUM shouldn’t be carried out for commercial diagnostic assays). Since editors are often seeking to limit the number of pages accepted, round one is almost by definition – rejection. Given the reviewer comments, I consider it lucky that the article was published.

The circulation of peer reviewed journals – is attractive, yet even if there are many subscribers, there is of course no guarantee that my article will be read. In journals to which I subscribe, the number of articles that I read is a small percentage of all articles. There are articles for which I don’t even understand the title! Moreover, it is possible that if the journal does not provide free access, that non subscribers who come across my article as an abstract will not read it as the pay per view charges are pretty steep.

Publishing on a commercial web site

The speed of publication is fast – actually, it takes about a half an hour to be satisfied that things look ok. Color figures are routine, unlike print journals. I choose HTML although PDF files are possible, which are often used by online journals. (I avoid PDF files because they are at times hard to open).

Some review is possible – although it is not really peer review. Occasionally, I send out articles for comment before they are posted. I often receive comments the same day and use some of them, with acknowledgements.

I can see some details about circulation – by analyzing log files from the web site. This shows me1:

  • how many people viewed the article
  • where they are from (the organization and country but not the specific person)
  • when they viewed the article
  • what else they viewed on the site
  • if they bookmarked the article
  • if they found the article through a search engine, what were the search terms
  • how else they got to the article (e.g., from another site for example)
  • occasionally useful codes, such as 404 (file not found) indicating a bad link

Sometime there are other interesting things. For one article, I noticed that unlike most articles which were viewed once, this article was often viewed twice at two different times during the day, as if the response were “huh, run that by me again.”

1Less of interest to me but nevertheless available from log files, are attempts to hack into the web site, operating system used, browser used, and so on.

The potential for a virtual quality web for hospital labs

I am not the only one who publishes articles about lab quality. Perhaps one of these days, “we” (whatever that means) can put together a web site that provides links to all such articles.

References

  1. https://jkrouwer.wordpress.com/publications/
  2. Krouwer, JS. A Critique of the GUM Method of Estimating and Reporting Uncertainty in Diagnostic Assays Clin Chem 2003;49:1218-1221.

What does “It’s up to the lab director” really mean

November 12, 2005

This phrase appears quite often in quality articles and usually means that a quality goal or the interpretation of an evaluation result, rather than being specified, is left up to the discretion of the laboratory director. Some examples of use of this phrase (or something close to it) follow:

Reference ranges (assay package inserts)

Under virtually any reference range section of a package insert, one will find the phrase, “Each laboratory should establish its own reference intervals based upon its patient population (1).”

Comment – This statement, while boilerplate for manufacturers, makes little sense for most assays. For example, for cholesterol the limit 200 mg/dL (5.17 mmol/L) is universal these days and I doubt that a laboratory would actually conduct a study to present a new limit to clinicians, as a clinician expects assays to conform to this limit. So, the task is that the manufacturer develop an assay with little bias (which they attempt to do and often achieve).

Standards

CLSI has a variety of evaluation protocols which avoid answering the question of how to define goals. Some examples:

EP6, Section 5.3 (Linearity) – “The laboratory must determine its own goals for measurement error for every analyte (see EP21).”

EP9, Section 7 (Bias) – “Each laboratory should develop its own criteria (in consultation with its medical staff and/or the technical literature).”

EP15, Forward (User Determination of Performance) – “First, the laboratory must specify the required performance for the method.“

EP21, Section 4.3 (Total Error) – “Laboratories should thus attempt to establish limits, for which there are many strategies suggested in the literature.”

Perhaps if CLSI were to add clinicians to its subcommittees, it would be in a better position to address goals.

EQC (Equivalent Quality Control)

Recently, EQC has been debated. From the CMS web site regarding EQC, “The laboratory director has the overall responsibility to determine the laboratory’s QC program” (2).

What’s missing

All of this “It’s up to the lab director” might make sense, but . . . When I have asked lab directors for their quality goals for assays, their response is usually a request for me to clarify the question. Afterwards, the usual response is, do you mean CLIA goals? Now CLIA goals are proficiency survey goals, not goals for patient results (see essay). In other words, the lab directors that I have spoken to don’t have their own laboratory quality goals.

Of course, this does not mean that laboratory directors don’t care about quality or that they do not recognize quality problems; however, it does impede measuring quality failure rates and establishing criteria to know when one has devoted enough effort to a quality program.

What is needed is for lab directors to establish lab quality goals and evaluate results. For example, the lab needs to:

  1. Classify all lab errors as to severity
  2. Establish goals for the rate of errors in each severity category
  3. Measure lab error rates
  4. For assay performance goals (e.g., failure to meet assay performance is a type of lab error under #1), use the new FDA waiver guidance (see essay) to set up goals including
    1. Use of glucose type error grids for each assay
    2. Goals for allowable total error
    3. Goals for limits for erroneous results
    4. Measure results against goals

Setting goals is a lot of work and could be facilitated by standards organizations. This is one reason that lab directors use CLIA goals – they are looking for guidance from regulatory1 providers. Since manufacturers tend to dominate these groups, it remains to be seen if this will be accomplished.

1Organizations such as CLSI have quasi regulatory status, since FDA adopts CLSI standards.

Acknowledgement

Helpful comments were provided by Glenn Fine, Executive Vice President of CLSI and Luann Ochs of Roche, the chairholder of the CLSI Evaluation Protocol Area Committee.

References

  1. See for example, http://www.beckman.com/customersupport/ifu/cis/389726/AA/EN_CHOL.doc
  2. See: http://www.cms.hhs.gov/CLIA/downloads/qchigh.pdf