How a Journal Club and a Blog Challenged the Mighty NEJM

Could this be end of the Tonka-Scope?

Much has been written about how Web 2.0 tools can change the healthcare landscape.  It would appear a recent set of circumstances has upped the ante.

This story begins with a recent study that attempted to tackle the problem of ICU infections. ICU infections are a challenging problem, patients who are admitted to the ICU are at risk of worsening illness and death from infections such as MRSA which can be acquired while in the ICU setting. To counteract this risk, current practice is the performance of surveillance cultures on people who are admitted to intensive care. If the person tests positive for certain infections they are placed in isolation (and health care providers are asked to wear silly gowns and share a useless stethoscope).

The success of this strategy is dubious, ranging from successful in some studies to nearly useless in others. Based upon my personal observations of my own hospital’s isolation practices, my only conclusion has been that yellow is not a good look for me.

But I digress. In this study, patients underwent “universal decontamination” with chlorhexidine, a commonly used antiseptic. The study found a dramatic drop in the numbers of MRSA infections and bloodstream infections. The study was peer reviewed and published in the flagship of medical publishing, the New England Journal of Medicine (sorry JAMA).

The results were so impressive in fact, that the accompanying editorial in the same issue was titled “Screening Inpatients for MRSA-Case Closed” . The Agency for Healthcare Research and Quality even advised on its website that hospitals begin implementing universal decontamination.

And why wouldn’t they? The 44% drop in ICU bloodstream infections resulted in a number needed to treat of 54, didn’t it?

Um, well, about those numbers . . .

An online journal club discussion found that the numbers didn’t quite add up.  The discussion was taken over to the Intensive Care Network, a relatively small blog in the outer rings of the healthcare galaxy, light years from the central galactic location of the mighty New England Journal. Their online community was called to look at the studies calculations and issued the following challenge:

“PLEASE have a go and see if you can match their NNT’s.

IF you can’t there is a serious problem, with practice changing implications.”

When none of them could get the numbers to add up, they contacted the studies lead author. Dr. Huang responded, and found the same problem as the rest of the community: there was an error in the calculations. As a result, the NNT increased from 54 patients to prevent one bloodstream infection to 99. Still encouraging, to be sure. But not nearly as impactful as the original results would suggest. This lead to the posting of a correction by NEJM.

This story makes me excited about the power of Web 2.0 tools, in particular social media sites that allow for rapid interactions between researchers and clinical providers so that they can exchange information in real time. In this case the combination of the traditional journal club, when combined with Web 2.0 tools, effectively lead to the crowdsourcing of the usual peer review process. By turning the usual peer review process on its head, trusted information that was potentially practice changing but factually erroneous was corrected so that practicing clinicians could make decisions based on good data

I don’t foresee that social media tools tools like blogs and crowdsourcing will ever replace the current process of academic peer review. But perhaps by replacing the one way exchange of information between researchers and clinicians with a two-way exchange of information between them, we can make the process better.

And if that results in me no longer  having to wear those ridiculous looking gowns, well so much the better.

Shout out to our friends at PulmCCM.org for reporting on this story.

– Deep Ramachandran is a Pulmonary, Critical Care, Sleep Medicine physician and a Social Media Section Editor at Chest Journal.  I am on twitter @caduceusblogger.