Evidence Based at NIH

Last year, NIH said it was undertaking a process to identify a research case definition for ME/CFS, but individuals like me were unable to get any additional information about what NIH intended. Now additional information has been made public in a February 27, 2013 letter from Assistant Secretary for Health Dr. Howard Koh to Kim McCleary of the CFIDS Association (pdf link). I’ve done some research, and I believe we need to monitor this case definition process closely.

Case definition has been the 600-pound gorilla in the room every time ME/CFS is discussed. There are five definitions currently in use (Oxford, Fukuda, Carruthers 2003, Carruthers 2011, and Jason Pediatric) and there is no consensus among researchers and policymakers on which one is the best. (This is such a complex issue, it deserves its own post.) NIH, as the largest funding source for biomedical research, can wield great influence over the definitions used by researchers. While it was a positive step to say NIH would undertake a research definition effort, there was no transparency or information available to us until now.

In his letter to Ms. McCleary, Dr. Koh said:

the [Trans-NIH] Working Group submitted a competitive application for an Evidence-based Methodology Workshop (EbMW) on ME/CFS coordinated by the NIH Office of Disease Prevention. The EbMW consists of a thorough, unbiased evidence review of the literature related to clinical research outcomes compared across case definitions and culminating in a workshop composed of experts and patients. The workshop participants and panel members will use the evidence review to evaluate the strength of evidence for case definitions with the goals of identifying the most consistent outcomes. The Working group successfully competed with proposals from other Working Groups across NIH (e.g. The NIH Pain Consortium) for funding provided by the NIH Office of Disease Prevention. The first organizational meeting for the EbMW on ME/CFS was held on February 19, 2013. (emphasis added)

It turns out that the NIH Office of Disease Prevention has a Consensus Development Program that produces, “evidence-based consensus statements addressing controversial medical issues important to researchers, healthcare providers, policymakers, patients, and the general public.” The Office of Disease Prevention provides infrastructure, funding, and coordination for the process. The process entails the following steps*:

1. A Steering Committee of ME/CFS scientific/research experts and federal partners coordinates the EbMW with logistical support from the Office of Disease Prevention.
2. A conference panel is selected. The panelists “give balanced, objective, and informed attention to the topic.” They cannot be HHS employees or have a financial or career interest in the conference topic. They may be familiar with the conference topic “but must not have published on or have a publicly stated opinion on the topic.”
3. The Agency for Healthcare Research and Quality conducts a “systematic evidence review” and produces a report for the panel members prior to the conference.
4. A two and a half day conference is held, consisting of presentations by subject matter experts, followed by discussion among the panelists, speakers and public in attendance.
5. Dr. Susan Maier told me, “ME/CFS science/research experts and patient advocate groups with scientific/research expertise will be invited to participate in the workshop; the workshop is open to the public and will be videocast and archived.”
6. The panel drafts a statement during the meeting and presents it for commentary on the third day. The final report is issued six weeks later. It is published online and in a major peer-reviewed journal.

The Office of Disease Prevention has recently followed this process for meetings on Polycystic Ovary Syndrome (conference held December 2012) and Diagnosing Gestational Diabetes Mellitus (conference held March 2013).  In scanning the materials for these two conferences, I was struck by the absence of patients or advocacy representatives on both the Steering Committees and Panels. I point this out not to raise a ruckus about it. Rather, I think we should educate ourselves about how the process has worked in other diseases, and prepare to operate within the constraints of the existing process. If EbMW never use patients as panelists, then we should accept that limitation of the method. Insisting the methodology be changed to suit us is not likely to succeed. It would be more productive to focus on extensive preparation for the public meeting, especially since there are opportunities for open discussion with the panelists and speakers.

There is no timeline for this Workshop available yet. Nor do we know the members of the Steering Committee. Much will depend on the evidence review conducted in advance of the meeting. The evidence based report prepared for the Gestational Diabetes meeting (pdf link) is more than 300 pages long, and distills thousands of scientific papers into recommendations for clinical practice. Will we have to worry about psychosocial research making its way into the evidence based report? Much will also depend on selection of the panelists. According to the Consensus Development Program’s website, those panelists “must not have published on or have a publicly stated opinion on the topic.” Who will those panelists be?

We may not be able to participate in this process until the public meeting, but I hope advocates will join me in monitoring how this evolves. I also hope NIH will be forthcoming about each step in the process. In my experience, the engagement between advocates and the government is more positive and productive when the advocates have access to accurate information. We have a lot to learn about this methodology, and we should adequately prepare to have the best possible influence on the process.

*My description of the conference steps is drawn from the general description of the Consensus Development Program on NIH’s website and an email from Dr. Susan Maier, Chair of the Trans-NIH ME/CFS Research Working Group. The actual process may differ from what I describe here.