Evidence Based at NIH

Last year, NIH said it was undertaking a process to identify a research case definition for ME/CFS, but individuals like me were unable to get any additional information about what NIH intended. Now additional information has been made public in a February 27, 2013 letter from Assistant Secretary for Health Dr. Howard Koh to Kim McCleary of the CFIDS Association (pdf link). I’ve done some research, and I believe we need to monitor this case definition process closely.

Case definition has been the 600-pound gorilla in the room every time ME/CFS is discussed. There are five definitions currently in use (Oxford, Fukuda, Carruthers 2003, Carruthers 2011, and Jason Pediatric) and there is no consensus among researchers and policymakers on which one is the best. (This is such a complex issue, it deserves its own post.) NIH, as the largest funding source for biomedical research, can wield great influence over the definitions used by researchers. While it was a positive step to say NIH would undertake a research definition effort, there was no transparency or information available to us until now.

In his letter to Ms. McCleary, Dr. Koh said:

the [Trans-NIH] Working Group submitted a competitive application for an Evidence-based Methodology Workshop (EbMW) on ME/CFS coordinated by the NIH Office of Disease Prevention. The EbMW consists of a thorough, unbiased evidence review of the literature related to clinical research outcomes compared across case definitions and culminating in a workshop composed of experts and patients. The workshop participants and panel members will use the evidence review to evaluate the strength of evidence for case definitions with the goals of identifying the most consistent outcomes. The Working group successfully competed with proposals from other Working Groups across NIH (e.g. The NIH Pain Consortium) for funding provided by the NIH Office of Disease Prevention. The first organizational meeting for the EbMW on ME/CFS was held on February 19, 2013. (emphasis added)

It turns out that the NIH Office of Disease Prevention has a Consensus Development Program that produces, “evidence-based consensus statements addressing controversial medical issues important to researchers, healthcare providers, policymakers, patients, and the general public.” The Office of Disease Prevention provides infrastructure, funding, and coordination for the process. The process entails the following steps*:

  1. A Steering Committee of ME/CFS scientific/research experts and federal partners coordinates the EbMW with logistical support from the Office of Disease Prevention.
  2. A conference panel is selected. The panelists “give balanced, objective, and informed attention to the topic.” They cannot be HHS employees or have a financial or career interest in the conference topic. They may be familiar with the conference topic “but must not have published on or have a publicly stated opinion on the topic.”
  3. The Agency for Healthcare Research and Quality conducts a “systematic evidence review” and produces a report for the panel members prior to the conference.
  4. A two and a half day conference is held, consisting of presentations by subject matter experts, followed by discussion among the panelists, speakers and public in attendance.
  5. Dr. Susan Maier told me, “ME/CFS science/research experts and patient advocate groups with scientific/research expertise will be invited to participate in the workshop; the workshop is open to the public and will be videocast and archived.”
  6. The panel drafts a statement during the meeting and presents it for commentary on the third day. The final report is issued six weeks later. It is published online and in a major peer-reviewed journal.

The Office of Disease Prevention has recently followed this process for meetings on Polycystic Ovary Syndrome (conference held December 2012) and Diagnosing Gestational Diabetes Mellitus (conference held March 2013).  In scanning the materials for these two conferences, I was struck by the absence of patients or advocacy representatives on both the Steering Committees and Panels. I point this out not to raise a ruckus about it. Rather, I think we should educate ourselves about how the process has worked in other diseases, and prepare to operate within the constraints of the existing process. If EbMW never use patients as panelists, then we should accept that limitation of the method. Insisting the methodology be changed to suit us is not likely to succeed. It would be more productive to focus on extensive preparation for the public meeting, especially since there are opportunities for open discussion with the panelists and speakers.

There is no timeline for this Workshop available yet. Nor do we know the members of the Steering Committee. Much will depend on the evidence review conducted in advance of the meeting. The evidence based report prepared for the Gestational Diabetes meeting (pdf link) is more than 300 pages long, and distills thousands of scientific papers into recommendations for clinical practice. Will we have to worry about psychosocial research making its way into the evidence based report? Much will also depend on selection of the panelists. According to the Consensus Development Program’s website, those panelists “must not have published on or have a publicly stated opinion on the topic.” Who will those panelists be?

We may not be able to participate in this process until the public meeting, but I hope advocates will join me in monitoring how this evolves. I also hope NIH will be forthcoming about each step in the process. In my experience, the engagement between advocates and the government is more positive and productive when the advocates have access to accurate information. We have a lot to learn about this methodology, and we should adequately prepare to have the best possible influence on the process.


*My description of the conference steps is drawn from the general description of the Consensus Development Program on NIH’s website and an email from Dr. Susan Maier, Chair of the Trans-NIH ME/CFS Research Working Group. The actual process may differ from what I describe here.


This entry was posted in Advocacy, Research and tagged , , , , , , , , , . Bookmark the permalink.

13 Responses to Evidence Based at NIH

  1. Tom Kindlon says:

    Thanks for this. Very interesting.

    Another definition used is the so-called “empiric” criteria (Reeves et al., 2005). They claim to be simply an operationalization of the Fukuda criteria, but really it produces an odd group that covers a much broader population e.g. prevalence in CDC study was 2.54% compared to .235% in an earlier study where most other things were the same.

    More info on the petition site I set up: http://www.ipetitions.com/petition/empirical_defn_and_cfs_research/index.html

    Hopefully this definition is dead in terms of new studies (I expect the CDC may still publish some CFS studies they did previously using this criteria).

    • Jennie Spotila says:

      Yes, you are right. CDC seems to have backed off using the empirical definition. Do you think they have, Tom? Or is empirical still something we need to worry about?

  2. Tom Kindlon says:

    My guess is they have backed away from it. But who knows? Maybe they might spin the data they collect from the various clinicians to try to justify it? But my guess is not but it’s only a guess. Bill Reeves being moved on and then passing away means I’m not sure it has anybody to actively promote it.

    However, the CDC collected lots of data in Georgia and a lot of the studies (from what I recall) haven’t been published so I imagine more studies will come out that use it.

  3. Tina says:

    Yes, patients are the subjects in deciding case definitions, not one of the authors or the scientists. A good definition is based on research. And you can’t be the research subject and the researcher. It would have no credibility if it was heavily influenced by patients. Now, I could be wrong. But this is the way I understand it.

  4. Deleted at Mary’s request

    • Jennie Spotila says:

      Mary, I believe Suzanne will be addressing drug repurposing at the FDA meeting. This makes sense because the Association is currently funding a grant on that precise issue. Suzanne has given at least one Faster Cures presentation on drug repurposing for ME/CFS as well.

  5. Kristina says:


    Thanks for explaining the process to us. Yes it’s very odd that the panel must not have a publicly stated opinion about the topic which probably leaves out about every ME/CFS expert, most of whom have served on CFSAC at one time or another for starters.

    Is the ICC definition not in contention?

    • Jennie Spotila says:

      I don’t know which definitions will be in or out of contention. I would hope the CCC (Carruthers 2003) and ICC (Carruthers 2011) would both be considered.

  6. KaLee says:

    I agree with your hope that Carruthers 2003 and 2011 would be considered. I attended two information seminars led by him during the 1990’s. Of all the ME/CFS experts I have met, Bruce Carruthers is the one who best seemed to understand the illness (at least the illness as I’ve experienced it). So accurate were his descriptions and explanations, I wondered at first if he was ill himself. Currently in the midst of the worst “crash” in years, I have found the recently-published “International Consensus Primer for Medical Practitioners” (2012) invaluable. Carruthers is listed as both a member of the Consensus Panel and one of two “Publishers”. The other Publisher (Marj van de Sande) lives only a few blocks away from me.

  7. Sandra says:

    What about the Ramsey case definition? Isn’t that where this whole thing started, and the elephant in the room that so many people want to conveniently ignore in order to reduce this illness to “fatigue,” when in reality it is a multi-systemic neurological disease.

  8. Justin Reilly says:

    This process doesn’t sound like it is well-suited for ME. It sounds like you need to be naive to the subject of a EbMW to get on the committee. I assume this is aimed at providing objectivity in dealing with controversial diseases. The problem is that you need to be a bona fide ME expert to really understand which ME definition is best. Someone who is not familiar with the disease is probably going to totally misapprehend the disease, the surrounding political morass and the fake, supposedly “evidence-based” “science.”

    Why are they going to choose a definition based on consistent outcomes? Wouldn’t accuracy of the definition be a better goal?

    This whole thing sounds like it’s going to be the typical NIH/CDC MEcfs train wreck.

  9. Deborah Waroff says:

    Good work Jennie. Can we find out how they will put this committee together so we can try to get people on it?

Comments are closed.