What is the purpose of the ME/CFS P2P meeting at NIH? You would think that we would know by now, since Assistant Secretary Dr. Howard Koh first announced the effort in October 2012. But to say the rhetoric has evolved over time would be a kind description.
HHS keeps changing the answers to questions about the purpose of the workshop, what kind of research is on the table, and whether the ME/CFS experts have meaningful input.
To me, it looks more like a bait and switch where the meeting sounded better the further back in time you look, and key information (like the panel being 100% non-experts) was withheld until the very last minute. The reality of this meeting is very different from the picture they portrayed early on.
Are we making a research case definition or not?
First they told us the workshop would create a new case definition:
The NIH has made a commitment to conduct an evidence‐based review of the status of ME/CFS research and also convene a dedicated workshop to address the research case definition for ME/CFS. Dr. Howard Koh, October 3, 2012 CFSAC Minutes, p. 5.
To address the highest priority identified, which was “case definition issues,” the Working Group submitted a competitive application for an Evidence-based Methodology Workshop (EbMW) on ME/CFS coordinated by the NIH Office of Disease Prevention. May 1, 2013, Response from Dr. Howard Koh to CFS Advisory Committee, p. 3
Then they told us it wouldn’t:
The purpose of the Pathways to Prevention Program and the ME/CFS workshop is not —and I repeat, not—to create a new case definition for research for ME/CFS. Dr. Susan Maier, December 11, 2013 CFSAC Minutes, p. 16.
But in the middle, they said the meeting might help with a research case definition:
This will not create a research case definition in the end, but will inform anyone who wants to do research in this area about what aspects of the case definition are really strong, which are really lacking, and how those holes might be filled. Dr. Beth Collins-Sharp, May 23, 2013 CFSAC Minutes, P. 16.
But the meeting is about research, right?
The answer might depend on the day, and the person you ask. Here are the research-oriented answers:
The purpose of an evidence-based methodology workshop is to identify methodological and scientific weaknesses in a scientific area and move the field forward through the unbiased and evidence-based assessment of a very complex clinical issue. Dr. Susan Maier, May 23, 2013 CFSAC Minutes, p. 6.
The takeaways from a systematic review are answers to the key questions that identify where there’s strong evidence, where there are gaps, and some ideas about how those gaps may be filled. Those are called research recommendations. Dr. Beth Collins-Sharp, May 23, 2013 CFSAC Minutes, p. 13.
It has the potential to be both [research and clinical], but understanding that we are a research organization and our focus is to improve the, um, the integrity of the science that is used for translation into clinical care means that we have to focus on besting the science that is used for the evidence. Dr. Susan Maier, Institute of Medicine Public Meeting, January 27, 2014, Minute 0:19.
Bob Miller, who served on the P2P Working Group, certainly thinks the meeting is about research:
NIH is hosting a Pathway to Prevention workshop to identify gaps in scientific research, to guide a path forward for NIH research. Bob Miller, March 11, 2014 CFSAC Transcript, p. 114.
But at other points, it appears the focus is back on the case definition:
The purpose of the Pathways to Prevention Program for ME/CFS is to evaluate the research evidence surrounding the outcome from the use of multiple case definitions for ME/CFS and address the validity, reliability, and ability of the current case definitions to identify those individuals with or without the illness or to identify subgroups of individuals with the illness who might be reliably differentiated with the different specific case definitions. Dr. Susan Maier, December 11, 2013 CFSAC Minutes, p. 16.
Doesn’t this assessment of multiple case definitions and what research tells us about subgroups sound like what the IOM panel is doing right now? And if IOM is already doing this, why do we need a separate process at NIH where the decision makers are ALL non-ME/CFS experts?
The expert gets to decide, right?
I went back through CFSAC minutes and other documents, looking for the first time Dr. Maier or another federal employee told an ME/CFS audience that the P2P Panel would be composed entirely of non-ME/CFS experts. It was January 27, 2014 in her presentation to the Institute of Medicine, when Dr. Maier offered her ill-fated “jury model” analogy. Dr. Susan Maier, Institute of Medicine Public Meeting, January 27, 2014, Minute 6:25.
Just to be clear, the earliest public discussion of P2P was October 2012, but it wasn’t until almost 16 months later that Dr. Maier finally told us that the P2P Panel would have no ME/CFS experts on it. Why did it take so long? Maybe the better question is why January 2014. Would Dr. Maier have talked about her jury model of “They don’t know, they don’t know anything” if I had not already exposed this here on January 6, 2014? Maybe, but it strikes me as more than odd that despite at least two opportunities to tell CFSAC about the “jury model,” she waited until the IOM meeting to actually disclose it.
But the government says Don’t Worry! Your experts are participating!
The working group will meet to develop the questions that will form the basis of the evidence-based review, develop workshop themes and structures, suggest speakers, and develop an agenda for the meeting. The deliverable from this meeting will be a list of questions for the evidence review, themes for the workshop, perhaps a draft agenda, and any speaker names for those who will speak at the meeting. Dr. Susan Maier, December 11, 2013 CFSAC Minutes, p. 16-17.
Our experts and I had real input into the agenda and questions. The Working Group drove the agenda, and we will participate in the Workshop. I believe the prep work for the Workshop is being done with strong representation from our illness, laying the foundation for a good outcome. Bob Miller, January 12, 2014.
It sure sounds like that Working Group finalized the questions for the evidence review:
The Key Questions were defined by the Working Group of content experts at a planning meeting organized by the NIH Office of Disease Prevention. May 2, 2014 Email from CFSAC listserv.
There’s a problem, everybody. Multiple sources who are in a position to know what happened at the January 2014 Working Group meeting told me that the questions in the study protocol were not the questions defined at the meeting. Did something happen between that January meeting and the release of the study protocol? I don’t know whether someone continued to tinker with the questions, or why the Working Group was not consulted. But either the questions have been significantly changed, or the information from my sources is deeply flawed.
Why is this a problem? Well, in addition to all the problems I documented with the study protocol, those questions form the structure of the P2P Workshop. Those questions give us a pretty good idea of what will be on the Workshop agenda, and I will supplement that with additional exclusive information in my next blog post.









P2P Agenda Fatigue
The Agenda Documents
I obtained these two agenda documents from NIH through a FOIA request. The first is titled “DRAFT Agenda” (previously obtained by another advocate, as well) and the second is titled “Agenda Example.” Neither document is dated, but circumstantial evidence suggests that both were drafted after the January 2014 Working Group meeting.
The Draft includes a list of possible speakers, including several advocates to address the “Patient Perspective.” My name is on that list, but I have not been contacted by anyone at NIH at any time about serving in that capacity. I don’t even know who put my name forward. Whether an invitation will be extended to me remains to be seen.
The Key Questions as presented in the Draft and Example documents are likely out of date, now that the systematic review protocol has been published. The Questions from the evidence review will structure the meeting, but the agendas are important indicators of NIH’s perspective and overall approach to the meeting.
Framing With Fatigue
Both of the documents include the same description of the overview that will begin the meeting:
Just to be sure you didn’t miss it, here’s is the framing for this meeting on ME/CFS: Overwhelming fatigue or malaise as a public health problem. Not ME/CFS as a public health problem. Not post-exertional malaise as a public health problem. Not cognitive dysfunction and disability as a public health problem. To NIH, “overwhelming fatigue” is the public health problem.
This is so wrong. It ignores what we’ve been saying about our experiences for years. It ignores the science on PEM and cognitive dysfunction. It ignores the fundamental question of what disease or diseases are being included in the CFS bucket. In fact, it steps back in time to the Oxford definition: overwhelming fatigue alone.
The real public health problem is that since 1988, CFS has been a wastebasket and dumping ground for people with unexplained chronic fatigue. Some of those people have depression, anxiety, MS, and other illnesses. Some of those people have medically unexplained fatigue. Some of those people have a disease characterized by PEM and cognitive dysfunction.
To lump all of that together as a public health problem of “overwhelming fatigue” completely and utterly misses the point. It perpetuates the hand waving and blurred lines in the government’s approach to my disease, and there’s just no excuse for it at this point.
Treatment Barges In
At the May 2013 CFSAC meeting, Dr. Maier said that treatment research was part of the evidence review, but she portrayed it as relating back to the case definition question:
But now we know that the evidence review will ask about treatment harms and benefits, and the characteristics of subgroups, responders and non-responders. The agenda documents reveal what this treatment focus will look like.
The Draft Agenda focuses on tools to measure outcomes, rather than comparative effectiveness of treatments. The Agenda Example document is very different. Here are the treatment presentations from that document (each topic allocated 20 minutes):
So we have an evidence review that lumps all the case definitions together, including Oxford. And we have an agenda that gives more time to CBT and GET than it does to symptom-based medication. And there is nothing here on disease-modifying treatment, like rituximab, Ampligen, or antivirals.
The topic selection and allocation of time among these treatment topics sends a subtle but powerful message to the Panel of non-ME/CFS experts, especially in light of the failure to distinguish among the case definitions at the outset. Previous evidence reviews, including AHRQ’s review from 2001 and the Brurberg, et al review published in February found no significant differences among case definitions or treatment outcomes, but those reviews were not set up to critically examine those differences. And as I’ve already pointed out, this current review assumes that differences among definitions represent subtypes and not separate diseases.
Design Flaws
The agenda documents show that the P2P Workshop is fundamentally flawed. The meeting is framed with the public health problem of “overwhelming fatigue.” The evidence review will include studies on adults with fatigue, and exclude those with unspecified underlying diagnoses. All the case definitions are lumped together for the purposes of assessing the reliability of those definitions and the effectiveness of treatments.
The evidence review and meeting agenda should begin with the proper scientific question: are ME and CFS the same disease, separate diseases, or points along a spectrum of fatiguing illnesses? That was the original starting question in the AHRQ evidence review contract, by the way. But it’s gone. The decision was made (not sure by whom) to assume the answer: that it is all one disease, separated only by subgroups. That assumption is the fatal flaw in this entire enterprise.
Remember that the P2P outcome will be decided by a panel of non-ME/CFS experts. We don’t know how they will be screened for bias. We won’t know who they are until shortly before the meeting. We will have no input into their selection.
This is not good science. This is sloppy, not precise. To revisit Dr. Maier’s jury analogy: this process will ask the allegedly impartial jury (selected by only one side) to reach conclusions based on evidence that has been marred by bias and assumptions. Maybe they will reach the right conclusions, or maybe the deck is stacked against us.
We have to find ways to speak out about this. I’m working on something right now, and there will be ways for you to express your own concerns. I hope you will join me.