The final P2P report is published, and now it’s time to evaluate the quality of the recommendations and how well the process served ME/CFS patients. There are many good things in the P2P report, and I’ll be focusing on those in my next post (now available here). Today, I want to discuss some serious flaws in the report and how, despite all the rhetoric, NIH and the P2P Panel did not engage with the public in a meaningful way. And I also want to discuss a lesson we can apply next time.
All That Input
At the telebriefing on Tuesday, June 16th, Dr. Green said the Panel members individually reviewed each of the public comments and discussed them in a conference call. Unfortunately, the Panel did not take most of our advice. The Panelists may have reviewed each comment, but only minor changes were made to the draft report in response to those comments.
For example, the final report appropriately deleted the sentence, “There is no agreement from the research community on what needs to be studied.” (Draft, lines 34-35) The statement that ME/CFS “overlaps with many other diseases” was also deleted. (Draft, line 33) The need to disseminate diagnostic and treatment recommendations to primary care providers was expanded, as many public comments suggested, to include specialties like neurology, rheumatology and infectious disease. (Draft, line 192) Other minor corrections were made, such as clarifying that Fukuda allows 163 combinations of symptoms, not that there are 163 symptoms of ME/CFS. (Draft, lines 50-51)
But the vast majority of comments were ignored. Even the statement that ME/CFS has an economic impact greater than $1 billion (Draft, lines 6-7) was not fully corrected. I lost count of the number of public comments that provided the correct number of more than $20 billion in direct and indirect costs. However, the final version estimates the burden at $2 to 7 billion, and does not clarify whether those are direct or indirect costs. When journalist Miriam Tucker asked for the reference for this estimate, Dr. Green said they used a number that was “scientifically based,” but she did not provide a reference.
But these are minor points compared to the failure to incorporate the major issues highlighted in multiple public comments. All the input on the critical points that most need attention was ignored.
The Money Elephant
The issue that receive more attention in public comment than any other was NIH funding. Comment after comment begged the Panel to acknowledge the paucity of funding and to recommend that NIH significantly increase its investment in ME/CFS research.
The IACFS/ME said, “[We] believe that the elephant in the room – research funding . . . needs to be addressed more strongly and specifically. . . . Thus we respectfully ask that the NIH Panel highlight the inadequate research funding of ME/CFS and link this core premise to specific recommendations for new funding initiatives, with dollar amounts, mechanisms, and deadlines, to begin to address the current underfunded status of this illness.”
The CFS Advisory Committee said, “We also ask that you take note of the fact that among the 234 disease categories supported by NIH in 2014, chronic fatigue syndrome ranked 228th with an estimated $5 million in funding. In order to move forward, it is vital that this issue be addressed. We ask that the Panel explicitly address the urgent need for government funding in order to advance the research for ME/CFS.”
The Massachussetts CFIDS/ME & FM Association said, “All of these recommendations for future directions in research require significant funding . . . NIH has never funded this illness commensurate with its impact on patients and society. While it may be possible to cobble together small amounts of funding from different agencies for some specific initiatives . . . why not just recognize the inequality here and deal with it? The types of research recommended here cannot be done on nickels and dimes.”
I could go on. Over and over and over, public comments begged the Panel to recommend an increase in NIH funding. If the Panel did indeed read the hundreds of pages of comments we submitted, they could not possibly have missed the blaring consensus on the urgent need to add zeroes to the amount of NIH’s ME/CFS investment. Despite all of this, the Panel made no such recommendation. I have no doubt that some people will conclude that NIH told the Panel they could not make the recommendation.
In fact, Dr. Green told Medscape News, “Money can be reallocated or better targeted.” That’s quite explicitly not an increase in funding, folks.
Eating the Oxford Cake
I blogged about the moment at the P2P Workshop when the Evidence Practice Center suggested retiring the Oxford definition, and the illogical reply to my question about retiring Oxford studies. Chris Heppner made a powerful case for declaring independence from the Oxford definition once and for all. But not only did the Panel’s final report fail to take that logical step, it actually toned down its language on the Oxford definition.
The draft report stated, “The Oxford criteria . . . are flawed and include people with other conditions, confounding the ability to interpret the science.” (Draft, lines 38-40) That sentence is gone from the final version. We are left with the recommendation from the Draft, unchanged in the final version: “Specifically, continuing to use the Oxford definition may impair progress and cause harm. Thus, for needed progress to occur, we recommend (1) that the Oxford definition be retired . . ” (Final, p. 16)
Personally, I am very disturbed by the deletion of the statement that Oxford includes people with other conditions. Why? Because the data establish that this is the case. The Panel should have retained their acknowledgement of the data. For the Panel to step back from acknowledging the facts strikes me as ominous, because obviously it helps ME/CFS advocates to have such an explicit declaration in the report and its removal makes no sense in light of the data. It will certainly raise questions in some minds about whether the Panel was pressured to remove the sentence.
I am also not satisfied with the recommendation to retire the Oxford definition without an accompanying recommendation to retire Oxford studies as applied to the ME/CFS population. Why? Because the largest trial of CBT and GET, the trial that is the basis for the claim that these therapies are effective treatments for ME/CFS, uses the Oxford definition. And that study is continually trotted out, including in the Annals treatment article this week, and no one will acknowledge the very basic logical fact that the study was done on a cohort that included an unknown number of people who did not meet criteria for ME/CFS (not Fukuda, not CCC, not ICC, and not SEID).
Multiple public comments, including mine, made this point to the Panel. The bottom line is, “You cannot take the results of studies done on subjects who do not have ME/CFS and simply apply those results to those that do.”
For some reason, the Panel did not agree with all of the comments making this point. Why? One possible reason is that if the PACE trial and similar Oxford “treatment” studies are discarded, then there is almost nothing left. This is especially true for the systematic evidence review, which excluded studies like the Rituximab trial. It would mean admitting that thirty years of research has not produced an effective treatment for ME/CFS, and that the only promising treatments are extraordinarily expensive. That’s a damning fact and quite an embarrassment for NIH and the research enterprise.
Not Our Wheelhouse
Multiple public comments urged the Panel to consider the IOM report while revising their recommendations. Even Dr. Nancy Lee, newly re-appointed DFO of the CFS Advisory Committee, asked them to discuss it.
Dr. Lee emailed her comments the day after the posting of the Draft Report. I think it’s worth quoting Dr. Lee at length: “To avoid the mistaken perception that NIH and the P2P Panel didn’t know about the IOM study (which NIH helped fund!), I urge you to mention the IOM study in your report. The discussion on lines 201-212 should be modified to acknowledge that the IOM’s report may make substantial progress in fulfilling P2P recommendation. There is a good chance that the IOM report will be released before the Panel’s report so you might even be able to modify your report at the last minute to reflect the content of the IOM’s report. HHS spent $1million on the IOM study to get much of what your panel recommends on line 202. I don’t want it to appear that NIH’s right hand doesn’t know what its left hand is doing!”
Despite the many requests, the Panel wrote that considering and incorporating the IOM’s recommendations was “beyond our scope and charge.” (Final, p. 18)
The charge to the P2P Panel was to provide “guidance to the NIH on research gaps and research priorities for ME/CFS.” (Final, p. 18) The final report notes the critical importance of case definition in ME/CFS research, and urges that a consensus be reached on a single case definition. (Final, pp. 9, 16) So how is the IOM report beyond the scope and charge of the P2P Panel?
I think the Panel members would agree that science requires examining all the data. My personal belief is that the phrase “beyond our scope and charge” is actually code for “we were not given the time and staff support to fully evaluate the IOM report.” Was it a matter of NIH putting its P2P process (evidence review + workshop = report) over the quality of the product? Was there some other reason NIH did not ask the Panel to review and consider the IOM report? I do not know.
Talking At The Back Of Their Heads
By now, we are all familiar with the NIH party line that ME/CFS patients and advocates were fully engaged by P2P. At the telebriefing, Dr. Ronit Elk said that patients and families were “completely a part of the process.” I won’t review all the evidence to the contrary from the last 18 months.
What I will do is point out that the “tremendous number of responses,” as Dr. Elk described it, seems to have had very little impact on the revisions to the draft report. Despite the extra time given the Panel, despite Dr. Green’s statement that each panelist read every single comment, despite the assurances from NIH that patients and advocates were integrated at every step, our comments were not accepted.
Many advocates might think that the dismissal of these comments prove that public comment is a waste of time. We comment, they don’t listen to us, so why bother? I say that’s a response that plays right into the government’s hands. This defeatist reaction has it all backwards.
Over 100 people and organizations submitted comment on the P2P report, and they were ignored. That gives us another link in the Thirty Years of Neglect Chain. We can embarrass them with this fact. We can tell the media, our Congressmen, CFSAC, and others that over 100 people spoke and we were dismissed and ignored. That’s the level of contempt NIH has for our input.
Dialing back our participation is exactly what they want us to do. They want us to go away. They think that if they just keep ignoring us, we will get frustrated and lose interest.
We should do the opposite. We should say, You ignored 100 comments? Let’s see you ignore 200 or 500 or 1,000. The more comments that go in, and the more they ignore them, the more ridiculous the situation becomes. That is a political weapon!
So yes, 100+ comments was not enough to get a stronger recommendation on funding into this report. There is still plenty here we can use. And we have one more incident of Government Ignores Constructive Response From Patient Community to add to the list. They are practically begging us to turn up the volume and intensity.
NIH is showing us the back of their heads. Ok, then we have to be louder. NIH is ignoring us. Then let’s be more forceful. Let’s increase the volume, increase the intensity, and build an ever growing wave of actions that will ultimately overwhelm resistance and swamp them. They ignore us because they can. So let’s make it impossible for them to do so.