Listen carefully to the agency updates at the June 16th CFS Advisory Committee meeting. Updates on the P2P Workshop and systematic evidence review may sound reasonable, but I suspect that hidden within will be some spin-doctoring to deflect our concerns about (and criticisms of) the P2P Workshop. So let’s just get it out of the way right now. Here are some possible spin arguments, and my responses.
The P2P Workshop is responsive to the CFS Advisory Committee recommendation to hold a meeting on case definition.
No, it is not. The recommendation was for a meeting to devise a clinical and research case definition. NIH has said that the P2P Workshop is expressly NOT devising a case definition.
ME/CFS experts are participating in every phase of the P2P Workshop.
Every phase except the one that really matters, and the expert participation is marginal. For example, there are twenty people on the P2P Working Group (that met in January to plan the meeting), but only five of them are ME/CFS experts. The full Working Group only meets once, if it follows the pattern of other P2P meetings. We don’t know how many experts are on the Technical Expert Panel (which helped with the protocol for the evidence review). The experts who will speak at the Workshop? Yeah, we don’t know who they are either. And the group that really matters is the P2P Panel and by design, NO experts whatsoever will be on that Panel. Ask yourself: would you have accepted an IOM committee with no ME/CFS experts on it? Then why is it ok for P2P?
Don’t worry, the P2P Panel of non-ME/CFS experts is just like a jury.
No, actually, it is nothing like a jury. For one thing, juries are selected in public, with both sides of a case having the opportunity to question potential jurors and move for their disqualification. The P2P Panel? Selected by NIH, behind closed doors, with no transparency for how they are screened for bias or preconceptions about ME/CFS. That’s like letting the prosecution choose a jury before anyone walks into the courtroom. Furthermore, this “jury” will make decisions based on an evidence base and field that is controversial, confusing, and laden with bias. No jury would ever be cast adrift in such murky waters without multiple layers of guidance and experts to assist them.
The systematic evidence review is the best methodology out there.
This may be true, although there is certainly controversy over the value of evidence reviews. But the issue is not the methodology. The issue is the study protocol that will be used to apply that methodology. I’ve already documented the multiple concerns with the protocol, beginning with the failure to ask if the case definitions capture more than one disease. Beyond that, remember that the evidence review is the PRIMARY source of information for the Panel of non-ME/CFS experts. If the review doesn’t ask the right questions, then there is little chance that the Panel will do so at the Workshop.
The P2P Workshop will cover more than the Key Questions as stated in the evidence review protocol.
I’m hearing this already from several sources, and I have three objections. First, the draft agenda documents follow the evidence review questions very closely. And if you compare the key questions and agenda for the P2P Opioid meeting, you will see that they match. Second, even if it is true that the Workshop will cover additional questions – where are those questions? Why is there no transparency about the content of the Workshop? If there are additional topics being covered, then NIH should release that information immediately. Third, if the evidence review does not include some of the questions, then how can the Workshop adequately cover those questions in the absence of the evidence report?
The Workshop will cover ME/CFS, not medically unexplained fatigue.
Then why is the evidence review covering studies on patients with fatigue and no underlying diagnosis? Why does the draft meeting agenda describe the overview section as “Overwhelming fatigue and malaise as a public health problem”?
Our experts and the public will be able to refute anything that goes wrong in the systematic evidence report.
The calendar matters here. The evidence review, conducted and written by a contractor with no ME/CFS expertise, will be published in draft form six weeks prior to the Workshop. At the same time, the contractor will present the evidence review to the P2P Panel (also with no ME/CFS expertise) in a three hour closed door meeting. The two groups of non-experts will discuss the evidence report. Not a single expert will be in the room to point out anything that went wrong with the review. So technically, yes, the public can comment on the draft evidence report. But in reality? The non-expert Panel may never see those comments and even if they do, they will assess the comments after spending three hours discussing the non-expert report with other non-experts.
Our experts will be able to provide all the necessary information at the Workshop meeting.
The speakers may have been suggested by the Working Group (only 25% of whom were ME/CFS experts), but the final agenda and speakers are selected by NIH. Each speaker has 20 minutes to talk about their assigned topic. There is time for discussion on the agenda, but we don’t know how it will work. Will the non-expert Panel control discussion with questions? Will there be an open mike that gives experts and patients an opportunity to point out everything that is wrong with the evidence report? We don’t know. Remember, the non-expert Panel takes the evidence report (which they review for 6 weeks) and the Workshop (which lasts 1.5 days and is controlled by NIH), and then they have to turn all that information around and in 24 hours produce a report with recommendations on the future directions of research, case definition, and possibly treatments.
To put this challenge in context, remember the 2011 State of the Knowledge meeting? That was a two day meeting and all the presenters were ME/CFS experts. The summary of research gaps and opportunities was presented by Dr. Suzanne Vernon with the Director of NIH, Dr. Francis Collins, in the room. And what has happened at NIH for ME/CFS research since then? An RFA? Nope. A strategic plan? Nope. Engagement with stakeholders, increase in funding, fair grant review? If a two day meeting of ME/CFS experts that provided input directly to the Director of NIH could not fix any of these problems, then a panel of non-ME/CFS experts using a flawed evidence review and 1.5 days of presentations of experts and non-experts combined will be facing a very steep climb to deliver a strong set of recommendations after 24 hours of discussion.
We should be glad NIH is paying this much attention to ME/CFS.
So any attention is good attention? No. In this case, attention in the form of bad key questions and the framing of the problem as “overwhelming fatigue or malaise” could be very damaging and counter-productive. The conclusions and report of the Panel of non-ME/CFS experts will shape NIH policy. NIH will put a media relations push behind the report. We will be stuck with it for some time to come.
Opposition to P2P is just nay saying and complaining.
As I’ve extensively documented here and in the letter to Dr. Collins, there are legitimate concerns that this process will not produce recommendations that move the field forward with the best of what science can offer. To dismiss these concerns as complaints or negativity is, quite frankly, disrespectful of the advocates who have taken the time to express them. Give my concerns a fair hearing. Then let’s talk about whether the P2P Workshop is the best way to collaborate with the ME/CFS research and clinical community. Anyone who thinks we’re whining or overreacting has not actually read the material submitted to Dr. Collins.
Don’t worry. We’re NIH and we know what we’re doing.
Even the most qualified scientist can get it wrong. Even the purest motivations can lead to mistakes. I’ve written before about the problems of mistrust and disrespect in the ME/CFS landscape. NIH should not tell us to just trust them. NIH should be demonstrating that it is safe and appropriate for us to trust them. Based on what I’ve documented about P2P thus far, they have not done so.
P2P Agenda Fatigue
The Agenda Documents
I obtained these two agenda documents from NIH through a FOIA request. The first is titled “DRAFT Agenda” (previously obtained by another advocate, as well) and the second is titled “Agenda Example.” Neither document is dated, but circumstantial evidence suggests that both were drafted after the January 2014 Working Group meeting.
The Draft includes a list of possible speakers, including several advocates to address the “Patient Perspective.” My name is on that list, but I have not been contacted by anyone at NIH at any time about serving in that capacity. I don’t even know who put my name forward. Whether an invitation will be extended to me remains to be seen.
The Key Questions as presented in the Draft and Example documents are likely out of date, now that the systematic review protocol has been published. The Questions from the evidence review will structure the meeting, but the agendas are important indicators of NIH’s perspective and overall approach to the meeting.
Framing With Fatigue
Both of the documents include the same description of the overview that will begin the meeting:
Just to be sure you didn’t miss it, here’s is the framing for this meeting on ME/CFS: Overwhelming fatigue or malaise as a public health problem. Not ME/CFS as a public health problem. Not post-exertional malaise as a public health problem. Not cognitive dysfunction and disability as a public health problem. To NIH, “overwhelming fatigue” is the public health problem.
This is so wrong. It ignores what we’ve been saying about our experiences for years. It ignores the science on PEM and cognitive dysfunction. It ignores the fundamental question of what disease or diseases are being included in the CFS bucket. In fact, it steps back in time to the Oxford definition: overwhelming fatigue alone.
The real public health problem is that since 1988, CFS has been a wastebasket and dumping ground for people with unexplained chronic fatigue. Some of those people have depression, anxiety, MS, and other illnesses. Some of those people have medically unexplained fatigue. Some of those people have a disease characterized by PEM and cognitive dysfunction.
To lump all of that together as a public health problem of “overwhelming fatigue” completely and utterly misses the point. It perpetuates the hand waving and blurred lines in the government’s approach to my disease, and there’s just no excuse for it at this point.
Treatment Barges In
At the May 2013 CFSAC meeting, Dr. Maier said that treatment research was part of the evidence review, but she portrayed it as relating back to the case definition question:
But now we know that the evidence review will ask about treatment harms and benefits, and the characteristics of subgroups, responders and non-responders. The agenda documents reveal what this treatment focus will look like.
The Draft Agenda focuses on tools to measure outcomes, rather than comparative effectiveness of treatments. The Agenda Example document is very different. Here are the treatment presentations from that document (each topic allocated 20 minutes):
So we have an evidence review that lumps all the case definitions together, including Oxford. And we have an agenda that gives more time to CBT and GET than it does to symptom-based medication. And there is nothing here on disease-modifying treatment, like rituximab, Ampligen, or antivirals.
The topic selection and allocation of time among these treatment topics sends a subtle but powerful message to the Panel of non-ME/CFS experts, especially in light of the failure to distinguish among the case definitions at the outset. Previous evidence reviews, including AHRQ’s review from 2001 and the Brurberg, et al review published in February found no significant differences among case definitions or treatment outcomes, but those reviews were not set up to critically examine those differences. And as I’ve already pointed out, this current review assumes that differences among definitions represent subtypes and not separate diseases.
Design Flaws
The agenda documents show that the P2P Workshop is fundamentally flawed. The meeting is framed with the public health problem of “overwhelming fatigue.” The evidence review will include studies on adults with fatigue, and exclude those with unspecified underlying diagnoses. All the case definitions are lumped together for the purposes of assessing the reliability of those definitions and the effectiveness of treatments.
The evidence review and meeting agenda should begin with the proper scientific question: are ME and CFS the same disease, separate diseases, or points along a spectrum of fatiguing illnesses? That was the original starting question in the AHRQ evidence review contract, by the way. But it’s gone. The decision was made (not sure by whom) to assume the answer: that it is all one disease, separated only by subgroups. That assumption is the fatal flaw in this entire enterprise.
Remember that the P2P outcome will be decided by a panel of non-ME/CFS experts. We don’t know how they will be screened for bias. We won’t know who they are until shortly before the meeting. We will have no input into their selection.
This is not good science. This is sloppy, not precise. To revisit Dr. Maier’s jury analogy: this process will ask the allegedly impartial jury (selected by only one side) to reach conclusions based on evidence that has been marred by bias and assumptions. Maybe they will reach the right conclusions, or maybe the deck is stacked against us.
We have to find ways to speak out about this. I’m working on something right now, and there will be ways for you to express your own concerns. I hope you will join me.