Listen carefully to the agency updates at the June 16th CFS Advisory Committee meeting. Updates on the P2P Workshop and systematic evidence review may sound reasonable, but I suspect that hidden within will be some spin-doctoring to deflect our concerns about (and criticisms of) the P2P Workshop. So let’s just get it out of the way right now. Here are some possible spin arguments, and my responses.
The P2P Workshop is responsive to the CFS Advisory Committee recommendation to hold a meeting on case definition.
No, it is not. The recommendation was for a meeting to devise a clinical and research case definition. NIH has said that the P2P Workshop is expressly NOT devising a case definition.
ME/CFS experts are participating in every phase of the P2P Workshop.
Every phase except the one that really matters, and the expert participation is marginal. For example, there are twenty people on the P2P Working Group (that met in January to plan the meeting), but only five of them are ME/CFS experts. The full Working Group only meets once, if it follows the pattern of other P2P meetings. We don’t know how many experts are on the Technical Expert Panel (which helped with the protocol for the evidence review). The experts who will speak at the Workshop? Yeah, we don’t know who they are either. And the group that really matters is the P2P Panel and by design, NO experts whatsoever will be on that Panel. Ask yourself: would you have accepted an IOM committee with no ME/CFS experts on it? Then why is it ok for P2P?
Don’t worry, the P2P Panel of non-ME/CFS experts is just like a jury.
No, actually, it is nothing like a jury. For one thing, juries are selected in public, with both sides of a case having the opportunity to question potential jurors and move for their disqualification. The P2P Panel? Selected by NIH, behind closed doors, with no transparency for how they are screened for bias or preconceptions about ME/CFS. That’s like letting the prosecution choose a jury before anyone walks into the courtroom. Furthermore, this “jury” will make decisions based on an evidence base and field that is controversial, confusing, and laden with bias. No jury would ever be cast adrift in such murky waters without multiple layers of guidance and experts to assist them.
The systematic evidence review is the best methodology out there.
This may be true, although there is certainly controversy over the value of evidence reviews. But the issue is not the methodology. The issue is the study protocol that will be used to apply that methodology. I’ve already documented the multiple concerns with the protocol, beginning with the failure to ask if the case definitions capture more than one disease. Beyond that, remember that the evidence review is the PRIMARY source of information for the Panel of non-ME/CFS experts. If the review doesn’t ask the right questions, then there is little chance that the Panel will do so at the Workshop.
The P2P Workshop will cover more than the Key Questions as stated in the evidence review protocol.
I’m hearing this already from several sources, and I have three objections. First, the draft agenda documents follow the evidence review questions very closely. And if you compare the key questions and agenda for the P2P Opioid meeting, you will see that they match. Second, even if it is true that the Workshop will cover additional questions – where are those questions? Why is there no transparency about the content of the Workshop? If there are additional topics being covered, then NIH should release that information immediately. Third, if the evidence review does not include some of the questions, then how can the Workshop adequately cover those questions in the absence of the evidence report?
The Workshop will cover ME/CFS, not medically unexplained fatigue.
Then why is the evidence review covering studies on patients with fatigue and no underlying diagnosis? Why does the draft meeting agenda describe the overview section as “Overwhelming fatigue and malaise as a public health problem”?
Our experts and the public will be able to refute anything that goes wrong in the systematic evidence report.
The calendar matters here. The evidence review, conducted and written by a contractor with no ME/CFS expertise, will be published in draft form six weeks prior to the Workshop. At the same time, the contractor will present the evidence review to the P2P Panel (also with no ME/CFS expertise) in a three hour closed door meeting. The two groups of non-experts will discuss the evidence report. Not a single expert will be in the room to point out anything that went wrong with the review. So technically, yes, the public can comment on the draft evidence report. But in reality? The non-expert Panel may never see those comments and even if they do, they will assess the comments after spending three hours discussing the non-expert report with other non-experts.
The speakers may have been suggested by the Working Group (only 25% of whom were ME/CFS experts), but the final agenda and speakers are selected by NIH. Each speaker has 20 minutes to talk about their assigned topic. There is time for discussion on the agenda, but we don’t know how it will work. Will the non-expert Panel control discussion with questions? Will there be an open mike that gives experts and patients an opportunity to point out everything that is wrong with the evidence report? We don’t know. Remember, the non-expert Panel takes the evidence report (which they review for 6 weeks) and the Workshop (which lasts 1.5 days and is controlled by NIH), and then they have to turn all that information around and in 24 hours produce a report with recommendations on the future directions of research, case definition, and possibly treatments.
To put this challenge in context, remember the 2011 State of the Knowledge meeting? That was a two day meeting and all the presenters were ME/CFS experts. The summary of research gaps and opportunities was presented by Dr. Suzanne Vernon with the Director of NIH, Dr. Francis Collins, in the room. And what has happened at NIH for ME/CFS research since then? An RFA? Nope. A strategic plan? Nope. Engagement with stakeholders, increase in funding, fair grant review? If a two day meeting of ME/CFS experts that provided input directly to the Director of NIH could not fix any of these problems, then a panel of non-ME/CFS experts using a flawed evidence review and 1.5 days of presentations of experts and non-experts combined will be facing a very steep climb to deliver a strong set of recommendations after 24 hours of discussion.
We should be glad NIH is paying this much attention to ME/CFS.
So any attention is good attention? No. In this case, attention in the form of bad key questions and the framing of the problem as “overwhelming fatigue or malaise” could be very damaging and counter-productive. The conclusions and report of the Panel of non-ME/CFS experts will shape NIH policy. NIH will put a media relations push behind the report. We will be stuck with it for some time to come.
Opposition to P2P is just nay saying and complaining.
As I’ve extensively documented here and in the letter to Dr. Collins, there are legitimate concerns that this process will not produce recommendations that move the field forward with the best of what science can offer. To dismiss these concerns as complaints or negativity is, quite frankly, disrespectful of the advocates who have taken the time to express them. Give my concerns a fair hearing. Then let’s talk about whether the P2P Workshop is the best way to collaborate with the ME/CFS research and clinical community. Anyone who thinks we’re whining or overreacting has not actually read the material submitted to Dr. Collins.
Don’t worry. We’re NIH and we know what we’re doing.
Even the most qualified scientist can get it wrong. Even the purest motivations can lead to mistakes. I’ve written before about the problems of mistrust and disrespect in the ME/CFS landscape. NIH should not tell us to just trust them. NIH should be demonstrating that it is safe and appropriate for us to trust them. Based on what I’ve documented about P2P thus far, they have not done so.