This post comes via Mary Dimmock, Claudia Goodell, Denise Lopez-Majano, and myself. You are welcome to publish it on your site with attribution and a link back to this post. You are also welcome to use this (and other material we’ve gathered) as a framework for your own comments on the draft evidence review – due October 20th.
It’s been a challenging few weeks, digesting and analyzing the AHRQ Draft Systematic Evidence Review on Diagnosis and Treatment of ME/CFS. We continue to be deeply concerned about the many flaws in the review, in terms of both the approach it took and how it applied the study protocol.
Our comments on the Review will reflect our significant concerns about how the Evidence Review was conducted, the diagnostic, subgroup, and harms treatment conclusions drawn by this report, and the risk of undue harm that this report creates for patients with ME. We believe a final version should not be published until these scientific issues are resolved.
Most fundamentally, the Evidence Review is grounded in the flawed assumption that eight CFS and ME definitions all represent the same group of patients that are appropriately studied and treated as a single entity or group of closely related entities. Guided by that assumption, this Evidence Review draws conclusions on subgroups, diagnostics, treatments and harms for all CFS and ME patients based on studies done in any of these eight definitions. In doing so, the Evidence Review disregards its own concerns, as well as the substantial body of evidence that these definitions do not all represent the same disease and that the ME definitions are associated with distinguishing biological pathologies. It is unscientific, illogical and risky to lump disparate patients together without regard to substantive differences in their underlying conditions.
Compounding this flawed assumption are the a priori choices in the Review Protocol that focused on a more narrow set of questions than originally planned and that applied restrictive inclusion and exclusion criteria. As a result, evidence that would have refuted the flawed starting assumption or that was required to accurately answer the questions was never considered. Some examples of how these assumptions and protocol choices negatively impacted this Evidence Review include:
- Evidence about the significant differences in patient populations and in the unreliability and inaccuracy of some of these definitions was ignored and/or dismissed. This includes: Dr. Leonard Jason’s work undermining the Reeves Empirical definition; a study that shows the instability of the Fukuda definition over time in the same patients; studies demonstrating that Fukuda and Reeves encompass different populations; and differences in inclusion and exclusion criteria, especially regarding PEM and psychological disorders.
- Diagnostic methods were assessed without first establishing a valid reference standard. Since there is no gold reference standard, each definition was allowed to stand as its own reference standard without demonstrating it was a valid reference.
- Critical biomarker and cardiopulmonary studies, some of which are in clinical use today, were ignored because they were judged to be intended to address etiology, regardless of the importance of the data. This included most of Dr. Snell’s and Dr. Keller’s work on two day CPET, Dr. Cook’s functional imaging studies, Dr. Gordon Broderick’s systems networking studies, Dr. Klimas’s and Dr. Fletcher’s work on NK cells and immune function, and all of the autonomic tests. None of it was considered.
- Treatment outcomes associated with all symptoms except fatigue were disregarded, potentially resulting in a slanted view of treatment effectiveness and harm. This decision excluded Dr. Lerner’s antiviral work, as well as entire classes of pain medications, antidepressants, anti-inflammatories, immune modulators, sleep treatments and more. If the treatment study looked at changes in objective measures like cardiac function or viral titers, it was excluded. If the treatment study looked at outcomes for a symptom other than fatigue, it was excluded.
- Treatment trials that were shorter than 12 weeks were excluded, even if the treatment duration was therapeutically appropriate. The big exclusion here was the rituximab trial; despite following patients for 12 months, it was excluded because administration of rituximab was not continuous for 12 weeks (even though rituximab is not approved for 12 weeks continuous administration in ANY disease). Many other medication trials were also excluded for not meeting the 12 week mark.
- Counseling and CBT treatment trials were inappropriately pooled without regard for the vast differences in therapeutic intent across these trials. This meant that CBT treatments aimed at correcting false illness beliefs were lumped together with pacing and supportive counseling studies, and treated as equivalent.
- Conclusions about treatment effects and harms failed to consider what is known about ME and its likely response to the therapies being recommended. This means that the PACE (an Oxford study) results for CBT and GET were not only accepted (despite the many flaws in those data), but were determined to be broadly applicable to people meeting any of the case definitions. Data on the abnormal physiological response to exercise in ME patients were excluded, and so the Review did not conclude that CBT and GET could be harmful to these patients (although it did allow it might be possible).
- The Evidence Review states that its findings are applicable to all patients meeting any CFS or ME definition, regardless of the case definition used in a particular study.
The issues with this Evidence Review are substantial in number, magnitude and extent. At its root is the assumption that any case definition is as good as the rest, and that studies done on one patient population are applicable to every other patient population, despite the significant and objective differences among these patients. The failure to differentiate between patients with the symptom of subjective unexplained fatigue on the one hand, and objective immunological, neurological and metabolic dysfunction on the other, calls into question the entire Evidence Review and all conclusions made about diagnostic methods, the nature of this disease and its subgroups, the benefits and harms of treatment, and the future directions for research.
As the Evidence Review states, the final version of this report may be used in the development of clinical practice guidelines or as a basis for reimbursement and coverage policies. It will also be used in the P2P Workshop and in driving NIH’s research strategy. Given the likelihood of those uses and the Evidence Review’s claim of broad applicability to all CFS and ME patients, the flaws within this report create an undue risk of significant harm to patients with ME and will likely confound research for years to come. These issues must be addressed before this Evidence Review is issued in its final form.
I’m sorry I con’t have time to write a good comment, but:
Are you sure about Rituximab being excluded because TREATMENT was less than 12 weeks, or because primary end-point was less than 12 weeks? I see two Rituximab studies: NCT01156909 and NCT00848692. The first has strange duration of primary end point, but seems to be looking at 6 week periods. The second looked at result after 3 months. Maybe they required effect to be seen at longer than 3 months, because they were excluding transitory effects in a disease with a lot of transitory effects? Both trials looked at secondary end points for much longer, but it might be that only the primary end point was considered for duration. –GKGK
The rituximab specifically listed in Appendix D with other excluded studies followed patients for twelve months, but the exclusion code says it was excluded for duration. No other explanation is given, and no mention or distinction is made in the report for any study’s primary or secondary endpoints.
Thank you for all your hard work, Jennie and Co. It is obvious that this P2P draft review is a major failure. I wonder how satisfied HHS/NIH is with it, being that they just spent $300,000 for a review that is meaningless…..or was that their intention to begin with? Was their intention to make a laughing stock out of ME/CFS? Being that they are the ones who chose this specific process of P2P for this study and they are the ones who designed the parameters for it and controlled the questions to be studied, doesn’t it follow that they pretty much knew where this was going?
If they didn’t know, then why not? They have been working with, funding and studying this illness for the past 30 years. They know exactly what is out there. If that is true, why go through this charade?
In the past 30 years, they have spent a pitiful amount in studying ME/CFS and a majority are psychologically based. Now, they are reviewing these studies based on any and all criteria out there with the singular emphasis on “fatigue”. What did they expect the outcome to show?
It is obvious that when they have stated, repeatedly, at CFSAC meetings how they know that this is a very disabling disease and that Chronic Fatigue syndrome is not an appropriate name because it is so much more than ‘fatigue” that those were empty words.
I propose that their intention is to further marginalize this disease into oblivion. This review will come to the conclusion that there is no evidence that ME/CFS is a biologically based disease and is rather an illness of “false belief”. They will recommend CBT and GET since these are the only treatments that show any progress at all.
This will be used for medical insurance and disability purposes and HHS will claim that it’s not their fault, it’s the result of an intensive evidence based review.
I think that we should hold HHS accountable for this corruption of power by contracting a process in a way that it has been directed to a result that will bring harm to patients.
i just want to say that you girls are AMAZING!! what an incredible job you’re doing! this is super helpful, and infinitely appreciated. i’ll be sending in comments too (very simple ones, based on yours + my own thoughts). will keep signal boosting your posts, keep on trying to get more people to open their eyes and get involved. strength and courage! <3 <3 <3
Thank you!!!!
I am working on comments too, so this is extremely helpful! Thank You!
I cannot begin to express how grateful I am for your brains and efforts! I will submit comment, and will base much of what I say on your summary. Thank you!!!
Thank you, Jane! The team worked really hard.
@Gabby
I N D E E D ! The name of the game and here are the results — all crafted by design within the set-up parameters — corruption and abuse of power — gross negligence to perpetuate a debilitating pandemic — destroying the lives of MILLIONS men, women and children. CRIMINAL!
P.S.
Therefore, this AHRQ Draft Systematic Evidence Review on Diagnosis of Treatment of ME/CFS is an outright LIE! And this is OUTRAGEOUS!
Pingback: Comments on Evidence Review due Oct 20th | Speak Up About ME
Just a question for those submitting comments, are you copying anyone on your report, and if yes who?
As always, THANK YOU to all who do so much…
(Corporate personhood. Corporate sociopath? DSM-IV criteria for sociopath includes: Repeated acts that could lead to arrest; Conning for pleasure or profit, repeated lying; Repeated assaults on others; Reckless when it comes to others safety; Poor work behavior or failure to honor financial obligations; Rationalizing the pain they inflict on others.)
@Ren
Oh, yes–Corporate Sociopathic behaviours–well-defined!
Pingback: Evidence Review Comments Due Monday October 20th | Speak Up About ME
The patient community cannot thank you all enough for your dedication to us and to a cure. Please believe each of you responsible for this work are loved and appreciated <3
Thank you so much, Sandy. It was a long month for all of us.
Thank you to all of you in your efforts to bring clarity to us. I will comment with these points as the basis. Best to all.
Thank you, Valerie!
I have emailed in a letter in response to the AHRQ Review and sighted my full support for your work in this matter. Thank you for being the guardians of this project and all the time you have put in to the review of their work.
Libby Potter (Elizabeth C Potter)
Thanks, Libby!
Pingback: ME/CFS P2P Review Is Deeply Flawed | Shout Out About ME | Latest news, opinion and extensive resources on ME/CFS
I don’t know how I missed this. Eight day flare, I guess. Even though I feel inadequate, I am glad to know you guys are amazing. Thank you. Between protests and critiques, I hope they get it. Outstanding brain power. I am now staying with the patient poplation, not qualified to advocate except from patient perspective. Thanks again!
Thanks Gina!