
The P2P ME/CFS Workshop has been approved and is scheduled for December 9-10th, 2014. The focus of this post is on analyzing four components of the information released by NIH yesterday:
- P2P is describing our disease as fatigue, without post-exertional malaise
- P2P is trying to clarify questions on the multiple case definitions, measurement tools, effective therapies and innovative research methods
- The P2P agenda uses questions beyond the evidence review, but not the most important question of all
- The P2P Working Group includes members with and without ME/CFS expertise
How Does P2P Describe ME/CFS?
Huge red flag, folks. Here is how the P2P website describes ME/CFS:
Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) is a complex, multifaceted disorder characterized by extreme fatigue and a host of other symptoms that can worsen after physical or mental activity, but do not improve with rest. In addition to extreme fatigue, people with ME/CFS may also experience:
Widespread muscle and joint pain
Sore throat
Tender lymph nodes in the neck or armpit
Headaches
Sleep problems
Difficulty with short-term memory or concentration
I added emphasis so you can’t miss the takeaway here. ME/CFS is characterized by extreme fatigue, and people with ME/CFS may also experience other symptoms. And what is missing from this list? POST-EXERTIONAL MALAISE. Even Fukuda lists post-exertional malaise as an optional symptom. But the way NIH has described the disease, it almost sounds like Oxford – extreme fatigue and maybe other symptoms.
The description also states, “sensitivity to environmental factors (e.g., noise, light, chemicals) may force many individuals with ME/CFS into seclusion or withdrawal from society.” These sensitivities can certainly be debilitating, but I think most (if not all) patients would agree that it is primarily PEM and all the other symptoms that keep us imprisoned in our homes or our beds.
Need more proof that NIH’s conception of ME/CFS does not question the assumption that they are the same fatiguing illness? They say the two names are for the same condition: “The name myalgic encephalomyelitis or ME is more commonly used in Europe and Canada, while the name chronic fatigue syndrome or CFS is used more often in the United States and Australia. Yet the acronym ME/CFS is increasingly being used worldwide.”
In all fairness, these descriptions do not automatically determine what the Panel’s report will say. But the paradigm of a single, fatiguing illness has been at the heart of my opposition to the way P2P was being put together, and this has not eased my concern.
What Will P2P Try To Do?
The P2P website describes four things that the Workshop will try to clarify, a weird sort of blend between the five questions presented by Dr. Susan Maier to IOM on January 27, 2014 (after the P2P Working Group planning meeting), and the Key Questions of the systematic evidence review protocol.
The first issue is how the research using multiple case definitions has contributed to the state of the current literature. It’s a good question, but the answer seems blindingly obvious. Perhaps there are more subtleties that outsiders would see that I do not. All I can see is the absolute muck of a contaminated evidence base that counts Oxford studies and CCC studies as one and the same, and has absolutely no consensus on how to diagnose or measure any of it. In my opinion, the use of multiple case definitions is responsible for the state of the current literature, which is why we are stuck in a hellish stalemate with no widely accepted criteria, biomarkers, or treatments.
The second issue is how measurements are able to distinguish among ME/CFS patients focused on subsets by duration, severity, onset, and “nature of the illness.” Two observations. First, what is “nature of the illness”? I do not understand whether this is referring to immunological vs. neurological, or something else. Second, this issue assumes that differences are automatically subsets! This is exactly what I’ve been harping on for months – that the failure to ask if ME and CFS are the same, different, or spectrum illnesses eliminates the most fundamental and foundational question of them all.
The third issue presents a big red flag. It asks how research on “therapies shown to be effective” will lead to an understanding the underlying pathology. What therapies have been shown to be effective? Are we talking CBT and GET? Rituximab? You will get two very different answers about underlying pathology if you consider CBT/GET to be effective instead of Rituximab (and vice versa). Just last week, the Solve ME/CFS Initiative told NIH that the search strategy will bias the evidence towards CBT and GET. If that prediction holds true, then asking what CBT and GET tell us about the underlying pathology is patently dangerous.
The fourth issue asks what “innovative research approaches” tell us about the pathophysiology of ME/CFS and how it can be used to develop treatments. What is an innovative research approach? Is this where Rituximab fits in? Or is this focused more on things like proteomics, microbiomics, or systems network analysis? Or something else? Without understanding the terms or context, it’s hard to tell.
Agenda Good or Agenda Bad?
You may recall that I got two draft agendas for the Workshop through FOIA. Circumstantial evidence suggested they were drafted at or soon after the January Working Group meeting. How do they stack up to the real thing posted on the P2P website? Answer: the draft agenda I got through FOIA is very very similar to the one posted yesterday.
A few overall observations: The time officially allocated to the “patient perspective” is 20 minutes. The Evidence Practice Center has a total of 1 hour, 20 minutes split between two days. Total time allocated for discussion: 2 hours, 40 minutes split between the two days. You may recall that Dr. Shirley said at CFSAC that there would be town hall-style discussion at the Workshop, and also said there would be “public testimony” but provided no details on that. With less than three hours for discussion, I expect tight facilitation as opposed to open mic. There is no indication of anything resembling “public testimony” as we know it from CFSAC or other federal meetings.
I must call out one change in particular. You probably recall that I have been decrying the framing of Dr. Maier’s overview of the topic, described as “Overwhelming fatigue and malaise as a public health problem.” On the agenda posted by NIH, Dr. Maier still has 20 minutes to present an overview, but that description of the overview is gone.
The five Workshop questions are identical to the draft agenda I obtained through FOIA. Here they are, with their sub-topics (each one gets 20 minutes), but I’ve left off EPC presentations and discussion time.
I. What is the Incidence and Prevalence of ME/CFS, and Who Does It Affect?
a) Incidence and Prevalence Data (Population-Based Studies)
b) Social Determinants of Health
c) Disease Across the Lifespan
II. What Tools, Measures, and Approaches Help Define Individuals with ME/CFS?
a) Overview of Existing Tools and Measures
b) Measures: Patient-Reported and Physiologic
c) Measures: Omics, Biomarkers and Imaging
d) Innovative Statistical Approaches
III. How Are Tools and Measures Used to Distinguish Subsets of Patients with ME/CFS?
a) Identification of Subsets of Individuals
b) Triangulating Quantitative and Qualitative Data (Quality of Life/Function)
c) What Outcomes Represent Improvement, Recovery, Prevention, Benefits, or Harms
IV. Given the Unique Challenges to ME/CFS, How Can We Foster Innovative Research to Enhance the Development of Treatments for Patients?
a) Incorporating Multiple Study Designs into ME/CFS Research
b) Maximizing Approaches and Results from the Study of Other Illnesses and Complex Chronic Conditions
c) Using Research on Comorbidities to Understand ME/CFS
V. What Does the Research on ME/CFS Tell Us About the Presentation and Diagnosis of ME/CFS in the Clinic?
a) Lessons from Current Treatments and Clinical Trials
b) Comparative Effectiveness Research
c) Health Services Research and Health Policy Relevant Research
I’m going to wave a few big flags here (you knew I would). First, this agenda does not ask if CFS and ME are the same illness, different illnesses, or different aspects of a spectrum. Does. Not. Ask.
You cannot answer a question if you refuse to ask it in the first place. If we have a pile of apples and oranges and we insist on talking about the incidence and prevalence of a fruit called “appanges,” for example, or the tools that will help distinguish the subsets of “appanges,” are we ever going to question whether “appanges” are actually a pile of apples and oranges????? No, we are not. We will continue to call them “appanges,” and argue about whether the number or shape or color of the seeds distinguishes subsets. We will not see what is right in front of us, because we did not bother to consider that “appanges” might be a made-up category of fruit truthiness.
Second, we keep hearing mixed messages about what this Workshop is really trying to accomplish. Is it to identify the gaps in research, as many people insisted at CFSAC? Is it to identify methodological weaknesses in the research, as Dr. Cook said on Tuesday? Is it to determine what treatment or clinical approach works best? I see shades of all three, with an emphasis on what is known and not what is unknown.
I must correct something I have been insisting was true. I have been saying that the agenda would mirror the questions for the systematic evidence review. That was incorrect. But while the agenda and systematic review questions are not identical, you can draw a lot of lines back and forth to connect one to the other.
When Carol Head (Solve ME/CFS Initiative) expressed concern at CFSAC about the elimination of the question of how CFS and ME differ, Dr. Collins Sharp – answering with the caveat that she is not at all involved in the P2P planning – said that the review questions are a subset of the Workshop questions. She said that any question that did not have sufficient literature to be included in the evidence review could still be addressed at the Workshop. This appears to be the case, but that most important and fundamental question is nowhere to be seen.
The P2P Working Group
The P2P Working Group is the committee that helps NIH plan the meeting. The Group met in person at NIH January 6-7, 2014 (that meeting agenda has been posted). Before now, the P2P Working Group roster was only available through FOIA. Here’s the breakdown of the full list:
Federal Employees, familiar with ME/CFS (6): Dr. Susan Maier (NIH), Dr. M. Katherine Jung (NIH), Dr. Janet Maynard (FDA), Dr. Eun-Chung Park (NIH), Dr. Leorey Saligan (NIH), and Dr. Mariela Shirley (NIH). The NIH employees are all members of the Trans-NIH ME/CFS Working Group. Dr. Park is the staff member contact for the Lipkin samples. Dr. Saligan’s research focus is acute and chronic fatigue, and he has done sample analysis for Dr. Baraniuk and others. Dr. Maynard is the FDA ex officio to CFSAC, and works in the FDA review division that handles ME/CFS drug applications.
Federal Employees, not familiar with ME/CFS (6): Jody Engel, Deborah Langer, Elizabeth Neilson, Wilma Peterson Cross, Paris Watson, and Dr. Jessica Wu all work at NIH’s Office of Disease Prevention. They also all serve on the P2P Working Group for the upcoming meeting on opioid use.
Non-Federal Members, familiar with ME/CFS (6): Dr. Mady Hornig (Columbia University), Dr. Leonard Jason (DePaul University), Dr. Nancy Klimas (NOVA Southeastern University), Robert Miller (Patient and Advocate), Dr. Peter Rowe (Johns Hopkins University), and Dr. Suzanne Vernon (Solve ME/CFS Initiative) are all familiar to the ME/CFS community.
Non-Federal Members, not familiar with ME/CFS (1): Dr. Carmen Green (University of Michigan) is an anesthesiologist and member of the HHS Interagency Pain Research Coordinating Committee. She is the chair of the P2P Panel.
Several names listed on the January roster (obtained through FOIA) as attending the meeting do not appear on this final Working Group roster. Missing are Dr. Suchitra Iyer (AHRQ), Dr. Heidi Nelson and Dr. Beth Smith (both of the Oregon Health & Science University Evidence Practice Center). I do not know for certain why they are not listed on the final Working Group roster, but they may have attended the meeting to discuss the evidence review questions rather than the planning as a whole.
Another odd omission: at the CFSAC meeting, Dr. Nancy Lee said that Marty Bond had attended “several” of the meetings for P2P. Yet Ms. Bond’s name is not listed on any of the documents posted or obtained through FOIA. So we cannot automatically assume that the only people attending Working Group meetings are the members themselves.
According to the P2P website, the Working Group drafted the questions for the evidence review, finalized the agenda, nominated speakers and panelists, selected the workshop date, and continue to be engaged in ongoing workshop planning. I am hearing conflicting things about that continued engagement and how extensive it will be.
Bottomline
Based on the information released yesterday, is P2P a worst case scenario? I have a vivid imagination, so I can definitely imagine something worse than this. But is P2P looking good? Absolutely not. If Mary Dimmock and I were writing our letter to Dr. Collins today, I would tweak some sections but all of my objections are basically unchanged.
P2P: The Question They Will Not Ask
by Mary Dimmock and Jennie Spotila
Boiled down to its essence, this set of questions is asking whether all the “ME/CFS” definitions represent the same disease or set of related diseases. The failure to ask this question puts the entire effort at risk.
This fundamental question was posed in the 2012 application for the Office of Disease Prevention to hold the P2P meeting (which I obtained through FOIA). It was posed in the 2013 contract between AHRQ and the Oregon Health & Science University for the systematic evidence review (which I obtained through FOIA). It was posed to the P2P Working Group at its January 2014 meeting to refine the questions for the evidence review and Workshop (according to Dr. Susan Maier at the January 2014 Institute of Medicine meeting).
And then the question disappeared.
The systematic evidence review protocol does not include it. Dr. Beth Collins-Sharp said at the June 2014 CFSAC meeting that the Evidence Practice Center is not considering the question because there is “not enough evidence” in the literature to answer the question. However, she said that the P2P Workshop could still consider the question.
But the draft agenda for the Workshop does not include it. Furthermore, every aspect of the P2P Workshop treats “ME/CFS” as a single disease:
A 20 minute presentation on “Case Definition Perspective” is the only lip service paid to this critical issue. This is completely inadequate, if for no other reason than because the presentation is isolated from discussions on the Workshop Key Questions and dependent topics like prevalence and natural history. As a result, it is unlikely to be thoroughly discussed unless one of the Panelists has a particular interest in it.
Why is this problematic? Because both the P2P Workshop and the evidence review are based on the assumption that the full set of “ME/CFS” case definitions describe the same disease. This assumption has been made without proof that it is correct and in the face of data that indicate otherwise, and therein lies the danger of failing to ask the question.
What if the case definitions do not actually describe a single disease? If there are disparate conditions like depression, deconditioning, non-specific chronic fatigue and a neuroimmune disease characterized by PEM encompassed by the full set of “ME/CFS” definitions, then lumping those together as one entity would be unscientific.
The most important part of designing scientific studies is to properly define the study subjects. One would not combine liver cancer and breast cancer patients into a single cohort to investigate cancer pathogenesis. The combination of those two groups would confound the results; such a study would be meaningful only if the two groups were separately defined and then compared to one another to identify similarities or differences. The same is true of the P2P evidence review of diagnostics and treatments: assuming that all “ME/CFS” definitions capture the same disease (or even a set of biologically related diseases) and attempting to compare studies on the combined patients will yield meaningless and confounded results if those definitions actually encompass disparate diseases.
There is a growing body of evidence that underscores the need to ask the fundamental question of whether “ME/CFS” definitions represent the same disease:
By contrast, there have not been any published studies that demonstrate that the set of “ME/CFS” definitions being examined in P2P encompass a single entity or biologically related set of entities. From Oxford to Fukuda to ME-ICC, there are significant differences in the inclusion and exclusion criteria, including differences in the exclusion of primary psychiatric illness. The magnitude of these differences makes the lack of such proof problematic.
Given that treating all “ME/CFS” definitions as a single entity is based on an unproven assumption of the clinical equivalence of these definitions, and given that there is ample proof that these definitions do not represent the same disease or patient population, it is essential that the P2P “ME/CFS” study start by asking this question:
The failure to tackle this cornerstone question up-front in both the agenda and the evidence review puts the scientific validity of the entire P2P Workshop at risk. If this question is not explicitly posed, then the non-ME/CFS expert P2P Panel will swallow the assumption of a single disorder without question, if for no other reason than that they do not know the literature well enough to recognize that it is an assumption and not established fact.
This post was translated into Dutch with my permission.