Credit: Robert Van Vranken
The IOM panel on ME/CFS held its second (and likely final) public meeting on May 5, 2014. On display near the meeting room was this painting by Robert Van Vranken: Untitled (Everything at once, or one thing at a time?). (read more about the painting here) Written on the wall within the painting is the beginning of a quote from Prometheus Unbound, “Hear the sum of the whole matter in the compass of one brief word . . .”
To hear the sum of the whole matter in one brief word: this is precisely the challenge before us. How do we convey the full sum of ME/CFS – all of the issues, science, suffering, truth – in a brief word? When we say ME or PEM or cognitive dysfunction, will the IOM hear the full compass of what we mean? And can their diagnostic criteria – in a few brief words – transmit the fullness of that knowledge to the world?
This was the overarching theme of the public session on May 5th: communicating the reality of ME/CFS, and the uncertainty over whether the message was heard.
Agenda Structure
The agenda for the meeting was organized around gaps in the panel’s knowledge. A patient panel was asked to address obstacles in interacting with the healthcare system. Dr. Leonard Jason presented data on challenges and issues in the diagnostic criteria. Dr. Akifumi Kishi presented on sleep disturbance and Dr. Gudrun Lange presented on neurocognitive impairment, two issues that rank high in studies of ME/CFS symptoms. Every invited speaker was given a list of questions by the panel to answer in their presentations.
Taken as a whole, the speakers emphasized the core symptoms of PEM, sleep disturbance and cognitive impairment, and Dr. Jason addressed key issues in devising diagnostic criteria. These are critical areas for the committee to consider. Some advocates asked why no immunologist or expert clinician like Dr. Peterson was invited. But there are committee members with that expertise: Dr. Klimas for clinical immunology, and Drs. Klimas, Bateman, Lerner, Natelson and Rowe for experience treating people with ME/CFS. The panel does not have expertise in sleep or cognitive impairment, and Dr. Jason’s expertise in ME/CFS criteria issues is unparalleled. The meeting agenda structure plugged some of those holes for the IOM panel.
Patient Presentations
Two patients (Joe Landson and Bob Miller) and two caregivers (Denise Lopez-Majano and Annette Whittemore) spoke on the obstacles that ME/CFS patients face in diagnosis and healthcare. The presentations were moving, and all four acknowledged their good fortune in being able to get as much from the healthcare system as they have.
Joe Landson and Annette Whittemore pointed out that patients usually know more than their doctors, but remain hopeful that they will find help and confirmation of the diagnosis. Annette told several stories about doctors refusing to offer treatment to her daughter Andrea because she carries the CFS label. Denise Lopez-Majano talked about how doctors gave her no guidance on how to care for her sons or help them recover. Even basic advice about PEM and cognitive impairment would have reduced their suffering. Bob Miller described how it wasn’t until he found Dr. Peterson fifteen years after getting sick that he finally received adequate diagnostic testing and treatment.
Both cognitive impairment and PEM featured prominently in the presentations. Joe talked about how the “clutch in his brain has snapped,” and that even simple cognitive tasks are now beyond his capabilities. Denise described her son Matthew losing focus while speaking or eating, as if “his brain was stuck on hold.” All the panelists described the unpredictability and severity of PEM, offering specific examples. I’m not sure if the IOM panelists keyed in on the significance of one of Denise’s comments: that her sons were prescribed exercise only after they were diagnosed with ME/CFS.
Dr. Lily Chu and Dr. Klimas were both interested in the presenters’ thoughts on the name of this disease. While all four were emphatic that “CFS” should be abandoned, only Annette Whittemore offered an alternative: Ramsay’s disease. Denise, Bob and Joe all declined to specify what name they preferred. Many advocates saw this as a squandered opportunity. None of the speakers addressed the fact that ME already has a diagnostic code, nor did they connect the name question back to the diagnostic criteria.
The sense I got was that the speakers felt put on the spot by the name question, as if they were being asked to speak for the entire advocacy community, but I think they could have offered their own opinions. The IOM requested we address the name question in written comments for this meeting. In my own written submission to the IOM panel, I said, “If your case definition resembles the traditional definition of myalgic encephalomyelitis, as I believe it must, then I urge you to retain the traditional name as well.” I hope many other people expressed their opinions, too.
Dr. Jason and Criteria Challenges
Dr. Jason was asked to address many questions in his presentation, and given only 30 minutes in which to do it. I strongly recommend watching the video of his presentation to get the full details. In my opinion, the most important take away was the issue of reliability and criterion variance.
Studies have shown that anywhere from 50-90% of people who meet the Fukuda definition also meet the Canadian Consensus Criteria. Dr. Jason said that one possible explanation for this wide variation is that doctors are not applying the criteria consistently. Without an agreed upon set of biomarkers or assessment tools, there is potential for great differences in the actual application of any case definition.
One way to solve this problem is with an assessment tool like the DePaul Symptom Questionnaire. Dr. Jason presented data showing that the DSQ can distinguish between ME/CFS and depression with 100% success. However, the DSQ is not as good at distinguishing ME/CFS from autoimmune diseases like lupus or multiple sclerosis, and his research is ongoing.
Another way to create better diagnostic criteria is to use frequency and severity cutoff scores. This means that we don’t just ask if the person has a symptom, such as dizziness. We have to ask how frequently the person is dizzy and how severe the dizziness is. Dr. Jason presented a powerful slide showing that when a definition requires symptoms to occur a little of the time and be mild (referred to as 1/1), almost 34% of healthy controls meet the Fukuda definition. But when symptoms must occur about half the time and be moderate in severity (referred to as 2/2), healthy controls are almost completely eliminated.
The many questions from the IOM committee members may give us clues about what they are grappling with behind closed doors.
- Why is PEM not reported by 100% of patients? Answer: self-report questionnaires are not perfect, patients may have adapted coping skills to reduce PEM, or it may not be present in 100%.
- Is both frequency and severity required? Answer: identifying low frequency is important because it is less debilitating to patients.
- Impact of even higher cutoffs, gradual onset, psychiatric comorbidity, gender, ethnicity, controls, and good day vs. bad day? Answer: much of this has not been studied, and data is limited by the collectors (such as biobanks).
- How do we create a definition that is not too complicated for clinicians to use? Answer: focus on the core symptoms, and give doctors tools like questionnaires and scoring rules to help them make the diagnosis.
This issue of creating a simple definition came up during the patient presentations, too. Several people supported a one sentence description of the essence of the disease. In order to be inclusive, subtypes were suggested as ways to separate patients within that broader category. To be honest, this makes me nervous. It opens the door to a broad definition, with subtypes used to truly delineate those with this disease. I’m not sure that’s much different from what we have now. However, Dr. Jason did present his data on the core symptoms (PEM and cognitive dysfunction), and showed that other symptoms may cluster patients with immune or autonomic abnormalities. That paradigm was well reflected in the patient presentations.
Sleep and Cognitive Dysfunction
Following in the theme of core symptoms, Dr. Akifumi Kishi presented his data on sleep dysfunction in ME/CFS. I found his presentation very difficult to follow, and was disappointed that he did not provide a more comprehensive survey of this area of research. His data show that ME/CFS patients have abnormalities in their sleep stage transitions, but not in the actual stages themselves. The data also show that ME/CFS patients slept better after exercise, which is completely at odds with what many of us have experienced.
Dr. Kishi was pressed hard by the panel on multiple issues related to study design, statistical power, outcome measures, and comparison to overtrained athletes. Dr. Kishi could not answer many of these questions due to a lack of data or failure to consider the issues during study design. It was disappointing that his studies have been so small and so preliminary, and I’m not sure how useful this data will be to the IOM panel.
Dr. Gudrun Lange gave an overview of cognitive dysfunction in ME/CFS, including her fMRI studies and patient evaluations. Overall, her data show that ME/CFS patients have impaired processing speed and working memory. In untimed tasks, they show little impairment. Decreased attention and concentration, less information available “online,” and an inability to organize learning and make quick decisions are all manifestations of the dysfunction. Her imaging studies show that even when ME/CFS patients complete the same tasks at the same level as healthy controls, they need to use more of the brain to do so.
Dr. Lange was questioned on comparisons to other groups, such as sleep deprived controls and patients with fibromyalgia or ADHD. She also highlighted the differences between ME/CFS cognitive dysfunction and those with dementia or stroke. ME/CFS patients are very different, and testing conducted by neuropsychologists not familiar with the disorder can appear to give normal results when the impairment may actually be severe. While Dr. Lange said that no short form testing has been validated in ME/CFS patients, she suggested giving a patient 6 or 7 steps of complicated driving directions and asking them to repeat it back. We rarely can.
It was fitting for the meeting to end on the subject of cognitive dysfunction, since that had been a significant focus of the patient/caregiver panel. And I can share one example of my own cognitive fallout from attending the meeting. I traveled to the meeting with a friend by car (unable to hold directions in my head, as Dr. Lange had highlighted). I had the following text exchange with my husband after a pit stop on the way home, perfect evidence of me losing focus and substituting words:
The Elephant Not In The Room
One audience member commented to me that this meeting was very different from the January public meeting because of the absence of protesting advocates. Bob Miller said during his remarks that patients are protesting the meeting because they have no faith in the system. If the patient community actually believed the IOM process would help them, Bob said, the room would be filled.
My position on participating in the IOM process is well-known. I’ve been criticized for it, as were the advocates who chose to participate as panelists at this meeting. I won’t revisit the controversy in detail here. All I can say is that while some people noted the absence of many advocates, that absence made it easy for the panel to focus on its own agenda. When they asked for opinions on the name, they only heard the uncertainty of several panelists and whatever was submitted in writing. Anything the other advocates would have said was simply lost. Those of us who have chosen to provide input to the committee are trying to represent the diversity of opinion in our community, but I’m sure we are not fully successful.
“Hear the sum of the whole matter in the compass of one brief word.” Have we done enough to tell the IOM panel the truth about this disease? Have we done enough to point them in the right direction, and give them evidence sufficient to reach the right conclusions? Were the protesters heard through their silence? Have we been effective in combining the power of that silent protest with input from those willing to speak and write to the panel? Will it be enough?
We can only guess, and hope, and try, and wait.
Protocol for Disaster?
Remind Me What This Is
The systematic evidence review is the cornerstone of the P2P process. The P2P meeting on ME/CFS will feature a panel of non-ME/CFS experts who will produce a set of recommendations on diagnosis, treatment, and research.
Because the P2P Panel members are not ME/CFS experts, they need background information to do their job. This systematic evidence review done by the Oregon Health & Science University under contract to AHRQ will be that background information. The systematic evidence report will be presented to the Panel in advance of the public P2P meeting, and will be used to establish the structure of the meeting as well.
The systematic review is the foundation. If done correctly, it would be a strong basis for a meaningful workshop. If done poorly, then everything that follows – the workshop and the resulting recommendations – will crumble. Based on the protocol published yesterday, I think “crumble” is putting it mildly.
The Key Questions
As I wrote in January, the original draft questions for the evidence review included whether CFS and ME were separate diseases. That question is GONE, my friends. Now the review is only looking at two things:
These questions are based upon a single and critical assumption: ME and CFS are the same disease. Differences among patient groups represent subtypes, not separate diseases. The first and most important question is whether the ME and CFS case definitions all describe one disease. But they’re not asking that question; they have already decided the answer is yes.
The study protocol and other communications from HHS (including today’s CFSAC listserv message) state that the P2P Working Group refined these study questions. The implication is that since ME/CFS experts and one patient served on the Working Group, we should be satisfied that these questions were appropriately refined. But what I’m piecing together from various sources indicates that the Working Group did not sign off on these questions as stated in the protocol.
Regardless of who drafted these questions, they cannot lead to the right answers because they are not the right questions. And when you examine the protocol of how the evidence review will be conducted, these questions get even worse.
Protocol Problems
The real danger signals come from the description of how this evidence review will be done. The issue is what research will be included and assessed in the review. For example, when asking about diagnostic methods, what definitions will be considered?
This evidence review will include studies using “Fukada [sic], Canadian, International, and others“, and the Oxford definition is listed in the table of definitions on page 2 of the protocol. That’s right, the Oxford definition. Oxford requires only one thing for a CFS diagnosis: six months of fatigue. So studies done on people with long-lasting fatigue are potentially eligible for inclusion in this review.
The description of the population to be covered in the review makes that abundantly clear. For the key question on diagnostic methods, the study population will be: “Symptomatic adults (aged 18 years or older) with fatigue.” There’s not even a time limit there. Three months fatigue? Four? Six? Presence of other symptoms? Nope, fatigue is enough.
There is a specific exclusion: “Patients with other underlying diagnosis,” but which conditions are exclusionary is not specified. So will they exclude studies of patients with depression? Because the Oxford definition does not exclude people with depression and anxiety. We’ve seen this language about excluding people with other underlying diagnosis before – and it results in lumping everyone with medically “unexplained” fatigue into one group. This protocol is set up to result in exactly that. It erases the lines between people with idiopathic chronic fatigue and people with ME, and it puts us all in the same bucket for analysis.
And what about the key question on treatment? What studies will be included there? All of them. CBT, GET, complementary/alternative medicine, and symptom-based medication management. It’s not even restricted to placebo trials; trials with no treatment, usual care, and head-to-head trials are all included.
Let’s do the math. Anyone with unexplained fatigue, diagnosed using Oxford or any other definition, and any form of treatment. This adds up to the PACE trial, and studies like that.
But it’s even worse. The review will look at studies published since January 1988 because that was the year “the first set of clinical criteria defining CFS were published.” (page 6) Again, let’s do the math: everything published on ME prior to 1988 will be excluded.
Finally, notice the stated focus of the review: “This report focuses on the clinical outcomes surrounding the attributes of fatigue, especially post-exertional malaise and persistent fatigue, and its impact on overall function and quality of life because these are unifying features of ME/CFS that impact patients.” (page 2) In other words, PEM = fatigue. And fatigue is a unifying concept in ME/CFS. Did anyone involved in drafting this protocol actually listen to anything we said at last year’s FDA meeting?
Bad Science
Credit: ElodieUnderGlass
Maybe you’re thinking it’s better for this review to cast a broad net. Capture as much science as possible and then examine it to answer the key questions. But that’s not going to help us in this case.
This review will include Oxford studies. It will take studies that only require patients to have fatigue and consider them as equivalent to studies that require PEM (or even just fatigue plus other symptoms). In other words, the review will include studies like PACE, and compare them to studies like the rituximab and antiviral trials, as if both patient cohorts were the same.
That assumption – that patients with fatigue are the same as patients with PEM and cognitive dysfunction – is where this whole thing falls apart. That assumption contaminates the entire evidence base of the study.
In fact, this review protocol makes an assumption about how the Institute of Medicine study will answer the same question. It is possible (though not assured) that IOM will design diagnostic criteria for the disease characterized by PEM and cognitive dysfunction. But this evidence review is based on an entirely different patient population that includes people with just fatigue. The conclusions of this evidence review may or may not apply to the population defined by the IOM. It’s ridiculous!
But it’s the end use that really scares me. Remember that this systematic evidence review report will be provided to that P2P Panel of non-ME/CFS experts. The Panel will not be familiar with the ME/CFS literature before they get this review. And the review will conflate all these definitions and patient populations together as if they are equivalent. I think it’s obvious what conclusion the P2P Panel is likely to draw from this report.
I would love to be wrong about this. I would love for someone to show me how this protocol will result in GOOD science, and how it will give the P2P Panel the right background and foundation for the recommendations they will draft. Please, scientists and policy makers who read this blog – can you show me how this protocol will produce good science? Because I am just not seeing it.
What Do We Do?
This protocol is bad news but it is by no means the last word. Plans are already in motion for how the advocacy community can respond. I will keep you posted as those plans are finalized.
Make no mistake, this evidence review and P2P process are worse than the IOM study. We must respond. We must insist on good science. We must insist that our disease be appropriately defined and studied.