IOM Panelists: Balance = Experts + Information – Bias

The issue on everyone’s mind is whether the Institute of Medicine panel is the right group to come up with a new ME/CFS clinical case definition. Our team has been debating this for nearly two weeks now, and we have not reached a unanimous conclusion. But our research and discussions illuminate some key issues, particularly on the inclusion of psychologists and the possibility of bias in three panelists.
 

Enough Experts?

 
The overall panel balance is 53% known and 47% unknown to the ME/CFS community. None of us think this is sufficient, but we do not agree on what the right split would be. Some of us think the panel should be 100% ME/CFS experts; others think the addition of at least one or two more experts would be appropriate.

Mary Dimmock has raised a very important point: in order to judge whether this panel is adequate, we have to think about the task. If this panel is going to create a case definition for the broader set of conditions that meet the various CFS criteria, then some might make the argument that this panel is appropriate. But if the panel is focused on the disease we described to the FDA in April, then Mary concludes that the panel needs more, and a broader cross-section, of ME/CFS experts and fewer psychologists. As Mary has reported, HHS and IOM have been vague in detailing which disease group is the focus of the panel.

We’re faced with a chicken-egg dilemma: identifying the disease focus should happen prior to the panel selection, but instead we have a panel with a seemingly vague task focus. This panel composition seems to hold a middle ground: there are not an overwhelming number of ME/CFS experts, but neither are there an overwhelming number of experts in unexplained chronic fatigue, psychogenic models, or “other multi-symptom, complex disorders” as described in the Statement of Work. To our team, the key issue is bias.
 

Psyched Out

 
There are two psychologists on the panel: Dr. Margarita Alegria and Dr. Charles Cleeland. None of us are particularly troubled by Dr. Cleeland, as the focus of his career has been chronic pain (a significant problem for many ME/CFS patients) and outcomes measures. Dr. Alegria is another story, as discussed in the next section.

In a comment on an earlier blog post, AJ asked if panels on other diseases have included psychologists or psychiatrists. We did some research into the question, and found mixed results:

• Psychologists/psychiatrists were included in panels on treatment for Gulf War Illness (5 of 15), epilepsy (3 of 17), living with chronic illness (3 of 17), chronic pain (5 of 19), and breast cancer and the environment (1 of 15).
• No psychologists/psychiatrists were included in panels on Gulf War and infectious disease, veterans and Agent Orange, cardiovascular and chronic lung disease, hypertension, hepatitis and liver cancer, and ALS in veterans.
• The only other IOM panel creating a case definition, the case definition for chronic multisymptom illness (aka GWI), includes 3 psychologists/psychiatrists out of 16 total members.

The ME/CFS panel includes 2 psychologists out of 15 total members.  Creating a case definition for ME/CFS will involve confronting the psychosocial literature and assessing its validity. For this reason, we do not categorically reject the presence of psychologists on the panel. However, we have serious concerns about psychologists who are ignorant of the methodological flaws in much of that literature. We are also concerned about the bias Dr. Alegria may bring to the task, and explore that in more detail below.
 

Bad Bias vs. Good Bias

 
The IOM Conflict of Interest Policy does not bar individuals from serving just because they have adopted positions on the issues facing the panel. In fact, the policy admits that members with potential biases might be necessary to ensure a competent and balanced panel. Bias will preclude committee service when “one is totally committed to a particular point of view and unwilling, or reasonably perceived to be unwilling, to consider other perspectives or relevant evidence to the contrary.” (p. 4)

There are multiple panelists with potential bias. For example, Drs. Bateman, Chu, Keller, Klimas, Lerner and Natelson have all taken public positions on ME/CFS case definitions. At least one team member personally knows each of these individuals, so we believe we can say that none of them are unwilling to consider other perspectives or relevant evidence.

Of greater concern to our team are three unknown panelists who may have biases that could be very harmful to the interests of the ME/CFS community. It can be challenging to assess bias at a distance based on publications, but we have legitimate concerns about three panelists: Dr. Alegria, Dr. Theodore Ganiats, and Dr. Cynthia Mulrow.

Dr. Alegria

As we mentioned in our profile of Dr. Alegria, she has co-authored three publications potentially relevant to her views on ME/CFS. The first two (here and here) address the relationship between somatic symptoms and psychiatric disorders like depression. Only 14 symptoms were assessed in these papers, and fatigue was not one of them. The papers found that having 3 or more of the 14 symptoms was associated with depression/anxiety and mental health service use. However, one of the papers correctly noted that the data are ill suited to infer causality between the physical symptoms and mental health issues.

After examining the third paper in its entirety, we came to mixed conclusions about whether it represents unreasonable bias on Dr. Alegria’s part. The paper builds on previous research showing that neurasthenia has the highest rate of comorbidity among ICD-10 disorders, and examines whether prevalence and comorbidity rates hold up across racial/ethnic groups. Comorbidity was examined only in depressive, anxiety and substance abuse disorders. Space does not permit a full parsing of the data here, including significant methodological limitations. Although the paper itself does not directly examine the relationship between neurasthenia and CFS, statements about CFS included in the paper have caused serious concerns among advocates. We carefully considered the statements and whether these are indications of unreasonable bias.

Likewise, as Lee [51] argued, the ‘‘disappearance’’ of culture-bound syndromes is related to changing sociocultural conditions, including economic and political factors and changes in managed care and pharmaceutical forces, to name a few. Indeed, chronic fatigue syndrome (CFS), also a controversial illness which has been argued to be a variant of neurasthenia [4, 21] has become increasingly diagnosed in the US [52, 53], whereas neurasthenia is virtually no longer diagnosed in the US context [51].

Many advocates are correctly concerned about the statement above that CFS is “a controversial illness which has been argued to be a variant of neurasthenia.” The first reference is to a thoroughly offensive article from 1991 that argued that CFS is a culturally sanctioned form of illness behavior.  The second reference is from 2007, but examined the concordance of CFS and three neurasthenia criteria in four medical practices in Pune, India. Neither reference can legitimately be used to support the statement that CFS should be seen as a variant of neurasthenia today.

After noting again that neurasthenia has almost disappeared from US clinical practice, the paper states:

This has implications for diagnosis and treatment of individuals who may present with symptoms of neurasthenia, but may otherwise be misdiagnosed as having depression, anxiety, or CFS.

This sentence can be read two ways. In one interpretation, neurasthenia patients are misdiagnosed with depression, anxiety or CFS because “neurasthenia” is not used in the US, and this does not imply that CFS as an entity is actually neurasthenia. In the second interpretation, people are diagnosed with CFS incorrectly because CFS is actually neurasthenia. Depending on which interpretation you choose, Dr. Alegria’s presence on the panel can be quite alarming, as she may have a simplistic, outdated view of ME/CFS or worse. But other team members see these few sentences in the paper within the context of Dr. Alegria’s research focus, which does not appear to reflect a tendency to classify patients with diagnosable physical conditions as having mental disorders.

Regardless of our personal interpretations of the meaning of these three papers, we all share a strong concern about her presence on the panel. She seems to have little experience relevant to an ME/CFS case definition at best, and potentially disastrous views if she does in fact believe that the physical symptoms of ME/CFS are equivalent to neurasthenia. Given the damaging legacy of psychogenic approaches to ME/CFS, we believe that her potential bias must be thoroughly investigated before the panel is finalized.

Dr. Ganiats

In our profile of Dr. Ganiats, we noted that Dr. Ganiats himself has not published anything directly indicating a bias on ME/CFS. However, he has been very closely associated with the American Academy of Family Practitioners. The AAFP has published problematic material about ME/CFS as recently as 2012. The AAFP’s consumer health information site presents a simplistic view of CFS, including the statement that “Most symptoms improve with time,” which is patently untrue, as many of us can personally attest.

We also noted Dr. Ganiats’ close association with colleague Dr. William Sieber. Dr. Sieber is a psychologist who has given presentations claiming that CFS symptoms are due to psychological problems. Dr. Ganiats has also published a number of papers on “quality of well-being” with Dr. Robert Kaplan, the Director of NIH’s Office of Behavioral and Social Sciences Research. We are wary of assuming bias by association, but Dr. Ganiats’s connections raise the possibility that he may have an outdated view of ME/CFS as merely fatigue of potentially psychogenic origin.

One team member suggested that Dr. Ganiats adds little to the panel as a general practitioner, especially since Dr. Bateman was a general practitioner for many years prior to opening her Fatigue Consultation Clinic. On the other hand, Dr. Ganiats brings his close relationship with AAFP and his experience in doctor-patient decision making to the table. But if Dr. Ganiats shares the views of AAFP or Dr. Sieber on ME/CFS, then he may have a strong bias towards the psychogenic view, or may at least be predisposed to broader views of the illness. This bias is unacceptable, and should be thoroughly considered by IOM before the panel is finalized.

Dr. Mulrow

Some readers took issue with our statement that Dr. Mulrow’s experience on IOM and in systematic reviews made her “very qualified to tackle the massive systematic evidence review that will be part of the IOM ME/CFS study.” And if her experience with IOM and reviews were the only question, that statement would be accurate on its own. However, as noted in our original profile, she has publications on ME/CFS that give us great pause.

Specifically, Dr. Mulrow led an evidence review on CFS in 2001 that concluded, in part, “The superiority of one case definition over another is not well established. The validity of any definition is difficult to establish because there are no clear biologic markers for CFS, and no effective treatments specific only to CFS have been identified.” It’s arguable whether this was true in 2001, but we certainly hope her view is different now.

Dr. Mulrow also co-authored a paper on treatment interventions in CFS in 2001 and based in part on that AHRQ review. Based on 2001 evidence, CBT and GET were considered promising treatments, despite some high drop out rates and lack of concrete outcome measures.

While the paper does say that effectiveness must be considered in light of those methodological issues, the analysis of these studies seems rather forgiving. The presence of multiple methodological problems did not preclude the studies being treated as highly valid. This points to potential risks in evidence-based medicine and meta-analysis approaches. The protocol for the review is established in advance, but if it is overly simplistic or has low inclusion criteria, the results of the review will reflect that lack of rigor.

Several team members feel very strongly that Dr. Mulrow is a potential problem on the panel precisely because of her systematic review expertise. They are willing to accept that she may bring an open perspective to ME/CFS itself, but fear that her process approach is ill-suited for an area that has been consistently underfunded for decades leading to an overemphasis on cheap (possibly less rigorous) science. Other team members feel that systematic review experience is necessary to help un-bias the view of the literature by setting evaluation criteria in advance. But again, if she is unwilling to approach the field as it is today then this is a problem.
 

Hiding in Plain Sight

 
There is one panelist who continues to stump us: Dr. Michael Shelanski. As we described in the original profile, he is a a pathologist focused on cell biology and animal models in Alzheimer’s disease. We don’t doubt his credentials in that field, but we fail to see how relevant that experience will be to creating a case definition for ME/CFS

Several big names in ME/CFS are missing from the IOM panel. The biggest head scratcher is Dr. Leonard Jason, who received at least seven nominations from the ME/CFS community. Dr. Jason is the authority on ME/CFS definition assessment and comparison. If he cannot serve on the panel for any reason, then his expertise must be harnessed by the panel another way.

The absence of infectious disease experts besides Dr. Lerner is another disappointment. Dr. Ian Lipkin and Dr. Jose Montoya each received at least five nominations. Dr. Lipkin is currently researching evidence for pathogens in ME/CFS, while Dr. Montoya is a clinician-scientist at Stanford University who spoke on a panel at the April FDA meeting. Either would be a strong addition to the panel to build on Dr. Lerner’s experience.

Dr. Christopher Snell received at least seven nominations, and is the leading expert on exercising testing in ME/CFS. But Dr. Keller is an adequate replacement, given her familiarity with Dr. Snell’s techniques and her own growing expertise in working with ME/CFS patients.

The IOM did not appoint a cardiologist or endocrinologist to the panel, and this is a disappointment. Cardiac and endocrine abnormalities have been part of the ME/CFS picture for decades, and both were listed in the Statement of Work (although not as requirements). While Dr. Natelson is a neurologist, inclusion of a researcher specializing in cognitive impairments in ME/CFS would also be advisable.
 

A Final Step

 
Feedback on the provisional panel is due December 23^rd. Our next post will include the feedback we intend to submit, and we hope that other advocates will also participate in whatever way they believe is appropriate.

Acknowledgements: This post was a group effort, and would not have been possible without the assistance and participation of Lori Chapo-Kroger, Claudia Goodell, Chris Heppner, Denise Lopez-Majano, Mike Munoz, Darlene Prestwich, Tamara Staples, WillowJ, and one advocate who wished to remain anonymous.