After my post on the basic process of an Institute of Medicine study, I received a number of questions in the comments and on Phoenix Rising about the definition of “conflict of interest” as distinguished from “bias.” I reached out to the IOM for more information, and they provided me with a copy of their conflict of interest policy. It’s long and a bit technical, so I’ve done my best to distill and explain it here. Disclaimer: I am not an expert, so this article reflects my understanding of the policy and its application. Please do not assume that my understanding is completely correct.
Why Does Conflict of Interest Matter?
The Institute of Medicine (and the other National Academies) pay attention to conflicts of interest for two major reasons: it is required by law and it affects the Institute’s credibility.
The Federal Advisory Committee Act prohibits any U.S. government agency from using recommendations from committees created by the IOM at the request of the government unless certain conditions are met, including several conditions regarding committee membership. First, no individual can have a conflict of interest relevant to the committee (with one exception explained below). Second, the committee membership must be fairly balanced. Third, the final report must be the result of the IOM’s independent judgment. In order for HHS to use the IOM’s conclusions on a clinical case definition for ME/CFS, the IOM has to ensure that these conditions were fulfilled, including no conflicts of interest among the committee members.
The IOM also recognizes that its credibility and the competence of its reports depend on a process free from conflicts of interest. In order for a report to be broadly accepted, it must not only be of the highest quality but must also be “the result of a process that is fairly balanced . . . and free of any significant conflict of interest.” (page 1) This is already an issue for the ME/CFS study. The predicted competence and balance of the panel members has been a significant contributing factor to the protests against the contract. The IOM recognizes that allegations of conflict of interest or a lack of balance will undermine a committee’s conclusions, even if those conclusions are correct. Thus, the IOM pays careful attention to conflicts of interest throughout the committee process.
What Is A Conflict of Interest?
Some of the concern over selection of ME/CFS panel members stems from confusion about what exactly constitutes a conflict of interest. The IOM policy covers this in detail. Here is the key definition:
For this purpose, the term “conflict of interest” means any financial or other interest which conflicts with the service of the individual because it (1) could significantly impair the individual’s objectivity or (2) could create an unfair competitive advantage for any person or organization. (page 4)
There are several other important factors in the assessment of conflict of interest, and the IOM policy covers this at length (see pages 4 and 9, particularly). I think this list is a fair summary:
- A conflict of interest is usually a financial interest.
- This applies not only to the individual’s financial interests, but also the interest of others like family members.
- This applies only to current interests. It does not apply to past or possible future interests.
- Conflict of interest must be determined on a case by case basis.
- A bias or point of view is usually not a conflict of interest.
There is one exception to this conflict of interest policy. A person might be allowed to serve on a panel if he/she has knowledge that is particularly valuable to the committee and no comparable person without a conflict can be found. The determination of whether the conflict is unavoidable is made at the Academy executive level, and must be disclosed to the public. (page 8)
Conflicts of interest are assessed in multiple ways by staff and the committee members both before and throughout the study. The goal of all these procedures is to identify “whether an individual . . . has identifiable interests that could be directly affected by the outcome of the project activities of the committee”. (page 9)
What Is Bias?
“The term ‘conflict of interest’ means something more than individual bias.” (page 4) This is an extremely important point to understand. Many of the questions and concerns advocates have raised about the IOM committee relate to the significance of bias. According to the IOM policy, the existence of a bias or point of view does not automatically disqualify someone from serving on a committee.
Bias is usually related to a person’s views or positions that “are largely intellectually motivated or that arise from the close identification or association of an individual with a particular point of view”. (page 3) Researchers or clinicians who have expressed support for the Canadian Consensus Criteria (or other ME/CFS definitions) would generally fall into this category, and are not automatically disqualified. In fact, the IOM policy recognizes that it may be necessary to appoint committee members with potential bias in order to ensure that the committee is fully competent and balanced.
Researchers or clinicians who have committed to a position through public statements, publications, or other activities are usually seen as having only a bias. It is also not uncommon for committee members to have their own published and professional work be part of the literature review for the committee. Neither of these potential biases rise to the level of a conflict of interest. (page 5)
But sometimes bias can be so significant or extreme that it does preclude service. For example, if a person’s employer has taken a fixed position on an issue, that might be a sufficiently direct interest of the individual to create a conflict of interest. (page 6). Another example is when the central purpose of a committee activity is to review and evaluate an individual’s own work, then that individual is deemed to have a conflict of interest. However, that individual can still provide relevant information to the committee. (page 5)
The test is whether the person is “unwilling, or reasonably perceived to be unwilling, to consider other perspectives or relevant evidence to the contrary”. (page 4) As with conflicts of interest, the specific circumstances of the individual and his/her positions must be considered in making this assessment.
Thee policies may be easier to understand through examples. These are my own hypothetical illustrations. I have not discussed any examples with IOM, and the disclaimer I stated at the outset applies here as well.
- Does an ME/CFS clinician have a conflict of interest because a new case definition may increase or decrease the number of patients? No. While this could be a financial interest because more patients might equal more business, it is not a current interest. The effect of a case definition on future business is speculative, and does not create a present conflict.
- Is a signatory to the expert letter barred from the IOM committee? On its own, signing the expert letter is a publicly expressed position that constitutes a bias but does not prohibit service. However, if the person is unwilling, or reasonably perceived as being unwilling, to consider other perspectives then that could be significant enough to preclude service on the committee.
- Are the authors of any of the ME/CFS case definitions prohibited from serving on the committee? Potentially yes. In order to fulfill its purpose of creating a clinical case definition, the committee will critically evaluate the existing definitions. This might create a conflict of interest for some or all of the co-authors because it involves reviewing one’s own work. (page 5) The IOM will have to determine the significance of this risk on a case by case basis. However, even if case definition co-authors cannot be appointed to the committee, they can still provide relevant information. There is also the option of the conflict of interest being waived and publicly disclosed, if it is determined that an individual has unique and essential expertise.
Assessing Conflicts of Interest and Bias
While the IOM pays close attention to conflicts of interest and bias, that does not guarantee that they get it right every time. Here is where ME/CFS advocates can play a role. The public will have twenty days to comment on the provisional committee slate. That will be our opportunity to express concerns about the committee balance, potential conflicts of interest, and potential bias. We must invest the effort and energy to identify and submit those concerns. The IOM recognizes that the credibility of a report can be undermined if the process was not competent, balanced, and free of conflicts of interest. Our perceptions of these issues matter, and we have to convey that to IOM.
Here are two hypotheticals to illustrate my point:
- A chronic pain researcher was paid for consultant services to a disability insurance company five years ago. This would not constitute a clear conflict of interest because it is a past financial interest, not a current one. But given the significance of a new case definition to disability insurance companies, I would argue that this creates a significant risk of perceived conflict of interest and the person should be removed from the committee.
- A researcher has made repeated public statements that ME/CFS is simply deconditioning and poor coping skills, and has repeatedly dismissed evidence to the contrary. IOM may consider this a bias, but not significant enough to prevent service on the committee. I would argue that the dismissal of contradicting evidence creates a significant risk that the researcher can be reasonably perceived as unwilling to to consider other perspectives, and should be removed from the committee.
As advocates, we must understand how IOM defines and applies its conflict of interest standards. We must voice our concerns over the provisional slate when it is announced. We have knowledge that IOM does not, and we should not withhold that information from the process.