After three years of controversy about the purported association between CFS and XMRV, and after two years of waiting for the definitive Lipkin study to be finished (full text of the paper is here), we have our answer. Stick a fork in it, people, because XMRV is done. There are plenty of places to get summaries (such as here and here and here, and a quite revealing interview with Dr. Lipkin here). I want to focus my analysis on the issues that come up the most in patients’ discussions about XMRV.
Did the Lipkin study use the right patients? As far as I can tell, yes. The Lipkin study used the Fukuda and Canadian Consensus Criteria, and only selected patients that had a sudden viral-like onset. Dr. Lipkin said today that this was done so that they chose subjects with a high likelihood of having an infection. The patients were selected by six clinicians around the country (to avoid any limitations based on geography): Dr. Cindy Bateman, Dr. Nancy Klimas, Dr. Anthony Komaroff, Dr. Susan Levine, Dr. Jose Montoya, and Dr. Dan Peterson. If you know anything about CFS, then you recognize these clinicians’ names. Cases were further examined for exclusionary conditions, like hepatitis, HIV, and thyroid dysfunction. Again, this was done to avoid confounding the results by introducing too many variables. Finally, cases had to demonstrate a required level of functional impairment based on responses to four clinical questionnaires. By comparison, the patients in the original Lombardi study met both the Fukuda and Canadian criteria, and presented with severe disability (Lombardi Supplemental). Those samples came from the Whittemore Peterson Institute’s national tissue repository (Lombardi), included samples from geographical diverse areas and included some cluster outbreaks. The Lombardi paper does not specify whether testing was done to exclude other infections, how long the samples had been stored prior to use in the study, or the mean length of illness. The Lipkin study appears to have done everything possible to identify a clean cohort of severely ill CFS patients who were likely to have evidence of infection, and we have much more information about this cohort than we do about the original Lombardi cohort.
Why not test the same patients as Lombardi, et. al? That’s been done. The Singh and Levy studies both retested reported positives from WPI. The Blood Working Group study also used reported positives from WPI and Lo, et al. Those studies were all negative. Dr. Lipkin did not explicitly address this issue in the press conference, but I assume that the point was to start with a fresh cohort and try to replicate the association between these viruses and CFS.
Were the samples handled correctly? As far as I can tell, yes. Dr. Lipkin spent significant time at the press conference this morning addressing the issue of why blood was used in this study. This is important, because after the negative replication attempts began to pile up, XMRV-theory supporters began claiming that the virus could not be detected in blood, but could be found in tissues. The question was raised again this morning by both Hillary Johnson and Deborah Waroff. Dr. Lipkin and Dr. Alter both said that the reason blood was used in this study is because both the Lombardi and Lo studies found XMRV/pMLVs in blood. I’m not sure why this gets lost in discussions about XMRV. The Lombardi paper found XMRV in 67% of the CFS blood samples they analyzed. It wasn’t tissue; it wasn’t lymph; it was blood. So an attempt to confirm the Lombardi and Lo findings has to look at the blood.
For the Lipkin study, blood was drawn fresh from patients and controls between 10am and 2pm, within the same season (to control for possible communal infections and diurnal variations). The blood was treated with EDTA (an anticoagulant) and shipped overnight to Columbia University where the coding and sample splitting was done. The Lombardi Supplemental says that blood samples were treated with sodium heparin, a different anticoagulant. There is no information provided about the time of day or time of year that samples were collected, nor the length of time samples were stored before the study. I’ve seen some claims online that the type of anticoagulant used makes a difference, but I have no information to say for sure either way.
After coding, each sample in the Lipkin study was divided into multiple aliquots. There were four testing sites (more on this in the next section): CDC, FDA, Dr. Hanson’s lab at Cornell, and Dr. Ruscetti’s lab at NIH. Each group received a double set of samples – 2 each of every patient and control. In addition, the groups received artificially spiked positive controls and known negative controls.
Did they use the right tests? Yes. Each of the four testing sites used their own assays in the Lipkin study. CDC used assays previously described in earlier papers to perform multiple rounds of PCR. FDA used a modified version of the assays described in the Lo paper to run PCR. Dr. Hanson performed PCR on samples that had first been cultured in the Ruscetti lab. Finally, the Ruscetti lab used a serologic assay that was slightly modified from what had been reported in Lombardi. Each lab used both negative and positive controls and got accurate results with those samples. Is it a weakness that these labs did not use the precise assays used in Lombardi and Lo? I suppose one could make that argument, but the flip side is that refinements in testing should be used to produce (hopefully) better results. If the labs had been required to use the exact tests used in Lombardi, and the results were negative, it would be a fair question why they were prevented from applying what had been learned since Lombardi. With this design, each lab could use the technique that it felt was most likely to produce accurate results.
Were the positive/negative results reliable? Yes. The Lipkin study testing involved PCR of plasma, PCR of PBMCs, PCR of cultured PBMCs, and serology testing. Remember that each subject sample was not only split among the labs (so subject x was tested by each group), but each sample was sent to each location twice (so subject x was tested twice by lab A, twice by lab B, etc). In order to be counted as a positive result, the Lipkin study states: “Subjects with two positive results in the same sample type were considered positive for XMRV/pMLV.” In other words, subject x had to test positive in two plasma samples or two PBMC samples, etc. to be counted positive. The original Lombardi and Lo studies did not use this redundancy.
The results were clear: No positives were found by CDC in plasma. No positives were found by FDA in plasma or PBMCs. No positives were found by Hanson in cultured PBMCs. (Lipkin study Table 3). Zero, zip, nada, nyet, nothing. Why do they think that PCR is reliable? Because Lombardi used PCR. This is another fact that has frequently been forgotten or swept under the rug during online discussions of XMRV. People have been claiming that you can’t find XMRV with PCR, when the original study used PCR. The Lipkin study included positive and negative controls, checks for contamination, and PCR found NOTHING (except in the positive controls).
The serology results were not clear cut. Approximately 6% of both the patients and the controls were found to be positive using a slightly modified version of the serology assay used in Lombardi. However, the Lipkin study points out that this antibody cannot be validated in a sample known to be positive for clinical XMRV infection, since there is no confirmed case of human XMRV infection. Furthermore, the antibody used may be cross-reactive, meaning it appears positive in the case of a non-XMRV infection. The CFIDS Association article on the study explains this nicely. In the paper, the authors state, “We posit that positive results represent either nonspecific or cross-reactive binding.” The fact that the same number of positives were found in both patients and controls is strongly suggestive that the result is not associated with CFS. In his interview with Nature, Dr. Lipkin said, “If you consider this in the context of the work that shows that XMRV originates in the laboratory, then I think we can probably close the door on this once and for all.”
The consensus reached by all of the study authors could not be clearer: “Our results definitively indicate that there is no relationship between CFS/ME and infection with either XMRV or pMLV.”
Where do we go from here? The Lipkin study ends as follows:
We remain committed to investigating the pathogenesis of CFS/ME and to ensuring that the focus on this complex syndrome is maintained. Studies under way include the search for known and novel pathogens and biomarkers through deep sequencing and proteomics.
There was much discussion at the press conference about the promising avenues for further inquiry, including additional pathogen discovery work, examining host response, and looking at protein and gene products. Dr. Lipkin is involved in some of that work through the Chronic Fatigue Institute. In his interview on This Week in Virology, Dr. Lipkin also said that he is professionally invested in the search for the pathogenesis of CFS.
Much was also made of the fact that the samples gathered for the Lipkin study represent an extraordinary resource for future research. The samples are stored in freezers at NIH and are available for qualified investigators. Application to use these samples must be made through NIH, and Dr. Lipkin stated that two investigators had already received samples and were working on them (although he offered no more specific information). In the TWiV interview, Dr. Lipkin said that it was the panel of investigators on this study that approved applicants for samples, but he didn’t explain how that works. He also said that there would be NIH money available for an RFA to use the samples, but this is the first I heard that. Obviously, an RFA from NIH with money attached would be big news and tremendous progress – so I hope we can get that confirmed by NIH. Finally, he said there is enough plasma stored for 50 labs to conduct studies. I think this is a huge positive outcome; more money was spent on this study by NIH than for any other CFS study in 2011. I am thrilled at the prospect that we might learn more from these samples.
Is XMRV really over? Yes. On top of the definitive statement in the paper, Dr. Mikovits and Dr. Alter firmly closed the door on XMRV/pMLV and CFS today. Dr. Alter said this study was “quite definitive.” Dr. Mikovits said the study rigorously excluded the earlier findings and that XMRV is “simply not there.” Furthermore, she said she was “100% confident in the results.” It can’t be said any plainer than that. Dr. Mikovits is on record as denying an association between XMRV/pMLVs and CFS. I’m fairly certain there will be some people who are still not convinced, but they will have to make their claims in spite of what Dr. Mikovits said herself.
What about the prostate cancer papers? That’s done, too. A study published today in PlosOne today concludes “In summary, our findings do not support any association between XMRV infection and prostate cancer, and by extension indicate that XMRV has never replicated outside of the laboratory setting. The initial discovery linking XMRV to prostate cancer in 2006 arose from laboratory contamination of clinical samples by an XMRV-infected LNCaP cell line. In turn, the LNCaP cells were most likely previously infected by 22Rv1, from which XMRV almost certainly originated through in vivo passaging of the CWR22 xenograft in mice.” Update at 8:10 pm: the original XMRV and prostate cancer study from 2006 is now retracted.
Another CFSAC Done Gone
The CFS Advisory Committee held its second meeting of the year on October 3-4, 2012. Last time, I organized my summary around the good, the bad, and the WTF moments. This time, I am organizing around the discussion themes. Overall, I felt this meeting was more substantive than in the past. There were even hints of introspection and data driven discussion.
Agency Updates
Assistant Secretary Dr. Howard Koh attended the opening of the meeting, and provided an update on the Department’s efforts since the last meeting. I was watching the meeting via webcast, and my feed froze during Dr. Koh’s comments. However, the portion I did see contained nothing new. Dr. Koh did not provide any details on the Ad Hoc Working Group beyond what we already know. Unlike previous meetings, he did not take questions from the members. Although he said “the committee has gotten stronger,” he did not announce the appointment of a new member to replace Dr. Rose. The committee bylaws require vacancies to be filled within 90 days, so the failure to appoint a replacement is a violation of the bylaws.
Both the FDA and Social Security Administration gave substantive presentations to the Committee. In my opinion, this was the high point of the meeting. Both Dr. Sandra Kweder (FDA) and Arthur Spencer (SSA) provided detailed information about their agencies and CFS related data. Dr. Kweder reported on the status of nine investigative new drug applications for CFS. Mr. Spencer provided disability data that the committee has been requesting for years. The overall approval rate for Social Security disability among cases where CFS is the primary diagnosis is 21%, in contrast to a national overall rate of 30%. I’m looking forward to seeing the slides from both these presentations because there was a lot of good information in them.
NIH and CDC also gave detailed updates. Dr. Susan Maier (NIH) reported that several new members were added to the Trans-NIH ME/CFS Working Group, including Dr. Harvey Alter. It’s very good news that Dr. Alter is staying involved in CFS despite the end of XMRV. Dr. Maier also provided (for the first time) data on the acceptance rates for CFS-related grant applications. The overall success rate is 25%, and in FY2012 the success rate is 18%. These rates are higher than the overall rate across NIH. Most of CDC’s report was focused on various education initiatives including CMEs offered through Medscape and CDC, as well as video of patient vignettes for the MedEd Portal that will be finished next year.
That ToolKit
CDC announced that after extensive debate, they have decided not to remove the ToolKit from the CDC’s website. Dr. Beth Unger said that they believe it should be available until it can be updated to reflect the other website revisions. Surprisingly there was little fanfare or reaction to this announcement. At its June meeting, the CFSAC had recommended that the ToolKit be removed. Dozens of patients testified in June and at this meeting that the ToolKit is harmful misinformation, and a coalition of groups and individuals submitted a detailed position paper to CDC in support of that June recommendation. Despite all that, the CDC has decided to keep the ToolKit. There was no pressure or reaction on the record from CFSAC members. No one asked why this decision was made, and no one besides Mr. Steve Krafchick pointed out that CDC is ignoring the CFSAC recommendation. CDC got off very lightly on this score, and I still can’t believe that no one raised a stink about it.
Chicken, Meet Egg
As I said above, Dr. Maier presented data on the approval rates for CFS applications to NIH. In light of that data, Dr. Gailen Marshall asked the committee why they thought NIH funding was so low. The high approval rate suggests that the problem is not NIH’s willingness to spend money but that there are few applications coming in. Dr. Mary Ann Fletcher spoke frankly about the perception in the research community that it is extremely difficult to get funding. She cited an unnamed researcher who left the field because of it, and pointed out that Dr. Nancy Klimas is quite successful in her applications for HIV and Gulf War Illness funding but not CFS. Eileen Holderman pointed out that the illness name, and particularly CDC’s definition and use of the name, dilutes our disease into simple chronic fatigue. This discussion tied in nicely with public comment by Matthew Lazell-Fairman and others that the decades of neglect and active denigration of the disease by CDC and other federal policy makers has created the climate where researchers believe they will not get funding. This circular discussion recurs at every single CFSAC meeting, but this time it led to the recommendation of creating a CFS study section at NIH.
Biomarkers
Biomarkers in CFS was a recurring theme on both days of the meeting. Dr. Jordan Dimitrikoff gave a presentation on the challenges faced by those studying Chronic Pelvic Pain Syndrome to identify biomarkers, not just for diagnosis but also to generate hypotheses about potential treatments. This is very similar to the approach of several CFIDS Association grants, in which a symptom or biomarker is queried to identify a possible drug to address that symptom or marker. Dr. Dimitrikoff acknowledged that the “true experts are the patients,” and he advocated setting aside cognitive bias to evaluate data and create learning networks. Dr. Fletcher then presented data from her research with Dr. Klimas and Dr. Gordon Broderick which identified different gene expression profile networks in CFS and Gulf War Illness patients, especially in immune pathways. This presentation complemented Dr. Dimitrikoff’s nicely, giving very specific examples of how biomarkers could lead to identifying potential drug treatments.
Discussion covered several very important points. First, that the case definitions are producing too much variability among patients. Without animal models, it is very difficult to study patients in a meaningful way without narrowing down the clinical presentation. Second, biomarkers must be distinctive in order to be useful. It is not enough to distinguish healthy controls from CFS patients. Biomarkers must distinguish between CFS and other chronic inflammatory conditions. In other words, if a biomarker cannot distinguish CFS from rheumatoid arthritis or lupus then it is of less utility than one that can. This has implications for both diagnosis and treatment trials.
Dr. Kweder’s presentation on the FDA and CFS treatment trials focused on the importance of outcome measures in order to quantify whether a treatment is having an effect. Outcome measures are not necessarily biomarkers; for example, there is no biomarker for migraines. But the more objective and quantifiable an outcome measure is, the more useful it is in clinical trials. Dr. Kweder pointed out that CFS has no single accepted case definition, no quantifiable way to measure symptoms and no biomarker for disease presence or activity. These are barriers to clinical trials, and partially to blame for the lack of trials in CFS. Dr. Kweder cited fibromyalgia, irritable bowel syndrome and depression as examples of conditions that received more clinical trials when those barriers were addressed. The FDA stakeholders’ meeting in spring 2013 will focus on identifying valid reliable outcome measures for CFS clinical trials.
There is a significant difference of opinion about whether we already have biomarkers and outcome measures for CFS. Dr. Fletcher and others cited a variety of measures already in use by researchers and clinicians. Dr. Fletcher was adamant that biomarkers did not need to be exclusive to CFS in order to be useful. Dr. Kweder said that a quantifiable biomarker or outcome measure must correlate to how the patient feels or fares. She also noted that heterogeneous conditions need larger clinical trials, so identifying subgroups can help target a treatment to those it is most like to help.
Case Definition
This is such a controversial topic, perhaps I should not expect a discussion of it to go smoothly, but the committee struggled once again to chart a way forward. Dr. Nancy Lee said that the case definition issue was discussed in at least one meeting with Secretary Sebelius, and that the Secretary was clear that the case definition must come from the medical community. Dr. Lee said that a recommendation from the committee that the Secretary endorse or adopt a specific definition will go nowhere. Dr. Marshall tried to focus discussion on designing a process that would produce a definition, but the committee quickly got snarled in the complexity of the problem.
One of the most contentious issues was whether the medical community has already endorsed a definition. Mr. Krafchick pointed out that the IACFS/ME used the 2003 Canadian Consensus Criteria in writing the Primer, and that it was the body of experts in this condition. Dr. Lee argued that the entirety of the medical community needed to endorse a definition, and Dr. Fletcher countered that this was not only unrealistic but was not a standard applied to any other illness. The root of this disagreement is the status of the IACFS/ME versus other medical societies. When the American College of Rheumatology endorsed a definition of fibromyalgia, the rest of the medical community accepted it because the ACR is a defined sub-specialty of medicine. Dr. Marshall drew a sharp distinction with the IACFS/ME, which is not a sub-specialty that offers board certification, and insisted that the American Colleges must have input into the definition in order for it to be widely accepted. This led to another vigorous argument over whether ME/CFS experts should address the definition or if non-experts should be invited to provide input and endorsement. The committee split over this, and in the end voted 5-4 (with one abstention) in favor of limiting input to the ME/CFS experts at this stage.
The other thorny question was whether to start with one of the existing definitions (Fukuda v. Canadian Consensus 2003 v. International Consensus 2011 – and different members referred to these papers by different names which made it even more confusing) or start from scratch. Dr. Dimitrikoff and Dr. Dane Cook recommended learning from definition processes in other illnesses such as lupus or IBS. Dr. Ermias Belay and Dr. Unger from CDC both advocated for a data driven process, relying on their multisite study that should be completed next year (although they did not promise a finished paper next year). This led to frustration among Dr. Fletcher, Mr. Krafchick and others about delay and the need for immediate action and leadership. After much wrangling, the committee settled on the 2003 Canadian Consensus Criteria as the starting point for a process to produce a clinical definition (see text of recommendation below).
One thing that got lost in this discussion was the role of patients. My impression from the preliminary discussion on October 3rd was that Dr. Marshall and others recognized patients as important stakeholders in this process. But the role of patients was not discussed on October 4th, and the final text of the recommendation did not specifically include or exclude us. I don’t think it is an exaggeration to say that there will be hell to pay if patients are excluded from the process of creating a new case definition.
Committee Effectiveness
At the end of the first day, Dr. Marshall invited committee discussion on recommendations or other issues for the next day’s session. This led to a discussion of how effective the Committee is in its work. Dr. Lisa Corbin and Mr. Krafchick expressed concern about the committee’s recommendations not receiving feedback or action. Dr. Lee stated that a response to the June recommendations is still in preparation, to which Mr. Krafchick noted that 111 days had passed since that meeting. It was also clear, once again, that members are not receiving materials related to the meetings. Dr. Krafchick had not seen Assistant Secretary Koh’s letter in response to the November 2011 meeting, and was told that “it’s on the website.” Does this mean that such key documents are not sent to members, or that they are not even notified when such information is posted to the website? Members were asked to read several articles in preparation for the case definition discussion the next day (again suggesting that they were not sent in advance), but the existing CFS case definitions were not among them. To be frank, I think it is appalling that more preparation is not done for these meetings and it clearly hampers the effectiveness of discussion. I also wish that members would give more thought to the phrasing of their recommendations in advance. The editing-by-committee process at the end of each meeting is frustrating to watch, and a little more care in proposing motions might help with that.
Final Recommendations
These are the recommendations to the extent I was able to capture the language. The final version may vary slightly: