PACE-Gate

It is International ME/CFS Awareness Day, and what is most on my mind is the status of PACE-Gate: the controversy over the design, conduct and results of the PACE trial. PACE is the largest clinical trial in ME/CFS, and it compared graded exercise therapy, adaptive pacing, supportive medical care, and cognitive behavioral therapy (explicitly targeting patients’ false beliefs about their illness). The first paper on the results was published in The Lancet in 2011, but there are dozens of flaws in the design and conduct of the study. Requests for data and reanalysis of conclusions are in process across many fronts. While PACE is of intense interest to the ME/CFS community because its results have been used as the basis of treatment recommendations, the issues have significant implications beyond ME/CFS for both scientists and the journals involved.

PACE-Gate was launched by academic/journalist David Tuller, DrPH (Lecturer at the School of Public Health and Graduate School of Journalism at University of California, Berkeley) with his 2015 article series, Trial By Error, which dismantled the bad science in PACE. Tuller was assisted by the work of many “citizen scientists,” ME/CFS patients who worked for years to carefully analyze the study and as much data as they could obtain.

For five years, ME/CFS patients have known and insisted that the results of the study  – even if it had been perfectly designed and conducted – are not applicable to us because the subjects were fatigued, and were not all suffering from ME/CFS. But as Dr. Tuller covered in such detail, the study was the opposite of perfectly designed and conducted. Subjects could worsen during the course of the trial and still be counted as recovered at the end because of post hoc changes in study design – and that is just one of dozens of fundamental flaws in the PACE trial. Yet still, the study is hailed as the largest clinical trial in our disease and as proof that we only need to exercise and shed the “belief” that we are sick in order to recover. Reminiscent of launching a thousand ships, Dr. Tuller’s articles kicked the bad science scandal into high gear, and efforts to expose the truth about PACE began to multiply.

Attempts to secure data from the PACE trial – anonymized to protect patients, of course – are of interest to scientists inside and outside the world of ME/CFS. As they have done for years, the PACE authors continue to block these efforts. In November 2015, Dr. James Coyne (Professor of Health Psychology at University Medical Center, Groningen, the Netherlands) filed a request for the data from the PLoS ONE article on the cost-effectiveness of the treatments evaluated in the PACE trial. King’s College London refused the request, claiming it was “vexatious.” This set off a flurry of protests from scientists, including those at Retraction Watch who called the reasons for rejection “absurd.” Dr. Coyne made PLoS aware of his request, since its policies require authors to make data available for papers published in PLoS journals. As of May 9, 2016, Dr. Coyne has still not received the data, and it is unclear if PLoS will actually enforce its data sharing policy. Dr. Coyne told me, “I am leaving things in PLoS’s hands. I am considering a range of responses if the Journal does not produce the data or retract the paper.”

Graphic credit: Radhika Bhatt, U.S. Department of Commerce Data Service

In December 2015, Dr. Tuller was joined by Dr. Vincent Racaniello (Professor of Microbiology and Immunology, Columbia University), Dr. Ron Davis (Professor of Biochemistry & Genetics, Stanford University) and Dr. Bruce Levin (Professor of Biostatistics, Columbia University) in filing a request for “the raw data for all four arms of the trial” on the original primary outcomes and measures of recovery. This request was denied by Queen Mary University of London in January 2016. Dr. Tuller told me that they appealed this decision, but QMUL declined to undertake a review. Tuller and the other scientists filed an appeal with the Information Commissioner’s Office, but they expect it to take many months for the ICO to come to a decision.

Valerie Eliot Smith has been tracking the slow progress in the case of another PACE data request to QMUL. Alem Matthees filed a request for PACE data, QMUL refused, and he appealed to the ICO. The Information Commissioner ruled in his favor, but QMUL appealed that decision to the First-Tier Tribunal (Information Rights). That hearing was held at the end of April and a decision is expected very soon.

In the absence of direct access to the data, multiple requests for independent review of the PACE data have been made. In November 2015, Dr. Vincent Racaniello and five colleagues published an open letter to Dr. Richard Horton, Editor of The Lancet, urging the publication to “seek an independent re-analysis of the individual-level PACE trial data.” After three months without a response from Dr. Horton, Dr. Racaniello published the open letter again, this time joined by an additional thirty-six scientists. Dr. Racaniello told me that Dr. Horton “invited us to submit it as an ‘official’ letter to Lancet, which would be published; and we did that. No response from him or Lancet yet.” Dr. Horton did not respond to my request for comment on when this letter will be published.

At the same time, twelve U.S. ME/CFS organizations asked the Centers for Disease Control and the Agency for Healthcare Research and Quality to examine the issues raised in Dr. Tuller’s investigation. In part, the organizations asked that the CDC remove all recommendations based on the PACE trial from its medical education material, and asked AHRQ to revise its systematic evidence review in light of the concerns regarding the PACE trial and the Oxford case definition. Mary Dimmock informed me that AHRQ agreed to “reanalyze results by case definition and also separate out CBT from general counseling,” but there is no timeline for this revision of the systematic review.

Advocates have not been silent, either. More than 2,200 people joined in the requests of CDC and AHRQ. More than 12,000 people signed an MEAction petition demanding retraction of claims made in PACE papers in The Lancet and Psychological Medicine, and further asking that independent researchers be given full access to the anonymized raw data from the trial.

I asked Dr. Tuller where he thinks the PACE controversy is headed. He said, “I have to believe that the scientific community will gradually demand that it get retracted or the pressure to release the data will be so overwhelming that QMUL will finally decide that its institutional interests diverge from those of the authors. Perhaps the tribunal will make a strong argument in favor of data release–or perhaps the opposite. I don’t know.”

Dr. Racaniello was not as optimistic. He said, “I think they are going to ignore, obfuscate, and give their usual responses until we are all dead. I don’t have hope that the PACE authors, or Lancet, will respond in any meaningful way until there is more of an outcry.”

What can we do? Dr. Racaniello said, “Share David Tuller’s articles on what is wrong with the study. If these are too complex, use his bullet point summaries. The patient community has been active but we need more scientists and physicians to weigh in on the problem.”

I think the opinions of scientists are the key to a solution here. Dr. Racaniello told me, “I think it’s bad science that was poorly reviewed. Why Lancet is not willing to re-investigate is a mystery.” The Lancet  has faced this problem before. PLoS appears to be back pedaling from enforcing its own data sharing policies. The PACE authors and their universities are refusing every data request at every turn. And unlike the previous five years, the Lancet, PLoS, PACE authors, and universities are not doing this to allegedly vexatious patients who are allegedly part of an allegedly vast conspiracy to make their lives difficult. The Lancet, PLoS, PACE authors and universities are doing it to scientists not previously associated with ME/CFS.

My experience as an ME/CFS advocate has taught me that this disease is the perfect case study for many issues in research, policy and medical care. The PACE trial is one such example. All scientists who care about strong peer review, open data, and verification of results should care about PACE, even if they don’t give a hoot about ME/CFS. Anyone who cares about translating good science into policy, medical education and healthcare should care about PACE, even if they are not interested in ME/CFS.

But as I said last year, and I say again this year: We will not see the change that we need and the resource investment that we need until we find a way to leverage the awareness into action. As advocates, we should do everything we can to communicate about the PACE issues to scientists outside the ME/CFS community. They can help boost the signal enough to get the Lancet, PLoS, PACE authors and universities to pay attention and take the necessary actions to bring PACE-Gate to the correct conclusion.