Update November 13, 2013: I have received a response from the CFIDS Association and have updated this post accordingly. Full comment from the Association below. My thanks to Carol Head, CEO of the CFIDS Association for her swift response to the concerns raised in the original blog post.
“You might have ME/CFS.” That’s the result I got on a diagnostic quiz on the CFIDS Association’s new website. Even apart from that rather ridiculous result – because there is absolutely no doubt that I have ME/CFS – this quiz is extremely problematic.
At best, it is inconsistent with the Association’s stated position on the ME/CFS case definition. At worst, it has implications for the Association’s approach to issues like the Institute of Medicine study on ME/CFS and the PCORI Patient-Powered Research Network. Updated: As you will see from the Association’s response, they have corrected the quiz to be consistent with their position on the case definition.
The Association’s new website launched this week with no fanfare or announcement. This has been in the works for many months, and it uses a cleaner and less cluttered design. The quiz “Do I Have ME/CFS?” is prominent in the site navigation and the home page slide show. I took the quiz this week because when I was a Board member, a patient approached me with concerns about a quiz on the old website.
The new quiz is almost identical to the old one, and my concerns were heightened from the very beginning. The first question of the test is “Have you felt generally unwell for six months or longer?” UNWELL. As so many of us have said so many times and in so many ways, unwell does not even begin to describe the experience of ME/CFS. The term is not defined in the quiz (the dictionary definitions include “being in poor health” and “undergoing menstruation”), nor does it appear in the case definition papers I examined (Fukuda, CCC, Reeves empirical, or ME-ICC). Why not just ask “Have you been sick for six months or longer?” Why use a vague term like unwell, that (to me, anyway) downplays the significance of our symptoms? To be honest, if I only felt unwell then I would be working. Updated: The updated quiz now asks “Have you experienced at least 6 months of severe physical or mental fatigue that you cannot explain, is not a result of ongoing exertion, is not relieved by rest and substantially reduces your activity levels?” Much more appropriate question that Are you unwell!
The test then progresses through ten more questions, assessing things like reduced tolerance for physical and mental activity, sudden onset, and overlapping conditions.
I don’t know who wrote this (or the older) quiz, how it was developed, and whether any attempt has been made to validate it. Question seven asks whether you have had any symptoms from a list of eight, and this is where the underlying case definition shows itself. The list of eight symptoms are the same eight assessed in the Fukuda definition, which requires at least four of them be present for six months along with debilitating fatigue. Because the quiz uses the same list, rather than the symptom clusters used in CCC, I have no doubt that the Association is using the Fukuda definition as the basis for this quiz. Updated: The Association has now updated the quiz to reflect the CCC, rather than Fukuda. See their full response at the end of this post.
This is where the quiz begins to spill over onto policy issues. First of all, the Association’s public position on the IOM study states, “We feel that the Canadian Clinical Consensus* can be optimized as a clinical case definition by applying a standardized methodology of execution, through validation of criteria, and a nationwide dissemination to health professionals.”
If the Association believes that the IOM study should optimize the CCC as a clinical case definition, why would they use Fukuda as the basis for the diagnostic quiz? If the CCC is the right starting point, if optimizing it for widespread clinical use is the Association’s goal, then shouldn’t the Association’s own “test” for ME/CFS use the same starting point? This inconsistency makes no sense to me. Updated: As the CEO of the Association explains below, the quiz is now based on the CCC and consistent with the Association’s position on case definition. However, using Fukuda as the basis of the quiz has deeper implications. You may recall that the Association, along with ten partner organizations, was invited to submit a full application for up to $1 million to build a patient-powered research network. If the Association were to win that grant, would they use Fukuda to identify participants? Since the launch of the SolveCFS BioBank, the Association has responded to questions from the patient community about eligibility criteria, including whether post-exertional malaise (not required under Fukuda) was necessary. The new website for BioBank eligibility simply says, “The SolveCFS BioBank has been built to be a robust resource for ME/CFS research. For this reason it is best that patients have a ME/CFS diagnosis from a healthcare provider. ” No criteria is specified. BioBank participants must complete extensive questionnaires, so information relevant to the various diagnostic criteria can be collected in that way. And one of the unique aspects of the potential PCORI grant is that patients share in the governance of the patient-powered research network. Using Fukuda for this quiz does not mean that the Association believes Fukuda should be used for the BioBank or the PCORI project, but it does at least raise the question. Updated: The quiz is no longer using Fukuda, and other Association activities include opportunities to apply the CCC.
Back to my result: You might have ME/CFS. The results page does not tell me why I received an equivocal answer. Is it because I have sleep apnea? Fukuda excludes me from a CFS diagnosis on that basis. The CCC says, “If a potentially confounding medical condition is under control, then the diagnosis of ME/CFS can be entertained if patients meet the criteria otherwise.” My sleep apnea is perfectly controlled, yet I am still sick. It’s difficult for a short internet quiz to capture this level of complexity, and I doubt the Association’s quiz is intended to do so.
Question 11 asks if I’ve seen a healthcare provider for basic lab tests and a physical exam. The question does not ask what the result of that visit was, nor does it list some of the tests that are increasingly used by ME/CFS specialists like two day CPET. Again, I doubt the quiz is intended to deal in that level of specificity.
Which brings me to my fundamental question about the quiz: What’s the point? An eleven question internet quiz cannot diagnose ME/CFS. If someone appears to have ME/CFS according to the quiz, are they more likely to donate to the Association or get involved in Association activities? Is the goal to encourage people to learn more about Association programs? Or is the goal to let someone know that they should see a doctor? If the latter, then this could be accomplished by saying that if a person experiences severe fatigue that reduces her ability to function for six months or more then she should see a doctor, and offer a fact sheet about ME/CFS diagnosis to assist in the conversation with the healthcare provider. Updated: The Association’s comments below offer more explanation of the intended purpose of the quiz, and the new version of the quiz includes appropriate caveats about its limitations and the need to seek adequate medical care.
Internet quizzes are inherently silly, and useless for a complex disease like ME/CFS.
So is this a bit of a gimmick that is sloppily inconsistent with the Association’s stated position on the case definition? If so, I hope the issue can be clarified as the alternative explanation of Association support for Fukuda is difficult to swallow in our current political climate. As you will see from the update below, the Association has updated the quiz to reflect their support for operationalizing the CCC, and have clarified the purpose of the quiz.
Response from Carol Head, CEO of The CFIDS Association, November 13, 2013:
First, we’ve now updated the quiz on our new SolveCFS.org site so that it is fully in line with CCC. (Note: It may take a short amount of time to propagate through the web.) We had always intended to do this and we’re pleased that it’s now done. I acknowledge that we goofed by missing the update by a couple of days as the new site went live in a “soft launch”. Your blog post pushed our timeline forward. FYI, as we transitioned a very large amount of content from the old site to the new site (more than a decade’s worth!), we’ve had to prioritize the work.
Importantly, the quiz is not new; it has been on our site for many years. We gave it more prominence with the new site because it was among the highest trafficked content. The feedback we have gotten over the years is overwhelmingly positive.
It is primarily meant for people who are newly sick, not for individuals, like you, who are already deeply knowledgeable about ME/CFS. It’s designed to be an aid to those exploring ME/CFS online, trying to figure out more about it and hopefully point them toward additional resources to explore further. While you KNOW you have CFS, many are searching for answers, have no idea what is plaguing them and far too many have doctors unwilling to look toward ME/CFS as a possible diagnosis. While such an elementary tool would not be useful for experienced patients like you, it can be invaluable for someone searching for answers. It can give them some direction, better questions to ask, as they search. Please note that it would be inappropriate to give anyone a definitive “You have ME/CFS diagnosis” online.