Insights from testing MYMOP and PEI
by Dr Mairead Murphy
Senior Research Associate
Centre for Academic Primary Care
In research studies, we use patient-reported questionnaires a lot to test if healthcare interventions work or not. Given that the correct completion of these questionnaires underpins our research conclusions, it’s important to know how patients interpret them.
We decided to look more closely at how people interpreted questionnaires. We asked people who had recently visited their GP to complete two commonly-used questionnaires in primary care and tell us their thought processes as they did this. The questionnaires we chose were:
- The patient enablement instrument (PEI): This questionnaire has six questions, all related to how “enabled” a patient feels following a GP consultation. It asks patient to rate their improvements in coping, understanding, self-help and other aspects of “enablement”.
- Measure Yourself Medical Outcomes Profile (MYMOP): This questionnaire measures symptoms, daily activities and wellbeing. Patients write down the symptoms and activities that matter to them, rather than selecting from a list.
We found that people don’t always interpret the questions as researchers think they do. Questionnaires which seem clear can mean completely different things to different people.
Broad concepts cause problems. For example, the first question on the PEI asks people about how they are “coping with life.” We noticed in our interviews that people take a long time to answer the question. In the end, “not coping” is interpreted by some as an inability to function and others as any minor irritation. In fact, each of the six questions in PEI had a similar dual interpretation. This might make the aggregate results questionable.
Specific words cause problems. For example, “illness” is a word commonly used in questionnaires. But half of the patients we interviewed didn’t think they were “ill”, even though they were seeking healthcare. This included people with epilepsy, polycystic ovaries, a heart condition with valve fitted, an allergic rash, a bruised leg and a baker’s cyst!
Unusual scales cause problems. Most questionnaires ask patients how they are now. But in the PEI, patients rate change (improvements) resulting from a GP appointment. Many people who like and respect their GP want to give a high score, so say they have improved even when they haven’t. Others can’t decide whether changes resulted from the GP appointment or not. And some simply forget that they are supposed to be rating their change and rate how they are feeling now instead.
Free-text questions cause problems. In MYMOP people are instructed to write down their symptoms (rather than selecting from list). Lots of people write down something other than symptoms. This happens even when they understand the question. This is probably why the author says MYMOP should be completed through interview only, guidance which researchers should follow.
Uncovering these kinds of issues through interviews with patients is important, because quantitative testing doesn’t always uncover ambiguities, and questionnaires may be deemed “psychometrically rigorous”, but still have underlying problems. Sometimes questionnaire developers don’t document how they intend the questions to be interpreted, which makes this kind of testing difficult.
In conclusion: questionnaire developers need to make sure they document how they intend questionnaires to be administered and interpreted and, where possible, researchers need to pre-test patient-reported questionnaires by asking people what their thought processes are as they complete them.
This blog is a summary of a paper published on 27 September 2018:
Patient understanding of two commonly used patient reported outcome measures for primary care: a cognitive interview study. Mairead Murphy, Sandra Hollinghurst, Chris Salisbury, BMC Family Practice 2018 19:162 September 2018
The research was funded by the NIHR School for Primary Care Research and the time spent writing the paper was funded by the Avon Primary Care Research Collaborative (now BNSSG Research and Evidence).
About the National Institute for Health Research
The National Institute for Health Research (NIHR): improving the health and wealth of the nation through research.
Established by the Department of Health and Social Care, the NIHR:
- funds high quality research to improve health
- trains and supports health researchers
- provides world-class research facilities
- works with the life sciences industry and charities to benefit all
- involves patients and the public at every step
For further information, visit the NIHR website www.nihr.ac.uk
This work uses data provided by patients and collected by the NHS as part of their care and support and would not have been possible without access to this data. The NIHR recognises and values the role of patient data, securely accessed and stored, both in underpinning and leading to improvements in research and care. Read more