Patient Questionnaires in Orthodontics

Purpose of questionnaires

Questionnaires can be used in research, quality improvement or to gather clinical data. They are tools for collecting data, rather than a specific methodology, so can be used in a diverse range of research study designs, including randomized controlled trials, observational studies and surveys, as well as for audit and service evaluation. A single questionnaire provides cross-sectional data for a group of people at a specific point in time, while repeat questionnaires allow data to be collected longitudinally.^2,3

Questionnaires can be used in healthcare research to assess:⁴

Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) are increasingly important for measuring the quality of care.⁵

Advantages and limitations of questionnaires

Questionnaires have gained widespread use, as they are relatively quick, easy and inexpensive to administer, and data can be collected from a large number of people across a large geographical area. Respondents can complete questions in their own time and preferred setting, with the reassurance of anonymity and confidentiality. Instruments can be used across different time points to measure change over time, for example, to monitor experience or identify change in behaviour or opinion.^2,3

Questionnaires are useful for gaining information about opinion or behaviour but are less able to explain why a phenomenon has occurred. As with all data collection tools, questionnaires require careful design, sampling and data collection methods to gain meaningful results. Poorly worded questions, inappropriate response formats, errors in selecting the respondent group and distributing the questionnaire can lead to poor quality data.^2,3

Types of data generated by questionnaires

Different types of data can be collected, depending on the intended purpose of the information gathered. Closed questions provide quantitative data, while open questions offer the opportunity to collect more detailed qualitative data. Quantitative data are often judged to be easier to manage; however, there can be complexities in correctly analysing and interpreting numerical data. It is important that the intended use of the data is clear when deciding what type of data are collected and the most appropriate response format, and that the appropriate analysis is then used for the type of data collected. For detailed information about appropriate statistical analysis, further reading is recommended.⁶

Categorical data include nominal and ordinal data. Nominal means labelled categories with no inherent order, for example, gender and ethnicity. Ordinal data are categorical data with an order, for example, age categories or a Likert scale. Ordinal data cannot be assumed to have interval properties, meaning that the relationship between the categories may vary across the scale. For example, on a 5-point Likert scale for measuring the experience of treatment (Figure 1), it cannot be assumed that a change from ‘Excellent’ to ‘Very good’ is the same as a change from ‘Fair’ to ‘Poor’.⁷ Categorical data produce counts, frequencies and proportions, for example 21 people, or 5% scored the service as ‘Fair’.

Discrete data are data that can only take certain values; there are a finite number of values possible. Discrete data are usually counts, for example, the number of people with a Class II malocclusion or the number of female respondents in a survey. Values cannot be subdivided meaningfully, for example, it does not make sense to describe 2.7 people with ClassIImalocclusion.

Continuous data are numerical data where any value is possible within a range, depending on the sensitivity of the measurement tool. Examples include age, height, overjet. Continuous data have interval properties, meaning the difference anywhere on the scale is the same, for example, the difference between 2−4 mm is the same as 4−6 mm.

Questionnaires can be misused if data are used erroneously, for example converting categorical responses into numerical data and treating them as continuous. Without a validated scoring system with appropriate item weighting, creating summative scores is likely to lead to statistical analysis that is difficult to interpret in a meaningful way.⁶

Questionnaire properties

It is essential that the chosen instrument is able to collect valid, reliable and unbiased information to answer the question.⁸ Validity is defined as the ability of the questionnaire to measure the variable of interest; this is essentially to decide whether the questionnaire results reflect ‘the truth’. For example, does the respondent interpret the questions and response options in the way intended and is the respondent able to give an answer that reflects their true opinion or experience? Reliability is the ability of the questionnaire to produce consistent data. Most commonly this is judged by whether repeat questions from an individual provide the same answer, or whether questionnaires that have been administered or analysed by different people find the same result. Issues with validity can impact on reliability, as respondents are more likely to answer in an erratic way if they are not able or willing to give a true answer.

Bias is the term given to an erroneous result arising from systematic errors in the collection of data. A number of factors may reduce the validity and reliability or increase the risk of bias of questionnaires (Table 1). Issues with validity can impact on reliability because respondents are more likely to answer in an erratic way if they are not able or willing to give a true answer. Other important concepts in questionnaire design are sensitivity, which is the ability of a questionnaire to measure change, and responsiveness, which is the ability of the questionnaire to measure change that is clinically significant and relevant.

Increasingly, including the target respondent group throughout the design and development of data collection tools is advocated to optimize relevance and acceptability.⁹ Ideally, any questionnaire should undergo a formal testing process with an adequate sample from the target population prior to use to allow the different properties of the questionnaire to be assessed.¹⁰

Selecting a questionnaire

When selecting or devising a questionnaire, it is important to define the question and perspective of interest, for example, patient, parent or clinician, and to identify the target population, the outcome of interest and the type of data required explicitly, based on how they will be used. It is tempting to use the opportunity afforded by a questionnaire to collect as much data as possible, but only data relevant and necessary to answer the question should be collected.

Where possible, it is preferable to use an existing questionnaire that has been validated with the target population and has been shown to be reliable. It may be necessary to alter questions or format to suit the desired use. The validity and reliability of a questionnaire following alteration, or if used with a new respondent group, should be considered during analysis and interpretation.¹¹ Where no suitable questionnaire exists, it may be necessary to develop an instrument. Best practice guidance for questionnaire design can support the development process.^12,13 Some of the key components of questionnaires are summarized in Table 2.

The questionnaire purpose and planned mode of administration will influence the approach to sampling, for example, whether a representative or purposive sample is preferable. Convenience sampling is often the most practical approach for audit and usually this will provide a satisfactory sample, but results may have limited generalizability to other settings or populations. For clinical questionnaires, the need for clinical information will determine when, how and to whom the questionnaire is distributed. The planned method of data collection may also influence which questionnaire is most appropriate. Remote data collection, for example, by post or online, provides limited opportunity to explain questions or explore responses, so simple, clear tools are essential. Face-to-face data collection allows clarification of questions and answers, so questionnaires may be selected that allow further exploration, for example questionnaires with open questions, or more complicated instruments.¹⁴

Patient questionnaires in orthodontic research

In research, questionnaires have been used to explore a number of areas including oral health-related quality of life and the impact of malocclusion, experience of treatment and treatment outcome. Quality of life (QoL) questionnaires may be generic or condition-specific, for example, questionnaires that explicitly measure the impact of hypodontia^15,16 or facial deformity.^17,18,19 Condition-specific QoL questionnaires have been shown to be more discriminative than generic measures owing to their ability to measure more subtle aspects of malocclusion²⁰ and a study testing the validity of a generic measure with orthodontic patients highlighted the need for child-centred, malocclusion-specific questionnaires.²¹ Surprisingly, to date, there are no condition-specific QoL measures for measuring the impact of cleft lip and palate,^22,23,24 although generic measures have been validated for this group.²⁵

Questionnaires can also be used in research to measure treatment experience or outcome for treatment as a whole, or for a single treatment modality. For example, satisfaction questionnaires have been used to gain patient feedback on different types of appliances^26,27 and in randomized controlled trials to provide a patient-centred outcome.^28,29

A summary of some orthodontic-specific questionnaires that have been developed and tested is given in Table 3. In addition, many non-validated questionnaires have been used in orthodontic research, particularly to assess satisfaction with treatment outcome. Usually a new tool was judged to be necessary because the existing tools were not suitable; however, the development and testing process is often poorly described. This can lead to challenges when attempting to assess the quality of the study and translate the findings into clinical practice.

Patient questionnaires in quality improvement in orthodontics

In the NHS, quality improvement aims to improve patient outcomes and the delivery of care through seven areas of activity, collectively called clinical governance.⁴² Data are required to measure and evaluate performance for clinical governance purposes and questionnaires are one method for obtaining these data, often from a patient perspective. An interesting paper from 2014 describes an experience-based design approach to measure feedback of patient experience in orthodontics to target service improvement.⁴³

One of the most widely used questionnaires is the Friends and Family Test (FFT), launched in 2013 and now in use in most NHS-funded services in England.⁴⁴ The FFT aims to provide patients with an opportunity to give feedback on their experience and identify areas for improvement. The results of the FFT are published regularly to promote transparency and allow comparison across services. There are, however, some concerns over the ability of the FFT to capture patient experience and its success as a performance measure⁴⁵ and, as with other generic measures, it may not identify service-specific issues.

The other key quality improvement area where patient questionnaires are regularly used is data collection for clinical audit. The audit process aims to monitor clinical practice against an agreed standard of care and questionnaires are commonly used to collect information about patient satisfaction with service delivery and treatment outcome. Often audit questionnaires are developed for one specific audit and the lack of standardization can make it difficult to compare findings across audits. Few audits report how the questionnaire was developed and tested. While this does not necessarily mean the information obtained is not useful for quality improvement purposes, if results do not reflect what is observed clinically, the instrument may require careful examination of its validity.

Patient questionnaires for clinical use

Patient questionnaires may be a useful tool to collect information prior to, during or after orthodontic treatment to assist clinicians with planning or delivering treatment. Pre-treatment questionnaires can be used to establish the impact of malocclusion and the motivation for treatment to determine the patient's expectations from treatment and to focus the consultation and relevant information provision. During treatment, questionnaires may be used to monitor treatment experience and to identify any changes in goals and expectations. Post-treatment questionnaires tend to focus on satisfaction with outcome and overall experience of treatment.

As with audit questionnaires, many clinical questionnaires are developed locally and are not standardized or validated through a formal process. In these cases, it is up to the clinician and patients to determine whether the questionnaire is able to gain information that is accurate and useful and often a pragmatic approach is taken. The Malocclusion Impact Questionnaire^30,31 is the only questionnaire identified by the authors that explicitly states that it is for both clinical and research purposes; however, other research questionnaires may be useful in routine clinical application to gain patient input.

Conclusions

Questionnaires have a useful role in research, quality improvement and clinical services. It is important that the purpose and perspective of the questionnaire is defined and the chosen instrument is relevant and able to engage the target population to gain meaningful results. The validity and reliability of an instrument is fundamental and measures should be taken to reduce potential biases as much as possible.