From Volume 5, Issue 3, July 2012 | Pages 82-88
There is an increasing volume of research undertaken within orthodontics and with this comes a need to evaluate what is available. This short series aims to help the orthodontist revise basic concepts of critical appraisal and pertinent statistics.
Information is collected for specified variables to provide the data to address a chosen research question. They are called variables as they provide information as to how individuals or items vary. The appropriate summary measures and statistics to use depend on how the variable is classified.
Classification is within one of several categories. If there are only two categories (for example, male or female) then the variable is known as binary or dichotomous. If there are more than two categories, then these may be either ordered (for example, low, average and high angle) so the variable is known as ordinal, or unordered, with the categories bearing no mathematical relationship to each other (for example, country of birth or ethnicity) when the variable is known as nominal. The number in each category may be expressed as proportions or percentages of the total, or as the risk or the odds of being in one category relative to another. A contingency table is often used to summarize the data.
For example, in a study examining the aetiology of temporomandibular joint ankylosis, the researchers investigated whether patients in their sample had previous middle ear infection. The data might be displayed in a contingency table (Table 1).
| Ankylosis | No ankylosis | TOTAL | |
|---|---|---|---|
| Infection | 44 | 8 | 52 |
| No infection | 7 | 41 | 48 |
| TOTAL | 51 | 49 | 100 |
Measurements are made on some form of numeric scale, and usually values can take either whole numbers (for example, number of clinic visits or parity) or lie on a continuous scale (for example, patient height or cephalometric data). Numerical data are usually summarized using two measures – one to give an average and the other to describe the spread.
These terms are closely related yet distinct. They describe how commonly a disease or condition occurs in a population.
Many studies collect information on two groups and make comparisons between them. Commonly, comparisons are made of new or standard treatment groups or of diseased individuals versus healthy controls. The appropriate comparison statistic to use depends on the type of variable being compared.
If the variable is numerical, then the difference in means or medians will provide a suitable summary to compare the groups.
However, if the outcome variable is categorical, several different summaries can be used to quantify the likelihood of being in one of the categories:
The difference in the likelihood of being in one category may then be compared between the groups in one of several ways:
To clarify this, suppose there are a+b individuals who are exposed to a factor, of whom a have the disease, and c+d individuals, who are not exposed and of whom c have the disease (Table 2).
| Disease status: Positive | Negative | TOTAL | |
|---|---|---|---|
| Exposed | a | b | a+b |
| Not exposed | c | d | c+d |
| TOTAL | a+c | b+d | a+b+c+d |
It can therefore be seen that:
The contingency table from the study looking at the aetiology of temporomandibular joint ankylosis may be enhanced (Table 3) to illustrate the concepts explained in Table 1. Here, a = 44, b = 8, c = 7 and d = 41.
| Disease status: Ankylosis (positive) | No ankylosis (negative) | TOTAL | |
|---|---|---|---|
| Infected (exposed) | (a) 44 | (b) 8 | 52 |
| Not infected (not exposed) | (c) 7 | (d) 41 | 48 |
| TOTAL | 51 | 49 | 100 |
44/52 = 0.846 or 84.6% of patients reporting past infection demonstrated ankylosis (EER) compared to 7/48 = 0.145 or 14.5% of uninfected patients (CER).
The absolute risk reduction is 14.5 – 84.6 = -70.1%. This means that 70.1 more of those reporting past infection have ankylosis (a negative risk reduction can be interpreted as an increase).
The number needed to treat is 1/-70.1 = -0.014. This means that on average, for every 1.4 who are infected, one extra will develop ankylosis.
The relative risk is (44/52) / (7/48) = 0.846/0.145 = 5.83. This means that those who are infected are, on average, 5.83 times more likely to develop ankylosis.
The odds of a previously infected patient having ankylosis are 44/8 = 5.5, and the odds of an uninfected patient having the same are 7/41 = 0.17.
The odds ratio of an uninfected patient having ankylosis is 5.5/0.17 = 32.35, which shows that the odds of ankylosis are considerably higher in those with a past history of infection.
A diagnostic or screening test is evaluated by assessing whether some potentially predictive feature is present or not in individuals with and without disease. If the feature is predictive of disease, then it will often be present in those with disease and absent in those without. There are a number of summaries, defined below, that are generally used to quantify how good the test is. All of these summaries can take values between 0 and 100% and an ideal test would have high values of all four.
In addition, two other terms are often used:
For example, the use of pulse oximetry to diagnose obstructive sleep apnoea (OSA) has been explored owing to the ease of administration and lower relative costs compared to more formal sleep studies. Sensitivity and specificity have been reported at 82% and 76%, respectively. Thus, even though these figures are quite high, 18% of those with OSA will be missed by the test and about a quarter of those without any OSA will test positive.
We now discuss ways in which sample data can be used to make inferences. How valid and generalizable these inferences are will depend on how the data were accrued and we should always bear in mind the design of the study when interpreting the results.
Research questions are generally concerned with finding differences. For example:
If we show that these statements (c and d) are unlikely to be true, then we will also have shown that the previous statements (a and b) are likely to be true.
Statistical analyses allow valid inferences to be made from a random sample to the population from which the sample was selected. The research question is phrased as a hypothesis, from which a null hypothesis is derived (one which predicts no difference, thus attributing any observed differences to chance) (Box 2).
The probability of how likely the null hypothesis is to be true is given by a p-value. Since p-values are probabilities they lie within the range zero to 1.
P-values close to zero are more likely to occur if the study has shown something positive (such as treatment works; disease has an effect; prevalence is different from that suggested by previous studies). The smaller the p-value, the greater the evidence against the null hypothesis. If the p-value is close to zero, we say the result is significant. P-values away from zero (usually >0.05) are said to be non-significant (or NS) at the 5% level.
Note that nothing is ever proven one way or the other – the p-value merely shows how likely the sample data was to have occurred if a certain situation (the null hypothesis) were true. Papers should ideally give the actual p-value for each statistical test performed. Be wary of studies that only give the results from statistically significant tests and/or omit to give confidence intervals alongside the significance tests (see later).
Continuing with the previous example, if the sample of children with osteogenesis imperfecta has an average height similar to that of healthy children, then this is a likely sample to obtain under scenario (c) and the p-value will be close to 1. That is, there is a high probability of obtaining the sample if (c) is true.
If the sample of osteogensis imperfecta children has a very low average height compared to the healthy children, then this would be unlikely under the scenario (c) and the p-value will be close to zero.
Similarly, if the treatment and placebo groups of orthodontic patients using a fluoride mouthwash have very different numbers of early decalcified lesions, then (d) appears to be false. The probability of obtaining samples by chance with such different numbers of lesions if the treatment does not work will be low (close to zero) and hence the p-value will be close to zero.
In statistical hypothesis testing, there are two types of incorrect conclusions or errors that can be drawn. These are explained below, together with important consequential applications.
A random sample should give an unbiased estimate of a population parameter. We usually want an estimate of a population mean (for example, the average height of 5-year-olds), population percentage (for example, the prevalence of molar-incisor hypoplasia), or the difference attributable to a treatment (for example, change in mandibular length). In each case, the sample gives an estimate. How precise an estimate this is depends on the sample size (larger samples give better estimates) and, in the case of numeric data, the variability of the observations made.
A standard error (SE or SEM) may be used to quantify the precision of an estimate (for example, a sample mean, difference in means or proportions between two groups, or odds ratio). Essentially, the SE quantifies the expected variability of the estimate given the sample size it is based on. Larger standard errors imply less precision of the estimate. Larger samples give more precise estimates (hence a smaller standard error).
A confidence interval (CI) can be constructed as follows:
95% CI for a population estimate = sample estimate +/- 1.96 x SEM
Confidence intervals are another approach to address the potential effect of chance on the results. Rather than phrasing the research question as a hypothesis, the value of a particular population parameter is estimated using data collected from a sample. Rather than a point estimate, it is more useful to have an interval estimate.
The confidence interval for a parameter is the range of values within which we are (usually 95%) confident that the true population parameter lies. It thus describes the degree of uncertainty around an estimate when quoted in relation to it (Box 4).
