Brooke PH, Shaw WC. The development of an orthodontic treatment priority index. Eur J Orthod. 1989; 11:309-320
Richmond S, Shaw WC, Roberts CT, Andrews M. The PAR Index (Peer Assessment Rating): methods to determine outcome of orthodontic treatment in terms of improvement and standards. Eur J Orthod. 1992; 14:180-187
Shaw WC, Richmond S, O'Brien KD, Brook P. Quality control in orthodontics: indices of treatment need and treatment standards. Br Dent J. 1991; 170:107-112
Richmond S, Shaw WC, O'Brien KD, Buchanan IB, Jones R, Stevens CD The development of the PAR Index (Peer Assessment Rating): reliability and validity. Eur J Orthod. 1992; 14:125-139
McMullen RE, Doubleday B, Muir JD, Harradine NW, Williams JK. Development of a treatment outcome standard as a result of a clinical audit of the outcome of fixed appliance therapy undertaken by hospital-based consultant orthodontists in the UK. Br Dent J. 2003; 194:81-84
Hand DP, Khalaf K, Mattick CR. Assessment of orthodontic treatment outcome using PAR score for patients treated at the orthodontic department of a teaching hospital. Br Orthod Soc Clin Effect Bull. 2010; 24:12-14
’Where did the time go?’ – an analysis of orthodontic care in a district general hospital over a 24-year period Lance R O Hale Dental Update 2024 7:4, 707-709.
A longitudinal prospective analysis was carried out within a District General Hospital's Orthodontic Department over the entire career of a Consultant Orthodontist, using the IOTN and PAR indices, to monitor continuously the ‘appropriateness’ and ‘quality’ of orthodontic treatment undertaken. Following the retirement of that Consultant, the cumulative results are presented. The results confirm that referrals to the Department were appropriate, demonstrate a generally high standard of treatment for a large number of patients over that period, and a trend towards the treatment of more complex cases, enabling some conclusions to be drawn.
Clinical Relevance: This paper demonstrates the recent level of activity and quality of an Orthodontic Service within a District General Hospital.
Article
Over the last 20 years or so it has become increasingly important for clinicians to be able to justify their treatment and demonstrate a high standard of outcome in that treatment. Quite apart from any natural wish on the part of the service provider that this should be so, this has now come to be encompassed by government objectives such as Clinical Governance, and the need to be able to demonstrate clinical effectiveness in the annual appraisal process.
Orthodontics was an early leader in this process with the development in the late 1980s of two indices, to allow reproducible qualitative assessment of orthodontic treatment need, and the clinical outcome of that treatment.
These two indices, the Index of Orthodontic Treatment Need (IOTN)1,3 and the Peer Assessment Rating (PAR)2,4 were developed from 1987 onwards by Shaw et al, and are now so well known as not to require detailed description. Briefly, the IOTN is an index of treatment need for individual patients, and can also be used to assist in their allocation to an appropriate treatment facility. It has two parts:
A Dental Health Component (DHC), scored from 1 (no need) to 5 (very great need); and
An Aesthetic Component (AC) scored from 1 (ideal) to 10 (very severe), based on standard photographs.
The PAR index is a numerical scoring system which allows measurement of occlusal change as a result of treatment. Both have been shown to be valid, easy to use, consistent and reproducible.3,4
The author became aware of these indices shortly after his appointment, in early 1987, as Consultant Orthodontist to the Luton and Dunstable and Hemel Hempstead Hospitals and, being keen to apply them to his practice, attended in January 1991 one of the earliest courses on their use, run by the Manchester Team.
Aim of this study
The aim of this study was to use these indices to determine:
The type and severity of cases being referred, using IOTN;
The quality of the treatment outcome, using PAR;
Any differences in treatment outcomes between various operator grades;
Any differences in outcome between routine and complex/multidisciplinary cases.
Also, that this should be an ongoing (prospective) process to allow the continuous monitoring of the quality of service.
Gold standards
The following gold standards were the objective:
The mean percentage reduction in PAR score from start to finish should be at least 70%;2
The percentage of cases in the ‘worse/no improvement’ category should be less than 5%;2
The percentage of cases that had been ‘greatly improved’ should be greater than 40%.2
Method and materials
Departmental staff
Over the years treatment has been provided by staff of various grades, including the author (Consultant A), a part-time associate specialist and clinical assistant, Oral Surgery SHOs, and latterly (2006–2009) a second (part-time) consultant (Consultant B).
Method and materials
All patients treated in the department from 1987 to mid 2011, for whom pre-treatment and debond study models were available, were scored on a longitudinal basis. In total, 3150 patients were included in this study.
At first, the scoring was undertaken at monthly audit meetings between the author and the associate specialist, and the results recorded on paper.
However, the department's maxillofacial technician was approached to write a computer program (Orthdex – marketed for a short period by Ortho-Care (UK) Ltd and used as the Audit tool in the Bedfordshire Orthodontic PDS pilot) to allow electronic scoring and storage of the data. Additionally, he and the orthodontic technician attended the IOTN and PAR course in the early years and, from then on, they performed most of the scoring, any cases for which there was concern being shown to the author, who had also been calibrated. Routinely, any case with a post-treatment PAR score of more than 10 was shown to the author for analysis of the less than ideal result.
Initially, the patients were scored following discharge from the department, two years post-treatment, which allowed all early patients to be captured, but latterly the scoring has been undertaken immediately after debond.
Recent analysis of this data has been undertaken using Orthdex and Microsoft ExcelTM.
Results
A total of 3150 patients have been identified and scored over a 24-year period. IOTN demonstrates that all cases treated over the years were appropriate for orthodontic intervention, but that there has been a gradual shift towards more severe cases as the years have gone by, shown in the 5-year blocks (Table 1a). Table 1b shows the distribution by operator in those blocks.
Table 1a. Start IOTN (DHC) of treated patients – 5-year blocks 1987–2011.Table 1b. Proportion of patients treated by operators – 5-year blocks 1987–2011.
Table 2 shows the pre- and post-treatment PAR scores, demonstrating a wide spread of scores before treatment, and a large shift to the left at debond.
Table 2. Pre- and post-treatment PAR scores for all operators.
The mean change in PAR for the whole group was 77.4% (SD 18.6%), exceeding the accepted standard of 70% (Standard 1).
This group was also analysed to demonstrate any inter-operator differences (Table 3). All grades can be seen to have equalled or exceeded the Standard. The Treatment Outcome (Nomogram) for the whole group is shown in Table 4a. This confirms very good compliance with Standards 2 and 3.
Table 3. Percentage PAR change by operator.Table 4a. Treatment outcomes (Nomogram) – whole group.
Analysis in 5-year blocks shows improving treatment outcomes, with a steady reduction in the ‘worse/no improvement’ grade, while the ‘greatly improved’ grade increased (Table 4b).
Table 4b. Treatment outcomes for all operators – 5-year blocks 1987–2011.
Further analysis shows that all operator grades achieved this result (Table 5). Seeking to analyse the group further, complex and multidisciplinary patients were extracted from the group (Table 6).
Table 5. Treatment outcomes by operator.Table 6. PAR change and outcomes – complex/multidisciplinary patients.
Impacted teeth requiring oral surgical intervention (IOTN 5i) provided by far the largest single group of patients, there being 1034 of these.
Orthognathic cases provided a further 265, analysis being undertaken for the group as a whole and by IOTN(DHC) categories (Table 7). The final column (Oddballs) in this table contains those who presented with other IOTN categories (ie 5i, 4h, 4f, etc), but also required orthognathic surgery owing to their skeletal pattern.
Table 7. Percentage PAR change for orthognathic patients by IOTN group.
There were 67 patients with all grades of cleft of the lip and/or palate, of whom 7 had undergone orthognathic surgery (Table 7).
A total of 31 patients presented with special educational needs, which made their treatment more complex.
Patients with hypodontia were not studied as a group as it was not possible to separate, on the basis of IOTN (4h/5h), those cases with anterior or posterior missing teeth and those requiring/not requiring restorative intervention.
Discussion
The results of this long-term study demonstrate, firstly, that those patients accepted into the department for treatment over the entire 24-year period were appropriate in terms of IOTN, and secondly, that the milder cases slowly decreased over the years, while the more severe malocclusions increased, especially those presenting with clefts of the lip and/or palate or requiring orthognathic surgery. It is entirely appropriate that the shift should occur in this direction, as more high street specialist practices were established in the area to undertake the more routine work.
Overall, the outcome of treatment using the PAR index showed a mean improvement for the whole group of over 77%, thereby more than satisfying the Standard 1 (Table 4). Further, all grades of operator exceeded this standard although, as one would expect, those with a postgraduate training and experience performed better (Table 3). Consultant B would appear to have performed on average to the highest standard (% PAR change 83.7 SD16.5) and there are several possible reasons for this:
More recent training;
Small patient numbers compared with Consultant A and the Associate Specialists; and
A very restricted case mix (mainly full fixed and orthognathic cases) by comparison with the longer established staff.
However, study of the treatment times in Table 3 would suggest that this was at the expense of longer treatments.
The Nomogram (Table 4a) also confirmed that the performance exceeded by some margin the Standards 2 and 3, that this was the case over the whole period studied, and in the 5-year blocks, which showed a gradual improvement in performance, with reduction in ‘worse/no improvement’ cases and an increase in the proportion of cases ‘greatly improved’ (Table 4b). It further showed that this applied to all operator grades (Table 5).
There is an anomaly with the Nomogram in that, while the ‘worse/no improvement’ and ‘improved’ outcomes are scored on a percentage change (< or >30%), it is not possible, whatever the percentage improvement in PAR, to achieve a ‘greatly improved’ outcome unless there is a PAR change of 22 points or more, so that cases with a low pre-treatment PAR score can never be ‘greatly improved’. The percentage of ‘greatly improved’ cases will therefore be much more affected by the pre-treatment PAR score than the other groups. In relation to the complex/multidisciplinary cases (Table 6), this was particularly evident with IOTN 5i cases. These cases presented a particular problem in the use of PAR since a large number had very low pre-treatment scores, often being well aligned Class I, or very mild Class II cases, for whom the primary problem was the impacted tooth, usually a canine. These patients also often presented late (14–16 years old), and declined intervention in the lower arch where there was mild crowding/irregularity, which was therefore not treated and adversely affected the post-treatment PAR score.
This therefore explains the rather disappointing Nomogram percentages, with the ‘greatly improved’ grade only just exceeding 40% (Standard 3).
The orthognathic cases appear to have by far the best results, both in terms of %PAR change and Nomogram outcome, and this is indeed the sort of outcome to be hoped for, in view of the complexity of their treatment and the quite major surgery to which they have been subjected. Of course it must be recognized that not all the credit for this outcome can be appropriated by the orthodontist! (Tables 6/7).
The patients presenting with clefts were a very varied group, since the mildest (posterior cleft or lip only) would be expected not to complicate the malocclusion greatly, while the severest would undergo orthognathic surgery, with the expected good outcome. The group in the middle, with complete clefts, would be expected to have the poorest results, often because they had only reached the stage of completion of the early ‘teenage treatment of expansion, bone grafting and simple orthodontic alignment of the upper teeth’, and had not yet reached an age where orthognathic treatment could be considered, or had refused such treatment when offered. Just such a case provided the worst post-treatment PAR score of 45, but despite this fell into that group scored as ‘improved’ on the Nomogram, indicating a change in PAR score of more than 30% (Table 6).
There might therefore be an argument for further subdividing this group for more detailed analysis. However, there has been some discussion as to whether PAR is an appropriate audit tool for patients with clefts.5
The results for the group of patients presenting with special needs was gratifying in view of the difficulties in handling them in treatment and, often, the selection of simpler treatment objectives at the outset. They demonstrated a full spectrum of malocclusions with one or two going as far as orthognathic surgery, with the expected excellent outcome (Table 6).
Conclusions
While previous PAR/IOTN studies have been ‘snapshots’ involving much smaller numbers, this study is a longitudinal one covering the whole period of a Consultant's tenure, and the data presented may therefore be unique.
In combination with other data, it has provided and, if continued by the author's successor, will continue to provide useful data for service planning and development.
Overall, the results demonstrate that a highly acceptable level of care can be achieved in a busy District General Hospital Orthodontic Department that does not purport to be a ‘Centre of Excellence’, and therefore support the Hospital Consultant-led Service for the care of complex/multidisciplinary cases.
This is further confirmed by the changing pattern of severity of cases treated over the 24-year period, with the opening of more specialist practices in the area.
It also demonstrates that good outcomes can be achieved by appropriately supervised junior staff.
This long-term study does not support the results of a smaller short-term Audit undertaken at Newcastle Dental School6 or the conclusions that the more complex multidisciplinary cases are insensitive to PAR and should be excluded from future studies.
The author is more of the view put forward by McMullen et al5 that some consideration be given to tightening the standards against which future studies might be judged.
