AIMS AND METHOD
There is interest in how outcome measures routinely used in mental health settings compare with each other in assessing risk and emotional disturbance. The relation between the Clinical Outcomes in Routine Evaluation - Outcome Measure (CORE-OM), a client-completed measure, and the Health of the Nation Outcome Scale (HoNOS), a clinician-completed measure, was explored using data from 297 clients referred to secondary services by a primary care mental health liaison team.
The correlation between CORE-OM and HoNOS was 0.50, with cluster and factor analyses revealing overlap between the measures in assessing risk to self and others and general emotional issues.
Although the measures are typically used in different settings, the overlap suggests that both might be useful in any setting where assessment for mental health problems and risk is needed.
Routine outcome measurement has been increasingly advocated for mental health and psychological services, with the Department of Health committed to the implementation of outcome measures in routine practice in mental health services (National Institute for Mental Health in England, 2002, 2004). This momentum has been evidenced by a report (Fonagy et al, 2004) building on the work of an expert advisory group on mental health outcomes, which concludes that the Health of the Nation Outcome Scale (HoNOS; Wing et al, 1998) ‘could provide a developmental anchor setting the minimum domains to be covered for the relevant population’ (Fonagy et al, 2004: p. 6). The report also states that ‘Trust[s] should also explore the use of other clinician rated and self report measures, including quality of life measures, so as to ensure that relevant domains can be appropriately measured across the populations served’ (Fonagy et al, 2004: p. 6).
These statements reflect a strategy whereby the appropriateness and utility of bonafide outcome measures should be investigated but that they need to be empirically anchored against HoNOS. Although no single measure could possibly capture the whole range of presenting problems and outcomes across mental health, establishing evidence of the relationship between measures and the extent to which they capture common and unique aspects of experience is important. This would add to validation evidence for the measures and clarify the extent to which they are useful for particular populations and in particular service settings. Accordingly, we focused on the HoNOS and a widely used self-report measure, the Clinical Outcomes in Routine Evaluation - Outcome Measure (CORE-OM; Barkham et al, 2001, 2005; Evans et al, 2002). Interestingly, each has been advocated for use at different ends of the severity range. Both measures can be used to assess various aspects of risk of self-harm or harm to others.
The HoNOS is advocated as suitable for use at the severe end of the spectrum to provide a practitioner-completed assessment across 12 domains, divided into four subgroups. Risk to self and others is assessed by two items in subgroup A (behavioural problems). This measure has derived support from some reviewers (e.g. Rees et al, 2004) and been criticised by others (e.g. Audin et al, 2001). The empirical relationship between HoNOS and other outcome measures has been investigated and significant associations have been reported. Orrell et al (1999) established significant correlations between HoNOS and a battery of six outcome measures on a sample of 100 patients: 0.59 for the Social Behaviour Scale (Wykes & Sturt, 1986), 0.51 for the Location of Community Support Scale (Kazarian & Joseph, 1994), 0.40 for the Brief Psychiatric Rating Scale (BPRS; Overall & Gorham, 1962), – 0.40 for the Global Assessment Scale (GAS; Endicott et al, 1976), 0.36 for the General Health Questionnaire (Goldberg, 1978) and – 0.33 for the Medical Outcomes Study Short Form-36 (Ware & Sherbourne, 1990). McClelland et al (1998) reported correlations on presentation to the service between HoNOS and GAS of 0.49 and with the BPRS of 0.44.
The CORE-OM is a 34-item self-report measure tapping four domains, including a risk sub-scale assessing risk to others (two items) and risk to self (four items). It has been widely used to measure emotional disturbance in service settings delivering psychological interventions in primary and secondary care (Barkham et al, 2005). Its relationship to other measures, including the Beck Depression Inventory (Beck & Steer, 1993), has been explored in large-scale studies (e.g. Leach et al, 2005).
Our study assesses the empirical relationship between these measures and their potential to complement each other across a range of settings.
Data were collected as part of routine service delivery for clients referred to the Primary Care Liaison, Assessment Treatment and Training (PLATT) team of a large mental health National Health Service (NHS) trust. This multidisciplinary team provides a service within two primary care trusts and serves a population of 330 000. The PLATT team receives 60 adult referrals a week, mainly from general practice, and provides an assessment filter for all mental health services in one locality. All those referred routinely complete the CORE-OM at assessment. For those receiving therapy within the PLATT service, the CORE-OM is also used before therapy and at discharge. For the first 6 months of the service, assessors completed HoNOS ratings of people assessed as requiring help from community mental health or psychiatry services.
Data for this study come from a service evaluation database for 1497 persons referred between October 2001 and March 2003. Of these, there were complete data for 1297 clients on CORE-OM and for 507 clients on HoNOS, with 315 clients having complete data for both measures. For some clients, CORE-OM and HoNOS were completed at different stages of therapy, so the study focused on 297 clients (175 women and 122 men) for whom the measures had been completed at assessment and within 30 days of each other. One client with an extreme outlying HoNOS total score of 37 was excluded from further analysis, leaving 296 clients in the study sample, whose HoNOS scores ranged from 0 to 25 (mean=9.03, s.d.=4.27, median=9.00). The CORE-OM scores ranged from 0.15 to 3.76 (mean=2.24, s.d.=0.69, median=2.28), with 274 clients (93%) scoring above the CORE-OM clinical cut-offs of 1.29 (female) and 1.19 (male), and 109 (37%) scoring above the ‘severe’ cut-off of 2.50 (see Barkham et al, 2001). The excluded client scored 2.93 on the CORE-OM.
Those included in the study sample (n=296) differed from those not included (n=1000) in CORE-OM score (mean=2.24, s.d.=0.69, for sample; mean=2.08, s.d.=0.76, for non-sample; t=3.28, d.f.=1294, P=0.001), but not in age (mean=37.5, s.d.=12.4, for sample; mean=36.9, s.d.=11.8, for non-sample; t=0.71, d.f.=1294, P=0.48) or gender (60% females in sample, 62% in non-sample; χ 2=0.86, d.f.=1, P=0.36). The difference in CORE-OM mean scores reflects the fact that HoNOS was completed only for individuals assessed as suitable for referral to psychiatric or community mental health team services.
The relation between the two measures was explored using Pearson product-moment correlations to compare total and sub-scale scores. At the item level, hierarchical cluster analysis (Ward’s method) and maximum likelihood factor analysis (with oblique rotation) were both carried out, all analyses being performed using the Statistical Package for the Social Sciences, version 12.0.
The correlation between CORE-OM overall score and HoNOS total score was 0.50 (P<0.001), the scales thus sharing 50% of the variance (Ozer, 1985), with CORE-OM correlating most highly with the HoNOS sub-scales A (behavioural problems; r=0.40) and D (social problems; r=0.39), with a slightly smaller correlation with the C sub-scale (symptomatic; r=0.34) and an effectively zero correlation with the B sub-scale (impairment; r=0.05). The same pattern is apparent for the CORE-OM risk sub-scale, which had correlations of 0.51 with the HoNOS total score, 0.57 with the A sub-scale, 0.39 with the D sub-scale, 0.22 with the C sub-scale and 0.02 with the B sub-scale.
Table 1 summarises the results of the hierarchical cluster analysis and factor analysis of the CORE-OM and HoNOS items. The cluster analysis shows two main clusters (1 and 2), each broken down into two further clusters (a and b). Cluster 1 features all the non-risk CORE-OM items and three of the HoNOS items (‘other mental and behavioural problems’, ‘problems with relationships’ and ‘problems with depressed mood’), whereas cluster 2 brings together the CORE-OM risk items and the remaining HoNOS items. Cluster 1a includes items mainly from the CORE-OM Problems sub-scale together with the three HoNOS items; cluster 1b includes items mainly from the CORE-OM Wellbeing and Functioning sub-scales. Cluster 2a includes the CORE-OM risk items and two HoNOS risk items (‘non-accidental self-injury’ and ‘overactive, aggressive, disruptive or agitated behaviour’), with the self-injury HoNOS item being closely related to the four CORE-OM ‘ risk to self’ items, and the aggression HoNOS item being closely related to the two CORE-OM ‘risk to others’ items. Cluster 2b contains the remaining HoNOS items.
The results of the factor analysis echo those of the cluster analysis, with some minor differences. A HoNOS item (‘problems with relationships’) grouped with the CORE-OM non-risk items by the cluster analysis is now grouped closer to the other HoNOS items. The CORE-OM risk to self item 34 (‘I have hurt myself physically...’) is closer to the risk to others items in the factor analysis than to the remaining risk to self items.
The findings that the HoNOS and CORE-OM total scores share 50% of the variance and that their respective risk items (CORE-OM risk sub-scale and HoNOS behavioural problems sub-scale) have 57% common variance suggest that both these widely used measures are identifying a common component of clients presenting with severe problems. Interestingly, the risk scale of the CORE-OM, comprising only six items, yielded as good if not better associations with the overall HoNOS and its behavioural scale than the CORE-OM total. Hence, the CORE-OM risk scale might prove a cost-efficient means of quickly obtaining clinically meaningful data.
The cluster 1a grouping of three HoNOS items, ‘other mental and behavioural problems’, ‘problems with relationships’ and ‘ problems with depressed mood’, with nine of the CORE-OM non-risk items reinforces the finding that these three HoNOS items are those that have shown evidence of being sensitive to change following psychological therapies (Audin et al, 2001).
The HoNOS is the recommended measure for clients with severe and enduring problems (Fonagy et al, 2004). In circumstances where a self-report measure is appropriate, for example where staff trained in the use of HoNOS are not available, CORE-OM can provide information relating to some of the key domains, particularly risk.
- © 2005 Royal College of Psychiatrists