Hostname: page-component-7c8c6479df-995ml Total loading time: 0 Render date: 2024-03-27T11:46:39.648Z Has data issue: false hasContentIssue false

Managing outcome performance in mental health using HoNOS: experience at St Andrew's Healthcare

Published online by Cambridge University Press:  02 January 2018

Philip Sugarman
Affiliation:
St Andrew's Healthcare
Lorraine Walker
Affiliation:
St Andrew's Healthcare
Geoff Dickens
Affiliation:
St Andrew's Academic Centre, King's College London, Institute of Psychiatry, Northampton NN1 5BW, email: gdickens@standrew.co.uk
Rights & Permissions [Opens in a new window]

Abstract

Aims and Method

Modern mental healthcare providers must demonstrate service-level clinical effectiveness to key stakeholders. We introduced two performance indicators of clinical effectiveness based on outcome data from routinely collected Health of the Nation Outcome Scales (HoNOS) and HoNOS–secure assessments across St Andrew's Healthcare, a charity providing in-patient services in Northampton and Essex. We present outcome data from the period 2004–2007.

Results

The indicators showed consistent 90-day improvement rates and increasing stability over time. the validity of results is supported by levels of change along predicted lines among different patient cohorts.

Clinical Implications

It is possible and beneficial to use routine outcome measures to demonstrate clinical effectiveness at service level. the future of managing mental health outcomes will be in electronic health records systems.

Type
Original papers
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © Royal College of Psychiatrists, 2009

Providers of modern mental healthcare are increasingly required to demonstrate the effectiveness of their services to a range of stakeholders including purchasers, service users and regulators. Measures of service effectiveness must of course cover value across a range of cost and quality indicators. The ‘balanced scorecard’ Reference Kaplan and Norton1 is a tool to bring a mixed bag of key performance indicators into sharp focus, aiming to provide managers with clear, guiding indicators of progress against key strategic aims. It has been likened to the dials in an aeroplane cockpit or dashboard. In healthcare management the scorecard should cover domains such as clinical quality and effectiveness, staffing, service demand, activity, financial performance and organisational learning and growth. Reference Sugarman2 The implementation of a balanced scorecard in psychiatric services has previously been described Reference Sugarman and Watkins3,Reference Coop4 but measures of clinical effectiveness have been problematical Reference Coop4 and have thus far been absent from these accounts. Below, we describe the integration into a balanced scorecard of two performance indicators using aggregated data from routinely performed outcome measures, and present service-level data demonstrating the usefulness of the key performance indicators for service providers and other key stakeholders.

Although the routine use of outcome measures has not been shown to bring about improvements in outcomes such as patient-rated unmet need and quality of life, there is promising evidence that in-patient days and thus service cost are reduced. Reference Slade, McCrone, Kuipers, Leese, Cahill and Parabiaghi5 There is consensus that outcomes measurement needs to be multidimensional Reference Cohen and Eastman6,7 both in terms of the outcomes measured and the viewpoints or perspectives considered, including those of service user, clinician or other stakeholders. Numerous items have been used or nominated to support outcome-focused clinical key performance indicators including frequency of unplanned discharges and readmissions, incidence of injurious behaviour and self-harm, transfers, use of restraint, seclusion and number of falls, Reference Thomson, Taber, Lally and Kazandjian8 provision of pre-admission care, Reference Dausey, Rosenheck and Lehman9 medication use and insurance claims. Reference Gilbody, House and Sheldon10 For those with severe mental illness, proposed outcome indicators include routine analysis of needs assessment measures Reference Phelan, Slade, Thornicroft, Dunn, Holloway and Wykes11 and Health of the Nation Outcome Scales (HoNOS) Reference Wing, Beevor, Curtis, Park, Hadden and Burns12 data at both service and individual level. 7

Health of the Nation Outcome Scales for working-age adults and the related family of variants includes outcome measures for children, for older people, and for those with a learning disability or acquired brain injury. All of the HoNOS variants aim to measure mental health and behavioural functioning in order to provide data to judge the effectiveness of mental health services. The tools perform adequately or better in terms of reliability, validity, sensitivity to change and utility. Reference Pirkis, Burgess, Kirk, Dodson, Coombs and Williamson13 In the UK, use is widespread and increasing: Reference James14 49% of all English mental health service providers were using HoNOS in more than one setting by 2002 and recently all members of the HoNOS family have been taken up into the National Health Service (NHS) Connecting for Health IT Programme. The HoNOS–secure Reference Sugarman and Walker15,Reference Dickens, Sugarman and Walker16 has been developed for users of secure psychiatric services, prisons and forensic community services, where diagnostic heterogeneity, multiple treatment needs and additional offence-related treatment needs adds further complexity to outcomes measurement. Reference Cohen and Eastman6 The HoNOS–secure contains modified versions of the original twelve HoNOS scales, and a separate ‘security scale’ comprising seven additional ratings to track change in the outcome of ongoing clinical risk assessment, and in the need for physical, relational and procedural security measures (Box 1). In short, HoNOS–secure is aimed at all users of secure and forensic services, and is designed to be applicable to these service users in the community also, allowing continuity in tracking outcomes.

Method

Setting

St Andrew's Healthcare is a charity operating in the UK independent healthcare sector, providing a range of specialist mental health services for NHS patients with many extended care pathways: men's, women's and adolescent services covering mental health and learning disability; Kemsley, the National Centre for Brain Injury Rehabilitation; Townsend services for older adults; and also the St Andrew's low secure adult mental health service in Essex. The six services, totalling approximately 500 beds, all provide a level of secure care.

Measures

Since 2004, all six services have used HoNOS outcome measures, with HoNOS assessments routinely conducted for all individuals at the end of a pre-admission assessment, within 7 days of admission, and thereafter at 3 monthly intervals until a final rating is made at discharge. Rating is undertaken by the individual's care team as part of the care programme approach process. Individuals are rated using the most appropriate HoNOS tool (HoNOS–secure, HoNOS–LD (for people with learning difficulties), HoNOS 65+ (for people aged 65 years and older), HoNOS–CA (for use with children and adolescents) or HoNOS–ABI (for people with acquired brain injury), collectively referred to as HoNOS specialty scales). Because all services provide a level of secure care, all individuals are rated on the seven-item HoNOS–secure ‘security scale’.

Box 1. HoNOS–secure seven-item ‘security scale’

  1. A Risk of harm to adults or children

  2. B Risk of self-harm

  3. C Need for building security to prevent escape

  4. D Need for a safely staffed living environment

  5. E Need for escort on leave (beyond secure perimeter)

  6. F Risk to individual from others

  7. G Need for risk management procedures

Each item is scored on a 0–4 scale using a glossary provided. Reference Sugarman and Walker15

Aims

Two key performances indicators of clinical outcome, covering overall progress and need for secure care, were added to an existing balanced scorecard at St Andrew's Healthcare in January 2004. We present aggregated outcome data across clinical services at St Andrew's Healthcare from 2004 to 2007 in order to demonstrate the value to service managers. Although the performance indicators are based on aggregated data, and not amenable to inferential statistics, we hypothesised that face validity of the service-level measures would be demonstrated if:

  1. change within different patient cohorts were to differ along reasonably predictable lines, with less change among more intractable patient groups such as those with dementia, and more among those with most potential to change such as those in the adolescent service;

  2. change over time, both within and across services, were to be relatively stable, and for this stability to increase as quality of data collection improved.

Key performance indicator calculations

Two key performance indicators, each indicating an average 90-day improvement rate, are calculated quarterly from individual patient HoNOS specialty and HoNOS–secure ‘security scale’ assessments. The method of calculation for the HoNOS–specialty scales and for the HoNOS–secure ‘security scale’ is the same, being the change in score as a proportion of the total possible score. This captures the direction and size of change in a cohort, and allows a broad comparison across services using different HoNOS specialty scales. Each quarterly data collection therefore captures the current patient cohort, and picks up at ward- and service-level any change in the patient group under treatment. Because the indicator is calculated to identify change rather than overall levels of well-being, it is not overly sensitive to fluctuations brought about by additions to and subtractions from the cohort through admission and discharge.

Results

Data for 2004–2007 were collated and 90-day change rates calculated as outlined above for HoNOS specialty and HoNOS–secure ‘security scale’ ratings. The mean successful completion rate of ratings within 3 months for the entire 1-year period for 2004 was 65% rising to 83.0% in 2007.

Figures 1 and 2 show 90-day change rates calculated as annual averages for the HoNOS specialty scales and the HoNOS–secure ‘security scale’ respectively. Relatively low completion rates in 2004 are reflected in the wide range of change across services, and this range generally narrows over time for both key performance indicators. The average 90-day change rate across the study period for the entire St Andrew's Healthcare service hovers around or just under 1% for HoNOS specialty scores and at around or just over 1% for the HoNOS–secure ‘security scale’. Figures 3 and 4 show average annual 90-day change rate by service for the HoNOS specialty and HoNOS–secure ‘security scale’ key performance indicators respectively. As predicted, most change is noted among people using the adolescent service and least among elderly people. The six services were ranked in order of ‘security scale’ total scores (‘need for security measure’) and in order of behavioural and psychiatric symptomatology totals (from HoNOS specialty scales). These rankings were positively correlated (Spearman's rank correlation coefficient 0.6). Positive change over time in symptomatology is demonstrated for the low secure mental health cohort, but not in the need for security measures (Fig. 4). This may be because this smaller service is integrated more with external care pathways, with less opportunity for individuals to progress through levels of security within the service. However, it should be noted that the small negative 90-day change rate on the HoNOS–secure ‘security scale’ key performance indicator for this service is largely because of a negative outcome in year 1 possibly as a result of data collection and calculation issues. Excluding year 1, the low secure service actually achieved modest improvements across the period 2005–7.

Fig. 1. Health of the Nation Outcome Scales (HoNOS) specialty scales: 90-day change scores.

Fig. 2. Health of the Nation Outcome Scales (HoNOS)–secure ‘security scales’: 90-day change scores.

Discussion

Two performance indicators based on routinely collected data from HoNOS specialty and the HoNOS–secure ‘security scale’ provided key information about the performance of clinical services and were an important addition to a balanced scorecard of indicators of business performance. It was possible to detect relatively small quarterly changes to overall health rating and security needs, suggesting HoNOS–secure is sensitive to change among a group of individuals referred to our specialist services from the NHS. As would be expected, greater change in both health and security status was found among people using the adolescent services where symptomatology and functioning problems are less entrenched.

Fig. 3. Health of the Nation Outcome Scales (HoNOS) specialty scales: 90-day change scores average by service.

Fig. 4. Health of the Nation Outcome Scales (HoNOS)–secure ‘security scales’: 90-day change scores average by service.

Limitations

Initial significant problems with data collection and completion have been identified, including return within appropriate time scales. Additionally, some errors in data processing have been identified and addressed. The solutions in these areas are for all HoNOS ratings to be ‘owned’ by the clinical teams and for easy-to-use electronic systems with good management. The now complete introduction across the charity of electronic patient records promises to further improve outcome data collection. There has been considerable recent debate about patients’ own subjective rating of outcome and its ability to effect change, and measures such as the ‘Recovery Star’ 17 could be included alongside existing performance indicators in a balanced scorecard using calculations demonstrated here.

Implications

The use of key performance indicators based on routinely collected outcome data promotes a transparent, open-book culture about outcomes. It allows healthcare providers to monitor performance at the same time as giving clinicians the freedom to develop therapeutic programmes that can best deliver real health improvement. More widespread use of similar performance indicators among other providers of secure and forensic services would facilitate benchmarking and further improvement.

Declaration of interest

P.S., L.W. and G.D. are employees of St Andrew's Healthcare.

Acknowledgements

We are grateful to all consultants, care coordinators and other clinical staff at St Andrew's Healthcare who have made this work possible by contributing to routine outcome measures collection using HoNOS specialty measures and the HoNOS–secure ‘security scale’ since 2004.

References

1 Kaplan, RS, Norton, DP. The Balanced Scorecard: Translating Strategy into Action. Harvard Business School Press, 1996.Google Scholar
2 Sugarman, P. Governance, strategy and innovation in mental health. Psychiatr Bull 2007; 31: 283–5.Google Scholar
3 Sugarman, PA, Watkins, J. Balancing the scorecard: key performance indicators in a complex healthcare setting. Clinician Manag 2004; 12: 129–32.Google Scholar
4 Coop, CF. Balancing the balanced scorecard for a New Zealand mental health service. Aust Health Rev 2006; 30: 174–80.Google Scholar
5 Slade, M, McCrone, P, Kuipers, E, Leese, M, Cahill, S, Parabiaghi, A, et al. Use of standardised outcome measures in adult mental health services. Randomised controlled trial. Br J Psychiatry 2006; 189: 330–6.Google Scholar
6 Cohen, A, Eastman, N. Needs assessment for mentally disordered offenders: measurement of ‘ability to benefit’ and outcome. Br J Psychiatry 2000; 177: 493–8.CrossRefGoogle ScholarPubMed
7 National Centre for Outcomes Development. Health Outcome Indicators: Severe Mental Illness. Report of a working group to the Department of Health. NCOD, 1999 (http://nchod.uhce.ox.ac.uk/mentalillness.pdf).Google Scholar
8 Thomson, R, Taber, S, Lally, J, Kazandjian, V. UK Quality Indicator Project (UK QIP) and the UK independent health care sector: a new development. Int J Qual Health Care 2004; 16 (suppl 1): i516.Google Scholar
9 Dausey, DJ, Rosenheck, RA, Lehman, AF. Preadmission care as a new mental health performance indicator. Psychiatr Serv 2002; 53: 1451–5.CrossRefGoogle Scholar
10 Gilbody, SM, House, AO, Sheldon, TA. Outcomes research in mental health. Systematic review. Br J Psychiatry 2002; 181: 816.CrossRefGoogle ScholarPubMed
11 Phelan, M, Slade, M, Thornicroft, G, Dunn, G, Holloway, F, Wykes, T, et al. The Camberwell Assessment of Need: the validity and reliability of an instrument to assess the needs of people with severe mental illness. Br J Psychiatry 1995; 167: 589–95.Google Scholar
12 Wing, JK, Beevor, AS, Curtis, RH, Park, SB, Hadden, S, Burns, A. Health of the Nation Outcome Scales (HoNOS). Research and development. Br J Psychiatry 1998; 172: 11–8.Google Scholar
13 Pirkis, JE, Burgess, PM, Kirk, PK, Dodson, S, Coombs, TJ, Williamson, MK. A review of the psychometric properties of the Health of the Nation Outcome Scales (HoNOS) family of measures. Health Qual Life Outcomes 2005; 3: 76.Google Scholar
14 James, M. The use of Health of the Nation Outcome Scales (HoNOS) in routine clinical practice by NHS mental health service providers in England: a summary of findings. The Approach 2002; 23: 13–6.Google Scholar
15 Sugarman, P, Walker, L. HoNOS–secure, Version 2b. Royal College of Psychiatrists, 2007 (http://www.rcpsych.ac.uk/researchandtrainingunit/honos/secure.aspx).Google Scholar
16 Dickens, G, Sugarman, P, Walker, L. HoNOS–secure: a reliable outcome measure for users of secure and forensic mental health services. J Forensic Psychiatry Psychol 2007; 18: 507–14.CrossRefGoogle Scholar
17 Mental Health Providers Forum. BAME project. Mental Health Providers Forum, 2008 (http://www.mhpf.org.uk/recoveryStar.asp).Google Scholar
Figure 0

Fig. 1. Health of the Nation Outcome Scales (HoNOS) specialty scales: 90-day change scores.

Figure 1

Fig. 2. Health of the Nation Outcome Scales (HoNOS)–secure ‘security scales’: 90-day change scores.

Figure 2

Fig. 3. Health of the Nation Outcome Scales (HoNOS) specialty scales: 90-day change scores average by service.

Figure 3

Fig. 4. Health of the Nation Outcome Scales (HoNOS)–secure ‘security scales’: 90-day change scores average by service.

Submit a response

eLetters

No eLetters have been published for this article.