Managing outcome performance in mental health using HoNOS: experience at St Andrew's Healthcare

AIMS AND METHOD

Modern mental healthcare providers must demonstrate service-levelclinical effectiveness to key stakeholders. We introduced twoperformance indicators of clinical effectiveness based on outcomedata from routinely collected Health of the Nation OutcomeScales (HoNOS) and HoNOS–secure assessments across StAndrew’s Healthcare, a charity providing in-patient servicesin Northampton and Essex. We present outcome data from theperiod 2004–2007.

RESULTS

The indicators showed consistent 90-day improvement rates andincreasing stability over time. The validity of results issupported by levels of change along predicted lines among differentpatient cohorts.

CLINICAL IMPLICATIONS

It is possible and beneficial to use routine outcome measuresto demonstrate clinical effectiveness at service level. Thefuture of managing mental health outcomes will be in electronichealth records systems.

Introduction

TOP

Introduction

Providers of modern mental healthcare are increasingly requiredto demonstrate the effectiveness of their services to a rangeof stakeholders including purchasers, service users and regulators.Measures of service effectiveness must of course cover valueacross a range of cost and quality indicators. The ‘balanced scorecard’¹ is a tool to bring a mixed bag of key performanceindicators into sharp focus, aiming to provide managers withclear, guiding indicators of progress against key strategicaims. It has been likened to the dials in an aeroplane cockpit or dashboard. In healthcare management the scorecard shouldcover domains such as clinical quality and effectiveness, staffing,service demand, activity, financial performance and organisationallearning and growth.² The implementation of a balanced scorecardin psychiatric services has previously been described^3,4 butmeasures of clinical effectiveness have been problematical⁴and have thus far been absent from these accounts. Below, wedescribe the integration into a balanced scorecard of two performanceindicators using aggregated data from routinely performed outcomemeasures, and present service-level data demonstrating theusefulness of the key performance indicators for service providersand other key stakeholders.

Although the routine use of outcome measures has not been shownto bring about improvements in outcomes such as patient-ratedunmet need and quality of life, there is promising evidencethat in-patient days and thus service cost are reduced.⁵ There is consensus that outcomes measurement needs to be multidimensional^6,7 both in terms of the outcomes measured and the viewpoints orperspectives considered, including those of service user, clinicianor other stakeholders. Numerous items have been used or nominatedto support outcome-focused clinical key performance indicatorsincluding frequency of unplanned discharges and readmissions,incidence of injurious behaviour and self-harm, transfers, use of restraint, seclusion and number of falls,⁸ provision of pre-admission care,⁹ medication use and insurance claims.¹⁰For those with severe mental illness, proposed outcome indicatorsinclude routine analysis of needs assessment measures¹¹ and Health of the Nation Outcome Scales (HoNOS)¹² data at bothservice and individual level.⁷

Health of the Nation Outcome Scales for working-age adults andthe related family of variants includes outcome measures forchildren, for older people, and for those with a learning disabilityor acquired brain injury. All of the HoNOS variants aim tomeasure mental health and behavioural functioning in orderto provide data to judge the effectiveness of mental healthservices. The tools perform adequately or better in terms ofreliability, validity, sensitivity to change and utility.¹³In the UK, use is widespread and increasing:¹⁴ 49% of allEnglish mental health service providers were using HoNOS inmore than one setting by 2002 and recently all members of theHoNOS family have been taken up into the National Health Service(NHS) Connecting for Health IT Programme. The HoNOS–secure^15,16 has been developed for users of secure psychiatric services,prisons and forensic community services, where diagnostic heterogeneity,multiple treatment needs and additional offence-related treatmentneeds adds further complexity to outcomes measurement.⁶ The HoNOS–secure contains modified versions of the originaltwelve HoNOS scales, and a separate ‘security scale’comprising seven additional ratings to track change in theoutcome of ongoing clinical risk assessment, and in the needfor physical, relational and procedural security measures (Box1). In short, HoNOS–secure is aimed at all users of secure and forensic services, and is designed to be applicable tothese service users in the community also, allowing continuityin tracking outcomes.

Method

TOP

Method

Setting
St Andrew’s Healthcare is a charity operating in the UKindependent healthcare sector, providing a range of specialistmental health services for NHS patients with many extendedcare pathways: men’s, women’s and adolescent servicescovering mental health and learning disability; Kemsley, theNational Centre for Brain Injury Rehabilitation; Townsend servicesfor older adults; and also the St Andrew’s low secureadult mental health service in Essex. The six services, totallingapproximately 500 beds, all provide a level of secure care.

Measures
Since 2004, all six services have used HoNOS outcome measures,with HoNOS assessments routinely conducted for all individualsat the end of a pre-admission assessment, within 7 days ofadmission, and thereafter at 3 monthly intervals until a finalrating is made at discharge. Rating is undertaken by the individual’scare team as part of the care programme approach process. Individualsare rated using the most appropriate HoNOS tool (HoNOS–secure,HoNOS–LD (for people with learning difficulties), HoNOS65+ (for people aged 65 years and older), HoNOS–CA (foruse with children and adolescents) or HoNOS–ABI (forpeople with acquired brain injury), collectively referred toas HoNOS specialty scales). Because all services provide alevel of secure care, all individuals are rated on the seven-itemHoNOS–secure ‘security scale’.

Box 1. HoNOS–secure seven-item ‘security scale’

Riskof harm to adults or children

Risk of self-harm

Need for buildingsecurity to prevent escape

Need for a safely staffed livingenvironment

Need for escort on leave (beyond secure perimeter)

Riskto individual from others

Need for risk management procedures

Eachitem is scored on a 0–4 scale using a glossary provided.¹⁵

Aims
Two key performances indicators of clinical outcome, coveringoverall progress and need for secure care, were added to anexisting balanced scorecard at St Andrew’s Healthcarein January 2004. We present aggregated outcome data acrossclinical services at St Andrew’s Healthcare from 2004to 2007 in order to demonstrate the value to service managers.Although the performance indicators are based on aggregateddata, and not amenable to inferential statistics, we hypothesisedthat face validity of the service-level measures would be demonstratedif:

change within different patient cohorts were to differ alongreasonably predictable lines, with less change among more intractablepatient groups such as those with dementia, and more amongthose with most potential to change such as those in the adolescentservice;
change over time, both within and across services,were to berelatively stable, and for this stability to increaseas qualityof data collection improved.

Key performance indicator calculations
Two key performance indicators, each indicating an average 90-day improvement rate, are calculated quarterly from individualpatient HoNOS specialty and HoNOS–secure ‘securityscale’ assessments. The method of calculation for theHoNOS–specialty scales and for the HoNOS–secure‘security scale’ is the same, being the change inscore as a proportion of the total possible score. This capturesthe direction and size of change in a cohort, and allows abroad comparison across services using different HoNOS specialtyscales. Each quarterly data collection therefore captures thecurrent patient cohort, and picks up at ward- and service-levelany change in the patient group under treatment. Because theindicator is calculated to identify change rather than overall levels of well-being, it is not overly sensitive to fluctuationsbrought about by additions to and subtractions from the cohortthrough admission and discharge.

Results

TOP

Results

Data for 2004–2007 were collated and 90-day change ratescalculated as outlined above for HoNOS specialty and HoNOS–secure‘security scale’ ratings. The mean successful completionrate of ratings within 3 months for the entire 1-year periodfor 2004 was 65% rising to 83.0% in 2007.

Figures 1 and 2 show 90-day change rates calculated as annualaverages for the HoNOS specialty scales and the HoNOS–secure‘security scale’ respectively. Relatively low completionrates in 2004 are reflected in the wide range of change across services, and this range generally narrows over time for bothkey performance indicators. The average 90-day change rateacross the study period for the entire St Andrew’s Healthcareservice hovers around or just under 1% for HoNOS specialtyscores and at around or just over 1% for the HoNOS–secure‘security scale’. Figures 3 and 4 show averageannual 90-day change rate by service for the HoNOS specialtyand HoNOS–secure ‘security scale’ key performanceindicators respectively. As predicted, most change is notedamong people using the adolescent service and least among elderlypeople. The six services were ranked in order of ‘securityscale’ total scores (‘need for security measure’)and in order of behavioural and psychiatric symptomatology totals(from HoNOS specialty scales). These rankings were positively correlated (Spearman’s rank correlation coefficient 0.6).Positive change over time in symptomatology is demonstratedfor the low secure mental health cohort, but not in the needfor security measures (Fig. 4). This may be because this smallerservice is integrated more with external care pathways, withless opportunity for individuals to progress through levelsof security within the service. However, it should be notedthat the small negative 90-day change rate on the HoNOS–secure‘security scale’ key performance indicator forthis service is largely because of a negative outcome in year1 possibly as a result of data collection and calculation issues.Excluding year 1, the low secure service actually achievedmodest improvements across the period 2005–7.

View larger version (8K):
[in this window]
[in a new window]
[as a PowerPoint slide]
Fig. 1. Health of the Nation Outcome Scales (HoNOS) specialty scales: 90-day change scores.

View larger version (8K):
[in this window]
[in a new window]
[as a PowerPoint slide]
Fig. 2. Health of the Nation Outcome Scales (HoNOS)–secure ‘security scales’: 90-day change scores.

View larger version (20K):
[in this window]
[in a new window]
[as a PowerPoint slide]
Fig. 3. Health of the Nation Outcome Scales (HoNOS) specialty scales: 90-day change scores average by service.

View larger version (21K):
[in this window]
[in a new window]
[as a PowerPoint slide]
Fig. 4. Health of the Nation Outcome Scales (HoNOS)–secure ‘security scales’: 90-day change scores average by service.

Discussion

TOP

Discussion

Two performance indicators based on routinely collected datafrom HoNOS specialty and the HoNOS–secure ‘securityscale’ provided key information about the performanceof clinical services and were an important addition to a balancedscorecard of indicators of business performance. It was possibleto detect relatively small quarterly changes to overall healthrating and security needs, suggesting HoNOS–secure issensitive to change among a group of individuals referred toour specialist services from the NHS. As would be expected,greater change in both health and security status was found among people using the adolescent services where symptomatologyand functioning problems are less entrenched.

Limitations
Initial significant problems with data collection and completionhave been identified, including return within appropriate timescales. Additionally, some errors in data processing have beenidentified and addressed. The solutions in these areas arefor all HoNOS ratings to be ‘owned’ by the clinicalteams and for easy-to-use electronic systems with good management.The now complete introduction across the charity of electronic patient records promises to further improve outcome data collection.There has been considerable recent debate about patients’own subjective rating of outcome and its ability to effectchange, and measures such as the ‘Recovery Star’¹⁷could be included alongside existing performance indicatorsin a balanced scorecard using calculations demonstrated here.

Implications
The use of key performance indicators based on routinely collectedoutcome data promotes a transparent, open-book culture aboutoutcomes. It allows healthcare providers to monitor performanceat the same time as giving clinicians the freedom to developtherapeutic programmes that can best deliver real health improvement.More widespread use of similar performance indicators amongother providers of secure and forensic services would facilitate benchmarking and further improvement.

Acknowledgments

We are grateful to all consultants, care coordinators and otherclinical staff at St Andrew’s Healthcare who have madethis work possible by contributing to routine outcome measurescollection using HoNOS specialty measures and the HoNOS–secure‘security scale’ since 2004.

References

TOP