Hostname: page-component-7c8c6479df-ws8qp Total loading time: 0 Render date: 2024-03-27T03:48:32.664Z Has data issue: false hasContentIssue false

Risk assessment is inseparable from risk management

Comment on Szmuckler (Psychiatric Bulletin, 2000, 24, 6–10)

Published online by Cambridge University Press:  02 January 2018

Harry Kennedy*
Affiliation:
Central Mental Hospital, Dundrum, Dublin 14, Ireland; e-mail: harry.kennedy@ireland.com
Rights & Permissions [Opens in a new window]

Abstract

Type
Opinion & Debate
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © 2001, The Royal College of Psychiatrists

Szmuckler (Reference Szmukler2000) has argued that patients in the community represent such low risks of grave outcomes that it is impossible to predict harm accurately. Others have made the same case for the impossibility of predicting suicide (Reference GeddesGeddes, 1999). Based on a 12-month prevalence of psychosis of 4 in 1000 (or 200 000 cases in England and Wales) and a homicide rate by people with schizophrenia (or severe mental illness) of approximately 20 per year in England and Wales (Reference Taylor and GunnTaylor & Gunn, 1999), Szmuckler arrives at a homicide rate of 1 per 10 000 ‘psychosis years’. This is about 10 times the average annual homicide rate for all of England and Wales (Reference Kennedy, Iveson and HillKennedy et al, 1999). At such low rates Szmuckler calculates that even predictions with sensitivity of 90% (the proportion of true negative predictions) would give 2000 false positives for every homicide accurately predicted. A recalculation employing Szmuckler's assumptions, using a standard probability matrix (Reference Bourke, Daly and McGilvrayBourke et al, 1985, p 248) gives 1111 false positives per homicide accurately predicted (Table 1).

Table 1. Homicide rate in ‘psychosis years’ using a standard probability matrix

Homicide (violence) actually occurs Homicide (violence) does not occur Row totals Homicide matrix (violence matrix)
(a) (b)
Homicide predicted (violence predicted) 18 (3600) 19 998 (19 960) 20 016 (23 560)
(c) (d)
Homicide not predicted (violence not predicted) 2 (400) 179 982 (176 040) 179 984 (176 440)
Column totals
Homicide matrix (violence matrix) 20 (4000) 199 980 (196 000) 200 000 (200 000)

Violence, not homicide

Recorded crimes of violence are much more common than homicide, by a factor of about 200 for the general population of London (Reference Kennedy, Iveson and HillKennedy et al, 1999) and the real rate of violence in the community is even greater. Anyone familiar with witness statements will know that the difference between an assault and a homicide is often merely a matter of chance. Violence, because it is more common, should be easier to predict than homicide. There are about 1000 hospital orders, restricted and unrestricted (Sections 37 and 37/41) per year in England and Wales, with a further 250 transfers from prison under Section 47/49 (Reference Kershaw and RenshawKershaw & Renshaw, 1998). These are not all sentenced for crimes of violence, but most are, and a further number of patients with severe mental illness are dealt with by making civil orders in Magistrates' Courts (Reference JamesJames, 1999) or by the police bringing patients directly to hospital under Section 138 (Reference Humphries and JohnstoneHumphries & Johnstone, 1993). An annual estimate of 4000 recorded crimes of violence by people with mental illness seems reasonable in this context.

Using the same assumed 12-month prevalence of schizophrenia, at 90% sensitivity and 90% specificity, Table 1 shows that there would be 4.95 false predictions of reported crimes of violence per accurate prediction. This would lead to the targeting of extra services to 11.8% of patients. Violence should therefore be easier to predict than homicide or suicide, but even so, Szmukler's use of homicide as an outcome measure is a straw-man argument with several related, but mistaken, assumptions.

Homicide and violence as system failures

Szmuckler has assumed that no risk assessment or risk management is currently in place. It would be more accurate to assume, at least for the sake of probability modelling, that the 20 homicides a year by those suffering from severe mental illness, like the 4000 violent offences, represent the failure rate of a mental health system for combined risk assessment and risk management, considered at the population level. Table 2 sets out sensitivity and specificity matrices for risk management in which patients either have or have not a risk of violence and homicide that for the sake of simplicity is either 0 (no risk at all) or 1 (certainty) for the coming year. Similarly, the process of risk management (a combination of clinical risk assessment, risk reduction through treatment and long-term risk management through the therapeutic uses of security) is assumed, for the sake of simplicity, to be 100% effective when applied. If the overall system, considered at the population level, has sensitivity and specificity of 90% such that 20 homicides and 4000 recorded crimes of violence would follow, then 100 patients are inappropriately treated for each homicide prevented. But only two are inappropriately treated for every five crimes of violence prevented.

Table 2. Sensitivity and specificity matrices for risk management

Homicide (violence) risk present Homicide (violence) risk not present Row totals Homicide matrix (violence matrix)
Preventive measures applied (a) (b)
Homicide (violence) 200 (40 000) 19 978 (15 600) 20 178 (55 600)
Preventive measures not applied (c) (d)
Homicide (violence) 20 (4000) 179 802 (15 600) 179 822 (144 400)
Column totals
Homicide matrix (violence matrix) 220 (44 000) 199 780 (156 000) 200 000 (200 000)

Since the therapeutic use of security includes a wide range of short- medium- and long-term interventions, including not only detention in secure hospitals, but also 24-hour nursed care units, high support hostels and assertive community care teams, the treatments inappropriately applied are not necessarily onerous and will have other health benefits.

Risk assessment and stratified therapeutic security as a system for risk management

Of course in practice, the risk of violence is a continuous variable rather than all or nothing, and no system of mental health care could be 100% effective in preventing violence by those with severe mental illness, in the community or in hospital. Table 3 shows a risk matrix that assumes that two groups of patients can be distinguished, one with a mean risk of violence in the next year of 0.5 (high) and the other with a mean risk of 0.02 (the population risk in community surveys). It is also assumed that clinical preventive measures, broadly defined, are 99% effective. In this model, the same number of recorded crimes of violence will follow if the mental health system operates with a sensitivity and specificity of 90%, while 5.6 patients receive preventive measures (4.6 inappropriately) for each episode of violence prevented.

Table 3. Risk matrix assuming two groups of patients (high and low risk) can be distinguished

Violence high risk present (P=0.5) Violence low risk present (P=0.02) Row totals
(a) (b)
Preventive measures applied (0.99 effective) 9542 (48 episodes of violence occur) 19 130 (4 episodes of violence occur) 28 672
(c) (d)
Preventive measures not applied 1084 (542 episodes of violence occur) 170 244 (3405 episodes of violence occur) 171 328
Column totals
10 626 (590 episodes of violence occur) 189 374 (3409 episodes of violence occur) 200 000 (3999 episodes of violence occur)

Theoretical and practical considerations

The model illustrated in Table 3 goes some way towards compensating for the deficiencies in the model employed by Szmuckler. The assumption that patients are not assessed for risk, and that their clinical management is not modified to take account of this, is common in existing research studies of risk assessment, for example, Powell et al (Reference Powell, Geddes and Deeks2000) and in many of the studies reviewed by Dolan and Doyle (Reference Dolan and Doyle2000).

Placement decisions are made to stratify patients according to risk

It is wrong to assume that all groups of patients really are at the average risk for patients nationally. Even in nonforensic populations presenting unselected from the community, it is a commonplace observation that about 3% of the patients account for about 50% of the violence in hospital (Reference Crichton and WalkerCrichton, 1996). All mental health services, general as well as forensic, respond by carefully stratifying patients according to the risk they present to themselves and to others. Snowden et al (Reference Snowden, McKenna and Jasper1999) point out that three-quarters of all restriction order patients in England and Wales (2744 and 3677, respectively) are in hospital at any one time. Even for populations of patients defined by local sectors, the most high risk will be in hospital owing to acute relapses, and for inner-city populations a higher proportion will be in secure forensic units (Reference Glover, Leese and McCroneGlover et al, 1999). Within forensic services, the aim is to stratify patients according to the risk of violence and the gravity of the violence. This allows the therapeutic use of security (relational, procedural and environmental) to be matched (high, medium, low, etc.) to the current need, determining the pace of progress through treatment settings, from admission to rehabilitation to community placement. Forensic patients are selected for discharge when stable, predictable and willing to tolerate a degree of intrusion and control by clinicians in the community. In community forensic populations the average risk of grave violence is higher than for patients in general psychiatric community services.

Risk is stratified in time

The risk of suicide is greatest in the year after discharge from hospital, of the order of 48 to 61 per 10 000 patient years (Reference Fawcett, Scheftner and FoggFawcett et al, 1990; Reference Geddes, Juszack and O'BRIENGeddes et al 1997; Reference Rossau and MortensenRossau & Mortensen, 1997). In a recent US study, the risk of violence in the year following hospital discharge was 1990/10 000 (19.9%) for patients with mental illness without substance misuse problems, and 3110/10 000 for those with mental illness and comorbid substance misuse (Reference Steadman, Mulvey and MonahanSteadman et al, 1998). This temporal stratification of risk should enable more accurate risk assessments for the period immediately after leaving hospital because of the higher average risk during that period. More intensive contact with mental health services (relational and procedural security) might be expected to manage the risk within the bounds of what is possible or predictable.

The variance of risk

The risk of violence or suicide is a statistically distributed variable in any population, whether or not there are distinct sub-populations with distinct distributions owing to disease or other factors. A wide variance in the distribution of risk can weaken the sensitivity and specificity of any method of assessment. However, just as it is wrong to assume that all patient populations are at the national average for a given risk, it is wrong to assume that the average risk in any patient group is made up of the same wide range of risk. Patients new to a mental health service are at unknown average risk and probably have a wide range of risk. In forensic and other services, patients are selected to stratify according to level of risk and gravity of the risk, but this stratification will also narrow the range of risk within each group. Those treated in the community should also be selected within a narrow range of risk. This narrow range should increase the accuracy of risk assessment.

Contextual factors

Clinical judgements and predictions are not made in a statistically pure vacuum. Such judgements are contextual, they are revised in the light of relevant information and they are often very specific to the experience of the clinician and of the patient. Bayesian approaches to measurement of risk are a better model for such assessments, in which probability is regarded as representing a degree of reasonable belief (Reference HeyHey, 1983). Powell et al (Reference Powell, Geddes and Deeks2000) employ Bayesian corrections for the likelihood ratios of risk-factors identified in their study so that the local context is included.

An important caveat regarding risk prediction concerns the assumption that all local populations from which patients are drawn are at the same average risk as the total population. This is untrue. In inner-city boroughs the population risk of homicide, suicide or recorded crimes of violence increases exponentially with population density or deprivation (Reference Kennedy, Iveson and HillKennedy et al, 1999). This variable background of environmental risk probably worsens the risk of adverse outcomes such as violence, homicide or suicide for a given patient. The non-linear relationship of the outcome to its (assumed) environmental determinants probably limits the clinician's capacity to make anything more than short-term predictions when the environmental risk is high.

Systems failure and the therapeutic uses of security

From admission ward through to locked intensive care to rehabilitation ward, 24-hour nursed care and hostel, all constituent parts of a mental health hospital service systematically stratify patients according to a hierarchy of risk, so that patients are allocated to places with high, medium or low levels of relational, procedural and environmental security. This long-standing structural risk management at the systems level may have been lost in modern sectorised community mental health teams and small generic in-patient units. If general psychiatrists' daily manage mixed populations of patients, many of whom are at a risk of committing homicide of 1/10 000 per year as Szmuckler suggests, (or 200/10 000 risk of violence) they may be unaware that a few of their patients are at a risk of 1/100 for homicide or 1/10 for violence. Even with little regard for risk management, most psychiatrists responsible for catchment populations of 30 000 to 50 000 (120 to 200 with severe mental illness) would still seldom experience adverse outcomes and may be unaware of the risks they run in some cases. This functional risk-blindness is likely to be particularly hazardous in high risk inner-city boroughs. It follows that the mixing of all patients from a small sector catchment area in small generic services is bad risk management at the systems level. Not only does it prevent the targeting of specialised care by stratifying according to need, it can make risk assessment less accurate and risk management too diffused.

Conclusions

The figures given by Szmuckler, and in Tables 1,2,3, depend on estimates and inferences. It would be possible to give real figures for the risk management matrices based on case registers and hospital activity data if health service information was accurately collated (Reference GloverGlover, 2000). The criminal justice data could similarly be collated. The use of presentation rates via the criminal justice system as a performance measure for mental health services is already possible (Reference Purchase, McCallum and KennedyPurchase et al, 1996).

The prevailing rate of homicide in the community by patients with schizophrenia is already modified by risk stratification in mental health services. Szmuckler is right to observe that there is little point in focusing on the prevention of homicide. But this is because the relationship between acts of violence and completed homicide is often a matter of bad luck. Risk assessment is worth the effort and can be improved on, although only by recognising that the process of assessment is, for practical purposes, inseparable from the process of managing risk by matching the patient's risk to the appropriate level of therapeutic security. The risk of violence should be more predictable and more easily managed, within the limits of what is possible with the available resources and the risks inherent in the local community. What really matters is the effectiveness of the mental health services as a risk management process for the patient population overall.

References

Bourke, G. J., Daly, I.E. & McGilvray, J. (1985) Interpretation and Uses of Medical Statistics (3rd edn). Oxford: Blackwell Scientific.Google Scholar
Crichton, J. (1996) Psychiatric inpatient violence, Appendix A. In: Dangerous People (ed. Walker, N.), pp. 144145. London: Blackstone Press.Google Scholar
Dolan, M. & Doyle, M. (2000) Violence risk prediction. Clinical and actuarial measures and the role of the Psychopathy Checklist. British Journal of Psychiatry, 177, 303311.CrossRefGoogle ScholarPubMed
Fawcett, J., Scheftner, W. A., Fogg, L., et al (1990) Time-related predictors of suicide in major affective disorder. American Journal of Psychiatry, 147, 11891194.Google Scholar
Geddes, J. (1999) Suicide and homicide by people with mental illness. British Medical Journal, 318, 11251226.CrossRefGoogle ScholarPubMed
Geddes, J., Juszack, E., O'BRIEN, F., et al (1997) Suicide in the 12 months after discharge from psychiatric in-patient care, Scotland 1968–92. Journal of Epidemiology and Community Health, 51, 430434.Google Scholar
Glover, G. (2000) The minimum data set. At last – informationl Psychiatric Bulletin, 24, 163164.CrossRefGoogle Scholar
Glover, G., Leese, M. & McCrone, P. (1999) More severe mental illness is more concentrated in deprived areas. British Journal of Psychiatry, 175, 544548.CrossRefGoogle ScholarPubMed
Hey, J. (1983) Data in Doubt. Oxford: Martin Robertson.Google Scholar
Humphries, M. & Johnstone, E. C. (1993) Dangerous behaviour preceding first admissions for schizophrenia. British Journal of Psychiatry, 163, 547.Google Scholar
James, D. (1999) Court diversion at ten years: can it work, does it work and has it a future? Journal of Forensic Psychiatry, 10, 507524.Google Scholar
Kennedy, H. G., Iveson, R. Y. & Hill, O. (1999) Violence, homicide and suicide: strong correlation and wide variation across districts. British Journal of Psychiatry, 175, 462466.Google Scholar
Kershaw, C. & Renshaw, G. (1998) Statistics of mentally disordered offenders, England & Wales 1997. Home Office Statistical Bulletin, 19.Google Scholar
Powell, J., Geddes, J., Deeks, J., et al (2000) Suicide in psychiatric hospital in patients. Risk factors and their predictive power. British Journal of Psychiatry, 176, 266272.Google Scholar
Purchase, N. D., McCallum, A. K. & Kennedy, H. G. (1996) Evaluation of a psychiatric court liaison scheme in North London. British Medical Journal, 313, 531532.Google Scholar
Rossau, C. D. & Mortensen, P. B. (1997) Risk factors for suicide in patients with schizophrenia: nested case-control study. British Journal of Psychiatry, 171, 355359.CrossRefGoogle ScholarPubMed
Snowden, P., McKenna, J. & Jasper, A. (1999) Management of conditionally discharged patients and others who present similar risks in the community: integrated or parallel? Journal of Forensic Psychiatry, 10, 583596.Google Scholar
Steadman, H. J., Mulvey, E., Monahan, J., et al (1998) Violence by people discharged from acute psychiatric inpatient facilities and by others in the same neighborhoods. Archives of General Psychiatry, 55, 393401.Google Scholar
Szmukler, G. (2000) Homicide inquiries. What sense do they make? Psychiatric Bulletin, 24, 610.Google Scholar
Taylor, P. J. & Gunn, J. (1999) Homicides by people with mental illness: myth and reality. British Journal of Psychiatry, 274, 914.Google Scholar
Figure 0

Table 1. Homicide rate in ‘psychosis years’ using a standard probability matrix

Figure 1

Table 2. Sensitivity and specificity matrices for risk management

Figure 2

Table 3. Risk matrix assuming two groups of patients (high and low risk) can be distinguished

Submit a response

eLetters

No eLetters have been published for this article.