Hostname: page-component-7c8c6479df-94d59 Total loading time: 0 Render date: 2024-03-28T22:39:14.422Z Has data issue: false hasContentIssue false

Criminally invalid: the treatment outcome profile form for substance misuse

Published online by Cambridge University Press:  02 January 2018

Jason Luty
Affiliation:
South Essex Partnership NHS Trust, the Taylor Centre, Queensway House, Essex Street, Southend-on-Sea, Essex SS1 2RB, email: sl006h3607@blueyonder.co.uk
Sabu Varughese
Affiliation:
Runwell Hospital, Wickford
Joby Easow
Affiliation:
Grays Community Drug and Alcohol Service, Grays
Rights & Permissions [Opens in a new window]

Abstract

Aims and Method

To audit completion of the Treatment Outcome Profile (TOP) form in individuals attending substance misuse services in England. Forms are completed at the start of treatment and every 3 months thereafter. All forms at 3-drug treatment services were inspected over 6 months.

Results

Forms were inspected for 200 service users; 86% were fully completed. Two-thirds (67%) of service users had no declared funding for illicit drug use in the previous month (mean spending £988; s.e. = 149) despite denying any paid employment and criminal activity.

Clinical Implications

The section on crime in the TOP form is unreliable and completely invalid.

Type
Original papers
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © Royal College of Psychiatrists, 2009

In October 2007, the Treatment Outcomes Profile (TOP): A Structured Interview for the Evaluation of Substance Misuse Treatment form Reference Marsden, Farrell, Bradbury, Dale-Perera, Eastwood and Roxburgh1 became a mandatory instrument that is to be completed for all individuals in treatment for illicit drug use with the National Health Service (NHS) substance misuse services. The results are reported back to local NHS commissioning organisations (drug and alcohol action teams). The TOP form is a single-page questionnaire with four domains: substance misuse, injecting behaviour, criminal activity and health/social functioning. It is to be completed by the patient's key worker at first contact and then every 3 months. The National Treatment Agency for Substance Misuse explicitly states in its patient information leaflet (Treatment Outcome Profile (TOP): Making Your Drug Treatment Better) that the TOP form ‘will enable treatment services to be assessed on the basis of the outcomes they achieve for their service users.’ The TOP form was developed from the Maudsley Addiction Profile Reference Marsden, Gossop, Stewart, Best, Farrell and Lehmann2 and tested on several hundred service users before its introduction in October 2007.

Method

The TOP forms are completed at the start of treatment and every 3 months thereafter in all individuals attending NHS substance misuse services in England. The form is based on a semi-structured interview and quantifies drug use in terms of estimated quantity or spending over the preceding 28 days. It also makes an assessment of physical and mental health, criminal activity and a variety of health and social indicators over the preceding 28 days.

All available forms were inspected over the period October 2007 to March 2008 at three drug and alcohol services in South Essex. This included a mixture of TOP forms from initial attendance and 3-monthly reviews, and was part of an audit to ensure that the forms were correctly completed. Data from the forms were summarised, including the reported spending on class A drugs (heroin and cocaine). Spending on cannabis, amphetamine and alcohol was not analysed, as less than 10% of total spending in the sample involved these drugs, whereas opiate and cocaine use was the primary problem for 86% of the sample. Where necessary, quantity used was converted to street drug prices using information collated from a variety of sources. 3 Individual TOP forms were completed by the 24 key workers based at the three teams.

Results

We inspected the TOP forms for 200 individuals; 86% were fully completed. However, analysis of the audit results showed that 67% of the sample had no declared funding for class A drug use (other than benefits), with a mean spending of £988 (s.e. = £149) in the previous month despite denying any paid employment and denying any criminal activity (69% male, n = 123; mean age 34 years, s.e. = 0.5). For the sample as a whole (n = 200), mean spending was also £988 over the previous month (s.e. = 137; 73% male; mean age 32 years, s.e. = 0.5).

Discussion

Over the past decade, the UK government has been committed to reducing crime especially by targeting illicit drug use. 4 Drug treatment agencies have been the flagship of this approach, with massive increases in government spending. There are currently around 180 000 illicit drug users (mainly opiate users) in treatment with the 200 UK community drug and alcohol services (http://drugs.homeoffice.gov.uk/treatment/strategy/?version=2). Government spending on these services in the 2006/07 financial year approached £600 million (www.nta.nhs.uk/about/funding/drug___treatment___services.aspx). Results from the TOP forms, especially indices of criminal activity, are likely to be used by the Department of Health to justify this level of spending and inform funding decisions in the future.

The audit confirmed that the vast majority of TOP forms were completed to an acceptable level. Funding for British drug treatment agencies is now dependent on submitting the results from these forms and they are routinely entered into the database or ‘return’ that are sent each month to fund-holders. It is therefore reassuring, although not entirely unexpected, that these forms are being completed.

How to explain the considerable disparity between self-declared drug use and available income?

The results of the study indicate a massive disparity between income (legal and criminal) and spending on illicit drugs, with half of our sample declaring no funding despite a mean spending of over £900 every 28 days. Most unemployed people, including class A drug users, claim benefits of £200-300 per month. 5 It seems extremely unlikely that drug use at this level is funded by gifts from relatives or friends. The street drug prices were compiled from a variety of sources, including the National Criminal Intelligence Agency and 80 front-line drug agencies, police forces and drug action teams in 20 towns and cities throughout the UK. 3 The prices of typical quantities of illicit drugs are displayed for the major conurbations throughout Britain. As this information was obtained from reports of staff, outreach workers and service users attending local treatment agencies, it is likely to be reliable.

The other explanation for the disparity between reported drug use and spending is the falsification of information provided at interview by the service users. Admittedly, it is possible that users could exaggerate their consumption, but they would have little reason to do this to the extent revealed by the study. Moreover, other studies have suggested that typical class A drug users in Britain spend £40 per day (around £1000 per month) on heroin and often more on cocaine. Reference Luty, Perry, Umoh and Gormer6,Reference Gossop7 Many surveys have concluded that self-reporting of drug use is reliable and valid when events are recent and individuals do not face negative consequences. Reference McLellan, Luborsky, Woody, Griffith, Evans and Barr8Reference Darke12 For example, the UK National Treatment Outcome Research Study reported that the concordance rate between self-reported use and results from a urine specimen for heroin and cocaine was 92%. Only 2-3% of clients who reported not using heroin or cocaine tested positive. Reference Gossop, Marsden and Stewart13 Hser et al assessed the reliability of retrospective self-report by 323 drug addicts at a 10-year interval with respect to opiate use and criminality. The diagnostic accuracy and predictive power of the self-report was found to be very high (over 80%). Reference Hser, Anglin and Chou14 In regard to self-report of cocaine use, Simpson et al reported that 8.8% of people denied cocaine use but subsequently tested positive. This was based on a random sample of 352 interviewees who presented with cocaine misuse as part of the US Drug Abuse Treatment Outcome Study (DATOS). Reference Simpson, Joe, Fletcher, Hubbard and Anglin15 However, these results are based on prolonged interviews in research samples rather than brief initial assessments in routine clinical teams where self-report may be less reliable.

The most likely explanation for the disparity between drug use and funding is that individuals are not reporting criminal activity. There is a vast body of research on self-report and concealment of delinquent and deviant activity including crime and sexual behaviour. Reference Farrington16Reference Weinhard, Forsyth, Carey, Jaworski and Durant20 It is also self-evident that many service users who are on court diversion programmes (such as the drug rehabilitation requirements) are unlikely to admit to illegal activity as this will be reported automatically to the court during their monthly reviews.

TOP and MAP

The Maudsley Addiction Profile (MAP) is a well-validated, brief, semi-structured interview developed to assess the substance use, risk-behaviour, health and social functioning of illicit drug users. Reference Luty, Perry, Umoh and Gormer6 A health professional is required to complete the various sections, which takes approximately 12 minutes. There is no requirement for formal training to use the MAP. There is considerable overlap between the MAP and the TOP form and they were produced by many of the same authors. Both instruments are completed by clinicians following an interview with the clients, both have excellent psychometric properties (with correlation coefficients exceeding 0.75 between different indices) and have been validated in samples of several hundred clients at more than 50 community drug and alcohol services. Reference Marsden, Farrell, Bradbury, Dale-Perera, Eastwood and Roxburgh1,Reference Luty, Perry, Umoh and Gormer6 They are relatively short (although TOP is significantly shorter) and are eminently practicable for use in front-line clinical services, although the researchers were frequently made aware of complaints by staff regarding the ever increasing and often irrelevant paperwork required by substance misuse services and mental health services in general. It would be highly desirable if the TOP form could be used to replace, rather than supplement, some of the other documentation often known as the ‘basic data-set’.

Are self-reports valid?

Problems with the validity of self-reported criminal activity in routine clinical settings have previously been observed in relation to the Maudsley Addiction Profile. Reference Luty, Perry, Umoh and Gormer6 Of the 206 service users in this study, 96 were not working and denied committing any acquisitive crimes in the previous month, although they spent an average of £595 (s.e. = 96) on heroin and cocaine over this period. It is difficult to see how they were able to fund this illicit drug use without recourse to criminal activity, particularly as their total income from benefits amounted to only £230 per month each, and this includes payments to cover food and accommodation. Even with enhancements for disability or child support, this amount remains less than £500 per month. 5 It is possible that these individuals were receiving an undeclared income from informal work as well as fraudulently claiming unemployment benefits. However, it seems unlikely that they were willing to reveal the amount they were spending on illicit drugs and yet withhold information on undeclared employment.

Conclusion

The effectiveness of NHS community drug and alcohol services will be assessed using data derived from the TOP forms. Moreover, future government policy and funding is likely to be determined by reports based on the TOP form (especially criminal activity), hence it is important that this information is valid. Unfortunately, our study indicates that data on criminal activity from the TOP form are invalid assuming the results of this study are typical throughout the UK. The criminal activity section of the TOP form is unreliable and completely invalid.

Declaration of interest

None.

Acknowledgements

The study was funded entirely by the authors and approved by the local audit and clinical governance committee.

References

1 Marsden, J, Farrell, M, Bradbury, C, Dale-Perera, A, Eastwood, B, Roxburgh, M, et al. Development of the treatment outcomes profile. Addiction 2008; 103: 1450–60.Google Scholar
2 Marsden, J, Gossop, M, Stewart, D, Best, D, Farrell, M, Lehmann, P, et al. The Maudsley Addiction Profile (MAP): a brief instrument for assessing treatment outcome. Addiction 1998; 93: 1857–68.CrossRefGoogle ScholarPubMed
3 Druglink. Street drug prices 2006. Druglink 2006; September/ October: 26.Google Scholar
4 Home Office. Drugs: Protecting Families and Communities: 2008–2018 Strategy. TSO (The Stationery Office), 2007.Google Scholar
5 Social Security Office. Social Security Benefit Rates. UK Department for Works and Pensions, 2006.Google Scholar
6 Luty, J, Perry, V, Umoh, O, Gormer, D. Validation and development of a self-report outcome measure (MAP–sc) inopiate addiction. Psychiatr Bull 2006; 30: 134–9.Google Scholar
7 Gossop, M. Drug Misuse Treatment and Reductions in Crime: Findings from the National Treatment Outcome Research Study (NTORS). National Treatment Agency, 2005.Google Scholar
8 McLellan, AT, Luborsky, L, Woody, GE, Griffith, J, Evans, F, Barr, HL, et al. New data from the Addiction Severity Index: reliability and validity in three centres. J Nerv Ment Dis 1985; 173: 412–23.Google Scholar
9 Mieczkowski, T. The accuracy of self-reported drug use. In Drugs and Crime and the Criminal Justice System (ed Weisher, R.): 275302. Anderson, 1990.Google Scholar
10 Poole, WK, Flynn, PM, Rao, AV, Cooley, PC. Estimating the prevalence of drug use from self-reports. Am J Epidemiol 1996; 144: 413–20.Google Scholar
11 Lundy, A, Gottheil, E, McLellan, AT, Weinsten, SP, Sterling, R, Serota, RD. Underreporting of cocaine use. J Nerv Ment Dis 1997; 185: 459–62.Google ScholarPubMed
12 Darke, S. Self-report among injecting drug users. Drug Alcohol Depend 1998; 51: 253–63.Google Scholar
13 Gossop, M, Marsden, J, Stewart, D. NTORS at One Year: the National Treatment Outcome Research Study – Changes in Substance Use, Health and Criminal Behaviours One Year after Intake. Department of Health, 1998.Google Scholar
14 Hser, Y, Anglin, MD, Chou, C. Reliability of retrospective self-report by narcotic addicts. Psychol Assess 1992; 4: 207–13.Google Scholar
15 Simpson, DD, Joe, GW, Fletcher, BW, Hubbard, RL, Anglin, D. A national evaluation of treatment outcomes for cocaine dependence. Arch Gen Psychiatry 1999; 56: 507–14.CrossRefGoogle ScholarPubMed
16 Farrington, DP. Self-reports of deviant behavior: predictive and stable? J Crim Law Criminol 1973; 64: 99110.Google Scholar
17 Huizinga, D, Elliott, DS. Reassessing the reliability and validity of self-report delinquency measures. J Quant Criminol 1986; 2: 293327.CrossRefGoogle Scholar
18 Bryant, CD. Deviant Behavior: Readings in the Sociology of Norm Violations. Hemisphere Publishing, 1990.Google Scholar
19 Anglin, D. Reliability and validity of retrospective behavioral self-report by narcotics addicts. Evaluation Review 1993; 17: 91108.Google Scholar
20 Weinhard, LS, Forsyth, AD, Carey, MP, Jaworski, BC, Durant, LE. Reliability and validity of self-report measures of HIV-related sexual behavior: progress since 1990 and recommendations for research and practice. Arch Sex Behav 1998; 27: 155–80.Google Scholar
Submit a response

eLetters

No eLetters have been published for this article.