Hostname: page-component-7c8c6479df-8mjnm Total loading time: 0 Render date: 2024-03-29T00:54:37.768Z Has data issue: false hasContentIssue false

Validation of the Social Satisfaction Questionnaire for outcome evaluation in substance use disorders

Published online by Cambridge University Press:  02 January 2018

Duncan Raistrick
Affiliation:
Leeds Addiction Unit, Leeds, email: Duncan.raistrick@leedsmh.nhs.uk
Gillian Tober
Affiliation:
Leeds Addiction Unit
Nick Heather
Affiliation:
Northumbria University
Jennifer A. Clark
Affiliation:
Newcastle General Hospital, Newcastle
Rights & Permissions [Opens in a new window]

Abstract

Aims and Method

To develop a scale to measure social satisfaction in people with substance use disorders and to test its psychometric properties. The rationale is that social satisfaction is more universal and relevant to treatment planning than assessing social problems. The new Social Satisfaction Questionnaire (SSQ) was derived from an existing social problems questionnaire and validation was undertaken on two large clinic populations.

Results

An eight-item SSQ was tested and found to have good psychometric properties in terms of test–retest reliability, internal consistency, distribution of responses and concurrent validity.

Clinical Implications

The SSQ is suitable for use as the social domain element of an outcome measures package.

Type
Original Papers
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © Royal College of Psychiatrists, 2007

Social functioning has consistently been associated with help-seeking and entry into treatment for alcohol and drug problems (e.g. Reference Moos and MoosMoos & Moos, 2004) and is an important domain to measure in the evaluation of treatment outcome; social problems are a key component of current models of the treatment process (Reference Gifford, Ritsher and MckellarGifford et al, 2006; Reference Orford, Hodgson and CopelloOrford et al, 2006). Poor levels of social functioning are associated with poorer treatment outcomes and risk of relapse (Reference Moos and MoosMoos & Moos, 2006). Substance use disorders span the socio-economic spectrum, and the task of identifying norms to measure social functioning components of treatment outcome is fraught with difficulty.

Standard measurement tools include the European Quality of Life (EuroQoL; Reference Brooks, Rabin and De CharroBrooks et al, 2003), which is useful for comparisons of health status across populations but is insensitive for specific social outcomes, and the Short Form Health Survey (SF–36; Reference Ware, Kosinski and DeweyWare et al, 2000), which is acceptable to service users (Reference Garratt, Ruta and AbdallaGarratt et al, 1993) and suggested as a standard for the National Health Service (NHS) but is too lengthy to be included in routine, substance misuse specific outcome measurement and does not inform treatment goals. Criteria suitable for outcome measurement in routine clinical practice (Reference Raistrick, Bradshaw and ToberRaistrick et al, 1994) are universality of scale items, change potential, items written in plain English, self-completion format to avoid therapist bias, satisfactory psychometric properties and brevity of completion. To include these and tap into the subjective experience of help seekers, we decided to measure satisfaction with social circumstances. Thus, the individual's objective social circumstances are less important than how they feel about them both for measurement and as the focus of treatment planning.

The relativity of social functioning was addressed by Corney & Clare (Reference Corney and Clare1985) who developed a 33-item Social Problems Questionnaire (SPQ) as a compact, comprehensible, valid and reliable self-report questionnaire for screening individuals in primary care or related settings who are particularly at risk for manifesting social maladjustment and dysfunction (Reference Corney and ClareCorney & Clare, 1985: p. 638).

Recognising the difficulties inherent in attempts to measure and compare social circumstances, the SPQ combines items enquiring about objective circumstances with items enquiring about the degree of satisfaction experienced with reference to these. The domains covered are: housing, occupation, finances, social activities and relationships, relatives, marital status, children, other domestic relationships, legal matters, and questions for those living alone. The scale was piloted and tested in primary care samples (including the responses of relatives), in psychiatric and epilepsy out-patient departments and in a social work case-load, and found to be a valid and reliable measure which is readily acceptable for use either by postal or face-to-face contact.

This paper describes the development and validation of the Social Satisfaction Questionnaire (SSQ), which was adapted from the SPQ (Reference Corney and ClareCorney & Clare, 1985) as a brief measure of satisfaction and designed for use in routine assessment and treatment outcome evaluation (Reference Tober, Brearley and KenyonTober et al, 2000).

Method

The SSQ was developed in four steps:

  1. step 1 - item reduction from the SPQ to create the eight-item SSQ

  2. step 2 - piloting the SSQ

  3. step 3 - main validation in clinical practice on two large samples

  4. step 4 - test-retest reliability.

Items which did not measure satisfaction directly, those measuring actual circumstances, failing to meet inclusion criteria, or referring to single occupational groups were removed from the SPQ, and some rephrasing of the remaining items was undertaken. This produced an eight-item questionnaire (see data supplement to the online version of this paper). The SSQ was piloted with 20 service users and minor adjustments made to wording and layout.

The psychometric properties of the scale were tested in three separate samples. Samples 1 and 2 were used for testing validity and internal consistency. Sample 1 consisted of 1665 new service users attending NHS services in Leeds and Newcastle. The assessment package comprised self-reported substance use, dependence as measured by the Leeds Dependence Questionnaire (LDQ; Reference Raistrick, Bradshaw and ToberRaistrick et al, 1994) and psychopathology as measured by the General Health Questionnaire (GHQ; Reference Goldberg and WilliamsGoldberg & Williams, 1988). Sample 2 consisted of 6732 new service users attending the Leeds service. For this sample the assessment package had been updated to measure psychopathology using the Clinical Outcomes in Routine Evaluation – Outcome Measure (CORE–OM; Reference Evans, Connell and BarkhamEvans et al, 2002). Sample characteristics are presented in Table 1.

Table 1. Characteristics of samples 1 (n=1665) and 2 (n=6732)

Sample 1 Sample 2
Newcastle Leeds
Gender, n (%)
     Male 743 (78) 569 (80) 4926 (73)
     Female 213 (22) 140 (20) 1806 (27)
Age, years: mean (s.d.) 34.9 (10.5) 33.3 (10.9) 38.4 (9.8)
Main reason for referral, n (%)
     Alcohol use 425 (60) 396 (47) 2512 (37.3)
     Drug use 283 (40) 453 (53) 4220 (62.7)
LDQ score: mean (s.d.) 19.19 (7.59) 20.37 (7.64) 17.4 (8.4)
GHQ score: mean (s.d.) 23.55 (8.63) 22.47 (8.46) NA
CORE—OM score: mean (s.d.) NA NA 57.1 (21.3)

Sample 3 was used for the purpose of estimating test–retest reliability. The scale was administered as part of a package of three questionnaires to 61 respondents attending an NHS clinic for treatment of their heroin use. The sample consisted of 40 males (65.6%) and 21 females (34.4%), with a mean age of 25.7 years (s.d.=4.9, range 17-37). The mean time between first and second completion of the scale was 5.3 days (s.d.=3.75) with a minimum of 2 days and a maximum of 14 days.

Results

Item analysis

An item analysis of samples 1 and 2 was conducted to examine the homogeneity and internal consistency of the scale. Each of the eight items correlated with total SSQ score (Spearman's rho ranged from 0.57 to 0.66 and 0.60 to 0.68 for samples 1 and 2 respectively) and all were significant at the P < 0.001 level. Cronbach's α was 0.73 and 0.79 respectively and similar values held with each item deleted. In order to examine the factor structure of the scale, responses given by sample 2 were subjected to exploratory principal components analysis, which yielded three factors with an eigen value >1. The first of these had an eigen value of 3.3 accounting for 41% of the variance with item loadings ranging from 0.59 to 0.69.

Factor 2 was specific to accommodation and factor 3 to employment and finance, therefore it was felt that these items did not make sense as stand-alone factors (see Table 2) and so a single factor is a better interpretation of the data.

Table 2. Principal components analysis for the Social Satisfaction Questionnaire (sample 2, n=6732)

Component
1 2 3
SSQ1 0.595 0.637 0.220
SSQ2 0.641 0.634 0.082
SSQ3 0.592 -0.259 0.552
SSQ4 0.620 -0.325 0.493
SSQ5 0.662 -0.333 -0.109
SSQ6 0.688 -0.315 -0.250
SSQ7 0.666 0.005 -0.466
SSQ8 0.644 0.005 -0.405

Test–retest reliability

Atest–retest reliability study was conducted with sample 3. The correlation between the total SSQ scores at the two data collection points was 0.85 and the mean correlation between individual item scores was 0.72. Mean scores at the first and second data collection points were compared in order to eliminate the possibility that scores have changed but were still correlated. No significant difference was found in the mean total score for the scale between the first and second scale completion (9.3 and 8.7 respectively) indicating that the responses had remained constant.

Validity

The relationships between the SSQ and other variables were examined, using a correlation technique to demonstrate construct validity. The expectation was that there would be significant but modest correlations given the distinctiveness of the domains of dependence, psychological health and social satisfaction. This was found to be the case. For sample 1 the SSQ was negatively and significantly correlated with measures of psychopathology (r=0.38, P < 0.001) and dependence (r=0.28, P < 0.001).

The above findings were confirmed by a stepwise multiple regression analysis, with SSQ score as the dependent variable and age, gender, GHQ score (psychopathology), LDQ score (dependence) and main reason for referral (alcohol or other drug) as independent variables. Results of this analysis showed that GHQ score and LDQ score predicted SSQ score (GHQ: t=9.01, P < 0.0001; LDQ: t=3.26, P <0.001). Thus, service users with higher psychopathology and dependence were most likely to have a low score for social satisfaction. As expected gender, main reason for referral and treatment site were not predictors of SSQ.

The correlations were reassessed on sample 2 which was large enough to analyse according to the drug groups of opiates, stimulants, cannabis and alcohol. Again the expectation was that there would be low-order but significant correlations between the SSQ and other assessment measures. This was found to be the case. The SSQ was negatively correlated with the LDQ in the range -0.16 to -0.28 and with CORE-OM in the range -0.40 to -0.52 (P=0.001) (Table 3). The scale achieves a close to normal distribution across sample 2, showing a capability to measure the full range of scores.

Table 3. Correlations of the Social Satisfaction Questionnaire score with LDQ and CORE—OM (sample 2, n=6732)

Opiates Stimulants Cannabis Alcohol All
LDQ -0.25 -0.16 -0.28 -0.27 -0.25
CORE—OM -0.45 -0.40 -0.52 -0.51 -0.45

Discussion

The aim of this study was to develop and validate an instrument to measure a dimension of social functioning that was meaningful in a help-seeking population and suitable for use in routine practice. We have presented evidence that the SSQ has robust internal psychometric properties. Validation of a subjective construct such as satisfaction is problematic, however, we have shown that the SSQ is a unidimensional construct which measures a separate domain to that of dependence and psychological morbidity and is independent of unrelated variables.

The case for a new scale is based on the need for a social domain measure that has clinical utility and acceptability in a busy clinic environment. The scale has face validity in pointing to social domains that are likely to be important to and engage service users in treatment. In sample 2 we combined the SSQ with the LDQ, the CORE-OM and the substance use section of the Maudsley Addiction Profile (MAP; Reference Marsden, Gossop and StewartMarsden et al, 1998) to create a comprehensive outcome measurement package which takes approximately 15-20 min to complete. Services less concerned with psychological morbidity can save time by replacing CORE-OM with the MAP measure for this domain. There is a strong argument for using a standard assessment package. For example, the Addiction Severity Index (Reference Mclellan, Luborsky and WoodyMcLellan et al, 1980) is popular but its reliability and validity have been questioned (Reference MäkeläMäkelä, 2004). There may be a stronger case for selecting from a menu of standardised outcome measures, each with strong reliability and validity, which can be packaged to suit local need, and it was with this approach in mind that we developed the SSQ.

The SSQ has been extensively used and found to have clinical utility in two NHS substance misuse services, however, there needs to be further testing of concurrent validity and in different service user groups, beyond problems of substance misuse. Therapists familiar with scale items can incorporate responses into treatment, for example to elicit motivational statements (Reference FordFord, 2003) and include service user concerns expressed through the SSQ in care plans. A critical test of the validity and utility of the SSQ will be its ability to predict treatment outcome in combination with other measures in the package and this will be reported in due course.

Declaration of interest

None.

References

Brooks, R., Rabin, R. E. & De Charro, F. (2003) The Measurement and Valuation of Health Status Using EQ–5D: A European Perspective. Kluwer.Google Scholar
Corney, R. H. & Clare, A.W. (1985) The construction, development and testing of a self-report questionnaire to identify social problems. Psychological Medicine, 15, 637649.CrossRefGoogle ScholarPubMed
Evans, C., Connell, J., Barkham, M., et al (2002) Towards a standardised brief outcome measure: psychometric properties and utility of the CORE–OM. British Journal of Psychiatry, 180, 5160.Google Scholar
Ford, P. (2003) An evaluation of the Dartmouth Assessment of Lifestyle Inventory and the Leeds Dependence Questionnaire for use among detained psychiatric inpatients. Addiction, 98, 111118.CrossRefGoogle ScholarPubMed
Garratt, A. M., Ruta, D. A., Abdalla, M. I., et al (1993) The SF36 health survey questionnaire: an outcome measure suitable for routine use within the NHS? BMJ, 306, 14401444.Google Scholar
Gifford, E.V., Ritsher, J. B., Mckellar, J. D., et al (2006) Acceptance and relationship context: a model of substance use disorder treatment outcome. Addiction, 101, 11671177.Google Scholar
Goldberg, D. P. & Williams, P. (1988) A User's Guide to the General Health Questionnaire. nferNelson.Google Scholar
Mäkelä, K. (2004) Studies of the reliability and validity of the Addiction Severity Index. Addiction, 99, 398410.Google Scholar
Marsden, J., Gossop, G., Stewart, D., et al (1998) The MaudsleyAddiction Profile (MAP): a brief instrument for assessing treatment outcome. Addiction, 93, 18571867.CrossRefGoogle ScholarPubMed
Mclellan, A., Luborsky, L., Woody, G. E., et al (1980) An improved diagnostic instrument for substance abuse patients: the Addiction Severity Index. Journal of Nervous and Mental Disease, 168, 2633.CrossRefGoogle Scholar
Moos, R. H. & Moos, B. S. (2004) The interplay between help-seeking and alcohol-related outcomes: divergent processes for professional treatment and self-help groups. Drug and Alcohol Dependence, 75, 155164.Google Scholar
Moos, R. H. & Moos, B. S. (2006) Rates and predictors of relapse after natural and treated remissions from alcohol use disorders. Addiction, 101, 212222.Google Scholar
Orford, J., Hodgson, R., Copello, A., et al (2006) The clients' perspective on change during treatment for an alcohol problem: qualitative analysis on follow-up interviews in the UK Alcohol Treatment Trial. Addiction, 101, 6068.Google Scholar
Raistrick, D., Bradshaw, J., Tober, G., et al (1994) Development of the Leeds Dependence Questionnaire (LDQ): a questionnaire to measure alcohol and opiate dependence in the context of a treatment evaluation package. Addiction, 89, 563572.CrossRefGoogle ScholarPubMed
Tober, G., Brearley, R., Kenyon, R., et al (2000) Measuring outcomes in a health service addiction clinic. Addiction Research, 8, 169182.CrossRefGoogle Scholar
Ware, J. E., Kosinski, M. & Dewey, J. E. (2000) How to Score Version 2 of the SF–36 Health Survey. QualityMetric Incorporated.Google Scholar
Figure 0

Table 1. Characteristics of samples 1 (n=1665) and 2 (n=6732)

Figure 1

Table 2. Principal components analysis for the Social Satisfaction Questionnaire (sample 2, n=6732)

Figure 2

Table 3. Correlations of the Social Satisfaction Questionnaire score with LDQ and CORE—OM (sample 2, n=6732)

Supplementary material: File

Raistrick et al. supplementary material

Supplementary Material

Download Raistrick et al. supplementary material(File)
File 509 Bytes
Supplementary material: PDF

Raistrick et al. supplementary material

Supplementary Material

Download Raistrick et al. supplementary material(PDF)
PDF 18.7 KB
Submit a response

eLetters

No eLetters have been published for this article.