1. No category

Psychiatry Research 208 (2013) 1–8
Contents lists available at SciVerse ScienceDirect
Psychiatry Research
journal homepage: www.elsevier.com/locate/psychres
Symptom severity scale of the DSM5 for schizophrenia, and other
psychotic disorders: diagnostic validity and clinical feasibility
Michael S. Ritsner n, Maria Mar, Marina Arbitman, Alexander Grinshpoon
Department of Psychiatry, Rappaport Faculty of Medicine, Technion—Israel Institute of Technology, Haifa and Sha’ar Menashe Mental Health Center, Israel
a r t i c l e i n f o
abstract
Article history:
Received 11 September 2012
Received in revised form
24 January 2013
Accepted 22 February 2013
Innovations in DSM5 include dimensional diagnosis of schizophrenia (SZ) and other psychotic (OP)
disorders using the symptom severity scale (SS-DSM5). We evaluated the psychometric properties and
diagnostic validity of the SS-DSM5 scale using a cross-sectional design and an unselected convenience
unselected sample of 314 inpatients and outpatients with SZ/OP and mood disorders who received
standard care in routine clinical practice. The SS-DSM5 scale, the Clinical Global Impression-Severity
scale (CGI-S), the Positive and Negative Syndrome Scale (PANSS), and the Bech-Rafaelsen Mania Scale
(BRMS) were administered. Factor structure, reliability, internal consistency, convergent and diagnostic
ability of the DSM5-SS were evaluated. Factor analysis indicated two latent factors underlying the SSDSM5 (Psychotic and Deficit sub-scales). Cronbach’s alpha was 4 0.70. Convergent validity of the SSDSM5 was highly significant. Patients with SZ/PO disorders were correctly diagnosed (77.9%) using the
SS-DSM5 scale (72% using PANSS). The agreement of the diagnostic decisions between the SS-DSM5 and
PANSS was substantial for SZ/PO disorders (Kappa ¼ 0.75). Classifying participants with SZ/PO versus
mood disorders using SS-DSM5 provided a sensitivity of 95%, and specificity of 34%. Thus, this study
suggests that the SS-DSM5 has acceptable psychometric properties and that its use in clinical practice
and research is feasible in clinical settings. The dimensional option for the diagnosis of schizophrenia
and related disorders using SS-DSM5 is discussed.
& 2013 Elsevier Ireland Ltd. All rights reserved.
Keywords:
DSM5
Schizophrenia
Other psychoses
Dimensional diagnosis
1. Introduction
Numerous studies in the last century have increasingly
emphasized that the boundaries between nosological entities
may not be categorical, and putative comorbidity of various
disorders may reflect impairments in common clinical dimensions, genetic variation, human behavior and neurobiological
functions (e.g., Peralta and Cuesta, 2007; Owen et al., 2007;
Ritsner and Gottesman, 2011). The categorical approach defines
subgroups within the disorder, whereas the dimensional
approach emphasizes the severity of different symptom clusters.
As such, it is important to explicitly include dimensional assessments of the core symptoms of psychotic disorders in order to
identify pertinent variability. Therefore, the most useful current
approach for the classification of schizophrenia (SZ), other
psychotic (OP), and mood disorders may be the complementary
use of categorical and dimensional representations of functional
psychoses (Salokangas, 2003; Brown and Barlow, 2005; Dikeos
n
Correspondence to: Acute Department Sha’ar Menashe Mental Health Center,
Mobile Post Hefer 38814, Hadera, Israel. Tel.: þ972 4 6278750;
fax: þ 972 4 6278045.
E-mail addresses: ritsner@sm.health.gov.il, ritsnerm@gmail.com (M.S. Ritsner).
0165-1781/$ - see front matter & 2013 Elsevier Ireland Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.psychres.2013.02.029
et al., 2006; Dutta et al., 2007; Helzer et al., 2008; van Os, 2009;
Kamphuis and Noordhof, 2009).
Recently, the American Psychiatric Association (APA) posted a
draft of the Diagnostic and Statistical Manual of Mental Disorders,
5th ed. (DSM5) on a special web site, www.dsm5.org. One of
innovations in the DSM5 is the extensive use of so-called dimensional assessments to account for severity of symptoms of
schizophrenia and other psychotic disorders. In particular, the
DSM5 Workgroup suggests that each of the diagnostic symptoms
for these disorders (‘Criterion A’) may be rated using a new
symptom severity scale that we called the Symptom Severity
Scale DSM5 (SS-DSM5) in the current study (http://www.dsm5.
org/Pages/Default.aspx)1.
The psychometric properties of the SS-DSM5 in terms of
reliability, and validity have not been presented. The purpose of
our study therefore was to establish the psychometric properties
(factor structure, reliability, internal consistency, convergent and
diagnostic ability) of the SS-DSM5 scale in an unselected sample
of patients with psychotic (SZ/OP) and mood disorders that were
1
Because the draft diagnostic criteria posted on www.dsm5.org are undergoing revisions, the specific criteria text has been removed from the website to
avoid confusion or use of outdated categories and definitions.
2
M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8
receiving standard care in inpatient and out-patient settings of a
large university hospital.
2. Methods
In this study, we used a cross-sectional design enrolling an unselected
convenience sample of inpatients and outpatients with SZ/OP and mood disorders
who received standard care from February 2010 to March 2011 in the routine
clinical practice settings at Shaar Menashe Mental Health Center affiliated to the
Rappaport Faculty of Medicine, Technion, Israel. The study included men and
women ages 18–80 years. The design adhered to the Declaration of Helsinki and
ICH/Good Clinical Practice guidelines. The Internal Review Board of Sha’ar
Menashe Mental Health Center approved the study. All participants provided
written informed consent for participation in the study, after receiving an
explanation of study procedures.
2.1. Assessment
Diagnosis was based on the Diagnostic and Statistical Manual of Mental
Disorders, 4th ed. (DSM-IV; codes: 295, 296, 297, and 298), on a face-to-face
interview, medical records, and consensus between two senior psychiatrists. Participants were assessed with the SS-DSM5 scale together with well-recognized and
established psychometric instruments: the Clinical Global Impression-Severity scale
(CGI-S; Guy, 1976), the Positive and Negative Syndrome Scale (PANSS; Kay et al.,
1987), and the Bech-Rafaelsen Mania Scale (BRMAS; Bech et al., 1978).
The SS-DSM5 scale includes nine items or domains (hallucinations, delusions,
disorganization, abnormal psychomotor behavior, restricted emotional expression,
avolition, impaired cognition, depression, and mania) that are rated for their
current severity (most severe in the past month) on a five-point scale ranging
from: 0¼ not present; 1¼ equivocal (severity or duration not sufficient to be
considered psychosis); 2 ¼ present, but mild (little pressure to act upon voices, not
very bothered by voices); 3 ¼present and moderate (some pressure to respond to
voices, or is somewhat bothered by voices); to 4¼ present and severe (severe
pressure to respond to voices, or is very bothered by voices). A score of 2 or higher
is considered sufficient severity to fulfill a diagnostic indicator for disorder.
Psychiatrists were requested to evaluate and provide their overall opinion about
the suitability and feasibility of the SS-DSM5 scale.
The following PANSS items were used for comparative analysis as diagnostic
indicators for schizophrenia and other psychoses (DSM-V Criterion A): delusions
(P1), hallucinations (P3), disorganized speech (P2), grossly abnormal psychomotor
behavior, such as catatonia (P4), negative symptoms (N1, N4, G13, G16). A PANSS
raw score of 3 and more for each PANSS item was used as a cut-off for a clinically
relevant symptom (Ritsner, 2011).
The BRMAS includes 11 items (elevated mood, pressure of speech, increased
social contact, increased motor activity, sleep disturbances, social activities and
distractability, hostility and irritability, increased sexual activity, increased selfesteem, flight of thoughts, and noise level of speech and other vocal activity). Each
item is scored on a scale of 0 (not present or no difficulties) to 4 (Bech et al., 1978).
Studies of the internal validity of the BRMAS have demonstrated that the simple
sum of the 11 items of the scale is a sufficient statistic for the assessment of the
severity of manic states. The inter-observer reliability was found to be high in a
number of studies conducted in various countries. The BRMAS has shown acceptable external validity, in terms of both sensitivity and responsiveness (Bech, 2002).
Prior to initiation of the study all raters were trained to produce acceptable
levels of reliability on rating scales. The Intraclass Correlation Coefficient (ICC) for
CGI-S was 0.90, for PANSS (0.89), and BRMAS total scores were 0.89 and 0.88,
respectively. All rating scales administered by the same raters (RMS, MM and MA).
2.2. Subjects
A total of 329 patients were screened and 314 individuals (226 inpatients and
88 outpatients) were enrolled in the study; nine of 15 subjects were excluded due
to serious medical illness; and six patients refused to participate. There were 192
male subjects (61.13%), with a mean age of 41.1 7 12.8 (range¼ 19–76); education
was 10.27 2.1 years, age of onset was 22.97 6.9 years, the mean duration of
illness was 15.9 711.7 years. All individuals met DSM-IV criteria; in particular,
148 patients met criteria for paranoid type of schizophrenia (295.3), 56 patients
met to criteria of schizoaffective disorder (295.7), 68 patients of other psychotic
disorders (disorganized type [295.1; n¼ 21], catatonic type [295.2; n ¼1], residual
type [295.6; n¼ 15], undifferentiated type [295.9; n¼13] types of schizophrenia;
4 delusional disorder and 10 brief psychotic disorder [codes 297/298]; schizophreniform disorder [295.4; n¼ 4]), and 42 patients of mood disorder (296; Major
Depressive Disorder [MDD: single episode, 296.2, n¼ 7; recurrent episode, 296.3,
n¼ 16]; [Bipolar I Disorder, BPD: single manic episode, 296.0, n¼ 13; recent
episode manic, 296.4, n¼ 5; most recent episode mixed, 296.6, n ¼1]). An
exacerbation of symptoms was observed among 99 patients, while stabilization
of the current mental condition was found among 215 participants. Illness course
was unspecified (14/314¼ 4.5%, first episode), continuous (117/314 ¼37.3%), or
episodic (183 ¼58.3%). Patients in the present sample (n¼ 314) had moderate
levels of illness and symptom severity (mean 7 S.D.), CGI-S was 4.8 71.0 score
(range¼2–7); PANSS total scores were 89.27 24.4 (range: 38–179), PANSS
negative symptom scores were 23.47 8.7 (range: 7–47), positive symptoms
22.07 8.7 (range: 7–43), and general psychopathology 43.8 710.6 (range:
22–90), BRMAS scores were 13.07 9.3 (range: 0–44).
Patients were treated with various antipsychotic medications as clinically
indicated. Ninety-seven patients were treated with first generation antipsychotic
agents (FGAs; chlorpromazine, haloperidol, haloperidol decanoate, perphenazine,
zuclopenthixol, zuclopenthixol decanoate, fluphenazine decanoate), 103 – with
second generation antipsychotics (SGAs; clozapine, risperidone, olanzapine, quetiapine, ziprasidone, amisulpride), and 90 – with a combination of FGAs and SGAs
(COMB). Chlorpromazine equivalent (CPZ) doses were calculated based on published
data (Foster 1989; Woods 2003). The mean CPZ (7S.D.) in the FGA group was
670753 mg/day; in the SGA group, 390794 mg/day, and in the COMB therapy
group 9707107 mg/day. In addition to antipsychotic medications, the patients took
mood stabilizers (valproate, carbamazepine, lamotrigine; n¼ 55), benzodiazepines
(n¼ 38), anti-Parkinson agents (n¼ 74), and antidepressants (n¼38).
2.3. Statistical analysis
Factor structure, reliability, internal consistency, convergent validity, diagnostic ability and accuracy of the DSM5-SS were evaluated.
The principal axis method of factor analysis with varimax rotated factor
matrix was applied to identify the factors underlying the SS-DSM5 dimensions.
This method can be used when the assumption of normality has been violated
(Fabrigar and Petty, 1999). The eigenvalue criterion 41.5 was used to determine
the number of factors to retain. Variables with an absolute loading greater than
the amount set in the minimum loading option (40.4) were selected. Reliability
testing included the identification of redundant items (item–item correlation
40.8), and the testing of internal consistency with Cronbach’s alpha (a) coefficient
for the entire SS-DSM5 scale and for each subscale.
The kappa (k) reliability test was used to study the agreement between SS-DSM5
scale and PANSS diagnostic decisions. Rules-of-thumb for kappa were: values less
than 0.20 indicate low agreement, values between 0.21 and 0.40 indicate fair
agreement, between 0.41 and 0.60 means moderate agreement, between 0.61 and
0.80 indicate substantial agreement, and 40.81 – almost perfect agreement between
the two scales (Viera and Garrett, 2005). In addition, the concordance in decisions
between the two diagnostic scales (SS-DSM5 and PANSS), and the P value were
calculated with the McNemar test of symmetry with continuity correction (it tests for
symmetry around the diagonal of the table). That is, reject the hypothesis of
symmetry of the diagonal if the reported probability level is less than 0.05.
To investigate convergent validity of the SS-DSM5 scale, we calculated Pearson’s
correlation coefficients of the SS-DSM5 scale scores with scores on the PANSS, BRMAS,
and CGI-S. The intraclass correlation coefficient (ICC) was used to measure inter-rater
reliability for the three raters; it was rated as fair (0.30 to 0.49), moderate (0.50–0.69),
or high (0.70–1.00) for the purposes of comparison (Landis and Koch, 1977).
In order to test the diagnostic validity and accuracy of the SS-DSM5 compared
to the PANSS scale, we used DSM5 ‘Criterion A’ for schizophrenia. The ‘Criterion A’
requires: (i) two or more of the characteristic symptoms: delusions, hallucinations,
disorganized speech, abnormal psychomotor behavior, negative symptoms, i.e.,
restricted affect or avolition/asociality; and (ii) at least one of the following:
delusions, hallucinations, disorganized speech. Diagnostic test evaluation was also
performed (http://www.vassarstats.net/clin1.html). Quantities typically used to
evaluate the diagnostic accuracy of binary variables are sensitivity, specificity,
positive predictive values (PPV), and negative predictive values (NPV), positive
likelihood ratio (þLR) and negative likelihood ratio ( LR), whereas an assessment
of the overall prognostic accuracy of ordinal or metric variables is typically assessed
by a geometric approach [the area under the corresponding receiver operating
characteristic (ROC) curve] (Zhou et al., 2002; Sheskin, 2004). In the absence of
general criteria of either minimum acceptable values of diagnostic accuracy
measures or the choice of quantity in symptom selection, emphasis was put on
both sensitivity and PPV, and the minimum acceptable value of both quantities was
defined according to studies of diagnostic/prognostic accuracy of symptoms in
¨
schizophrenia (Klosterkotter
et al., 2001). Thus, a slightly lower value of sensitivity
than required for symptoms (Z0.25) and a value of PPV (Z0.70) were chosen as
selection criteria. Furthermore, statistic procedure of the receiver operator characteristic (ROC) analysis generates both binormal and empirical (non-parametric)
ROC curves. The ROC curve shows the characteristics of a diagnostic test by
graphing the false-positive rate (specificity) on the horizontal axis and the truepositive rate (sensitivity) on the vertical axis for various cut-off values. The area
under an ROC curve (AUC) is considered as an effective measure of inherent validity
of a diagnostic test. Other things being equal, the larger the AUC, the better the test
predicts the existence of the disorder. The possible values of AUC range from 0.5
(no diagnostic ability) to 1.0 (perfect diagnostic ability).
Finally, the feasibility and acceptability of SS-DSM5 scale was examined by
eight clinical psychiatrists and compared to PANSS and BRMAS scores.
M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8
Continuous variables were compared using the two-tailed t-test, or the
Wilcoxon signed-rank test (z) for assessing the difference in medians, as well as
using the general model of an analysis of variance (ANOVA). Post hoc analysis was
carried out in cases of significant outcomes, using the Tukey–Kramer method. For
all analyses, the level of statistical significance was defined as an alpha less than
0.05. All analyses were performed using the Number Cruncher Statistical System
(NCSS Statistical Software, Kaysville, UT) (Hintze, 2006).
3
item pairs consistently suggested redundancy across diagnostic
groups. Specifically, positive correlations between six SS-DSM5
symptoms (hallucinations, delusions, disorganization, psychomotor behavior, emotional expression and avolition) ranged from
0.25 (Po0.05) to 0.59 (Po0.001), excepting correlation between
psychomotor behavior, and emotional expression (P40.05). Similarly, disorganization, emotional expression and avolition positively correlated with impaired cognition scores (r ranged from
0.27–0.50), while disorganization and psychomotor behavior
scores correlated with mania scores (r ¼0.38 and 0.46, respectively, p o0.001). Negative correlations were observed for
depression symptom scores with delusions (r ¼ 0.30, Po0.01),
disorganization (r ¼ 0.32, P o0.01), and mania scores (r ¼ 0.27,
po0.05). Finally, correlation coefficients did not reach significant
levels between: (a) impaired cognition scores and three symptoms; (b) mania scores and five symptoms, and (c) depression
scores and six symptoms (Table 2).
3. Results
3.1. Factor structure
Results from an exploratory factor analysis of SS-DSM5 dimensions are summarized in Table 1. We found a model with two
latent factors (scales), which were labeled as ‘Psychotic’ and
‘Deficit’ scales. The first factor included negative loadings of
delusions, disorganization, abnormal psychomotor behavior and
mania scores. The second factor was constructed using restricted
emotional expression, avolition and impaired cognition scores
with negative loadings. Correspondingly, they accounted for
50.8%, and 48.4% of the total variance of the nine SS-DSM5 items.
Two SS-DSM5 items (‘hallucinations’ and ‘depression’) did not
reach the minimum loading option (40.4).
3.3. The internal consistency
Cronbach’s alpha coefficients for the SS-DSM5 scale total score
ranged from 0.38 to 0.79. In particular, Cronbach’s a was 0.67 for the
whole sample. Among individuals with paranoid type of schizophrenia, and with other psychotic disorders, alpha coefficients were
0.70, and 0.76, respectively. Among participants with schizoaffective
and mood disorders, alpha coefficients for the total score were
similarly low (0.58 and 0.38, respectively). Cronbach’s a coefficient
was 0.75 and 0.71 for ‘psychotic’, and ‘deficit’ scales of the SS-DSM5,
3.2. Reliability
Item–item Pearson’s correlation coefficients of 40.8 were
used to identify redundant items. No highly correlated SS-DSM5
Table 1
Factor loadings and communalities after varimax rotation of variable values SS-DSM5 items among 314 patients with schizophrenia spectrum and mood disorders.
SS-DSM5 items
Intraclass correlation coefficient (ICC) Factor 1 (eigenvalue ¼ 2.19) ‘psychotic
syndrome’
Hallucinations
Delusions
Disorganization
Abnormal Psychomotor Behavior
Restricted Emotional Expression
Avolition
Impaired Cognition
Depression
Mania
Factors’ contribution (%)
0.92
0.91
0.87
0.81
0.81
0.79
0.80
0.95
0.97
0.87
Factor 2 (eigenvalue ¼ 2.09) ‘deficit
syndrome’
Factor loading
Communalities
Factor loading
Communalities
0.3282
0.5560
0.7518
0.7168
0.0864
0.3067
0.0109
0.3425
0.6905
50.8
0.1077
0.3092
0.5652
0.5138
0.0074
0.0941
0.0001
0.1173
0.4767
–
0.3734
0.3468
0.4788
0.1608
0.8744
0.6422
0.5268
0.0614
0.3383
48.4
0.1394
0.1202
0.2293
0.0258
0.7646
0.4124
0.2776
0.0037
0.1145
–
Table 2
Pearson correlations coefficientsc between SS-DSM5, CGI-S, BRMS, and PANSS scores (n¼ 314).
CGI-S
Hallucinations
Delusions
Disorganization
Psychomotor Behaviora
Emotional Expressionb
Avolition
Impaired Cognition
Depression
Mania
PANSS total score
Negative scale
Positive scale
General psychopathology
Bech-Rafaelsen Mania Scale
a
b
c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
0.39
0.25
0.42
0.38
0.29
0.17
0.28
0.07
0.15
0.48
0.39
0.43
0.43
0.31
0.44
0.40
0.25
0.28
0.33
0.09
0.08
0.01
0.58
0.42
0.61
0.49
0.13
0.49
0.38
0.31
0.38
0.02
0.30
0.23
0.66
0.40
0.80
0.53
0.40
0.59
0.43
0.44
0.33
0.32
0.38
0.71
0.55
0.76
0.55
0.47
0.08
0.31
0.08
0.03
0.46
0.54
0.27
0.60
0.52
0.61
0.56
0.50
0.16
0.23
0.59
0.79
0.35
0.43
0.13
0.27
0.14
0.05
0.67
0.69
0.46
0.60
0.02
0.10
0.16
0.38
0.51
0.10
0.36
0.13
0.27
0.13
0.10
0.39
0.10
0.23
0.16
0.13
0.43
0.13
0.87
0.83
0.84
0.91
0.32
0.51
0.66
0.00
0.68
0.57
0.27
Abnormal Psychomotor Behavior.
Restricted Emotional Expression.
Significant levels of correlation coefficients: r¼ 0.11–0.29 (Po 0.05); r ¼ 0.30–0.37 (P o 0.01); r 40.37 (P o 0.001).
4
M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8
participants with paranoid schizophrenia, schizoaffective and
mood disorders. These symptoms were used to examine diagnostic ability of the SS-DSM5 and PANSS symptoms.
respectively, with a small correlation between them (r¼0.29,
Po0.05).
3.4. Convergent validity
3.6. Dimensional diagnosis: validity and accuracy
Table 2 displays correlations of SS-DSM5 dimensions with
other rating scales, which demonstrated low (r ¼0.31) to high
positive correlations (r¼0.79). The SS-DSM5 scale total score
highly correlated with PANSS dimensions (ranged between
r ¼0.73 and 0.90, Po0.001), and moderately with BRMAS scores
(r¼ 0.43, Po0.001). The SS-DSM5 psychotic scale highly correlated with the PANSS positive scale (r¼ 0.86, Po0.001), general
psychopathology (r ¼0.58, Po0.001), and BRMAS scores (r ¼0.87,
Po0.001), and moderately with the PANSS negative scale
(r¼ 0.37, Po0.001). The SS-DSM5 deficit scale highly positively
correlated with the PANSS negative scale (r¼ 0.85, Po0.001) and
general psychopathology (r ¼0.60, P o0.001), and moderately
with the positive scale (r ¼0.40, Po0.001). Correlation coefficient
between the SS-DSM5 deficit scale and BRMAS scores did not
reach significant levels (r ¼ 0.10, P¼0.070).
The mean SS-DSM5 total score for our sample was 15.1 76.2
(range: 1–31). As anticipated, the patients with a more severe
disorder had the most severe symptoms as measured with
SS-DSM5 scale: a total score of 8.573.5 approximately corresponded to being considered ‘‘mildly ill’’ according to the CGI-S
score, 12.674.1 as ‘‘moderately ill’’, 16.075.5 as ‘‘markedly ill’’,
18.476.4 as ‘‘severely ill’’, and 22.9 77.8 scores as the ‘‘most
extremely ill’’.
The concurrent validity of the SS-DSM5 symptoms was constructed by comparison with the PANSS scores and the DSM-IV
diagnosis made by the clinicians. As can be seen in Table 4, using the
SS-DSM5 symptoms, 77.9% of 272 patients with psychotic disorders
were correctly diagnosed, in particular, 79.8% of the patients with
paranoid schizophrenia, 71.4% with schizoaffective disorder, 79.4%
with other psychotic disorders (using PANSS symptoms: 82.0%,
81.7%, 85.7%, 79.4%, respectively). Fig. 2 depicted the falsenegative diagnostic decisions: the SS-DSM5 scale revealed falsepositives for 26.2% of the patients with mood disorders (16.7%
for PANSS), and 22.1% and 18.0% for psychotic disorders, respectively. The greatest discrepancy was observed among patients with
schizoaffective disorder (28.6% and 14.3%, respectively). The concordance of the decisions between SS-DSM5 and PANSS scales,
measured with the reliability coefficient Kappa (k 7SE), was
0.7570.06 for all psychotic disorders (t¼16.7, Po0.001),
0.7870.08 for paranoid type of schizophrenia (t¼7.4, P¼0.007),
0.5770.12 for schizoaffective disorder (t¼5.4, P¼ 0.020), and
0.8470.12 for other psychotic disorders (t¼ 4.9, P¼0.038).
The sensitivity, specificity, negative and positive predictive
values and negative and positive likelihood ratios are presented in
the Table 5. As can be seen, classifying participants with psychotic
and mood disorders using SS-DSM5 provided a sensitivity of 95%,
specificity of 34%, PPV 77.9%, NPV 73.8%, clinically important þLR
3.53, and small difference in –LR 0.35. The receiver operator
characteristic (ROC) curves used to find the best cut-off points for
classification. Fig. 3 depicts two receiver ROC curves for SS-DSM5
and PANSS scales. The area under the ROC curve was 0.91 70.016
(SE) with z-value (AUC 40.5)¼24.7 (po0.001) for PANSS, and
3.5. Raw scores and frequency of symptoms
An inspection of Table 3 reveals consistent differences in raw
rating scale scores, especially, between psychotic and mood
disorders, excepting BRMAS and CGI-S scores. Fig. 1 posted
frequency of SS-DSM5 symptoms (a score of 2 or higher) among
Table 3
Clinical characteristics of patient’s sample (n¼ 314).
Rating scales
CGI-S score c
PANSS total score d
Negative subscale
Positive subscale
General Psychopathology
Mania Scale e
SS-DSM5 total scoref
Hallucinations
Delusions
Disorganization
Behaviorg
Expressionh
Avolition
Cognition
Depression
Mania
a
Paranoid
schizophrenia
(n¼ 148)
Schizoaffective
disorder (n ¼56)
Other psychotic
Mood
disorders (n¼ 68) disorders
(n¼ 42)
ANOVAa
(d.f. ¼3,314)
Mean
S.D.
Mean
S.D.
Mean
S.D.
Mean
S.D.
F
Po
Mean
S.D.
t/z
P
4.7
93.7
24.9
23.8
45.0
11.4
15.9
1.5
3.0
2.0
2.0
2.5
2.6
1.0
0.5
0.5
0.9
21.9
7.7
7.5
10.0
7.4
5.5
1.5
1.0
1.1
1.2
1.0
1.0
1.0
0.8
0.9
4.7
92.0
22.5
24.0
45.4
19.2
16.1
1.0
2.9
2.1
2.5
1.6
2.3
0.7
0.9
1.8
0.8
18.6
7.3
6.8
9.3
11.8
5.3
1.3
0.9
1.3
1.1
1.0
1.2
1.0
1.2
1.5
5.0
94.1
26.2
23.1
44.6
11.9
16.5
1.4
2.5
2.5
2.2
2.4
2.5
1.3
0.5
0.7
1.1
28.1
9.8
9.2
12.9
9.1
6.8
1.4
1.3
1.4
1.4
1.4
1.3
1.1
0.9
1.0
4.8
60.4
13.9
10.5
35.8
11.7
8.6
0.1
0.7
0.6
2.0
0.4
0.9
0.4
2.5
0.7
0.8
10.3
4.1
5.2
6.0
8.6
4.0
0.3
1.1
0.9
1.1
0.7
1.0
0.6
1.6
1.3
2.0
26.9
27.5
35.4
9.4
11.3
20.8
21.0
46.4
20.9
2.2
40.5
21.8
7.3
37.1
16.3
0.12
0.001
0.001
0.001
0.001
0.001
0.001
0.001
0.001
0.001
0.084
0.001
0.001
0.001
0.001
0.001
4.8
93.6
24.8
23.7
45.0
13.2
16.1
1.4
2.9
2.2
2.2
2.3
2.5
1.0
0.6
0.8
1.0
22.8
8.3
7.8
10.6
9.4
5.8
1.5
1.1
1.2
1.5
1.2
1.1
1.1
0.9
1.2
0.3
9.2
8.3
10.6
5.5
1.0
8.0
5.6
11.7
7.5
1.0
9.8
8.2
3.1
10.8
0.4
0.77
o 0.001
o 0.001
o 0.001
o 0.001
0.34
o 0.001
o 0.001
o 0.001
o 0.001
0.32
o 0.001
o 0.001
0.002
o 0.001
0.65
ANOVA (d.f. ¼3.272) comparisons between four subgroups of patients.
Significance (t-test) between psychotic and mood disorders.
CGI-S: Clinical Global Impression-Severity scale.
d
PANSS: Positive and Negative Syndrome Scale.
e
BRMAS: Bech-Rafaelsen Mania Scale.
f
SS-DSM5: Symptom severity scale of DSM5.
g
Abnormal Psychomotor Behavior.
h
Restricted Emotional Expression.
b
c
Significanceb (between
psychotic and mood
disorders)
Psychotic
disorders
(n¼ 272)
M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8
5
70
60
Percent
50
40
30
20
ia
n
an
M
D
C
ep
og
re
ni
ss
tio
io
n
on
iti
ol
Av
ga
is
D
H
Paranoid schizophrenia (n = 148)
Schizoaffective disorder (n = 56)
Mood disorders (n = 42)
E
Ex mo
pr tio
es n
si al
on
Ps
y
Be ch
h a om
v i o to
or r
n
tio
za
ni
us
al
or
lu
D
ci
el
na
tio
io
ns
0
ns
10
SS-DSM5 symptoms
Fig. 1. Frequency of SS-DSM5 symptoms across diagnostic groups of patients.
Table 4
Diagnostic psychotic and mood disorders using DSM-V ‘Criterion A’ and DSM5-SS and PANSS scales.
Number of characteristic symptoms Paranoid schizophrenia (
n¼ 148)
SS-DSM5 scale
0a
1a
2
3
4
5
PANSSb
0a
1a
2
3
4
5
Schizoaffective disorder
(n ¼56)
Other psychotic
disorders (n¼ 68)
Total psychotic disorders Mood disorders
(n¼ 272)
(n¼42)
n
%
n
%
n
%
n
%
n
%
7
23
33
31
21
33
4.7
15.5
22.3
20.9
14.2
22.3
1
15
9
16
9
6
1.8
26.8
16.1
28.6
16.1
10.7
5
9
9
7
13
25
7.4
13.2
13.2
10.3
19.1
36.8
13
47
51
54
43
64
4.8
17.3
18.8
19.8
15.8
23.5
19
12
10
1
0
0
45.2
28.6
23.8
2.4
0
0
6
21
40
40
15
26
4.1
14.2
27.0
27.0
10.1
17.6
1
7
15
15
12
6
1.8
12.5
26.8
26.8
21.4
10.7
5
9
12
8
21
13
7.4
13.2
17.6
11.8
30.9
19.1
12
37
67
63
48
45
4.4
13.6
24.6
23.2
17.7
16.5
18
17
5
2
0
0
42.9
40.5
11.9
4.8
0.0
0.0
a
False negative decision refers to a test result that tells you a disease or condition is not present, when in reality, there is disease.
PANSS characteristic symptoms: delusions (P1), conceptual disorganization (P2), hallucinatory behavior (P3), excitement (P4), negative symptoms, i.e., blunted affect
(N1), emotional withdrawal (N2), disturbance of volition (G13), and active social avoidance (G16).
b
35
SS-DSM5
PANSS
28.6
30
26.2
25
20.6 20.6
Percent
20.2
20
16.7
22.1
18.3
18
14.3
15
10
5
0
Mood disorders
(n = 42)
Paranoid SZ
(n = 148)
Schizoaffective
(n = 56)
Other psychotic
(n = 68)
Total psychotic
(n = 272)
Diagnostic groups (DSM-IV)
Fig. 2. False-negative decisions are based on the SS-DSM5 and PANSS scales (Criterion A).
6
M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8
Table 5
Diagnostic test evaluation using SS-DSM5 of the probability that it is psychotic (SZ/PO) disorders versus mood disorders.
Parameters
Estimated value
95% Confidence interval
Lower limit
Upper limit
0.6560
0.9110
0.2466
0.7590
0.9738
0.4483
For any particular test result, the probability that it will be psychotic or mood disorder
Positive (psychotic disorders)
0.8662
Negative (mood disorders)
0.1337
0.822376
0.099115
0.9008
0.1776
For any particular positive test result, the probability that it is psychotic disorder
True positive (Positive Predictive Value) c
0.7794
False Positive
0.2205
0.724479
0.173725
0.8262
0.2755
For any particular negative test result, the probability that it is mood disorder
0.7380
True negative (Negative Predictive Value)d
False Negative
0.2619
0.576769
0.143863
0.8561
0.4232
Likelihood Ratios (weighted by prevalence)e
Positive Likelihood Ratio (þ LR)
Negative Likelihood Ratio (–LR)
2.80
0.21
4.45
0.59
Prevalence
Sensitivitya
Specificityb
0.7101
0.9506
0.3406
3.53
0.35
a
The sensitivity, i.e. correctly identified ‘psychotic disorders’ (true positive rate)
The specificity, i.e. correctly identified ‘mood disorders’ (true negative rate).
Positive predictive value (PPV) is probability that the ‘psychotic disorders’ is present when the test is positive.
d
Negative predictive value (NPV) is probability that the ‘mood disorders’ is not present when the test is negative.
e
Positive likelihood ratio (þ LR)¼true positive rate/false positive rate¼sensitivity/ (1 specificity). Negative likelihood ratio (–LR) ¼false negative rate/true negative
rate¼ (1 sensitivity)/specificity.
b
c
0.85þ0.024, z ¼14.3 (p o0.001) SS-DSM5, which means that from
91% to 85% of the patients were correctly classified.
(iii)
3.7. Feasibility and acceptability
The intraclass correlation coefficient (ICC) for individual items of
the SS-DSM5 and PANSS ranged from 0.79 to 0.97, and those for the
total score was 0.87; that is quite comparable with CGI-S (0.90),
PANSS (0.89), and BRMAS (Table 1). The feasibility and acceptability
of the SS-DSM5 scale was examined compared to the PANSS and the
BRMAS. After completing these scales, eight clinical psychiatrists
reported that the SS-DSM5 scale eased their assessment of the
intensity of symptoms, it was easier to understand, less burdensome
to administer, and quite acceptable to use in routine clinical
practice. The amount of time needed to administer the SS-DSM5
during psychiatric examination was 1077.5 min, compared to the
BRMAS (22713.8 min.), and the PANSS (3879.4 min). They
considered the SS-DSM5 scale useful. However, the dimensional
diagnostic procedure was somewhat burdensome for psychiatrists
since this procedure included three steps: (1) assessment of the
mental health state with SS-DSM5, (2) transformation of the raw
scores to symptoms, and (3) checking the diagnostic ‘Criterion A’ of
DSM5 with the obtained dimensional symptoms.
4. Discussion
This study aimed to establish the psychometric properties and
diagnostic validity of the SS-DSM5 scale for dimensional diagnosis of SZ/OP disorders.
The key psychometric properties of the SS-DSM5 scale are:
(i) The factorial analysis demonstrates a two-factorial structure
of the SS-DSM5 dimensions, which were labeled ‘psychotic’,
and ‘deficit’ scales, respectively.
(ii) The reliability and internal consistency of the SS-DSM5 total
score and its scales were shown to be strong. Cronbach’s
alpha was 40.70 for the entire sample; 0.75 and 0.71 for the
(iv)
(v)
(vi)
(vii)
‘psychotic’, and ‘deficit’ scales, respectively, with a small
correlation between them (r ¼0.29, Po0.05).
The convergent validity was confirmed statistically: correlations of SS-DSM5 and its scales with CGI-S and the PANSS
counterpart were all highly significant (r range from 0.73–
0.90, Po0.001), and moderate with BRMAS scores (r¼ 0.43,
Po0.001). The intraclass correlation coefficient (ICC) for
individual items of the SS-DSM5 PANSS ranged from 0.79–
0.97 (0.87 for the total score) comparable to other rating
scales.
The patients with more severe disorders had the most severe
symptoms as measured with the SS-DSM5 scale: CGI-S
‘‘mildly ill’’ corresponded to SS-DSM5 total scores of
8.5 73.5, and the CGI-S ‘‘extremely ill’’ to SS-DSM5 scores
of 22.9 77.8.
The agreement of the diagnostic decisions between the
SS-DSM5 and PANSS was substantial for all psychotic disorders (77.9% vs. 72% using PANSS; Kappa k¼0.75,
Po0.001).
Classifying participants with ‘psychotic’ or ‘mood’ disorders
using SS-DSM5 provided a sensitivity of 95%, specificity of
34%, PPV 77.9%, and NPV 73.8%.
Clinical psychiatrists found that the SS-DSM5 scale eased
their assessment of the intensity of symptoms, was easier to
understand, less burdensome to administer, and quite acceptable for use in routine clinical practice.
Since, according to the authors’ best knowledge, this is the first
article that reports the psychometric properties and feasibility of
the SS-DSM5 scale, it is not possible to compare the current
results with those of other studies. Results of this study suggest
that the SS-DSM5 compares well to the PANSS and may be validly
employed in the diagnostic assessment of schizophrenia and
other psychotic disorders. A reliable and valid measure of symptom severity is needed to match the growing interest in dimensional diagnostic procedures in mental health care. Indeed, a
standardized rating system for individual symptoms would contribute to knowledge of the severity of the polymorphic symptomatology and other presentations of mental health disorders.
M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8
1. ROC Curve of psychotic versus mood disorders
1.00
Sensitivity
0.75
0.50
0.25
0.00
0.00
Criteria
PANSS
SS-DSM5
0.25
0.50
Specificity
0.75
1.00
Sensitivity
2. ROC Curve of psychotic versus mood disorders
7
N1, N4, G13, and G16) as dimensional options for the diagnosis of
schizophrenia and related disorders. Indeed, these PANSS items
revealed only 18% false-positives for psychotic disorders and
16.7% for mood disorders (21.1% and 26.2% for SS-DSM5,
respectively).
There are several limitations in this study. First, test–retest
reliability and sensitivity to change were not tested because of the
cross-sectional design, covering symptoms during the last four
weeks. Second, the diagnosis was made by the clinicians according
to DSM-IV criteria, but not using standardized diagnostic tools.
In conclusion, the SS-DSM5 scale appears to have solid
psychometric features that are likely to make it useful in mental
health services. Although clinical psychiatrists found the SS-DSM5
scale quite acceptable and very feasible, its utility for improving
diagnostic practice remains open. Future research with more
diverse clinical samples is underway and should further identify
the strengths and weaknesses of the measure.
1.00
Authors’ contributions
0.75
RMS originated the study design. RMS and AG were responsible for collection of data, statistical analysis, and an interpretation of data and drafted the manuscript. MM and MA participated
in acquisition of data and interpretation of data. All authors have
given final approval of the version to be published.
0.50
Acknowledgments
0.25
The authors especially thank R. Kurs, B.A. for editing this
manuscript.
Criteria
PANSS
SS-DSM5
0.00
0.00
0.25
0.50
Specificity
0.75
References
1.00
Fig. 3. Receiver operator characteristic (ROC) analysis. The first plot shows the
empirical ROC curve. The second plot shows an ROC curve based on the binormal
assumption.
However, there are two problems associated with the dimensional approach to diagnosis in an official nomenclature: (a) there
is no widely tested and accepted system of dimensional diagnosis,
and (b) clinicians find the added work of rating dimensions
burdensome (Frances 2009). Furthermore, different categorical
cut-off points may lead to different or even opposing clinical
conclusions (Kraemer et al., 2004). It is important to keep in mind
that although a dimensional approach has been offered for the
diagnoses of various disorders (e.g., Peralta, Cuesta, 2007; Helzer
et al., 2007; Allardyce et al., 2007), practical use and clinical
relevance in psychiatry remain unclear. The SS-DSM5 dimensions
do not appear to be diagnosis specific for categorical diagnoses in
the schizophrenia spectrum. There could be some concerns about
the proposed SS-DSM5 dimensions. For instance, they do not
possess a standardized metric measure for: (1) what the rather
crude scoring ranging from 0 to 4 represents; (2) issues that need
to be resolved regarding dimension weighting in different psychotic disorders; and (3) the lack of dimensions for the assessment of severity of emotional distress, and quality of life
impairment.
The PANSS is a commonly used instrument to assess symptom
severity in SZ/OP disorders; however, it has not yet been used in
the dimensional diagnostic procedure of psychotic disorders. The
present study raises the question regarding PANSS items (P1-P4,
Allardyce, J., Suppes, T., Van Os, J., 2007. Dimensions and the psychosis phenotype.
International Journal of Methods Psychiatric Research 16 (1), S34–S40.
Bech, P., 2002. The Bech-Rafaelsen Mania Scale in clinical trials of therapies for
bipolar disorder: a 20-year review of its use as an outcome measure. CNS
Drugs 16, 47–63.
Bech, P., Rafaelsen, O.J., Kramp, P., Bolwig, T.G., 1978. Mania Rating Scale: scale
construction and inter-observer agreement. Neuropharmacology 17, 430–431.
Brown, T.A., Barlow, D.H., 2005. Dimensional versus categorical classification of
mental disorders in the fifth edition of the diagnostic and statistical manual of
mental disorders and beyond: comment on the special section. Journal of
Abnormal Psychology 114, 551–556.
Diagnostic and Statistical Manual of Mental Disorders, 5th ed. Available from:
(DSM-5) web sites, http://www.dsm5.org. and /http://www.dsm5.org/Pages/
Default.aspxS.
Dikeos, D.G., Wickham, H., McDonald, C., Walshe, M., Sigmundsson, T., Bramon, E.,
Grech, A., Toulopoulou, T., Murray, R., Sham, P.C., 2006. Distribution of
symptom dimensions across Kraepelinian divisions. British Journal of Psychiatry 189, 346–353.
Dutta, R., Greene, T., Addington, J., McKenzie, K., Phillips, M., Murray, R.M., 2007.
Biological, life course, and cross-cultural studies all point toward the value of
dimensional and developmental ratings in the classification of psychosis.
Schizophrenia Bulletin 33, 868–876.
Fabrigar, L.R., Petty, R.E., 1999. The role of the affective and cognitive bases of
attitudes insusceptibility to affectively and cognitively based persuasion.
Personality and Social Psychology Bulletin 25, 91–109.
Frances, A., 2009. Whither DSM-V? The British Journal of Psychiatry 195, 391–392.
Foster, P., 1989. Neuroleptic equivalence. The Pharmaceutical Journal 243,
431–432.
Guy, W., 1976. ECDU Assessment Manual for Psychopharmacology. US Department of Health and Human Services, Rockville, MD.
Helzer, J.E., Bucholz, K.K., Gossop, M., 2007. A dimensional option for the diagnosis
of substance dependence in DSM-V. International Journal of Methods in
Psychiatric Research 16 (1), S24–S33.
Helzer, J.E., Kraemer, H.C., Krueger, R.F., Wittchen, H.U., Sirovatka, P.J., Regier, D.A.,
2008. Dimensional Approaches in Diagnostic Classification: Refining the
Research Agenda for DSM-V. American Psychiatric Association, Arlington, VA.
Hintze, J.L., 2006. NCSS-2007. Statistical System for Windows. User’s Guide.
Number Cruncher Statistical Systems, Kaysville Utah.
Kamphuis, J.H., Noordhof, A., 2009. On categorical diagnoses in DSM-V: cutting
dimensions at useful points? Psychological Assessment 21, 294–301.
8
M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8
Kay, S.R., Fiszbein, A., Opler, L.A., 1987. The Positive and Negative Syndrome Scale
(PANSS) for schizophrenia. Schizophrenia Bulletin 13, 261–276.
¨
Klosterkotter,
J., Hellmich, M., Steinmeyer, E.M., Schultze-Lutter, F., 2001. Diagnosing schizophrenia in the initial prodromal phase. Archives of General
Psychiatry 58, 158–164.
Kraemer, H.C., Noda, A., O’Hara, R., 2004. Categorical versus dimensional
approaches to diagnosis: methodological challenges. Journal of Psychiatric
Research 38, 17–25.
Landis, J.R., Koch, G.G., 1977. The measurement of observer agreement for
categorical data. Biometrics 33, 159–174.
Owen, M.J., Craddock, N., Jablensky, A., 2007. The genetic deconstruction of
psychosis. Schizophrenia Bulletin 33, 905–911.
Peralta, V., Cuesta, M.J., 2007. A dimensional and categorical architecture for the
classification of psychotic disorders. World Psychiatry 6, 36–37.
Ritsner, M.S., 2011. Toward a Multidimensional Continuum Model of functional
psychoses for research purposes. In: Ritsner, M.S. (Ed.), Handbook of Schizophrenia Spectrum Disorders, vol. I. Springer-Verlag, Dordrecht, pp. 85–113.
Ritsner, M.S., Gottesman, I.I., 2011. The schizophrenia construct after 100 years of
challenges. In: Ritsner, M.S. (Ed.), Handbook of Schizophrenia Spectrum
Disorders, vol. I. Springer-Verlag, Dordrecht, pp.1–44.
Salokangas, R.K.R., 2003. Symptom dimensions and outcome in schizophrenia.
World Psychiatry 2, 172–178.
Sheskin, D.J., 2004. Handbook of Parametric and Nonparametric Statistical Procedures, 3rd ed. Chapman & Hall, CRC, Boca Raton.
Viera, A.J., Garrett, J.M., 2005. Understanding Interobserver agreement: the Kappa
Statistic. Family Medicine 37, 360–363.
Woods, S.W., 2003. Chlorpromazine equivalent doses for the newer atypical
antipsychotics. Journal of Clinical Psychiatry 64, 663–667.
Zhou, X.-H., Obuchowski, N.A., McClish, D.K., 2002. Statistical Methods in
Diagnostic Medicine. Wiley & Sons Interscience, New York 437p.
van Os, J., 2009. Salience syndrome’ replaces ‘schizophrenia’ in DSM-V and ICD-11:
psychiatry’s evidence-based entry into the 21st century? Acta Psychiatrica
Scandinavica 120, 363–372.