Because prevalence is measured as a proportion of the whole population with a given problem, general population surveys are the only way to generate prevalence rates. The survey participants have to be randomly selected, to ensure that the result is an accurate representation of the population as a whole. The general rule is the bigger the sample, the more accurate the survey.
Given the low prevalence rate of “pathological” gambling in most jurisdictions, samples well into the thousands are often necessary to provide an acceptable margin of error. For example, even with a sample of 1,000, the error margin is about 3.2%. That means that, 19 times out of 20, the real value in the population will be within 3.2% of the percentage figure given in the survey. For example, the prevalence of pathological gambling has typically been about 1-2%, and this means that, with a sample of 1,000 the real prevalence of problem gambling is somewhere between 0% and 5%. This kind of variability makes it very difficult to track small changes in prevalence rates. Sample sizes have to quadruple to cut the margin of error in half. A sample size of 5,000 gives an error margin of 1.5%. To get down to an error margin of 1%, a sample of 10,000 would be necessary.
About once in every 20 surveys, even in a random sample, the sample drawn is not representative, just by chance, and so provides a distorted picture of the prevalence rates of the population as a whole. What this means is that prevalence rates should be interpreted with care, keeping in mind what makes sense. All general population surveys are published with an estimate of the error margin, and it is important to check survey error margins to avoid making decisions based on differences in prevalence rates that may be within the error margin of the survey, and so not necessarily “real.”
Problem and pathological gambling are often lumped together, because they add up to about 5-8% of the population. This means the error margin for inclusion in the new melded group is smaller. However, the ability to track movement between the groups, or differences in the relative sizes of the two groups, suffers. In Alberta, for instance, between 1994 and 1998, the overall size of the group (problem and pathological together) went from 5.4% to 4.8%, which looks like a decline (Wynne Resources, 1998). But if the two groups are looked at separately, it appears that the pathological group went from 1.4% in 1994 to 2.0% in 1998. The decline was apparently in the problem group, which went from 4.0% down to 2.8% in 1998. In this case, the differences for the groups separately were not statistically significant, which means that the changes were within the margin of error. But it is possible to use this as an example of the differences that may be obscured if the two groups are combined. If the two groups are combined, the headline reads “problem gambling down in Alberta, despite increases in availability.” Yet looked at separately, the results could be reported as “pathological gambling up 42% since 1994.”
Issues with measures used for prevalence
Most of the measures that are currently used to provide prevalence rates of problem, pathological or compulsive gambling are derived from psychiatric clinical assessments. The DSM-IV (Diagnostic and Statistical Manual of the American Psychiatric Association — fourth edition) criteria are based on advice from psychiatrists, and are grounded in the principles of clinical assessment. The SOGS (South Oaks Gambling Screen), a scale developed at the South Oaks Hospital in the United States that treats alcohol, drug and gambling problems, and its derivatives, such as the SOGS-RA(Revised Adolescent) and the NODS (NORC DSM Screen for Gambling Problems, National Opinion Research Centre at the University of Chicago) were all developed with reference to these same clinical principles, and to the DSM criteria in particular. There are some differences in emphasis, and in wording, among these scales — the SOGS-RA, for instance, is intended to measure problem gambling among adolescents — but the origins of the models used are the same. In this section, the strengths and weaknesses of these measures of problem gambling are discussed.
The SOGS has been by far the most widely used measure of gambling problems in the literature, both for the assessment of clinical populations and in the general population. There are several reasons for this. First, until recently it was the only gambling measure that had been adequately validated and reliability-tested. Second, as Shaffer, Hall and Vander Bilt (1997) point out, many of the prevalence studies conducted in the US have been designed or conducted by Rachel Volberg, who uses the SOGS. As the body of literature using the SOGS grew, it became more and more difficult to use another instrument, because cross-jurisdictional comparisons are often an important goal of prevalence studies, and a common measurement instrument is necessary in order to make these comparisons.
The SOGS is composed of 20 items. A “yes” response to between three and four items is considered indicative of problem, possibly pathological, gambling, while “yes” responses to five or more items is considered diagnostic of probable pathological gambling (Volberg & Steadman, 1988).
The SOGS has been widely used by gambling researchers as well as clinicians. Lesieur (1994) cites studies in five countries using this measure (Canada, US, Spain, New Zealand, and Australia), and Shaffer et al.’s (1997) meta-analysis of 120 studies shows 55.1% used the SOGS or a SOGS derivative such as the SOGS-RA for adolescents as a measurement instrument.
The criticisms of the SOGS have centred around the fact that it was developed in a clinical setting, and yet is used in general population studies (Lesieur & Blume, 1993). Culleton (1989) has suggested that using this type of screening test for a general population survey is an inappropriate way to establish a prevalence rate, and may not be accurate, given the very low rate of occurrence of gambling problems in the general population. Culleton and other researchers have suggested that the SOGS, when used with the general population, results in a high number of false positives (Culleton, 1989; Abbott & Volberg, 1992; and Dickerson, 1993). The questions, while meaningful to a clinical sample, may be misinterpreted by others. For example, asking questions about borrowing (the SOGS includes a large number of questions on borrowing from various sources) may capture the casual gambler who has borrowed a few quarters from a friend at a casino.
On the other hand, it has also been suggested that the SOGS results in an undercount of gambling problems in epidemiological surveys, because treatment and institutional populations are not sampled in this type of survey (Lesieur, 1994;Volberg & Boles, 1995), nor are pathological gamblers likely to be home to participate in a telephone survey, which is usually how general population surveys are conducted in North America. Several studies have suggested that those who are in treatment for drug or alcohol problems (Gambino, Fitzgerald, Shaffer & Renner, 1993; Rosenthal, 1992; Lesieur, Blume & Zoppa, 1986;) and those in prisons (Templer, Kaiser & Siscoe, 1993) are much more likely than the general population to report gambling problems, and again, these populations tend not to be included in general population surveys. Other research has shown that adolescents are more likely to report gambling problems than adults (Shaffer & Hall, 1996; Insight Canada, 1994; Wallisch, 1993; Ladouceur & Mireault, 1988; Lesieur & Klein, 1987). Typically, epidemiological surveys have separated adults and adolescents, and have not reported a prevalence rate that includes both groups.
The SOGS has been revised to suit a number of purposes and situations. In particular, there is a South Oaks–Revised Adolescent (SOGS-RA), which was developed in order to obtain a more accurate measure of adolescent gambling problems. The wording of the questions and the scoring of the borrowing items were adjusted to emphasize the frequency of gambling activity and behavioural signals rather than borrowing as the most important marker of problems. This screen was also satisfactorily tested for reliability and validity (Winters, Stinchfield & Fulkerson, 1993). While the scale has been shown to be psychometrically adequate for adolescent males, its adequacy for adolescent females has not been demonstrated, given the low rate of gambling and severity of problems in Winters et al.’s sample (1993).
More recently, the SOGS has been replaced as the standard in the measurement of gambling problems by measures adapted from the DSM-IV criteria. Diagnosis for pathological or problem gambling is based on a set of 11 criteria, the presence of five or more of the criteria signalling the presence of pathological gambling.
In epidemiological studies, these criteria are operationalized into varying numbers of questions, with 10 of the criteria included. The last criterion, that “the gambling behaviour is not better accounted for by a manic episode” (APA, 1994), is not included in telephone surveys, for obvious reasons, although this has been a major criticism of the use of the DSM-derived measures (Shaffer et al., 1997). It has been pointed out that there has not been even one documented case of an individual experiencing a manic episode that also met the other criteria for pathological gambling. In fact, the members of the Work Group on Disorders of Impulse Control who developed the DSM-IV criteria protested including the manic episode as a criterion at all.
The DSM criteria are less focused on the financial aspects of gambling problems than the SOGS, and more focused on the loss of control aspects. This behavioural focus is a strength of the measure, as it results in a more balanced view of gambling problems than is possible with the SOGS. The end result is a more conservative estimate of gambling problems. Summing yes responses means that the DSM-IV accords a maximum score of 2 to financial difficulties, while with the SOGS, financial difficulties and borrowing money can count for up to 14 points. The DSM-IV has been used in at least six studies in the us and Canada between 1993 and 1997 (see Shaffer et al., 1997— table 14) and several since then (nods, for example).While the criteria are the same, question wording to address the DSM-IV criteria has tended to vary depending on the researchers and target populations involved.
Methodological concerns in the measurement of problem gambling
Many of the problems with the measurement of gambling problems are in fact problems with prevalence studies in general. The most serious problem facing researchers using general population surveys is the decline in response rates, that is, the proportion of people contacted who actually agree to complete the telephone survey. In Canada, a response rate of 60% is now considered “good.” Even reputable survey firms acknowledge that response rates in large cities are often significantly lower than 60%.We know that the people who agree to participate and those who do not are different demographically. The extent of this bias can be evaluated by comparing the sample achieved against Statistics Canada numbers for a variety of demographic variables like age, sex, education level and income. In Canada, telephone surveys typically result in an under-representation of households where little English or French is spoken, households at the high and low ends of the income spectrum, and households with unlisted telephone numbers. Also, most survey samples do not include individuals in university dorms or institutions (e.g., military bases, prisons, treatment centres, etc.). Even the use of the telephone introduces some bias, as the 3% of Canadian households without telephones are known to be demographically different from the 97% with telephones — lower income, more mobile, more likely to be unemployed, and more likely to be young (18–24).
Over and above these difficulties are some issues that are specific to gambling studies in particular. Gambling may be a sensitive topic, and it may not be possible for the respondent to answer the survey in private. If a family member is present, that may affect how the respondent answers the questions. It has also been suggested by clinicians that those with serious gambling problems may not answer their telephones because they fear it may be a bill collector calling.
Why review methodological issues with general population surveys? Because, while these surveys are the best measurement of problem gambling available, they do have flaws, which means that problem gambling and people affected by problem gambling may be under-represented in general population surveys.
Back to Definitions and Prevalence of Gambling and Problem Gambling