Gang Prevention Strategy

Gang Prevention Strategy - PDF Version (222 KB)

ISBN: 978-1-100-20349-2

Building the Evidence - Evaluation Summaries 2012-ES-23

1. Introduction^{Footnote 1}

Information provided by the Integrated Gangs and Weapons Enforcement Unit of the Hamilton Police Service (HPS) showed a growing trend in youth gang involvement in the Hamilton area. The 2009 HPS report showed a 7% increase in crime among youth (ages 12-17) and increased accessibility to gangs due to internet recruitment efforts.

To address the issue of youth gangs, Living Rock Ministries, a non-profit Christian outreach based in downtown Hamilton, delivered the Gang Prevention Strategy (GPS) with approximately 2.3M in funding from the National Crime Prevention Centre (NCPC). The program was implemented from April 2007 to March 2011. It targeted youth aged 13 to 25 who were at risk of gang involvement, or were already gang-involved.

2. Program Description

The objective of the GPS was to reduce the risk of gang involvement among at-risk youth. Specifically, the program aimed to:

• Increase awareness of consequences of gang involvement
• Encourage youth to adopt a less positive attitude toward gangs
• Increase motivation to participate in pro-social behaviours
• Decrease risk factors that contribute to interest in gang activity
• Increase protective factors that contribute to youth's interest in pro-social activity

GPS was based on a wraparound approach. Key elements of the program included being assigned a coach, developing a case management plan, meeting with the coach for at least one hour per month for the first three months, and participating in at least one hour of programming activities each week for a defined three month period.

Eligibility for the program was determined by a Self Discovery Quiz (SDQ) administered to potential participants. This screening tool assessed both “positive” (e.g., I am in school) attributes, and “negative” ones (e.g., I have friends who are in gangs). The coaches considered positive responses to determine the youth's assets in building their plan of care, while negative responses helped determine their level of risk for becoming involved in gang activity.

A youth who had a negative score of -46 or lower was considered to be at higher risk and was eligible to participate in GPS. GPS staff were able to override the cut off score if they found youth were exaggerating their levels of risk in order to qualify for the program. The override protocol was also used when the risk score was not consistent with valid external information that suggested that the youth was involved in a gang.

Eligible youth, after providing the required consent, were assigned a unique case management number and completed a Youth Pre Survey. The Pre Survey, elaborating on questions in the SDQ, elicited information about youths' behaviour, education, peer associations, and relationships with the community and their family.

After six months in case management, the youth took the Youth Post Survey, which followed-up on the questions asked in the SDQ and the Youth Pre Survey. Researchers then compared pre- and post-treatment responses.

Treatment in the GPS program was considered to be complete once the youth had:

• Months 1-3: Met with their coach one hour during each of first three months, to establish the plan of care, build trust and jointly establish goals
• Months 4-6: Met once per week for GPS activities in the next 3 months, to engage the youth in activities aimed at achieving their goals and minimizing their risk factors
• Completed a post survey at the end of treatment (six months)

Program participants

The GPS program specifically targeted youth aged 13 to 25 who were at risk of gang involvement. 41% or 230 participants were selected from the pool of visitors to Living Rock.

Records show that 3,620 individual youth visited Living Rock from 2002-2010. 56% were under 19 years of age and 80% had not completed high school. 50% lived with family or friends, while 20% were living in a shelter, hostel, drop-in, or were homeless and 18% had lived by themselves in a rental room. Approximately 10% of youth who visited Living Rock reported some degree of involvement with the legal system or noted that they had a criminal background. 23% reported using marijuana.

Participants were recruited into the GPS through a combination of outreach activities, word-of-mouth and financial incentives. They were then asked to take the SDQ to determine eligibility. 41% of youth who took the SDQ were eligible based on their score alone, while another 22% were admitted through an override by a coach.

Overall, 230 youth were considered eligible for the program. Of these, 10% were not interested and 3% declined to provide consent, leaving 201 youth who were accepted into the program. Drop out rates were significant, with 43% of youth becoming inactive at some point after providing consent. Reasons for attrition included moving, becoming incarcerated, and getting a full time job.

Total program length was six months, and the median number of hours in case management among those who completed the program was 242 hours with 86 youth completing treatment.

3. Evaluation of the Program

A quasi-experimental design was initially chosen for the evaluation of the GPS. This design would have involved pre and post measures with both treatment and comparison groups. However, due to difficulties in recruiting sufficient numbers of youth for the comparison group, the methodology was changed to a single group repeated measures design.

The revised design involved comparing pre and post surveys administered six months after the treatment ended. Additional follow up surveys were administered every six months to participants who were still available.

The study incorporated both quantitative and qualitative data. Quantitative data was collected during the screening period (through a Self Discovery Quiz), at program entry (Youth Pre Survey), and at program completion six months later (Youth Post Survey). Additional quantitative data was collected from program activity and case management logs, surveys administered to seminar attendees, youth crime statistics provided by the HPS, and administrative and financial project documents.

Evaluators assessed survey responses for reliability and administered tests to analyze the differences between proportions in groups with different levels of risk and dosage. The purpose of these tests was not to compare the sample to a population, but rather to determine the effectiveness of dosage between different subsets of the sample group.

The evaluators also assessed the marginal frequencies of binary outcomes, and conducted factor analyses to group attributes in the surveys. Where appropriate, Analysis of Variance was also used to determine significant differences between changes at pre, and post intervals.

The evaluators attempted to clearly define dosage and clarify risk levels so that these variables could be correlated with each outcome of interest and utilized in a regression equation. 242 hours in case management was decided as the cut-off value for low versus high dosage. This was based on the median score for youth who had completed the GPS Program.

Qualitative data was used to support and elaborate upon quantitative findings. It was obtained through 15 key informant interviews (with the GPS manager, coaches and partners), 10 site visits, six focus groups (four with youth and two with GPS coaches), and observational research conducted onsite at a Citywide Freestyle Friday event held at a coffee house in Hamilton, at a daytrip to Medieval Times in Toronto, and two retreats at Circle Square Ranch in Brantford.

4. Evaluation Findings

Process Findings

Implementation of the GPS

The implementation of the program deviated in some ways from the program initially conceived by Living Rock. The program was not able to implement the family-targeted component as originally planned. This was due to the fact that many families were dysfunctional or unsupportive.

Coaches were ultimately satisfied with the training they received, noting that although it was inadequate at the beginning of the program, quality of training improved significantly over time. Key informant interviews also revealed that coaches would have preferred more training specific to case management techniques and case management software.

Regarding case management tools, coaches noted that they effectively captured information about the youth and allowed for sufficient customization to youth needs. However, they also noted that tools were too clinical and that the youth did not respond well to them.

Other implementation challenges related to inconsistent data entry, file management and file transfer challenges.

Target Group and Risk Factors

Originally, the project aimed to target youth who were at risk of becoming gang-involved, but were not gang members. However, over the course of the program, it became clear that recruitment activities were also attracting some youth who were already gang-involved. This had implications for coaches and staff who were not necessarily prepared to target these higher-risk youth, particularly in the early stages of the program.

A variety of outreach methods was used to recruit participants. The Citywide Freestyle Friday event, for example, was a rap event with strong appeal for youth. Staged during the second year of the project, it attracted over 100 attendees and a large number of new recruits. Outreach through partnering organizations also introduced the GPS to youth.

Qualitative findings showed that word-of-mouth was the primary means through which youth became aware of the program, with 65% of participants hearing about GPS from their friends. Youth were drawn to the program for the incentives and food, without which they would have been less likely to attend. Nonetheless, youth were likely to follow coaches' recommendations and work towards goals once in the program.

A risk assessment tool to assess type and level of risk was utilized during the final stages of the evaluation. However, evaluators agree that further reliability testing and piloting was needed to ensure incorporated risk factors were relevant for youth in the Hamilton region.

Partnerships

Overall, partnerships were useful and appropriate and allowed GPS to offer a full range of programs and services. In some cases, it would have been beneficial if points of contact within partnering organization had formal training in social work. This would have improved interactions between organizations and may have increased benefits to the youth.

Outcome Findings

Community Knowledge and Awareness

Awareness of risk factors associated with gang involvement, alcohol and drug abuse

Results from a conference delivered by Living Rock suggested an increase in community knowledge about drug abuse and gang prevention. Post-conference results indicated that the majority of community members reported an increase in their knowledge of street and gang culture (71%), the association between youth gangs and addiction (55%), and the possible risk factors associated with youth gangs (73%). However, it should be noted that most participants already worked with street youth or had an academic interest in the subject of gangs or street youth although the networking opportunities contributed to building knowledge.

Risk and protective factors

Strengthened bonds with coaches

Focus group and interview data suggests that youth formed close bonds with their coaches, often viewing them as mentors. Bonds developed slowly, and strengthened over the six month treatment period.

Positive life changes via the Plan of Care

Qualitative findings show that use of the Plan of Care was implemented somewhat differently by each coach. This made it difficult to compare or standardize the plan and the goals it contained. Some coaches mentioned that the plan was more appropriate for clinical/administrative treatment styles, as opposed to the conversational/informal style that they favoured.

Community engagement

There was a statistically significant 13% increase in the level of volunteering among youth between the pre survey and post survey. It is unclear whether the increase is sustainable in the absence of the opportunities and incentives provided by the program.

Social and Employment Skills

Data for this outcome was very limited, however qualitative findings seem to suggest that the youth felt as though their social skills were improving, and that they were better equipped to handle problems in a socially acceptable manner.

Attitudes

Attitudes toward education, community gangs and drug abuse

Although there was a directional increase in positive attitudes towards schools, these changes were not statistically significant. There was a statistically significant 18% increase in the extent to which youth felt put down by other students between the Youth Pre Survey and the Youth Post Survey. There was no statistically significant change in the way youth felt about gangs.

Self esteem and depressive symptoms

There was no statistically significant change in overall self-esteem among participants. Self reported emotional and psychological issues, including anger, anxiety, depression and self-injury actually worsened over the course of the program, though the increases were not significant. Dosage and risk had no significant effect on emotional and psychological issues. In-depth interview data suggested a strong need for on-site mental health practitioners to be made available.

Behaviour

Gang involvement^{Footnote 2}

The evaluation found a statistically significant reduction in the percentage of youth who were in a gang. At the time of screening (SDQ), 40% of youth reported that they were not currently in a gang. At the time of the post survey, the percentage had risen to 74%, representing a 34% increase in the number of youth who were not gang-affiliated. Exit rates were higher among high-risk youth (defined as those reporting a score of 95 or higher). Among the high-risk group, the number of youth who reported that they were not in a gang increased by 36%, versus 30% among the lower-risk group.

Dosage also had a statistically significant effect on gang involvement, wherein the median score of 242 hours in case management was decided as the cut off value for low versus high dosage. Among the youth that received the most hours of programming, the number of youth who reported that they were not in a gang between the SDQ and the post survey increased by 41%. Among the low dosage group, the increase was 26%.

Delinquent behaviours

Statistically significant changes were observed between the SDQ and post tests to the extent to which youth tried to illegitimately obtain money from someone without earning it (34% decrease); acted out at school (31% decrease); tagged property (29% decrease); carried illegal weapons (27% decrease) physically assaulted someone (22% decrease); issued threats to people or groups (16% decrease) and damaged or destroyed property (15% decrease).

There were no statistically significant reductions regarding the extent to which youth robbed someone, stole something, verbally bullied or harassed someone, associated with gang members, took drugs and alcohol, skipped school, gambled or stayed out all night.

Education

Due to the lack of administrative data from schools, and the use of self-reported data indicates that measures may have a low level of reliability. There was an increase in youth moving to higher levels of math, but a decrease in English levels between the Youth Pre and Post test survey. Qualitative findings suggest that the program may have fostered a more positive attitude towards education among participants.

Employment

Statistically significant differences were observed in youth's sources of income between the pre and post surveys. The results indicate that 13 % more youth were able to obtain income from legal/legitimate sources, and 13% fewer youth obtained income from illegal sources. Varying risk levels and dosage (hours of programming) had no significant effect on changes to employment and income source.

Drug use

There was a statistically significant 18% decrease in marijuana use between the pre and post surveys. There was also a statistically significant 17% decrease in ecstasy use. It should be noted that drug use was self-reported. Tests for change in all other types of drug and alcohol use showed no changes after youth completed the GPS program.

Youth's risk levels had a statistically significant association with their marijuana use, with a 25% reduction in drug use among higher-risk youth versus a 2% reduction among lower-risk youth. During focus groups, youth expressed that to some degree the program served as a networking opportunity and increased their exposure to different drug dealers and drugs.

Peer associations

Overall, youth were decreasing their association with negative peer influences, with 10% noting that their group of friends were less accepting of illegal activities. The evaluation was not able to identify whether the youth changed friends or whether the friends themselves changed their behaviour.

Reduction in the number of gangs in downtown Hamilton

Police data for the area in which Living Rock was expected to have the greatest impact was not available, so it was not possible to determine whether the program affected the number of gangs in downtown Hamilton.

Unintended Outcomes

Negative peer associations

Interviews suggested the need to separate activities for gang-involved youth and the low-risk youth so as not to expose the latter to the gang lifestyle. One coach revealed that he had witnessed youth develop new negative peer associations. Others coaches expressed, however, that higher-risk youth could provide important mentorship to lower-risk youth.

Exclusion due to eligibility requirements

Some coaches noted that some lower-risk youth who would have benefitted from the program were excluded due to eligibility requirements that prioritized higher-risk youth.

Reliance on financial incentives

Qualitative data revealed that many youth became dependent on the incentives for survival and their program goals became secondary to incentives. Responses were mixed when youth were asked if they would attend the program without incentives. Some suggested that incentives should be paid out incrementally as program participants achieved their goals. The financial incentives may have also influenced participants' responses to screening tools to guarantee program entry.

Knowledge exchange

The program successfully promoted knowledge sharing between program partners, and the benefits of all programs were communicated to the youth.

Cost Analysis Findings

The total cost of the project from 2008 to 2010 was $1,600,258 (excluding the cost of the evaluation, of approximately $250,000).

Evaluation Limitations

The validity of the study was negatively affected by the following methodological limitations:

Lack of a comparison group

The change in evaluation methodology from a quasi-experimental to single group repeated measures design negatively impacted the validity of the study. Without a comparison group, attributing outcomes to program activities is problematic. Triangulation of survey data with qualitative sources partially mitigated this concern.

Sample size

Statistical power was limited by a relatively small sample size (n=86) of completed post-test surveys.

Inconsistency of data

Initially, records including time sheets, plans of care, activity logs, attendance records, case management files and youth-coach correspondence were paper-based, with coaches using different approaches to recording case management notes. The use of an electronic data management system, adopted during the final year of the program, improved the availability and consistency of the data. Records may have been limited due to the transient nature of participants, and delays in transferring data to the evaluators.

Reporting bias

Interviews with the coaches provided evidence that youth bonded with coaches. Youth were more likely to admit to particular behaviours once trust had been established (i.e. during the post-survey). This may have compromised attribution of positive impacts to the program by lessening the ability of the evaluators to identify positive impacts of the GPS program. In two cases, the coaches re-administered the Youth Pre-Survey after the youth admitted to drug use they had previously denied. There is also the potential for interviewer bias, as coaches who had formed connections with participants were responsible for administering surveys.

5. Lessons Learned and Recommendations

Program Delivery

The evaluation produced the following recommendations to improve similar interventions in future:

• Focus on activities and locations that generate the highest amount of word-of-mouth awareness to best target transient populations.
• Organize activities that provide youth with positive alternatives to their regular Friday/Saturday night activities, to allow youth to interact with their peers in a controlled environment.
• Similar programs should view six-month post-program retention of knowledge as an indicator of the effectiveness of community educational initiatives.
• Trust takes time to build. More time should be allotted for relationship building and nurturing. Qualitative data suggest that most bonding occurred within three months of program entry, as very little occurred before the end of the first month.
• Develop clear, standardized usage guidelines for the plan of care. This is necessary to increase fidelity, define responsibilities of the youth and coach, and document progress.
• Community involvement and volunteerism should be considered a longer-term outcome for similar future programs.
• The program referred youth to a large and diverse range of agencies. As a result, they may have received varying levels of service. In future, the program should make referrals to a smaller number of agencies with a wider spectrum of care to maintain a standardized method of referral for all youth.
• Mental health supports must be factored into the program design, either through partnerships or on-site staff.
• Incorporate means of replacing illegally obtained income for youth, whether through incentives, job programs or assistance in obtaining government financial support.
• Revise the definition of treatment to reflect the developmental window for youth and the transient nature of high-risk youth.
• Future programs would benefit from a comprehensive study of incentives.
• Incorporate time management software at the project launch to provide increased granularity in how the personnel's time was included in the individual project activities.
• Future projects should anticipate training needs and budget for them in advance of launch.

Evaluation

The following recommendations should also be considered when evaluating crime prevention programs:

• Ensure that all data related to program components are tracked during the whole program. Efforts to clearly define and track dosage and risk levels will increase the ability to attribute the program to the outcomes of interest. Clearly defining program duration by articulating an end date would also allow the evaluators to isolate program attribution more effectively.
• Evaluation activities that focus on clarifying the program components would increase the opportunities to identify a feasible comparison group. This program was not able to identify a comparison group, as the program activities were too similar to the potential comparison group.
• Efforts to upgrade a data collection system will increase the accuracy and efficiency of the data collection and analysis process.
• Future evaluations should focus on measuring changes in the individual youths' attitudes and behaviours with their peer group, rather than measuring the peer group's attitudes and behaviours. This is especially relevant for programs where the primary target groups are youth at risk of gang involvement and not gang-involved youth.
• Youth survey data should be corroborated with more reliable school or police data.
• Future iterations of the GPS Program should be implemented on a larger scale. This will help to ensure that sample sizes are large enough to be statistically reliable.

6. Conclusion

The outcome evaluation revealed that the GPS program achieved several of its target outcomes. Notably, it resulted in significant reductions in drug use, gang involvement, delinquent behaviour and reliance on illegal sources of income.

In addition, participants often developed strong relationships with coaches, and made some positive life changes. Results from this evaluation should be interpreted cautiously. In the absence of a comparison group, it is not possible to attribute impacts to treatment provided by the program.

The evaluation suggests that future projects of a similar nature should standardize treatment plans, lengthen treatment periods, reconsider approaches for using financial incentives and ensure that appropriate resources are available to address mental health issues.

For more information or to receive a copy of the final evaluation report, please contact the National Crime Prevention Centre by e-mail at ps.prevention-prevention.sp@canada.ca.

If you wish to register for the NCPC mailing list to receive information from the Centre, please visit the subscription page at: http://www.publicsafety.gc.ca/cnt/bt/mlng-lst-eng.aspx

Date modified:

2022-07-21