Regina Anti-Gang Services

Regina Anti-Gang Services PDF Version (247 KB)

ISSN: 978-1-100-20406-2

Table of contents

• 1. Introduction
• 2. Program Description
• 3. Evaluation of the Program
• 4. Evaluation Findings
• 5. Lessons Learned and Recommendations
• 6. Conclusion
• End Notes

Building the Evidence - Evaluation Summaries 2012-ES-26

1. Introduction^{Footnote 1}

Regina Anti-Gang Services (RAGS) was developed in 2007 in response to the high level of gang activity in the North Central neighbourhood of Regina. It offered support to gang-involved youth in one of the most deprived, gang-filled, and criminally active neighbourhoods in Canada.

Although Ontario has more gang members than any other province, Saskatchewan has the second largest number of gang members. However, with one-tenth the population of Ontario, Saskatchewan has the largest number of gang members per capita, with 1.34 youth gang members for every 1,000 people. In Saskatchewan, almost all gangs were reported to be Aboriginal.

When the RAGS project proposal was being developed, Regina had the highest murder rate of all Canadian metropolitan areas, with 4.5 murders per 100,000 individuals. Common assault accounted for approximately 3 in 4 assaults, and 3 in 5 violent offences. Sexual assaults accounted for 6% of all violent crimes. Robberies accounted for 7%.

Regina's North Central neighbourhood has been estimated to be one of the most deprived communities in Canada, second only to Vancouver's downtown east side. It has the highest Crime Location Quotient (LQV) in Canada. Almost one-half of residents are Aboriginal. Roughly one-third of all residents are on social assistance. Many depend on food banks to make it through the month. The teenage pregnancy rate is estimated to be nearly the highest in Canada. Relative to the rest of Regina, the average family earns roughly half the income compared to the city's average. North Central has the highest level of crime relative to other parts of Regina, with exceptionally high levels of serious and violent crime. One-quarter of all police calls originate from this community.

The National Crime Prevention Centre (NCPC) awarded $2.6 million to the North Central Community Association (NCCA) to implement RAGS from October 2007 until March 2011. The program integrates elements of several evidence-based models, including Wraparound, Multi-Systemic Therapy (MST), harm reduction, and also, cultural and faith-based support.

2. Program Description

The Regina Anti-Gang Services Project (RAGS) aimed to help gang-involved youth leave their gangs safely. Towards this objective, RAGS sought to:

• Increase youth attach ment to the labour force, schools and retraining;
• Increase their attachment to healthy adult role models;
• Reduce their involvement in gang related violence and crime;
• Decrease gang-related sex-trade activity.

Participants of RAGS took part in intensive daily services. Although these services were based on evidence-based models such as Wraparound and MST, staff adapted these models to better suit the needs of Aboriginal youth. Focusing on the social context in which gang-related behaviours develop while targeting individual change, interventions used the family as the primary area of work to build the youth's and family's strength. An intensive case management model was used to target problems that predict known risks and protective factors, both for individuals and groups. As a result, the intensity of programming was much higher than that of Wraparound and MST. For example, while MST averages 60 hours of programming over 16 weeks, RAGS averaged roughly 385 hours of programming over 77 weeks.

The program provided four core activities:

Life skills programming for young men

Group training, education and skill-based learning on topics such as exiting gangs, violence, personal awareness, problem solving, healthy relationships, parenting and fathering, addictions, team building, empowerment, behaviour modifications, and literacy.

Circle keeper program for young women

Gender-specific life skills and traditional cultural training for women who are in a gang, or connected to one, to support exit from the sex-trade and gangs through education.

Intensive gang exit counselling

Individual, crisis, and family counselling sessions targeting specific goal areas, such as safe exit from gangs, parenting, self-esteem, life skills development and others.

Outreach to schools and institutions

Engaging potential RAGS participants through schools, correctional centers, courts, and occasionally on the streets.

Program participants

The program engaged gang-involved youth (primarily Aboriginal) and young adults aged 16 to 30 years old (mean age 23.9 years) as well as their parents, their family or both. All participants demonstrated major risk factors, with 100% being current or former gang members or being females affiliated through their boyfriend. 82% of participants had serious drug and alcohol addictions. Overwhelmingly, they held strong social ties to gang life, with 88% having friends who were gang members and 90% having family who were gang members.

Moreover, the majority had been involved in very serious crimes, including murder, manslaughter, attempted murder, assault causing bodily harm, assault with weapons, robberies, home invasions, auto theft, drug tracking and prostitution-related offences.

In total, 99 clients (66 males and 33 females) participated in RAGS from January 2008 to March 2011. Within this group, 74 participants received intensive counseling and completed the surveys. On average, youth who completed the program (n = 51) spent 86.8 weeks in the program and participated in 306 therapeutic sessions, equalling 439.8 hours. Of the primary participants, 7 dropped out of the program.

3. Evaluation of the Program

An independent process and impact evaluation was conducted to determine the effectiveness and efficiency of the RAGS program. A cost analysis was included.

The evaluation covered the period March 2008 to January 2011. A mixed approach was used, combining both quantitative and qualitative methods.

The quantitative methodology employed a non-randomized comparison group design, with treatment and comparison group members matched on key variables. The treatment group comprised 74 RAGS participants, while the comparison group had 29 gang-involved, Aboriginal high-risk youth and adult offenders. The comparison group members were minimally involved in RAGS. They received five or fewer hours of contact per month, and only participated in recreational activities, but not in the counseling services.

The researchers used several instruments and data collection methods, including:

• Survey instruments measuring gang membership, substance use, risk and prevention, identity, beliefs/attitudes about conflict, violence, guns and aggression;
• Focus groups with target youth, young adults, and elders;
• Program and client records synthesizing summaries of involvement and contact logs;
• Analyzing client plans including counseling plans and records of involvement;
• Observations by team members;
• Third party data from the Ministry of Corrections, Public Safety, and Policing, and the Regina Police Service.

Over 24 months, data was collected on five occasions, covering pre-, mid-term, post-, and follow-up measures. The follow-up measure was taken at 24 months, six months after participation in the program typically ended.

The evaluators employed two statistical methods to assess the program's impact. Matched pair t-tests and Independent Samples t-tests were used to determine whether significant changes occurred among youth in treatment, and whether differences between the groups (program participants and non-participants) were statistically significant at a particular point in time.

4. Evaluation Findings

Outcome Findings

A summary of key outcomes is presented in the table below.

Awareness and Beliefs

Beliefs about conflict

Results show a positive change (59% decrease) in beliefs about conflict in program participants six months into the program but this effect did not last. While no significant results were found at 12 and 18 months, trends show a 41% decline in beliefs about conflicts at 12 months and a 63% decline at 18 months. Results also show that the matched comparison group did not differ from program participants at baseline and at six months. While it would appear that the program did not change participants' conflict resolution beliefs, the low sample size at 18 months may explain results.

Attitudes

General approval of aggression

Only assessed at six months, program participants' results show a 57% significant decline in general approval of aggression.

Approval of retaliation

Results at six months show a 71% significant decline in program participants' approval for retaliation.

Dislike of guns

This measure assessed participants discomfort with guns. Results at 6 months show a 65% increase in dislike of guns.

Gender stereotyping

This measure looked at participants' gender stereotyping attitudes. Results show a 67% increase in program participants' propensity towards gender stereotyping at six months. No significant improvements were made between the pre-test and the six-month measure.

Overall it appears there were some positive changes in attitudes. However, program effects cannot be ascertained as comparisons with the matched comparison group were not made.

Risk and Protective Factors

Risk index scores

This composite index contained a number of risk factors, including gang affiliation, substance abuse, non-violent crime, violent crime, adult role models, and gang involved peers. Nonetheless, both the within-group and comparison group statistics showed a significant decline in participants risk index scores over time.

Employment

No significant changes in program participants' attachment to the labour force at 6, 12, or 18 months were found. Comparisons with the matched comparison group at baseline and six months show no differences between the two groups with respect to increased attachment to the labour force. However, relative to the comparison group (whose scores dropped over time), RAGS participants had achieved a greater level of attachment to the labour force.

Substance use

Substance use declined by between 61% and 68 % between time intervals. That is, more than twice as many youths in the program decrease their drug consumption as increased it. Only one of these measures shows a significant improvement at six months. Moreover, while the comparison group showed a higher level of substance use, no significant differences were found between the groups at 6 months. It would appear that the program did not influence participants' substance use, and the low reliability of the measure (Cronbach alpha of .60) used may explain the lack of significant results.

Depression

Program participants' scores on the depression measure declined over time, but only the first measure, taken at six months, showed a significant decrease in depression. Control group participants showed higher levels of depression at the six-month time interval, no significant differences were found between youth in the RAGS program and the matched comparison group. The program appears to have had no effect in improving symptoms of depression in youth.

Behaviours

Gang affiliation survey scores

Results show a steady decline in gang affiliation over time. Gang affiliation declined for about 46% of the participants at six months, 63% at 12 months, and 71% at 18 months. While no differences in gang affiliation were found between program participants and the matched comparison group, youth in the matched comparison group showed greater affiliation with gangs at six months, indicating that the program had a positive impact on gang affiliation.

Violent crime

This score was assessed by counting how many types of violent crimes participants reported they were involved in during the previous six months. These crimes either included the use of violence or the threat of using it. Results show that program participants significantly reduced their involvement in violent crimes. Involvement declined by 50%, 75%, and 63% at 6, 12, and 18 months respectively. However, the one six month comparison group measure show no difference between RAGS participants and the matched comparison group in the level of involvement in violent crimes. With this mixed evidence, we cannot ascertain with some degree of confidence that RAGS had an effect on program participants' level of violent crime.

Non-violent crime

This measure assessed how many different types of crime participants were involved in during the previous six months that did not include violence towards others. Results show significant decreases in non-violent crimes over time, with declines of 56%, 63% and 63% at 6, 12, and 18 months respectively. Results of group comparisons show that youth in the comparison group were more involved in non-violent crimes at six months than RAGS participants at six months. No other comparisons at 12 and 18 months were made. Results indicate that the program likely had an effect in reducing participants' involvement in non-violent crimes.

Cost Analysis Findings

The total cost of the Project including in-kind contributions was $2,872,560. The average cost total cost per participant (n = 99) was $8,290.20 per year, or $690.85 per month.

Evaluation Limitations

While the selection of a matched comparison group design in the current study would be deemed a sound research method^{Footnote 2}, it is limited in its ability to attribute outcomes to program activities. The following threats to validity should be considered when interpreting the results.

Limitations of the statistical tests used

The outcomes were measured multiple times, between groups, over a number of timeframes and assessed through two separate statistical methods; the Matched pair t-tests and the Independent Samples t-tests. While these methods are appropriate for measuring change between two groups or between two time periods, they are less appropriate for measuring change over multiple times and between groups. The repeated measurement between times, and then separately between groups, could result in showing a positive change when there was none or not showing a positive change when there was one.

Low sample size

The participation rate between-time periods and between-group comparisons was quite low. For between-time comparisons, participation dropped from 54 to 21 cases in some instances and to 18 in others at the 18-month follow-up. The participation rate in between group comparisons was even more severe. No results were reported to show differences between program participants, and the matched comparison group for the time periods beyond 12 months, and the sample size at 12 months is very small, averaging 10 to 12 cases.

Instrumentation

RAGS staff administered the questionnaires with young people because many of the participants had low levels of literacy. This may have introduced bias into the surveys such that the participants may have attempted to show themselves in the best possible light. There could also be inconsistencies in the way the instruments were administered and interpreted over time.

Construct validity

The survey instruments employed were largely from the list of acceptable tools proposed by the NCPC, drawing on standardized instruments. However, when a number of outcome indexes were composed from these instruments, a statistical test (Cronbach's alpha) showed that some of them were measuring similar constructs while others were possibly measuring multiple constructs.

Nonetheless, while reliability was slightly below the .70 threshold for conflict resolution, substance abuse, gender stereotyping, and discomfort with guns, results indicate that most measures were reliable. Cultural identity and total risks were found to be unreliable measures of these constructs.

In spite of these numerous limitations, the evaluation has enough strength to attribute some outcomes to program activities, specifically outcomes related to youth leaving their gangs. The critical outcome measures, gang affiliation and total risk scores, showed a positive outcome for both the within- and between-group measures.

5. Lessons Learned and Recommendations

Program Delivery

Supplemental external services

RAGS participants required addictions counseling and employment programming. To ensure the program could meet participants' complex needs, it was necessary to develop partnerships with other service providers who could provide a broader spectrum of support. Also case management was a key part of the program. This broader continuum of services should focus on the individual, their health, family, school, workforce, and community. This should be planned at the outset.

Program intensity

There were key times when the evaluation showed that the program had made its largest impact. Primarily, this occurred within the first six months. To maximize the program's impact, RAGS should deliver a higher intensity of programming during these key times.

Engaging family

Some participants were tied to gangs through their families. This placed them in a difficult position where they would have to cut family ties in order to cut their gang ties. To address potential obstacles and opportunities arising from participants' families, a family and parent component should be added to the RAGS program, and services delivered to them.

Traditional Aboriginal teachings

Cultural programming is important to gang-involved Aboriginal youth. Nonetheless, there may be a sizable portion of participants who reject cultural activities such as communicating with community leaders. To increase the odds that youth build attachments to their culture rather than their gangs, all aspects of programming should be permeated with traditional teachings and practices.

Building relationships through activities

Fostering partnerships with police and corrections was critical to the program's success. Successful relationship building can lead to the participants viewing police as mentors and counselors, and typically, this can come about when officers volunteer their free time to engage the youth in recreational, artistic or employment-related activities. Staff also benefited by engaging participants in recreational and artistic activities. These shared activities offered many benefits, including facilitating open dialogue between staff and participants, and it helped participants better relate, even those who may have been enemies on the streets.

Gender-responsive programs for women

Female clients do not feel comfortable mixing with male participants in the same program space. In the eyes of traumatized young women, any male can potentially be an abuser. It is critical that programming address the physical and sexual violence that these women experience at the hands of male partners. There is a need to develop gender-responsive programs for young women that address their unique risks and protective factors. Such programs should be separate and distinct from those programs for young men, implemented in a separate and secure location, and delivered by women.

Staffing considerations

Staffing RAGS was difficult. The program required staff to be on call at difficult times, resulting in high turnover. Although staff with prior gang experience had greater clout among participants, this benefit also brought some conflicts of interest. To reduce a variety of risks, staff should be trained in maintaining appropriate boundaries with young people. This is important for staff who have past experience with gangs and who may also share the same social circles as the participants.

Managing the risk of violence

Gang exit projects are risky to operate. Gang leaders do not want their members to leave the gang, and may resort to threats and violence against staff and participants. In the case of RAGS, participants feared that gang members were trying to infiltrate the program to spy on participants. These risks should be mitigated by ensuring there is secure office space, conducting intake assessments off-site, assessing participants' level of motivation, conducting regular safety audits and consultations, and reviewing all “near misses” where acts of violence were narrowly avoided.

Evaluation

Create an evaluation culture

It is not uncommon for the staff from youth service agencies to have little exposure to outcome evaluations. This can create many evaluation challenges. To overcome these challenges, the evaluation team should spend considerable time with agency staff to demystify the concept of evaluation and address their concerns. Moreover, there is a need to create an evaluation culture that engages and collaborates with agency staff from the start of the project.

Engage participants

From the start of the evaluation, young people were engaged through four main methods. These included piloting tools, consulting with the evaluators, participating in focus groups, and providing feedback on the results of each annual report. When youth were meaningfully engaged in the evaluation, they developed a sense of ownership over the tools and methodology. Young people need to know that their voices are important and that their ideas and concerns will be recorded and addressed.

Cultural sensitivity

The evaluation methodology and tools should reflect First Nations and Métis cultures, including language and cultural traditions. To achieve this, evaluators should consult with Aboriginal staff and young people. To demonstrate the evaluation team's commitment to cultural sensitivity, the evaluation team should participate in cultural activities with youth, including feasts, ceremonies, sweats and circles.

Gender sensitivity

Evaluation tools were designed in a way that permitted gender sensitive assessment and follow-up tools. For example, survey questions addressed issues related to care of children, involvement in the sex trade, and depression. When assessing gang membership and affiliation, questions should be sensitive to the gendered experiences of gang involvement and to women's unique risks and protective factors.

Develop control group options from the start

Matching the treatment group to a control group is essential to rule out alternative explanations of program impacts. However, recruiting a control group sample is challenging and maintaining contact with high-risk gang members is very difficult. It is best to pursue at least two options for control groups from the beginning to ensure the participation of an adequate control group.

Maintain contact with participants over time

Although it is difficult to maintain contact with high-risk gang members over time, it is essential. As the sample sizes of participants completing follow-up surveys declines, the power of the statistical tests also declines. This can undermine the ability to assess if any impacts are maintained over time. In order for the evaluators to have enough data to test program impacts over time, staff have to make more of an effort to find those who have already completed the program and may have moved on.

Use mixed methods

Often, evaluations of gang intervention projects rely solely on quantitative methods. Although important, these tools cannot identify the dynamics and fine details of complex issues such as mental health, gang involvement, gang exit, the sex trade, and cultural attachment or identity. Case studies, in-depth interviewing, observation of program activities, client file reviews and focus groups are methods well-suited to complement quantitative measures.

6. Conclusion

RAGS was successful in reaching its targeted population and in helping participants exit from gangs. However, there were inconclusive results on RAGS' ability to help gang-involved women leave the sex trade.

Overall, the program positively influenced participant's beliefs about conflict and it appears to have improved attitudes toward aggression, retaliation and guns. The program significantly reduced participant's overall risk scores.

However the impact on employment, substance use and depression was less significant or inconclusive. Most notably, RAGS influenced key behavioral outcomes such as gang affiliation scores, non-violent crime, and possibly violent crime. Implementing and evaluating similar projects in other Aboriginal communities might help improve its outcomes.

For more information or to receive a copy of the final evaluation report, please contact the National Crime Prevention Centre by e-mail at ps.prevention-prevention.sp@canada.ca.

If you wish to register for the NCPC mailing list to receive information from the Centre, please visit the subscription page at: http://www.publicsafety.gc.ca/cnt/bt/mlng-lst-eng.aspx

Date modified:

2022-08-16