Youth Alliance Against Gang Violence

Youth Alliance Against Gang Violence PDF Version (222 KB)

ISSN: 978-1-100-20348-5

Table of contents

• 1. Introduction
• 2. Program Description
• 3. Evaluation of the Program
• 4. Evaluation Findings
• 5. Lessons Learned and Recommendations
• 6. Conclusion
• End Notes

Building the Evidence - Evaluation Summaries 2012-ES-22

1. Introduction^{Footnote 1}

The Youth Alliance against Youth Violence (YAAGV) program, also known as the Warrior Spirit Walking program, was developed in 2007 in response to the high number of street and gang-involved youth in Prince Albert, Saskatchewan.

It offered support to Aboriginal youth, aged 12 to 21, who were gang-involved or at high risk of gang involvement. The project aimed to assist these youth to safely leave gangs or to resist gang activities.

Saskatchewan is believed to have the highest concentration of youth gangs in Canada per capita. In addition, approximately one in ten of the province's youth are involved in the criminal justice system. The majority of gangs in Saskatchewan are Aboriginal.

Prince Albert, the third-largest city in Saskatchewan, has a population of some 35,000. Approximately two-thirds (67%) of its population is under 25 years of age. One-third of Prince Albert residents identify themselves as Aboriginal. One-quarter of all families are single parents. The city's unemployment and teen pregnancy rates are highest in the province. Approximately 400 children are in foster care or in staffed group care facilities.

The city has the highest crime rate among small cities in Western Canada. It has experienced an increase over the past number of years in gang violence involving aggravated assaults, shootings, and murders. Seventy percent of arrests made by the Prince Albert Police Service are drug and substance related.

Prince Albert also has a high concentration of correctional facilities, with more than 1,000 inmates at any time. Many families move to Prince Albert to be closer to incarcerated family members. Recent figures from the Correctional Service of Canada indicate that 75% to 80% of offenders admitted to custody in Saskatchewan are of Aboriginal descent.

In April of 2007, the National Crime Prevention Centre (NCPC) awarded $1,657,141 under the Youth Gang Prevention Fund (YGPF) to the Prince Albert Outreach Program Inc. (PAOPI), a registered non-profit charitable organization, to implement and evaluate the YAAGV. The Project began August 15, 2007 and ended March 31, 2011.

2. Program Description

YAAGV is a community-based crime prevention program that uses the Circle of Courage Approach as its foundation, and incorporates elements of Wraparound and Multi Systemic Therapy (MST) into the design.

The program was designed specifically to suit the needs of Aboriginal youth who were gang-involved or at risk of gang involvement. The program sought to:

• Increase youth attachment to school
• Increase youth employability and life skills
• Reduce youth involvement in gang-related violence and crime
• Increase literacy skills and high school completion rates

The Circle of Courage is a traditional indigenous model of youth development. Based on the four parts of the medicine wheel, the approach draws from indigenous philosophies of child rearing and education, and resilience research.

YAAGV focused on the multiple determinants of criminal and anti-social behaviour, and provided services in the youth's own neighbourhood instead of in the family unit.

In MST, the family is the primary area of work. Building the family's strengths is a main focus of intervention, but the vast majority of YAAGV youth had little to no contact with their parents. Many had grown up in care, and/or been incarcerated for lengthy periods of time, while others had no contact with families due to severe abuse. Therefore, it was not deemed feasible to focus interventions on the family unit.

Finally, compared to MST, which averages 60 hours of contact with families over a four month period, YAAGV offered a much more intensive case management program, with youth receiving on average 684 hours of service.

The Wraparound approach is used with adolescents who have serious emotional disturbances and are at risk of out-of-home placement. It refers to a specific set of practices to develop individualized care plans based on the strengths, values, norms and preferences of the child, family and community. Like Wraparound, YAAGV used individualized care plans. However, for reasons discussed above, only a minority of cases involved parents.

YAAGV involved six components:

Counselling

Services included individual crisis, employment, substance abuse, female assistance group, and community school-based counselling services. A variety of socio-recreational and group activities were also offered to address peer pressure, conflict resolution, and gang resistance issues.

Presentation Team

Under staff supervision and support, selected program participants disengaging from gangs made presentations to other young people about the dangers of youth violence, bullying and gang involvement. The primary goal for the presenters was to develop employment and life skills.

Senior and Junior Won Ska Cultural Schools

This is an alternative school program to help high-risk Aboriginal youth complete their high school training, earn elementary and high school credits, increase literacy skills, increase life skills, and participate in employment training.

Youth Activity Centre

This centre provides a safe environment for recreational, arts, musical, and cultural activities.

Van Outreach

Four times a week staff used a van to make contact with high-risk youth on the streets. A nurse joined the staff twice a week to provide needle exchange services. Other services included providing healthy meals and drinks, and condom and bad-date list distribution, counselling, information, and referrals for more intensive services.

Court Outreach

Court house visits provided by a justice worker offering legal, counselling, information, and referral services.

Program Participants

The program served 147 participants aged 12 to 21 between November 2007 and March 2011. Of these, 41% percent were female and 59% were male. The average age was 17.8 years.

Risk Factors

Fifty one percent reported being at risk of gang involvement while 49% reported being gang involved. All participants reported at least one risk factor, including substance abuse, violent crime, unemployment, prior arrests or suicide ideation.

Results of a survey conducted at entry indicate that 45% of youth had grown up in the care of child welfare and/or youth justice facilities, 33% had sold drugs in the six months prior to entering the program, 39% had attacked someone with the idea of seriously hurting them, 82% had a close family member with a severe drug or alcohol problem, 45% had severe alcohol problems (frequent binge drinking), 76% frequently used marijuana, 42% frequently used ecstasy, 26% reported having suicide ideation, and 13% had attempted suicide.

Program Participation

Between March 2008 and January 2011, 86 cases were closed. Of this number, twelve had not completed the program, nine had dropped out part way through, two committed suicide, and one was murdered. Of the remaining cases, 74 had successfully completed the program. All 74 participants had completed counselling and were gang-free at case closure.

At file closure, all 86 participants had received an average of 100.2 weeks of intervention. A total of 58,782 hours of face-to-face programming was received, with each of the 86 participants receiving 683.5 hours of services on average. Across programs, dosage ranged from a low of 13.8 hours (Van Outreach) to a high of 443.6 average hours of service per youth at the Junior Won Ska School. The majority of the active youth participated in at least four of the six program components.

The 74 youth who successfully completed the program received, on average, 749 hours of programming whereas the 12 youth who did not complete the program had received, on average, 290 hours of programming.

3. Evaluation of the Program

An independent process and impact evaluation was conducted to determine the effectiveness and efficiency of the YAAGV project. The evaluation study ran from November 2007 to March 2011.

A mixed method approach was used, combining both quantitative and qualitative methods. Quantitative methods were used to determine the impact of the program on outcomes, and qualitative methods, which included client file review, focus groups, police records, and in-depth interviews with youth, were used to provide context to the quantitative findings.

The evaluation employed a non-randomized control group design. A total of 147 YAAGV youth were selected to join the treatment group. The control group consisted of 48 gang-involved or high-risk youth selected from the group of PAOPI court outreach cases. They were matched to the treatment group on key characteristics such as age, gang involvement, gender, Aboriginal status, place of residence, offending history, and employment and school status.

Outcomes were measured using various US-based standardized instruments adapted from the Centre for Disease Control: Measuring Violence-related Attitudes, Behaviors, and Influences among Youths compendium; the Rutgers Teenage Risk and Prevention Questionnaire; the Rochester Youth Development Study Attachment to Teacher, and Depression Questionnaires; and the Seattle Social Development Project Parental Supervision, Attachment to Parents, Commitment to School scales.

Data from program participants was collected on six occasions, over 30 months, at entry (denoted T1), 6 months (T2), 12 months (T3), 18 months (T4), 24 months (T5), and 30 months (T6). Data from the control group was collected at entry (T1), 6 months (T2), and 12 months (T3).

Matched pair t-tests were used to determine whether differences between time periods were statistically significant for program participants. Independent samples t-tests were also used to determine group differences on each outcome at three time points: entry, 6 and 12 months.

4. Evaluation Findings

Process Findings

Program delivery

Field observations showed that interventions were gender responsive, culturally competent, and were implemented as planned. Observations also demonstrated high demand for each type of service provided to youth. The exceptions were Van Outreach and the Youth Activity Centre, which had relatively low rates of participation. In response, the hours of operation of the Van Outreach and YAC were changed to better serve clients.

Satisfaction with the Program

Focus group participants spoke highly of the Won Ska Cultural School, expressing that they felt respected by staff and trusted them. Individual attention from teachers, learning at one's own pace, completing credits and positive role modeling provided by program staff were identified as key success factors by program participants.

Outcome Findings

Attitudes

Cultural identity

No significant changes in cultural identity were found among program participants.

Reductions in pro-violent attitudes and beliefs

Four indices were used to measure pro-violent attitudes and beliefs. These were: Approval of Aggression, Retaliation to Aggression, Approval of Aggression and Retaliation Combined, and Conflict Resolution Skills. Data for these indicators were only available for T1 and T2. During this period, no significant changes were observed in general approval of aggression and retaliation. A significant improvement in conflict resolution skills (a 65% increase) was found between entry and six months.

Acceptance of gangs

Acceptance of gangs measured the extent to which participants believed it was acceptable or cool to be in a gang. Findings suggest that acceptance of gangs declined with time spent in the program. Significant changes were observed at T5 and T6. Acceptance of gangs had declined by 38% and 42% between entry and 24 and 30 months respectively.

Attachment to parents

Three indices were used to measure attachment to parents and parental figures. These were the Attachment to Parent, the Positive Parental Supervision, and the Pro-social Parental Involvement scales. While some positive changes (between 34% and 55% increase) were observed between entry into the program and the different measurement periods, none were found to be statistically significant.

Risk Factors

Bonding to adult role models

A 75% significant increase in bonding to adult role models was found for a very small number of YAAGV participants between T1 and T6 (30 months).

Depression symptoms

A significant 58% decrease in depression symptoms was found between T1 and T4 (18 months). Depression scores remained unchanged between T1 and subsequent measurement at 24 and 30 months.

Attachment to school

Attachment to school was based on three indicators: attachment to teachers, commitment to school, and number of suspensions. Results show that for a small number of program participants, attachment to teachers significantly increased by 63% between T1 and T5 (24 months) and by 75% at T6. No significant increases in commitment to school were found at 6, 12, 18, 24, or 30 months. Finally, the number of suspensions significantly decreased by 14% between T1 and T3 (12 months), but showed no significant subsequent changes over time.

Attachment to the labour force

The results of frequency analyses show that approximately 50% or more of the youth who were not working at T1 were employed at 6, 12, 18, and 30 months.

Involvement with criminal and anti-social peers

The level of involvement with criminal and anti-social friends remained unchanged for YAAGV participants. No significant differences were found between entry and the various measurement periods for this expected program outcome.

Total risk

An overall risk index was constructed to assess the extent to which the program reduced the attitudes and behaviours that might predispose youth to gang involvement. The index combined seven important indices; levels of non-violent crime; violent crime; present gang membership; school suspensions; anti-social peer behavior; substance abuse; and lack of access to adult role models. It had a reliability index of .66 which is slightly below the acceptable .70 criterion (Nunnally, 1978^{Footnote 2}). A significant reduction in risk was observed among the targeted youth between entry to the program and T3 (33% decrease), T4 (26%), and T5 (42%), but not at T6.

Behaviours

Substance use

The index of substance abuse indicated the number times over the previous six months a youth had used various types of drugs. A 60% significant decrease in substance abuse was observed for a very small number of participants between T1 and T6 (30 months). Comparisons across other times did not show any significant reductions.

Crime and violence

The evaluation found some reduction in participants' involvement in non-violent and violent crimes, however findings were inconsistent. The low reliability of these scales, and the smaller sample sizes at 24 and 30 months, may explain the lack of results.

Significant decreases in non-violent crimes were observed between T1 and T3 (47%) and between T1 and T4 (51%), but as mentioned above did not persist at 24 and 30 months. A significant decrease (33%) in involvement in violent crimes was found between T1 and T4 but, again, differences observed across other time comparisons were not significant.

Gang affiliation

The rate of exit from gangs among YAAGV participants increased over the six follow-up periods. Approximately 67% of youth who were gang members on entry to the program had left their gangs by six months, and 100% of youth who were gang members on entry had left by the 24 month follow-up. Overall, all youth who had successfully completed the program (n = 74) had exited gangs or had resisted involvement with gangs at case closure.

The table below provides a summary of key results. Only significant results are reported. None of the between-group analysis is reported as the comparison group was not equivalent to the YAAGV group.

Cost Analysis Findings

Including in-kind contributions and funding from NCPC, the total cost for delivering the program was $2,623,222. The average cost per participant who successfully completed the program (n = 74) was $35,448 or $2,954 per month per participant.

Evaluation Limitations

While the selection of a matched comparison group design in the current study would be deemed a sound research method^{Footnote 3}, it is limited in its ability to attribute outcomes to program activities. The following threats to statistical conclusion and internal validity should be considered when interpreting the results.

Limitations of the statistical tests used

The outcomes were measured multiple times, between groups, over a number of timeframes and assessed through two separate statistical methods, the Matched pair t-tests and the Independent samples t-tests. While these methods are appropriate for measuring change between two groups or between two time periods, they are less appropriate for measuring change over multiple times and between groups. The repeated measurement between times, and then separately between groups, could result in showing a positive change when there was none or not showing a positive change when there was one.

Low sample size

The participation rate between time periods and between group comparisons was quite low. For between-time comparisons, program participation dropped quite dramatically from 147 to 109 at 6 months, and then to 75 at 12 months, 49 at 18 months, 24 at 24 months and less than 12 at 30 months. For control group participants, the participation rate was even greater, dropping by almost half at 6 months and to just a few cases (less than 8) at 12 months.

Unequivalent groups

Results of Independent t-tests comparing the performance of control group participants on key matching characteristics show that the two groups were significantly different from each other at baseline, and therefore non-equivalent.

Instrumentation

YAAGV staff administered the questionnaires with young people because many of the participants had low levels of literacy. This may have introduced bias into the surveys as a result of the participants attempting to show themselves in the best possible light or where they were reluctant to provide honest answers.

Low reliability of the measures

A reliability analysis, performed on survey results used to measure outcomes, revealed that some of the scales had low-(Violent Crime scale)-to- moderate reliability (Non-violent Crime, Attachment to Teachers, Total Risk scales).

Maturation

As the study did not control for age or include analyses by age group, it is difficult to determine with confidence the impact of the program on certain age groups. Some of the participants were quite young when they started the program and had gone through a natural life maturation process by the time the evaluation ended. The older age category also went through some changes, becoming fathers or mothers, which may have added to their decision to exit the gang.

Low reliability of measurement

Characteristics of gang involved members were not analyzed separately from those of high-risk youth. It is possible that the differences between the two were significant enough to produce mixed results.

5. Lessons Learned and Recommendations

The evaluation produced many recommendations to improve implementation of similar interventions in future and to inform future program evaluations.

Program Delivery

Holistic youth services

Gang-involved and high-risk youth often have needs that far exceed those of a usual crime prevention program. Gang projects serving youth are most effective when nested within a larger youth-serving organization and/or within a community-based spectrum of services. Food, clothing, shelter, medical care, schooling, employment, leisure and specialized health problems also need to be considered as part of the support provided to youth wanting to exit gangs or youth at high risk of joining a gang.

Informed written consent

Partnerships were developed with local health, counselling, shelter, recreational, artistic, employment and school programs to better address the complex needs of youth. It is important that case-related information be shared with relevant professionals in these organizations, and do so in a manner that respects the privacy and confidentiality of participants. A written consent process was put in place, and proved useful to ensure a safe exchange of case-specific information.

Partnerships with schools

Developing solid relationships with schools was a key success factor for this project. School principals regularly provided project staff with referrals of students who were experiencing difficulties and would benefit from intervention. This process allowed project staff to prioritize their work and ensured they knew which youth needed immediate assistance. Principals and teachers were knowledgeable about youth gang issues and were able to collaborate with YAAG program staff on assessment and ongoing therapeutic work with participants.

Life skills training as a way to reach youth

Life skills training sessions, such as teaching youth to cook, sew or plan their day, provided opportunities for YAAGV staff to initiate in-depth discussions with youth who were reluctant to share personal information in more traditional counselling settings. This, in turn, led to a better understanding of the complexity of needs presented by individual youth. Program staff also reported that learning life skills creates resiliency in youth, particularly when they are able to put into action newly learned skills when faced with difficult situations.

Engagement of high-risk parents into the program activities

Engaging parents of gang-involved or high-risk youth can be challenging. Some successes were achieved by having program staff make home visits at the start of the school year to meet with the parents or guardians of each program participant. Their purpose was to engage family in school life, and open the door to regular communication. Inviting parents to family activities, such as holiday feasts, was also a successful method to engage parents or guardians.

Gender responsiveness

Implement separate therapeutic programs for girls and young women as many have experienced sexual violence by men. It was hard for the young women to relate to male staff.

Opportunities for meaningful youth engagement

The organization provided multiple opportunities for youth to get engaged in the development and design of the program, such as sitting on Advisory Committees, becoming peer mentors, making presentations to youth in schools and at other youth gathering events. This encouraged youth to become more engaged in program activities and have greater ownership of service solutions developed for them.

Relationships with local police

For a variety of reasons, developing a solid working relationship with the local police was difficult. Given the serious nature of criminal activity engaged in by many gang-involved youth, developing partnerships with police early into the project are critical for the success of gang projects.

Evaluation

Create an evaluation culture

Service agency staff often has minimal experience with evaluation, which can present challenges. Evaluators should engage staff in evaluation activities and evaluation products from the start.

Engage youth

Young people need to know that their voices are important and that their ideas and concerns will be addressed. Informing youth of their rights as well as the progress and outcomes of evaluation may be an effective way to keep them engaged in evaluation. Engaging members of the target group in the evaluation activities and products (e.g., review of survey instruments, data collection strategies) may help ensure greater validity and reliability in the findings.

Consider gender issues

Evaluation tools should be responsive to women's needs. Risk assessments and survey questions should address issues related to care of children, involvement in the sex trade, and depression.

Adapt to cultural characteristics

To ensure that evaluation methodology and tools are reflective of First Nations and Métis cultures, standardized instruments need to be adapted to the characteristics of that audience, and their construct validity ensured. It cannot be assumed that standardized instruments adopted from other programs and implemented in different settings and for different populations will work in the Canadian context, and more specifically, the Aboriginal context.

Triangulate qualitative and quantitative data

Purely quantitative data cannot capture the complexity of issues like gang involvement, the sex trade, mental health and culture. Quantitative methods should be supplemented with qualitative methods. Case study designs have been found to be particularly effective when conducting research and evaluation studies in Aboriginal contexts.

6. Conclusion

YAAGV was successful in reaching its targeted population and in helping youth exit gangs or resist involvement with gangs. All youth who had completed the program had exited gangs or resisted involvement with gangs.

YAAGV had some success impacting the risk factors and behaviors of participating youth. In areas, such as depression levels and attitudes toward gangs, positive changes were observed but these were sporadic and limited to specific follow-up periods.

Similarly, a 60% significant decrease in substance abuse was observed for a small number of participants. The evaluation also found some reduction in participants' involvement in non-violent and violent crimes, however findings were inconsistent. Most notably, consistent positive results were observed in increasing youth's attachment to the labour force and exiting youth from gangs.

For more information or to receive a copy of the final evaluation report, please contact the National Crime Prevention Centre by e-mail at ps.prevention-prevention.sp@canada.ca.

If you wish to register for the NCPC mailing list to receive information from the Centre, please visit the subscription page at: http://www.publicsafety.gc.ca/cnt/bt/mlng-lst-eng.aspx

Date modified:

2022-07-26