Results of Crime Prevention Programs for 12 to 17 Year Olds

By D. Laliberté, G. Rosario, L. Léonard, D. Smith-Moncrieffe and A. Warner

1. Introduction

One of the fundamental objectives of Public Safety (PS) Canada's crime prevention activities is to assist those responsible for the implementation and delivery of crime prevention in making the best informed decisions. In order to achieve this objective, Public Safety Canada develops and disseminates practice-oriented knowledge on effective and cost-effective preventative interventions. In turn, this is done through rigorous impact evaluation studies of selected community-based prevention projects funded by PS to determine what works, how it works and at what cost (Smith-Moncrieffe, Lauzon and Jobin, 2008).

Through the program funding made available under the National Crime Prevention Strategy (NCPS), Public Safety Canada supports the implementation of community-based projects that respond to specific priorities such as youth gangs, youth violence and bullying. Most of these community-based projects aim to implement interventions that address criminogenic (risk) factors that are known to be related to an increased likelihood of offending, and protective factors that decrease this probability.Footnote 1 These projects aim to test programs that have been evaluated in other countries, but their effectiveness is not known in Canada. The NCPS also evaluates projects that are innovative or promising to better understand what elements of the interventions are evidence-based.Footnote 2

Since 2008, PS has focused most of its efforts on developing knowledge of effective practices in the following domains: early risk factors among at-risk children and youth; youth gangs; recidivism among high risk groups; and prevention in Aboriginal communities. The particular focus on children and youth is explained by well-established knowledge that discernible risk factors can be identified when children enter school, that the earlier the intervention the larger the benefits, and that if nothing is done, a proportion of these children will be at risk of entering into chronic delinquency patterns. Furthermore, delinquency peaks at age 18, and so it is very appropriate that preventative interventions would largely focus on this group.Footnote 3

Using the knowledge base of effective practices developed in other countries, especially in the USA, PS funded crime prevention projects aimed to replicate and evaluate promising and model programs in Canada, focusing on this group of the population. Wherever possible, similar interventions were implemented and evaluated in multiple sites. Overall, since 2010, PS has been conducting 11 evaluation studies of 10 different models implemented for 12 to 17 year olds in 16 different sites across Canada (see names of programs, locations and dates in Table 1). The following programs have been tested: Alternative Suspension; Intervention Rethink Refocus Reintegrate; Prevention Intervention Toronto; Multisystemic Therapy; Programme de suivi intensif de Montréal / Gangs de rue; Youth Inclusion Program; Leadership and Resiliency; Velocity; Life Skills Training; and Towards No Drugs.

Table 1: List of programs evaluated for their impacts on 12 to 17 year olds
Program Location
Programs addressing school-related issues
Alternative Suspension (AS)
2009–2014
Alberta (Edmonton), British Columbia (Chilliwack) and New Brunswick (Moncton)
Intervention Rethink Refocus Reintegrate (iR3)
2007–2012
British Columbia (Surrey)
Programs addressing aggression and violence
Prevention Intervention Toronto (PIT)
2009–2012
Ontario (Toronto)
Multisystemic Therapy (MST)
2009–2014
Ontario (Toronto)
Youth Inclusion Program (YIP-Maritimes)
2010–2014
New Brunswick (Saint-John) and Nova Scotia (North Sydney and Spryfield)
Youth Inclusion Program (YIP-Quebec)
2012–2016
Quebec (Montréal and Salaberry-de-Valleyfield)
Programs addressing personal and social skills
Outdoor adventure programs
Leadership and Resiliency (LRP)
2011–2015
North West Territories (Hay River and Yellowknife)
Velocity
2009–2014
Newfoundland and Labrador (St. John's)
Social skills focus
Life Skills Training (LST)
2010–2013
Alberta (Edmonton)
Programme de suivi intensif de Montréal /
Gangs de rue
(PSI-MTL/GDR)
2009–2014
Quebec (Montréal)
Substance abuse programs
Towards No Drugs (TND)
2009–2014
Ontario (Hamilton)

This paper presents preliminary findings from 9 of these 11 impact evaluation studies.Footnote 4 The following section briefly describes these interventions, including their implementation milieu and their target groups. The third section describes the approach taken by PS to perform the evaluation studies and synthesis of findings. The fourth section presents the preliminary findings, and the last section provides some tentative conclusions.

2. The interventions

The programs tested offered a variety of activities and services for youth and young adults ranging from individual case management, education and employment support, skill building, and recreational activities. Most projects were developed and implemented in communities by voluntary not-for-profit organizations, in collaboration with key crime prevention stakeholders such as schools and local police. These interventions usually target youth between the ages of 12 and 17Footnote 5 who display multiple risk factors associated with criminal behaviour such as substance abuse, limited attachment to school, associating with delinquent peers, violent and aggressive tendencies, and early contact with the justice system. Their risk profile is often complex and many of the youth also have low literacy rates, poor parental supervision, mental health issues, unstable housing, and low academic achievement.

2.1 Programs addressing school-related issues

It is well established that youth who are at-risk of following a delinquent trajectory experience various difficulties in school. These include: difficulty adjusting to school, low attachment to school, poor performance, truancy and systematic suspensions, bullying, etc. Several different types of interventions have been designed, in particular, to address systematic suspensions. Traditional suspensions involve removing a youth from school for a period of time. An abundance of research challenges the effectiveness of this form of suspension and shows that sending a youth to an unsupervised environment may increase disciplinary problems, and in the long run, may even contribute to an unsafe environment in the school and neighbourhood (Costenbader and Samia, 1997; Morrison, Antony, Storino and Dillon, 2001; Atkins, McKay et al., 2002).

PS's National Crime Prevention Centre had already tested an alternative to suspension program in Quebec, showing promising results (CAC International, 2005). According to its 2005 evaluation, Alternative Suspension successfully decreases early school departure and improves attitudes and behaviours at school (conflict and anger management, attendance, suspension, etc.). In 2009, the PS crime prevention program supported and evaluated two alternative-to-suspension programs: the YMCA's Alternative Suspension (AS) program which targets youth suspended or at risk of suspension because of their behaviours, and the Intervention, Rethink, Refocus, Reintegrate (iR3), a program for youth receiving a first time suspension for violence or substance use that is also inspired by the original YMCA's AS program.

At the time of this synthesis, the YMCAs of Quebec have established AS Programs in over 41 Canadian locations. Since 2009, PS has been funding the national expansion phase of the program and has supported the implementation in 16 communities across the country.Footnote 6 During the 2011–2012 school-year, 186 youth (65% male) participated in AS in the three sites that are involved in an impact evaluation. Over 25% of students were then referred to the AS Program because of disruptive and impulsive behaviours (27.2%) and approximately 20% were referred for physical or verbal violence (21.5%), risky behaviours (22.1%), apathy and lack of motivation (17.4%) (R.A. Malatest & Associates Ltd, 2013a).

iR3 is an alternative to the traditional suspension of youth who are found to be engaging in anti-social behaviours at school (Doley and Gagnon, 2012). iR3 targets at-risk youth (grades 6 to 8) who are facing a first-time suspension for aggressive behaviours and/or use of/association with drugs or alcohol. This program includes an individual follow-up, a set of workshops, guest speakers and activities allowing students to reflect on their behaviour and to learn specific knowledge and skills. The program is also designed to create opportunities for youth to engage in pro-social activities and to connect with adult mentors in the program, at their school and within the community. Between 2006–2007 and 2011–2012, 638 students (82% male) were admitted to iR3 in Surrey, BC (Doley and Gagnon, 2012).

Through these programs, suspended students with disciplinary problems are removed to an off-site location and participate in a curriculum designed to address specific at-risk factors and reintegrate them into the school. This includes academic support, thematic workshops, one-on-one meetings, and sports activities. The anticipated outcomes of both programs include improved attitudes toward school and delinquency and a reduction in suspensions and disciplinary actions among program participants.

2.2 Programs addressing aggression and violence

Research on offending trajectories has established that youth who demonstrate patterns of violent behaviour and offending are at elevated risks of long-term and chronic offending trajectories into adulthood. Youth who are in gangs or who have committed more serious offences have been involved in several projects funded under the NCPS.

The Prevention Intervention Toronto (PIT) model includes a carefully chosen set of complementary and integrated components designed to facilitate change among targeted youth (Wortley et al, 2012). This case management program, developed by the City of Toronto to promote youth development, prevent and treat health and behaviour problems among young people, is based on a variety of evidence-based sources supporting gang-intervention and violence-prevention programs.Footnote 7 PIT consists of a needs assessment phase, a group training phase that includes one-on-one counselling and an integration phase that allows youth to meet with their case manager to achieve program goals, in particular, their transition to a pro-social lifestyle. The 306 participants (72% male) who were accepted into PIT were between the ages of 12 and 24 years old. To be accepted into the PIT program, youth had to meet at least one of the five following criteria: 1) self-identification as a former or current gang member; 2) case manager identification as a former or current gang member; 3) met Eurogang criteria;Footnote 8 4) moderate to high risk score; 5) qualitative exemption including family gang involvement, residence in a high-risk community, or the case manager's belief that the youth was lying during the screening, artificially lowering their score (Wortley et al, 2012).

Multisystemic Therapy (MST)(Henggeler et al, 2006) is an intensive, short-term intervention designed for at-risk youth and their families whose intended outcomes include reduced substance use and aggression, improved school performance and positive family relationships. Each family is assigned a case worker who provides regular support and counselling based on the particular needs of the youth and their family. The treatment includes individual, family, peer, school, and community interventions such as parent and skills training (Welsh, 2007: 22). Previous evaluations have shown the program to be effective at reducing a number of risk factors such as substance use and aggressive behaviour, and keeping youth involved in positive activities such as school and employment (Borduin et al., 1995 quoted in Welsh, 2007: 22). As of March 2012, 58 youth aged 12–18 years were accepted into the MST program in Toronto, of whom 44% were at high risk and 41% at moderate risk (Harry Cummings and Associates Inc., 2012). The average age of 40 discharged youth is 15.7 years and 65% of youth are male.

2.3 Programs addressing personal and social skills

Youth who are at risk of delinquency often demonstrate fewer positive social skills, including low empathy, higher aggressiveness, elevated risk-taking behaviour, etc. Several types of interventions address these factors.

Outdoor adventure programs

Outdoor adventure programs are a means of removing youth at risk from their typical environments to provide them with challenges where they can learn new skills and feel positive about their accomplishments. These programs have been shown to have a number of positive outcomes, including reduced recidivism rates, improved social skills, and reduced substance use (West and Crompton, 2001). These types of projects are popular in Canadian communities, and a number of funded projects focus on outdoor learning and adventure. They generally include but are not limited to camps, day outings into wilderness areas and outdoor skill building. This thematic paper covers two programs addressing skills through outdoor activities: the Leadership and Resiliency Program (LRP) and the Velocity Program (originally named Adventure Youth Initiative).

The theory behind the LRP created by Fairfax-Falls Church is to replace risky behaviour with healthy risk taking in the form of outdoor activities, community service learning and a classroom based curriculum (Fishbein and Boylan, 1997 and 1998). LRP is based on three premises: social competence fosters resiliency against substance use and violent behaviour; positive social influences facilitate the establishment of pro-social behavioural norms; and associations with positive role models and peers protect adolescents from negative influences. LRP is being implemented in multiple schools in two sites in the NWT. During the 2011–2012 school year, most LRP youth were between 13 to 18 years old, evenly distributed by gender, and the majority were Aboriginal. Most of the youth had multiple risk factors such as substance use and behavioural issues (R.A. Malatest & Associates Ltd, 2013b).

The Velocity Adventure Program (Velocity) is an adventure-based program for youth who are at risk of, or have already been involved in, criminal activity (Public Safety, 2013a). Based on community consultations and by the design of the Community Youth Network in St. John's (Newfoundland and Labrador), a pilot project was implemented to fill a gap in services and to enhance the engagement of youth age 13 and above, since this group was not receiving appropriate support. The project combines evidence from effective models and practices involving structured outdoor adventure activities, life skills training and mentorship (Wilson, S.J. and Lipsey, 2000; Tarolla, Wagner, Rabinowitz and Tubman, 2002). The Velocity project aims at reducing aggressive and violent behaviour, substance use, lack of attachment to school, and offending among at-risk youth, most of them having had contact with the criminal justice system. The program includes group work, a seven-day wilderness camp and day-long wilderness outings and drop-in activities.

During these activities, youth are provided with individual support, referrals to other community services, and are encouraged to make healthy life choices. Velocity was funded by PS from 2009 through 2014. The project served 71 participant youth during the 3 year evaluation, of which 63% were boys between 13–17 years of age. The program fell short of its initial goal to deliver Velocity to 90 youth over 36 months. The evaluation of Velocity covers three cohorts with each cohort participating for about one year in the program (Ference Weicker & Co., 2013).

Social skills focus

The Youth Inclusion Program (YIP), originally developed by the Youth Justice Board in the UK, is expected to reduce crime and antisocial behaviours by helping youth acquire new skills, take part in social activities and get help with their studies. It includes one-on-one sessions that address individual needs along with group activities, which can include academic support, recreational activities, mentoring, and life skills training (Mackie, Hubbard and Burrows, 2008; Mackie, Burrows and Hubbard, 2003). The YIP has been implemented in the Maritimes, Québec and British Columbia (Table 1). The three YIP-Maritimes projects covered by this synthesis have admitted 217 participants of whom 55% are male (NRG Research Group, 2013).

The Life Skills Training (LST) program was designed for a school setting and aims to enhance personal competence, basic life skills and skills related to resistance to social influences that promote substance use (EPIS Center, 2013). The program draws on strength-based approaches to working with youth, by reinforcing functional behaviour and emphasizing building skills that can be utilized in all areas of a youth's life. It includes workshops and curricula targeting specific risk factors and has been tested in several studies (Botvin and Griffin, 2005; MacKillop, Ryabchenko and Lisman, 2006).

The LST program uses these activities to reach its objectives of decreasing drug abuse risks by reducing personal motivation to use drugs and reducing the susceptibility of youth to social factors that promote drug use. The LST program in Alberta, which took place after school in an at-risk neighbourhood school, ran from 2010 through 2013. It reached approximately 22 youth per year for a total of 87 youth of which 26% were male (latter two cohorts), with 37% being Aboriginal. The youth were in grades 4 through 9 (ages 10–16) and adaptations were made to the LST curriculum to tailor it to the target population (adapting from written to verbal tasks, incorporating cultural elements, etc.) (R.A. Malatest & Associates Ltd, 2013c).

Substance abuse programs

As a result of early initiation or addictions, substance abuse is a well-established factor in long-term offending trajectories.

Towards No Drugs (TND) is based on a motivation-skills-decision-making model developed by the Institute for Health Promotion and Disease Prevention Research, University of Southern California (Sussman, Dent and Stacy, 2002). The program consists of interactive classroom sessions and aims to address issues such as substance use and delinquency by strengthening decision-making and motivation to change. The program was implemented in two schools and in a community where substance abuse, specifically marijuana and alcohol use were disproportionately high. The target group in the community was at highest risk and displayed multiple risk factors that put them at high risk for criminal involvement in the justice system. The expected outcomes of the program included a reduction in substance use and criminal behaviour. The 1623 participants in TND were 53% female and 10% were high-risk, 22% medium risk and 68% low risk (Goss Gilroy Inc. Management Consultants, 2013).

3. Evaluation approach

3.1 Conducting evaluations

To measure program impacts in such a way as to rule out alternative explanations and determine if the measured intervention contributed to the positive changes in the outcomes of interest, rigorous designs, such as quasi-experimental designs, are encouraged. The challenges of experimental designs that incorporate some form of comparison are well documented (cost, recruitment and retention, attrition, etc.) and it is not feasible to expect that every evaluation will be able to reach this level of rigour. As a minimum, pre and post program tests with follow-up assessments up to six months post program have been required when comparison groups are not possible. In addition, evaluators are encouraged to utilize qualitative techniques to strengthen the findings (case studies, in-depth interviews).

Collecting data for an evaluation is both critical and challenging. All of the evaluations draw on multiple sources and data collection methods to assess the impact of the program. Research methods include individual-level risk assessments, questionnaire-based surveys administered by evaluators or program staff, focus groups, semi-structured interviews, case files, secondary data from schools and the police. The evaluators were encouraged to use standardized and validated data collection tools as much as possible. Evaluation firms contracted by the Department have used the most rigorous design that was available, given the specific circumstances of the evaluated programs (Table 2).

Table 2: Evaluation design, referral process and availability of secondary data
Program Pre/Post and follow-up measures Experimental group/Referral process Matched comparison group Secondary data
AS Pre-post and 4–6 weeks and end of school year follow-ups Students referred by schools at 3 project service centers Students referred who did not participate in AS or quit ASFootnote 9 Project and schools
iR3 Pre-post Students referred by participating schools across the School Board Provincial school data matched to youth in the program Provincial school data verified by individual school suspension data
LRP Pre/Post 4 schools (2 sites) Other school Project and schools
LST Pre-Post After-school program From a similar agency and neighbourhood Project
MST Pre-post and follow-ups at 6 and 12 months post-program Referred by the district school board, community organizations and parents Youth who did not complete the program were used Project
PIT Pre-post and follow-up at 12 months post program Youth referred from a variety of sources including schools, community organizations, youth workers and parents Comparison group used from a comparable high-risk neighbourhood located in North West Toronto Project, schools and police
TND Pre-post and follow-up at 12 months post program School-based program (42 schools) Several classes in the same schools were used as a comparison group Project and schools
Velocity Pre-post and follow-up at 12 months post program Participants referred by community agencies and partners. Referral forms completed for each youth. Comparison group not matched for risk factors Project and police
YIP-Maritimes Pre-post and follow-up at 12 months post program Students referred to 3 community organizations by schools. NA Project, schools and police

Notes:

  1. Secondary data is generally used to validate self-reported information. Unlike primary data, secondary data is not directly collected by the evaluators.
  2. At this stage, not all evaluations are able to determine the level of significance and the effect size with accuracy.

3.2 Synthesizing the evaluation studies for this reportFootnote 10

The studies needed to satisfy the following criteria in order to be included in this report: 1) availability of at least some quantitative information (descriptive or inferential statistics); 2) measures related to key criminogenic related outcomes; 3) the study meets at least the basic evaluation design; 4) the study has at least some interim data; and 5) the study includes youth aged 12–17.

A descriptive analysis was conducted for the domains related to knowledge, attitudes, risk factors, protective factors and behaviours. Each table provides the following key information: 1) name of the study; 2) description of the measure examined; 3) sample size of the experimental and comparison group (if any); 4) type of change in the outcome being measured; and 5) the design and/or source of data. Appendix 1 provides summary tables with the detailed results.

The results in these studies are based on a variety of statistical tests used to convey the level of impact the program had on the variables being measured. These statistics include percentages and various coefficients that were convertedFootnote 11 into four categories so that the reader can interpret the results.Footnote 12 The definitions are as follows:

  1. A positive change is defined as a result that demonstrates a statistically significant (p<0.05) reduction in risk factors or offending behaviours or an increase in positive knowledge, attitudes or protective factors. When there was a trend demonstrating more frequently occurring favourable outcomes, the study was labelled "positive". For example, a project may have six different types of outcomes measuring the risks related to aggression, as it is good practice to use a variety of measures to confidently determine if changes are occurring in the outcomes of interest. If four outcomes were positive, one demonstrated no change, and the final one showed a negative change, the trends would be labelled favourable overall.

  2. A negative change is defined as a result that demonstrates that there is a statistically significant (p<0.05) increase in risk factors or offending behaviours or it demonstrates a decrease in positive knowledge, attitudes or protective factors. For example, a project may have four different types of outcomes measuring changes in attitudes. If three of the four outcomes demonstrate a negative result, this outcome related to attitudes was labelled as a "negative" trend.

  3. A "neutral" finding is defined as a result that does not demonstrate a statistically significant (p<0.05) change in the outcomes tested prior to and after the program. For studies that report descriptive frequencies, a "neutral finding" indicates that the frequency distribution prior to and after the program is similar. For example, a project can use five different outcomes to measure changes for reducing non-violent offending. If three of the five outcomes demonstrate no change, the trend would be labelled "neutral". A related footnote is generally provided if this trend was noted in the initial pre and post test period or whether it was identified in the post-program follow-up period. When a "neutral" trend is identified in the post program period (six months after the program is complete), it is possible that the immediate post program trends are positive and these changes have been maintained up to six months after the program. In this scenario, the "neutral" trend is not necessarily negative, as this means the initial positive trends have been maintained.

  4. Findings are classified as "mixed" when there is an equal number of negative and positive outcomes, an equal number of neutral and negative outcomes, or an equal number of neutral and positive outcomes.

3.3 Limitations of the studies

There are a few limitations that should be considered when interpreting the data in this synthesis report. Since youth are in the program for approximately six months, we usually assess several groups prior to making conclusions. However, since the projects may have been on the ground for a limited time period and the evaluation studies for an even shorter time period, data may currently be available for only a few groups. In this report, many of the evaluations have half of the groups, requiring further replications prior to making strong conclusions about program efficacy.

The external validity of the findings may differ between studies due to the variety of evaluation methods used. For example, comparison groups may not always be equally strong. For some indicators, between-group comparisons were not feasible at the time when this paper was written. Greater weight was not applied to findings derived from more rigorous evaluations, indicating that the findings were treated equally when the trends were being assessed. Since weightings were not used for this synthesis, interpretations about what models are more effective should be delayed until weightings are applied to the final evaluation results.

Finally, not all of the evaluations are at the same stage. For example, the PIT evaluation is fully complete while other evaluations included in this report will not be completed until 2015. This variation in timing means that trends have often consolidated evaluations that have pre and post-test measures (T1 and T2) with evaluations that have completed pre, post and post program measures up to six months and in some cases one year (T1, T2, T3 and in some cases T4). We know in the evidence-based literature that immediate post program measures and long term maintenance measures naturally differ so the findings need to take into consideration when the measurements were assessed in relation to the intervention.

4. Findings

It is generally accepted among both researchers and practitioners that change will be incremental over the course of the intervention. Prior to reaching behavioural changes such as reduced criminal and delinquent behaviour, it is anticipated that youth will gain knowledge and improve attitudes as a first step toward changing behaviour. As youth gain knowledge about and change attitudes toward risky behaviour, they will begin to address behavioural risk factors that will eventually lead to changes in criminal and delinquent behaviour.

4.1 Knowledge and attitudes

Seven of the evaluation studies in this synthesis report measure knowledge and attitudes with a majority (71%) of the results showing positive changes for the youth in these various interventions (iR3 show neutral results and LST mixed results). Table 3 indicates that a variety of knowledge and attitudes have been changed. These include: 1) being motivated to change; 2) having a better understanding about the negative consequences of substance abuse; 3) having a positive attitude towards the justice system; and 4) having an appropriate attitude towards offending.

Prochaska and DiClemente's (1983) model suggests that participants potentially move out of the pre-contemplation (not ready to change) and contemplation phase (thinking about change) when their attitudes begin to shift from negative to positive thinking. Reduced risk factors and increased protective factors are more likely to be observed among youth who have successfully absorbed new knowledge and changed their attitudes.

Table 3: Findings – Changes in knowledge and attitudes
Program Description of variables Changes/Trends
AS Thought given to situation at school and willingness to improve behaviour Favourable
iR3 Attitudes toward violence and school Neutral
LST Cultural knowledge Neutral
Understanding effects of substance use Favourable
PIT Attitude towards gangs, and the justice system Favourable
TND Knowledge about use of drugs and consequences Favourable
Velocity Attitude towards schooling Favourable
YIP Motivation to change Favourable
Thinking and behaviour
Attitudes to offending

Notes:

  1. Preliminary results except for iR3 and PIT. Detailed results and sample size are included in appendix Table A-1.
  2. The denominator for this table is 7 studies as LST has two measures that were merged according to the definition provided in section 3.2.

4.2 Risk and protective factors

Eight of the eleven evaluation studies in this synthesis report measure risks and protective factors with a majority of them showing positive changes for several variables related to seven domains. As indicated in Table 4, 57% of the domains show favourable changes (Skills, Education/Academic performance, Parenting/Family relationships, Externalizing behaviours/Pro-social behaviours), 29% show neutral results (Self-esteem/Emotional regulation, Alcohol and Drug abuse) and 14% show mixed results (Anti-social peers/Gang membership).

1) Skills: TND and Velocity (out of three evaluation studies) report favourable changes in decision making skills or the ability to handle and reduce substance use. The LST program has had neutral trends on the skills variables measured.

2) Anti-social/Peers/Gang membership: Two of the three studies (PIT and YIP) report mixed trends for this domain. The YIP program addresses lifestyle (which is favourable), neighbourhood and friends (neutral). The PIT program has had favourable changes associated with anti-social peers, but mixed results with regard to the association with gang-involved peers. The MST study reported favourable changes at the time of discharge.

3) Education/Academic performance: Four of the studies (AS, iR3 , MST and YIP) demonstrate favourable trends for behaviours related to school and education. Key variables are related to performance, course completion, attendance, behaviour, disciplinary actions, absenteeism and suspension. However, PIT has mixed results for school attendance and disciplinary problems.

Table 4: Findings – Risk and protective factors
Program/Domain Description of variables Change/
Trend
Skills
LST Decision making skills Neutral
Drug refusal skills
TND Decision making skills Favourable
Velocity Ability to handle substance use problems and motivation to reduce substance use Favourable
Anti-social peers/Gang membership
MST Youth involved with pro-social peers & activities (at time of discharge) Favourable
PIT Association with gang-involved peers Mixed
Association with anti-social peers Favourable
YIP Lifestyle Favourable
Neighbourhood and friends Neutral
Education/Academic performance
AS Performance at school Favourable
% courses completed
Class attendance
Behaviour at school
Disciplinary action at school
Subsequent school suspension
iR3 Disciplinary action Favourable
Suspensions
MST Youth Success in academic/vocational setting (at program discharge) Favourable
PIT School attendance, Disciplinary problems Mixed
YIP School and education Favourable
Performance at school
Absenteeism
Self-esteem/Emotional regulation
LST Self-image Favourable
Coping skills Neutral
PIT Risk seeking Neutral
Velocity Ability to handle stress Neutral
YIP Perception of self and others Favourable
Emotional and mental risk factors
Parenting/Family relationships
MST Risk of out of home placement/improved family relations, parenting skills, and family social supports Favourable
PIT Family relationships Neutral
YIP Family and personal relationships Favourable
Externalizing behaviours / Pro-social behaviours
Velocity Aggressive and anti-social behaviours Favourable
Alcohol and Drug abuse
PIT Alcohol Neutral
TND Cocaine, Marijuana, Prescription drugs, Alcohol Neutral
Velocity Alcohol and Drugs Neutral
YIP Substance use Neutral

Notes:

  1. Preliminary results except for iR3 and PIT. Detailed results and sample size are included in appendix Table A-2.
  2. The denominator for this table is 7 domains as a result of synthesizing variables based on predetermined categories. In the case of the domain for Anti-social peers and Gang membership, PIT and YIP were categorized as mixed based on the definitions provided in section 3.2. For the domain Self-esteem and Emotional regulation, LST was classified as mixed based on the definitions provided in section 3.2.

4) Self-esteem/Emotional regulation: The YIP program has had a positive impact (perception of self and others, emotional and mental risk factors) whereas two other studies report neutral results (risk seeking for PIT and ability to handle stress for Velocity). LST has mixed results, being favourable on self-image but neutral for coping skills.

5) Parenting/Family relationships: Two (MST and YIP) of the evaluation studies that measure parenting and family relationships report favourable changes whereas PIT is rather neutral.

6) Externalizing behaviours/Pro-social behaviours: Impacts of Velocity on aggressive and anti-social behaviours are favourable.

7) Alcohol and drug abuse: For all of the programs (PIT, TND, YIP and Velocity), results on substance use are neutral.

4.3 Behaviours related to crime

The impact evaluations have also measured changes in behaviours related to crime (Table 5) and found that three programs (MST, Velocity and YIP) have had a favourable impact on youth arrests or police contacts, whereas TND had a mixed impact (weapon carrying) and PIT has been neutral (arrests, non-violent offending and criminal victimizations).

Table 5: Findings – Behaviours related to crime
Program Description of variables Change/Trend
MST Youth not arrested for an offence committed during MST (3 months after intake) Favourable (at 3 months post-intake)
PIT Arrests, Non-violent offending, Criminal victimizations Neutral
TND Weapon carrying Mixed
Velocity Police contacts Favourable
YIP Police contacts Favourable

Note: Preliminary results except for PIT. Detailed results and sample size are included in appendix Table A-3.

5. Preliminary conclusions

Despite the challenges of conducting impact evaluations in a community setting, all of the evaluations discussed in this report have produced data to assess the impact on key factors and contribute to knowledge about effective programming. The results so far are able to identify some programs that appear to work, and others that may need some adjustment to meet the needs of youth and increase the likelihood of achieving positive results in the community.

The initial expectation was that in the short term, youth participants would increase knowledge and attitudes related to crime and delinquency as a first step toward changing behaviour. Several programs demonstrate positive change across a number of measures, including improvements in knowledge and attitudes toward substance use, school and violent/ aggressive behaviour.

The most positive impact of the programs is on youths' behaviour. Most of the interventions showed a decrease in at least one of the behaviours that led to the youth being referred to the program. Particularly positive was the impact of the program on reductions in police contacts. All of the evaluations that were able to collect data showed an improvement among the program participants. As such, these project evaluations provide useful information about what results local organizations, schools and youth-serving agencies are achieving in implementing promising and model crime prevention programs.

Appendix

Table A-1: Detailed findings for crime prevention programs measuring knowledge and attitudes
Program Description of variables Sample size Results Changes/
Trends
Design/Source
AS Thought given to situation at school and willingness to improve behaviour E=119 Serious or some thought and willingness to change: 72.4%
Little thought and willingness to change: 23.3%
No thought and willingness to change: 4.3%
Favourable Post-questionnaire filled by youth worker
iR3 Attitudes toward school/teacher E=122 No change Neutral Pre and post measures
Attitudes toward substance use E=122 No change Neutral Pre and post measures
LST Cultural knowledge E=87
C=27
p>0.05 Neutral Pre/post measures
Understanding effects of substance use E=87
C=27
r=-0.65
p=.02
Favourable Pre/post measures
PIT Attitude towards gangs and the justice system E=188
C=99
F coefficients range from .587 to 6.546
P<0.05
Favourable Matched comparison group with a 6 month post program follow-up
TND Knowledge about use of drugs and consequences E=868
C=59
F=112.61
P<0.05
Favourable Matched comparison group with a 12 month follow-up
Velocity Attitude towards schooling E=71
C=45
P<0.05 compared to pre-test Favourable Pre and post measures with follow-up
YIP Motivation to change E=41 Favourable: 46%
Neutral: 44%
Unfavourable: 10%
Favourable Pre and post measures
Thinking and behaviour E=41 Favourable: 61%
Neutral: 20%
Unfavourable: 20%
Favourable Pre and post measures
Attitudes to offending E=41 Favourable: 51%
Neutral: 39%
Unfavourable: 10%
Favourable Pre and post measures

Note: Preliminary results except for iR3 and PIT. E= experimental group and C= control group

Table A-2: Detailed findings for crime prevention programs measuring changes in risks and protective factors
Program Description of variables Sample size Results Changes/
Trends
Design/Source
Skills
LST Decision making skills E=87
C=27
p>0.05 Neutral Pre/post measures
Drug refusal skills E=87
C=27
p=0.28
r=-0.41
Neutral Pre/post measures
TND Decision making skills E=868
C=59
F= 4.88
P<0.05
Favourable Matched comparison group
Velocity Ability to handle substance use problems and motivation to reduce substance use E=71
F=45
P<0.05 compared to pre-test Favourable Pre and post measures with follow-up
Anti-social peers/Gang membership
MST Youth involved with pro-social peers & activities (at time of discharge) E=28
C=12
71.4% of the treatment group vs. 25% of the control group Favourable Single group repeated design
PIT Association with gang-involved peers E=76
C=43
F ranges from 0.461 to 5.761
P<0.05
Mixed (similar declines observed among the experimental and the comparison group) Matched comparison group
Association with anti-social peers E=76
C=43
F ranges from 0.461 to 5.761
P<0.05
Favourable Pre/post measures with comparison group
YIP Lifestyle E=41 Favourable: 68.29%
Neutral: 24.39%
Unfavourable: 7.32%
Favourable Pre and post measures
Neighbourhood and friends E=41 Favourable: 32%
Neutral: 66%
Unfavourable: 2%
Neutral Pre and post measures
Education/Academic performance
AS Performance at school E=106 Improvement for 89% of cases Favourable Follow-up with schools 5 weeks post-end of school year
Percentage of courses completed E=164 78% youth completed all courses Favourable School data
Class attendance E=98 On average youth attended 95% of their classes and 36% had perfect attendance Favourable School data
Behaviour at school E=111 Improvement: 58.6% of youth Favourable Follow-up with schools at end of school year
Disciplinary action at school E=111 Decrease: 59.5% of youth Favourable Follow-up with schools at end of school year
Subsequent school suspension E=121 53% of youth have not had subsequent suspension Favourable Follow-up at end of school year
iR3 Disciplinary action E=126
C=126
18% disciplinary action compared to 43% in comparison group Favourable Matched comparison group
Suspensions E=122
C=122
22.6% suspended compared to 74% in comparison group Favourable Matched comparison group
MST Youth success in academic/ vocational setting E=28
C=12
67.9% of treatment group participants vs. 33.3% of control group participants showed evidence of academic success at program discharge Favourable Single group repeated design
PIT School attendance, Disciplinary problems E=76
C=43
F ranges from .587 to 3.5558 Mixed Matched comparison group
YIP School and education E=41 Improvement: 61%
Neutral: 27%
Unfavourable: 12%
Favourable Pre-post measures
Performance at school E=29 Increase in GPA for 76% of cases Favourable School data, pre-program
Absenteeism E=37 57% of youth have a reduced number of school days missed Favourable School data
Self-esteem/Emotional regulation
LST Self-image E=87
C=27
r=-.44 Favourable Mixed design with pre/post measures
Coping skills E=87
C=27
r=-.37 Neutral Mixed design with pre/post measures
PIT Risk Seeking Scale E=76
C=43
F=2.226

P=0.110
Neutral Matched comparison group
Velocity Ability to handle stress E=71
C=45
p>0.05 Neutral Pre and post measures with follow-up
YIP Perception of self and others E=41 Favourable: 51%
Neutral: 32%
Unfavourable: 17%
Favourable Pre-post measures
Emotional and mental risk factors E=41 Favourable: 63%
Neutral: 29%
Unfavourable: 7%
Favourable Pre-post measures
Parenting/Family relationships
MST Risk of out of home placement/improved family relations, parenting skills, and family social supports E=28
C=12
Experimental group ranges were 25%-88% vs. control group ranging from 16.1 to 60% Favourable Single group repeated design
PIT Family relationships E=75
C=43
F=.519
P=0.551
Neutral Matched comparison group
YIP Family and personal relationships E=41 Favourable: 63%
Neutral: 32%
Unfavourable: 5%
Favourable Pre and post measures
Externalizing behaviours and Pro-social behaviours
Velocity Aggressive and anti-social behaviours E=71
C=45
Favourable: 41%
Neutral: 20%
Unfavourable: 39%
Favourable Pre and post measures with follow-up
Alcohol and Drug abuse
PIT Alcohol E=76
C=43
F=1.345
P=.263
Neutral Matched comparison group
TND Cocaine, Marijuana, Prescription drugs, Alcohol E=868
C=59
F ranges from 0.22 to 0.86
P>0.05
Neutral Matched comparison group
Velocity Alcohol and drugs E=71
C=45
Favourable: 28%
Neutral: 59%
Unfavourable: 13%
Neutral Pre and post measures with follow-up
YIP Substance use E=41 Favourable: 32%
Neutral: 58%
Unfavourable: 10%
Neutral Pre and post measures

Note: Preliminary results except for PIT. E= experimental group and C= control group

Table A-3: Detailed findings for crime prevention programs measuring changes in contact with the justice system (behaviours)

Program
Description of variables Sample size Results Trends Design/Source
PIT Arrests, Non-violent offending, Criminal victimizations E=76
C=41
F ranges from 0.291 to .761 Neutral Matched Comparison Group
MST Youth not arrested for an offence committed during MST (3 months after intake) E=28
C=12
89.3% of the treatment group vs. 66.7% not arrested for an offence committed during MST (3 months after intake) Favourable (at 3 months post-intake) Single group repeated design
TND Weapon carrying E=847
C=54
F ranges from 0.07 to 0.63 Mixed Matched Comparison Group
Velocity Police contacts E=71
C=45
49% overall reduction in participant police contact Favourable Pre and post measures with follow-up
YIP Police contacts E=50 68% youth with no suspect/charge during post period Favourable Police post-data

Note: Preliminary results except for PIT. E= experimental group and C= control group

References

End Notes

1 For more information, consult PS publications (Yessine, 2011; Public Safety Canada, 2008) and PS web site for a more thorough description of the risk and protective factors associated with crime and delinquency.

2 Organizations such as Substance Abuse and Mental Health Services Administration (SAMHSA) and Blueprints for Healthy Youth Development recommend intervention programs that have been evaluated with this level of rigour. These are model and promising programs that have already been evaluated and shown to be effective in preventing and reducing crime.

3 The current thematic paper focuses on 12 to 17 year olds whereas another upcoming paper will target 6 to 11 year olds.

4 Evaluation results of PSI-MTL/GDR, YIP-Québec and LRP are not available at this time.

5 The majority of funded projects worked with youth in this age range although a few accepted individuals up to the age of 30.

6 There are currently 16 service sites in the following communities and locations: British-Columbia (Abbotsford, Chilliwack and Surrey), Alberta (North Edmonton and South Edmonton), Saskatchewan (2 sites in Regina and 1 in Moose Jaw), Manitoba (Winnipeg and Seven Oaks), New Brunswick (Moncton), Nova Scotia (Dartmouth and Glace Bay), Newfoundland and Labrador (St. John's, Corner Brook and Grand Falls-Windsor).

7 Social Development Research Group:
http://www.sdrg.org; Breaking the Cycle: http://www.canadiantraininginstitute.com/breaking-the-cycle/; “Healthy Relationships Curriculum” developed by Men for Change: http://www.m4c.ns.ca/index.html; Oakland Men's Project: http://www.paulkivel.com/resources/articles/item/70-the-oaklands-men-project

8 According to the Eurogang criteria, youth belong to a gang or troublesome group of friends if they regularly hang out with a group of three or more people; AND they have been part of this group for more than one month; AND their group has engaged in at least one illegal activity; AND their group engaged in illegal activities on two or more occasions (Wortley et al, 2012).

9 Only AS participants that quit very early in the program were considered for inclusion in the comparison group.

10 For more details on methodology and interim findings, consult Public Safety Canada publications (2012, 2013a, b, c, d, e and f) and Laliberté (2013).

11 To ensure this paper could be used for a lay audience, the data was converted to user friendly classifications. However, we could have used the Comprehensive Meta-Analysis Software (CMA) that would have converted to effect sizes all the percentages, F and d coefficients to ensure the data was standardized. This is another valid method to synthesize different types of data to make strong conclusions about trends in the data.

12 Smith-Moncrieffe (2013) for a description of innovative synthesis methodologies applied at Public Safety Canada.

Date modified: