Results of Crime Prevention Programs for 12 to 17 Year Olds
By D. Laliberté, G. Rosario, L. Léonard, D. Smith-Moncrieffe and A. Warner
1. Introduction
One of the fundamental objectives of Public Safety (PS) Canada's crime prevention activities is to assist those responsible for the implementation and delivery of crime prevention in making the best informed decisions. In order to achieve this objective, Public Safety Canada develops and disseminates practice-oriented knowledge on effective and cost-effective preventative interventions. In turn, this is done through rigorous impact evaluation studies of selected community-based prevention projects funded by PS to determine what works, how it works and at what cost (Smith-Moncrieffe, Lauzon and Jobin, 2008).
Through the program funding made available under the National Crime Prevention Strategy (NCPS), Public Safety Canada supports the implementation of community-based projects that respond to specific priorities such as youth gangs, youth violence and bullying. Most of these community-based projects aim to implement interventions that address criminogenic (risk) factors that are known to be related to an increased likelihood of offending, and protective factors that decrease this probability.Footnote 1 These projects aim to test programs that have been evaluated in other countries, but their effectiveness is not known in Canada. The NCPS also evaluates projects that are innovative or promising to better understand what elements of the interventions are evidence-based.Footnote 2
Since 2008, PS has focused most of its efforts on developing knowledge of effective practices in the following domains: early risk factors among at-risk children and youth; youth gangs; recidivism among high risk groups; and prevention in Aboriginal communities. The particular focus on children and youth is explained by well-established knowledge that discernible risk factors can be identified when children enter school, that the earlier the intervention the larger the benefits, and that if nothing is done, a proportion of these children will be at risk of entering into chronic delinquency patterns. Furthermore, delinquency peaks at age 18, and so it is very appropriate that preventative interventions would largely focus on this group.Footnote 3
Using the knowledge base of effective practices developed in other countries, especially in the USA, PS funded crime prevention projects aimed to replicate and evaluate promising and model programs in Canada, focusing on this group of the population. Wherever possible, similar interventions were implemented and evaluated in multiple sites. Overall, since 2010, PS has been conducting 11 evaluation studies of 10 different models implemented for 12 to 17 year olds in 16 different sites across Canada (see names of programs, locations and dates in Table 1). The following programs have been tested: Alternative Suspension; Intervention Rethink Refocus Reintegrate; Prevention Intervention Toronto; Multisystemic Therapy; Programme de suivi intensif de Montréal / Gangs de rue; Youth Inclusion Program; Leadership and Resiliency; Velocity; Life Skills Training; and Towards No Drugs.
Program | Location |
---|---|
Programs addressing school-related issues | |
Alternative Suspension (AS) 2009–2014 |
Alberta (Edmonton), British Columbia (Chilliwack) and New Brunswick (Moncton) |
Intervention Rethink Refocus Reintegrate (iR3) 2007–2012 |
British Columbia (Surrey) |
Programs addressing aggression and violence | |
Prevention Intervention Toronto (PIT) 2009–2012 |
Ontario (Toronto) |
Multisystemic Therapy (MST) 2009–2014 |
Ontario (Toronto) |
Youth Inclusion Program (YIP-Maritimes) 2010–2014 |
New Brunswick (Saint-John) and Nova Scotia (North Sydney and Spryfield) |
Youth Inclusion Program (YIP-Quebec) 2012–2016 |
Quebec (Montréal and Salaberry-de-Valleyfield) |
Programs addressing personal and social skills | |
Outdoor adventure programs | |
Leadership and Resiliency (LRP) 2011–2015 |
North West Territories (Hay River and Yellowknife) |
Velocity 2009–2014 |
Newfoundland and Labrador (St. John's) |
Social skills focus | |
Life Skills Training (LST) 2010–2013 |
Alberta (Edmonton) |
Programme de suivi intensif de Montréal / Gangs de rue (PSI-MTL/GDR) 2009–2014 |
Quebec (Montréal) |
Substance abuse programs | |
Towards No Drugs (TND) 2009–2014 |
Ontario (Hamilton) |
This paper presents preliminary findings from 9 of these 11 impact evaluation studies.Footnote 4 The following section briefly describes these interventions, including their implementation milieu and their target groups. The third section describes the approach taken by PS to perform the evaluation studies and synthesis of findings. The fourth section presents the preliminary findings, and the last section provides some tentative conclusions.
2. The interventions
The programs tested offered a variety of activities and services for youth and young adults ranging from individual case management, education and employment support, skill building, and recreational activities. Most projects were developed and implemented in communities by voluntary not-for-profit organizations, in collaboration with key crime prevention stakeholders such as schools and local police. These interventions usually target youth between the ages of 12 and 17Footnote 5 who display multiple risk factors associated with criminal behaviour such as substance abuse, limited attachment to school, associating with delinquent peers, violent and aggressive tendencies, and early contact with the justice system. Their risk profile is often complex and many of the youth also have low literacy rates, poor parental supervision, mental health issues, unstable housing, and low academic achievement.
2.1 Programs addressing school-related issues
It is well established that youth who are at-risk of following a delinquent trajectory experience various difficulties in school. These include: difficulty adjusting to school, low attachment to school, poor performance, truancy and systematic suspensions, bullying, etc. Several different types of interventions have been designed, in particular, to address systematic suspensions. Traditional suspensions involve removing a youth from school for a period of time. An abundance of research challenges the effectiveness of this form of suspension and shows that sending a youth to an unsupervised environment may increase disciplinary problems, and in the long run, may even contribute to an unsafe environment in the school and neighbourhood (Costenbader and Samia, 1997; Morrison, Antony, Storino and Dillon, 2001; Atkins, McKay et al., 2002).
PS's National Crime Prevention Centre had already tested an alternative to suspension program in Quebec, showing promising results (CAC International, 2005). According to its 2005 evaluation, Alternative Suspension successfully decreases early school departure and improves attitudes and behaviours at school (conflict and anger management, attendance, suspension, etc.). In 2009, the PS crime prevention program supported and evaluated two alternative-to-suspension programs: the YMCA's Alternative Suspension (AS) program which targets youth suspended or at risk of suspension because of their behaviours, and the Intervention, Rethink, Refocus, Reintegrate (iR3), a program for youth receiving a first time suspension for violence or substance use that is also inspired by the original YMCA's AS program.
At the time of this synthesis, the YMCAs of Quebec have established AS Programs in over 41 Canadian locations. Since 2009, PS has been funding the national expansion phase of the program and has supported the implementation in 16 communities across the country.Footnote 6 During the 2011–2012 school-year, 186 youth (65% male) participated in AS in the three sites that are involved in an impact evaluation. Over 25% of students were then referred to the AS Program because of disruptive and impulsive behaviours (27.2%) and approximately 20% were referred for physical or verbal violence (21.5%), risky behaviours (22.1%), apathy and lack of motivation (17.4%) (R.A. Malatest & Associates Ltd, 2013a).
iR3 is an alternative to the traditional suspension of youth who are found to be engaging in anti-social behaviours at school (Doley and Gagnon, 2012). iR3 targets at-risk youth (grades 6 to 8) who are facing a first-time suspension for aggressive behaviours and/or use of/association with drugs or alcohol. This program includes an individual follow-up, a set of workshops, guest speakers and activities allowing students to reflect on their behaviour and to learn specific knowledge and skills. The program is also designed to create opportunities for youth to engage in pro-social activities and to connect with adult mentors in the program, at their school and within the community. Between 2006–2007 and 2011–2012, 638 students (82% male) were admitted to iR3 in Surrey, BC (Doley and Gagnon, 2012).
Through these programs, suspended students with disciplinary problems are removed to an off-site location and participate in a curriculum designed to address specific at-risk factors and reintegrate them into the school. This includes academic support, thematic workshops, one-on-one meetings, and sports activities. The anticipated outcomes of both programs include improved attitudes toward school and delinquency and a reduction in suspensions and disciplinary actions among program participants.
2.2 Programs addressing aggression and violence
Research on offending trajectories has established that youth who demonstrate patterns of violent behaviour and offending are at elevated risks of long-term and chronic offending trajectories into adulthood. Youth who are in gangs or who have committed more serious offences have been involved in several projects funded under the NCPS.
The Prevention Intervention Toronto (PIT) model includes a carefully chosen set of complementary and integrated components designed to facilitate change among targeted youth (Wortley et al, 2012). This case management program, developed by the City of Toronto to promote youth development, prevent and treat health and behaviour problems among young people, is based on a variety of evidence-based sources supporting gang-intervention and violence-prevention programs.Footnote 7 PIT consists of a needs assessment phase, a group training phase that includes one-on-one counselling and an integration phase that allows youth to meet with their case manager to achieve program goals, in particular, their transition to a pro-social lifestyle. The 306 participants (72% male) who were accepted into PIT were between the ages of 12 and 24 years old. To be accepted into the PIT program, youth had to meet at least one of the five following criteria: 1) self-identification as a former or current gang member; 2) case manager identification as a former or current gang member; 3) met Eurogang criteria;Footnote 8 4) moderate to high risk score; 5) qualitative exemption including family gang involvement, residence in a high-risk community, or the case manager's belief that the youth was lying during the screening, artificially lowering their score (Wortley et al, 2012).
Multisystemic Therapy (MST)(Henggeler et al, 2006) is an intensive, short-term intervention designed for at-risk youth and their families whose intended outcomes include reduced substance use and aggression, improved school performance and positive family relationships. Each family is assigned a case worker who provides regular support and counselling based on the particular needs of the youth and their family. The treatment includes individual, family, peer, school, and community interventions such as parent and skills training (Welsh, 2007: 22). Previous evaluations have shown the program to be effective at reducing a number of risk factors such as substance use and aggressive behaviour, and keeping youth involved in positive activities such as school and employment (Borduin et al., 1995 quoted in Welsh, 2007: 22). As of March 2012, 58 youth aged 12–18 years were accepted into the MST program in Toronto, of whom 44% were at high risk and 41% at moderate risk (Harry Cummings and Associates Inc., 2012). The average age of 40 discharged youth is 15.7 years and 65% of youth are male.
2.3 Programs addressing personal and social skills
Youth who are at risk of delinquency often demonstrate fewer positive social skills, including low empathy, higher aggressiveness, elevated risk-taking behaviour, etc. Several types of interventions address these factors.
Outdoor adventure programs
Outdoor adventure programs are a means of removing youth at risk from their typical environments to provide them with challenges where they can learn new skills and feel positive about their accomplishments. These programs have been shown to have a number of positive outcomes, including reduced recidivism rates, improved social skills, and reduced substance use (West and Crompton, 2001). These types of projects are popular in Canadian communities, and a number of funded projects focus on outdoor learning and adventure. They generally include but are not limited to camps, day outings into wilderness areas and outdoor skill building. This thematic paper covers two programs addressing skills through outdoor activities: the Leadership and Resiliency Program (LRP) and the Velocity Program (originally named Adventure Youth Initiative).
The theory behind the LRP created by Fairfax-Falls Church is to replace risky behaviour with healthy risk taking in the form of outdoor activities, community service learning and a classroom based curriculum (Fishbein and Boylan, 1997 and 1998). LRP is based on three premises: social competence fosters resiliency against substance use and violent behaviour; positive social influences facilitate the establishment of pro-social behavioural norms; and associations with positive role models and peers protect adolescents from negative influences. LRP is being implemented in multiple schools in two sites in the NWT. During the 2011–2012 school year, most LRP youth were between 13 to 18 years old, evenly distributed by gender, and the majority were Aboriginal. Most of the youth had multiple risk factors such as substance use and behavioural issues (R.A. Malatest & Associates Ltd, 2013b).
The Velocity Adventure Program (Velocity) is an adventure-based program for youth who are at risk of, or have already been involved in, criminal activity (Public Safety, 2013a). Based on community consultations and by the design of the Community Youth Network in St. John's (Newfoundland and Labrador), a pilot project was implemented to fill a gap in services and to enhance the engagement of youth age 13 and above, since this group was not receiving appropriate support. The project combines evidence from effective models and practices involving structured outdoor adventure activities, life skills training and mentorship (Wilson, S.J. and Lipsey, 2000; Tarolla, Wagner, Rabinowitz and Tubman, 2002). The Velocity project aims at reducing aggressive and violent behaviour, substance use, lack of attachment to school, and offending among at-risk youth, most of them having had contact with the criminal justice system. The program includes group work, a seven-day wilderness camp and day-long wilderness outings and drop-in activities.
During these activities, youth are provided with individual support, referrals to other community services, and are encouraged to make healthy life choices. Velocity was funded by PS from 2009 through 2014. The project served 71 participant youth during the 3 year evaluation, of which 63% were boys between 13–17 years of age. The program fell short of its initial goal to deliver Velocity to 90 youth over 36 months. The evaluation of Velocity covers three cohorts with each cohort participating for about one year in the program (Ference Weicker & Co., 2013).
Social skills focus
The Youth Inclusion Program (YIP), originally developed by the Youth Justice Board in the UK, is expected to reduce crime and antisocial behaviours by helping youth acquire new skills, take part in social activities and get help with their studies. It includes one-on-one sessions that address individual needs along with group activities, which can include academic support, recreational activities, mentoring, and life skills training (Mackie, Hubbard and Burrows, 2008; Mackie, Burrows and Hubbard, 2003). The YIP has been implemented in the Maritimes, Québec and British Columbia (Table 1). The three YIP-Maritimes projects covered by this synthesis have admitted 217 participants of whom 55% are male (NRG Research Group, 2013).
The Life Skills Training (LST) program was designed for a school setting and aims to enhance personal competence, basic life skills and skills related to resistance to social influences that promote substance use (EPIS Center, 2013). The program draws on strength-based approaches to working with youth, by reinforcing functional behaviour and emphasizing building skills that can be utilized in all areas of a youth's life. It includes workshops and curricula targeting specific risk factors and has been tested in several studies (Botvin and Griffin, 2005; MacKillop, Ryabchenko and Lisman, 2006).
The LST program uses these activities to reach its objectives of decreasing drug abuse risks by reducing personal motivation to use drugs and reducing the susceptibility of youth to social factors that promote drug use. The LST program in Alberta, which took place after school in an at-risk neighbourhood school, ran from 2010 through 2013. It reached approximately 22 youth per year for a total of 87 youth of which 26% were male (latter two cohorts), with 37% being Aboriginal. The youth were in grades 4 through 9 (ages 10–16) and adaptations were made to the LST curriculum to tailor it to the target population (adapting from written to verbal tasks, incorporating cultural elements, etc.) (R.A. Malatest & Associates Ltd, 2013c).
Substance abuse programs
As a result of early initiation or addictions, substance abuse is a well-established factor in long-term offending trajectories.
Towards No Drugs (TND) is based on a motivation-skills-decision-making model developed by the Institute for Health Promotion and Disease Prevention Research, University of Southern California (Sussman, Dent and Stacy, 2002). The program consists of interactive classroom sessions and aims to address issues such as substance use and delinquency by strengthening decision-making and motivation to change. The program was implemented in two schools and in a community where substance abuse, specifically marijuana and alcohol use were disproportionately high. The target group in the community was at highest risk and displayed multiple risk factors that put them at high risk for criminal involvement in the justice system. The expected outcomes of the program included a reduction in substance use and criminal behaviour. The 1623 participants in TND were 53% female and 10% were high-risk, 22% medium risk and 68% low risk (Goss Gilroy Inc. Management Consultants, 2013).
3. Evaluation approach
3.1 Conducting evaluations
To measure program impacts in such a way as to rule out alternative explanations and determine if the measured intervention contributed to the positive changes in the outcomes of interest, rigorous designs, such as quasi-experimental designs, are encouraged. The challenges of experimental designs that incorporate some form of comparison are well documented (cost, recruitment and retention, attrition, etc.) and it is not feasible to expect that every evaluation will be able to reach this level of rigour. As a minimum, pre and post program tests with follow-up assessments up to six months post program have been required when comparison groups are not possible. In addition, evaluators are encouraged to utilize qualitative techniques to strengthen the findings (case studies, in-depth interviews).
Collecting data for an evaluation is both critical and challenging. All of the evaluations draw on multiple sources and data collection methods to assess the impact of the program. Research methods include individual-level risk assessments, questionnaire-based surveys administered by evaluators or program staff, focus groups, semi-structured interviews, case files, secondary data from schools and the police. The evaluators were encouraged to use standardized and validated data collection tools as much as possible. Evaluation firms contracted by the Department have used the most rigorous design that was available, given the specific circumstances of the evaluated programs (Table 2).
Program | Pre/Post and follow-up measures | Experimental group/Referral process | Matched comparison group | Secondary data |
---|---|---|---|---|
AS | Pre-post and 4–6 weeks and end of school year follow-ups | Students referred by schools at 3 project service centers | Students referred who did not participate in AS or quit ASFootnote 9 | Project and schools |
iR3 | Pre-post | Students referred by participating schools across the School Board | Provincial school data matched to youth in the program | Provincial school data verified by individual school suspension data |
LRP | Pre/Post | 4 schools (2 sites) | Other school | Project and schools |
LST | Pre-Post | After-school program | From a similar agency and neighbourhood | Project |
MST | Pre-post and follow-ups at 6 and 12 months post-program | Referred by the district school board, community organizations and parents | Youth who did not complete the program were used | Project |
PIT | Pre-post and follow-up at 12 months post program | Youth referred from a variety of sources including schools, community organizations, youth workers and parents | Comparison group used from a comparable high-risk neighbourhood located in North West Toronto | Project, schools and police |
TND | Pre-post and follow-up at 12 months post program | School-based program (42 schools) | Several classes in the same schools were used as a comparison group | Project and schools |
Velocity | Pre-post and follow-up at 12 months post program | Participants referred by community agencies and partners. Referral forms completed for each youth. | Comparison group not matched for risk factors | Project and police |
YIP-Maritimes | Pre-post and follow-up at 12 months post program | Students referred to 3 community organizations by schools. | NA | Project, schools and police |
Notes:
- Secondary data is generally used to validate self-reported information. Unlike primary data, secondary data is not directly collected by the evaluators.
- At this stage, not all evaluations are able to determine the level of significance and the effect size with accuracy.
3.2 Synthesizing the evaluation studies for this reportFootnote 10
The studies needed to satisfy the following criteria in order to be included in this report: 1) availability of at least some quantitative information (descriptive or inferential statistics); 2) measures related to key criminogenic related outcomes; 3) the study meets at least the basic evaluation design; 4) the study has at least some interim data; and 5) the study includes youth aged 12–17.
A descriptive analysis was conducted for the domains related to knowledge, attitudes, risk factors, protective factors and behaviours. Each table provides the following key information: 1) name of the study; 2) description of the measure examined; 3) sample size of the experimental and comparison group (if any); 4) type of change in the outcome being measured; and 5) the design and/or source of data. Appendix 1 provides summary tables with the detailed results.
The results in these studies are based on a variety of statistical tests used to convey the level of impact the program had on the variables being measured. These statistics include percentages and various coefficients that were convertedFootnote 11 into four categories so that the reader can interpret the results.Footnote 12 The definitions are as follows:
- A positive change is defined as a result that demonstrates a statistically significant (p<0.05) reduction in risk factors or offending behaviours or an increase in positive knowledge, attitudes or protective factors. When there was a trend demonstrating more frequently occurring favourable outcomes, the study was labelled "positive". For example, a project may have six different types of outcomes measuring the risks related to aggression, as it is good practice to use a variety of measures to confidently determine if changes are occurring in the outcomes of interest. If four outcomes were positive, one demonstrated no change, and the final one showed a negative change, the trends would be labelled favourable overall.
- A negative change is defined as a result that demonstrates that there is a statistically significant (p<0.05) increase in risk factors or offending behaviours or it demonstrates a decrease in positive knowledge, attitudes or protective factors. For example, a project may have four different types of outcomes measuring changes in attitudes. If three of the four outcomes demonstrate a negative result, this outcome related to attitudes was labelled as a "negative" trend.
- A "neutral" finding is defined as a result that does not demonstrate a statistically significant (p<0.05) change in the outcomes tested prior to and after the program. For studies that report descriptive frequencies, a "neutral finding" indicates that the frequency distribution prior to and after the program is similar. For example, a project can use five different outcomes to measure changes for reducing non-violent offending. If three of the five outcomes demonstrate no change, the trend would be labelled "neutral". A related footnote is generally provided if this trend was noted in the initial pre and post test period or whether it was identified in the post-program follow-up period. When a "neutral" trend is identified in the post program period (six months after the program is complete), it is possible that the immediate post program trends are positive and these changes have been maintained up to six months after the program. In this scenario, the "neutral" trend is not necessarily negative, as this means the initial positive trends have been maintained.
- Findings are classified as "mixed" when there is an equal number of negative and positive outcomes, an equal number of neutral and negative outcomes, or an equal number of neutral and positive outcomes.
3.3 Limitations of the studies
There are a few limitations that should be considered when interpreting the data in this synthesis report. Since youth are in the program for approximately six months, we usually assess several groups prior to making conclusions. However, since the projects may have been on the ground for a limited time period and the evaluation studies for an even shorter time period, data may currently be available for only a few groups. In this report, many of the evaluations have half of the groups, requiring further replications prior to making strong conclusions about program efficacy.
The external validity of the findings may differ between studies due to the variety of evaluation methods used. For example, comparison groups may not always be equally strong. For some indicators, between-group comparisons were not feasible at the time when this paper was written. Greater weight was not applied to findings derived from more rigorous evaluations, indicating that the findings were treated equally when the trends were being assessed. Since weightings were not used for this synthesis, interpretations about what models are more effective should be delayed until weightings are applied to the final evaluation results.
Finally, not all of the evaluations are at the same stage. For example, the PIT evaluation is fully complete while other evaluations included in this report will not be completed until 2015. This variation in timing means that trends have often consolidated evaluations that have pre and post-test measures (T1 and T2) with evaluations that have completed pre, post and post program measures up to six months and in some cases one year (T1, T2, T3 and in some cases T4). We know in the evidence-based literature that immediate post program measures and long term maintenance measures naturally differ so the findings need to take into consideration when the measurements were assessed in relation to the intervention.
4. Findings
It is generally accepted among both researchers and practitioners that change will be incremental over the course of the intervention. Prior to reaching behavioural changes such as reduced criminal and delinquent behaviour, it is anticipated that youth will gain knowledge and improve attitudes as a first step toward changing behaviour. As youth gain knowledge about and change attitudes toward risky behaviour, they will begin to address behavioural risk factors that will eventually lead to changes in criminal and delinquent behaviour.
4.1 Knowledge and attitudes
Seven of the evaluation studies in this synthesis report measure knowledge and attitudes with a majority (71%) of the results showing positive changes for the youth in these various interventions (iR3 show neutral results and LST mixed results). Table 3 indicates that a variety of knowledge and attitudes have been changed. These include: 1) being motivated to change; 2) having a better understanding about the negative consequences of substance abuse; 3) having a positive attitude towards the justice system; and 4) having an appropriate attitude towards offending.
Prochaska and DiClemente's (1983) model suggests that participants potentially move out of the pre-contemplation (not ready to change) and contemplation phase (thinking about change) when their attitudes begin to shift from negative to positive thinking. Reduced risk factors and increased protective factors are more likely to be observed among youth who have successfully absorbed new knowledge and changed their attitudes.
Program | Description of variables | Changes/Trends |
---|---|---|
AS | Thought given to situation at school and willingness to improve behaviour | Favourable |
iR3 | Attitudes toward violence and school | Neutral |
LST | Cultural knowledge | Neutral |
Understanding effects of substance use | Favourable | |
PIT | Attitude towards gangs, and the justice system | Favourable |
TND | Knowledge about use of drugs and consequences | Favourable |
Velocity | Attitude towards schooling | Favourable |
YIP | Motivation to change | Favourable |
Thinking and behaviour | ||
Attitudes to offending |
Notes:
- Preliminary results except for iR3 and PIT. Detailed results and sample size are included in appendix Table A-1.
- The denominator for this table is 7 studies as LST has two measures that were merged according to the definition provided in section 3.2.
4.2 Risk and protective factors
Eight of the eleven evaluation studies in this synthesis report measure risks and protective factors with a majority of them showing positive changes for several variables related to seven domains. As indicated in Table 4, 57% of the domains show favourable changes (Skills, Education/Academic performance, Parenting/Family relationships, Externalizing behaviours/Pro-social behaviours), 29% show neutral results (Self-esteem/Emotional regulation, Alcohol and Drug abuse) and 14% show mixed results (Anti-social peers/Gang membership).
1) Skills: TND and Velocity (out of three evaluation studies) report favourable changes in decision making skills or the ability to handle and reduce substance use. The LST program has had neutral trends on the skills variables measured.
2) Anti-social/Peers/Gang membership: Two of the three studies (PIT and YIP) report mixed trends for this domain. The YIP program addresses lifestyle (which is favourable), neighbourhood and friends (neutral). The PIT program has had favourable changes associated with anti-social peers, but mixed results with regard to the association with gang-involved peers. The MST study reported favourable changes at the time of discharge.
3) Education/Academic performance: Four of the studies (AS, iR3 , MST and YIP) demonstrate favourable trends for behaviours related to school and education. Key variables are related to performance, course completion, attendance, behaviour, disciplinary actions, absenteeism and suspension. However, PIT has mixed results for school attendance and disciplinary problems.
Program/Domain | Description of variables | Change/ Trend |
---|---|---|
Skills | ||
LST | Decision making skills | Neutral |
Drug refusal skills | ||
TND | Decision making skills | Favourable |
Velocity | Ability to handle substance use problems and motivation to reduce substance use | Favourable |
Anti-social peers/Gang membership | ||
MST | Youth involved with pro-social peers & activities (at time of discharge) | Favourable |
PIT | Association with gang-involved peers | Mixed |
Association with anti-social peers | Favourable | |
YIP | Lifestyle | Favourable |
Neighbourhood and friends | Neutral | |
Education/Academic performance | ||
AS | Performance at school | Favourable |
% courses completed | ||
Class attendance | ||
Behaviour at school | ||
Disciplinary action at school | ||
Subsequent school suspension | ||
iR3 | Disciplinary action | Favourable |
Suspensions | ||
MST | Youth Success in academic/vocational setting (at program discharge) | Favourable |
PIT | School attendance, Disciplinary problems | Mixed |
YIP | School and education | Favourable |
Performance at school | ||
Absenteeism | ||
Self-esteem/Emotional regulation | ||
LST | Self-image | Favourable |
Coping skills | Neutral | |
PIT | Risk seeking | Neutral |
Velocity | Ability to handle stress | Neutral |
YIP | Perception of self and others | Favourable |
Emotional and mental risk factors | ||
Parenting/Family relationships | ||
MST | Risk of out of home placement/improved family relations, parenting skills, and family social supports | Favourable |
PIT | Family relationships | Neutral |
YIP | Family and personal relationships | Favourable |
Externalizing behaviours / Pro-social behaviours | ||
Velocity | Aggressive and anti-social behaviours | Favourable |
Alcohol and Drug abuse | ||
PIT | Alcohol | Neutral |
TND | Cocaine, Marijuana, Prescription drugs, Alcohol | Neutral |
Velocity | Alcohol and Drugs | Neutral |
YIP | Substance use | Neutral |
Notes:
- Preliminary results except for iR3 and PIT. Detailed results and sample size are included in appendix Table A-2.
- The denominator for this table is 7 domains as a result of synthesizing variables based on predetermined categories. In the case of the domain for Anti-social peers and Gang membership, PIT and YIP were categorized as mixed based on the definitions provided in section 3.2. For the domain Self-esteem and Emotional regulation, LST was classified as mixed based on the definitions provided in section 3.2.
4) Self-esteem/Emotional regulation: The YIP program has had a positive impact (perception of self and others, emotional and mental risk factors) whereas two other studies report neutral results (risk seeking for PIT and ability to handle stress for Velocity). LST has mixed results, being favourable on self-image but neutral for coping skills.
5) Parenting/Family relationships: Two (MST and YIP) of the evaluation studies that measure parenting and family relationships report favourable changes whereas PIT is rather neutral.
6) Externalizing behaviours/Pro-social behaviours: Impacts of Velocity on aggressive and anti-social behaviours are favourable.
7) Alcohol and drug abuse: For all of the programs (PIT, TND, YIP and Velocity), results on substance use are neutral.
4.3 Behaviours related to crime
The impact evaluations have also measured changes in behaviours related to crime (Table 5) and found that three programs (MST, Velocity and YIP) have had a favourable impact on youth arrests or police contacts, whereas TND had a mixed impact (weapon carrying) and PIT has been neutral (arrests, non-violent offending and criminal victimizations).
Program | Description of variables | Change/Trend |
---|---|---|
MST | Youth not arrested for an offence committed during MST (3 months after intake) | Favourable (at 3 months post-intake) |
PIT | Arrests, Non-violent offending, Criminal victimizations | Neutral |
TND | Weapon carrying | Mixed |
Velocity | Police contacts | Favourable |
YIP | Police contacts | Favourable |
Note: Preliminary results except for PIT. Detailed results and sample size are included in appendix Table A-3.
5. Preliminary conclusions
Despite the challenges of conducting impact evaluations in a community setting, all of the evaluations discussed in this report have produced data to assess the impact on key factors and contribute to knowledge about effective programming. The results so far are able to identify some programs that appear to work, and others that may need some adjustment to meet the needs of youth and increase the likelihood of achieving positive results in the community.
The initial expectation was that in the short term, youth participants would increase knowledge and attitudes related to crime and delinquency as a first step toward changing behaviour. Several programs demonstrate positive change across a number of measures, including improvements in knowledge and attitudes toward substance use, school and violent/ aggressive behaviour.
The most positive impact of the programs is on youths' behaviour. Most of the interventions showed a decrease in at least one of the behaviours that led to the youth being referred to the program. Particularly positive was the impact of the program on reductions in police contacts. All of the evaluations that were able to collect data showed an improvement among the program participants. As such, these project evaluations provide useful information about what results local organizations, schools and youth-serving agencies are achieving in implementing promising and model crime prevention programs.
Appendix
Program | Description of variables | Sample size | Results | Changes/ Trends |
Design/Source |
---|---|---|---|---|---|
AS | Thought given to situation at school and willingness to improve behaviour | E=119 | Serious or some thought and willingness to change: 72.4% Little thought and willingness to change: 23.3% No thought and willingness to change: 4.3% |
Favourable | Post-questionnaire filled by youth worker |
iR3 | Attitudes toward school/teacher | E=122 | No change | Neutral | Pre and post measures |
Attitudes toward substance use | E=122 | No change | Neutral | Pre and post measures | |
LST | Cultural knowledge | E=87 C=27 |
p>0.05 | Neutral | Pre/post measures |
Understanding effects of substance use | E=87 C=27 |
r=-0.65 p=.02 |
Favourable | Pre/post measures | |
PIT | Attitude towards gangs and the justice system | E=188 C=99 |
F coefficients range from .587 to 6.546 P<0.05 |
Favourable | Matched comparison group with a 6 month post program follow-up |
TND | Knowledge about use of drugs and consequences | E=868 C=59 |
F=112.61 P<0.05 |
Favourable | Matched comparison group with a 12 month follow-up |
Velocity | Attitude towards schooling | E=71 C=45 |
P<0.05 compared to pre-test | Favourable | Pre and post measures with follow-up |
YIP | Motivation to change | E=41 | Favourable: 46% Neutral: 44% Unfavourable: 10% |
Favourable | Pre and post measures |
Thinking and behaviour | E=41 | Favourable: 61% Neutral: 20% Unfavourable: 20% |
Favourable | Pre and post measures | |
Attitudes to offending | E=41 | Favourable: 51% Neutral: 39% Unfavourable: 10% |
Favourable | Pre and post measures |
Note: Preliminary results except for iR3 and PIT. E= experimental group and C= control group
Program | Description of variables | Sample size | Results | Changes/ Trends |
Design/Source |
---|---|---|---|---|---|
Skills | |||||
LST | Decision making skills | E=87 C=27 |
p>0.05 | Neutral | Pre/post measures |
Drug refusal skills | E=87 C=27 |
p=0.28 r=-0.41 |
Neutral | Pre/post measures | |
TND | Decision making skills | E=868 C=59 |
F= 4.88 P<0.05 |
Favourable | Matched comparison group |
Velocity | Ability to handle substance use problems and motivation to reduce substance use | E=71 F=45 |
P<0.05 compared to pre-test | Favourable | Pre and post measures with follow-up |
Anti-social peers/Gang membership | |||||
MST | Youth involved with pro-social peers & activities (at time of discharge) | E=28 C=12 |
71.4% of the treatment group vs. 25% of the control group | Favourable | Single group repeated design |
PIT | Association with gang-involved peers | E=76 C=43 |
F ranges from 0.461 to 5.761 P<0.05 |
Mixed (similar declines observed among the experimental and the comparison group) | Matched comparison group |
Association with anti-social peers | E=76 C=43 |
F ranges from 0.461 to 5.761 P<0.05 |
Favourable | Pre/post measures with comparison group | |
YIP | Lifestyle | E=41 | Favourable: 68.29% Neutral: 24.39% Unfavourable: 7.32% |
Favourable | Pre and post measures |
Neighbourhood and friends | E=41 | Favourable: 32% Neutral: 66% Unfavourable: 2% |
Neutral | Pre and post measures | |
Education/Academic performance | |||||
AS | Performance at school | E=106 | Improvement for 89% of cases | Favourable | Follow-up with schools 5 weeks post-end of school year |
Percentage of courses completed | E=164 | 78% youth completed all courses | Favourable | School data | |
Class attendance | E=98 | On average youth attended 95% of their classes and 36% had perfect attendance | Favourable | School data | |
Behaviour at school | E=111 | Improvement: 58.6% of youth | Favourable | Follow-up with schools at end of school year | |
Disciplinary action at school | E=111 | Decrease: 59.5% of youth | Favourable | Follow-up with schools at end of school year | |
Subsequent school suspension | E=121 | 53% of youth have not had subsequent suspension | Favourable | Follow-up at end of school year | |
iR3 | Disciplinary action | E=126 C=126 |
18% disciplinary action compared to 43% in comparison group | Favourable | Matched comparison group |
Suspensions | E=122 C=122 |
22.6% suspended compared to 74% in comparison group | Favourable | Matched comparison group | |
MST | Youth success in academic/ vocational setting | E=28 C=12 |
67.9% of treatment group participants vs. 33.3% of control group participants showed evidence of academic success at program discharge | Favourable | Single group repeated design |
PIT | School attendance, Disciplinary problems | E=76 C=43 |
F ranges from .587 to 3.5558 | Mixed | Matched comparison group |
YIP | School and education | E=41 | Improvement: 61% Neutral: 27% Unfavourable: 12% |
Favourable | Pre-post measures |
Performance at school | E=29 | Increase in GPA for 76% of cases | Favourable | School data, pre-program | |
Absenteeism | E=37 | 57% of youth have a reduced number of school days missed | Favourable | School data | |
Self-esteem/Emotional regulation | |||||
LST | Self-image | E=87 C=27 |
r=-.44 | Favourable | Mixed design with pre/post measures |
Coping skills | E=87 C=27 |
r=-.37 | Neutral | Mixed design with pre/post measures | |
PIT | Risk Seeking Scale | E=76 C=43 |
F=2.226 P=0.110 |
Neutral | Matched comparison group |
Velocity | Ability to handle stress | E=71 C=45 |
p>0.05 | Neutral | Pre and post measures with follow-up |
YIP | Perception of self and others | E=41 | Favourable: 51% Neutral: 32% Unfavourable: 17% |
Favourable | Pre-post measures |
Emotional and mental risk factors | E=41 | Favourable: 63% Neutral: 29% Unfavourable: 7% |
Favourable | Pre-post measures | |
Parenting/Family relationships | |||||
MST | Risk of out of home placement/improved family relations, parenting skills, and family social supports | E=28 C=12 |
Experimental group ranges were 25%-88% vs. control group ranging from 16.1 to 60% | Favourable | Single group repeated design |
PIT | Family relationships | E=75 C=43 |
F=.519 P=0.551 |
Neutral | Matched comparison group |
YIP | Family and personal relationships | E=41 | Favourable: 63% Neutral: 32% Unfavourable: 5% |
Favourable | Pre and post measures |
Externalizing behaviours and Pro-social behaviours | |||||
Velocity | Aggressive and anti-social behaviours | E=71 C=45 |
Favourable: 41% Neutral: 20% Unfavourable: 39% |
Favourable | Pre and post measures with follow-up |
Alcohol and Drug abuse | |||||
PIT | Alcohol | E=76 C=43 |
F=1.345 P=.263 |
Neutral | Matched comparison group |
TND | Cocaine, Marijuana, Prescription drugs, Alcohol | E=868 C=59 |
F ranges from 0.22 to 0.86 P>0.05 |
Neutral | Matched comparison group |
Velocity | Alcohol and drugs | E=71 C=45 |
Favourable: 28% Neutral: 59% Unfavourable: 13% |
Neutral | Pre and post measures with follow-up |
YIP | Substance use | E=41 | Favourable: 32% Neutral: 58% Unfavourable: 10% |
Neutral | Pre and post measures |
Note: Preliminary results except for PIT. E= experimental group and C= control group
Program |
Description of variables | Sample size | Results | Trends | Design/Source |
---|---|---|---|---|---|
PIT | Arrests, Non-violent offending, Criminal victimizations | E=76 C=41 |
F ranges from 0.291 to .761 | Neutral | Matched Comparison Group |
MST | Youth not arrested for an offence committed during MST (3 months after intake) | E=28 C=12 |
89.3% of the treatment group vs. 66.7% not arrested for an offence committed during MST (3 months after intake) | Favourable (at 3 months post-intake) | Single group repeated design |
TND | Weapon carrying | E=847 C=54 |
F ranges from 0.07 to 0.63 | Mixed | Matched Comparison Group |
Velocity | Police contacts | E=71 C=45 |
49% overall reduction in participant police contact | Favourable | Pre and post measures with follow-up |
YIP | Police contacts | E=50 | 68% youth with no suspect/charge during post period | Favourable | Police post-data |
Note: Preliminary results except for PIT. E= experimental group and C= control group
References
- Atkins, M.S., McKay, M.M., Frazer, S.L., Jakobson, L.J., Arvanitis, P., Cunningham, T., Brown, C. and L. Lanbrecht (2002). Suspensions and detentions in urban, low-incomed school: punishment or reward? Journal of Abnormal Child Psychology, 30, (4): 361-371.
- Borduin, C. M., Mann, B. J., Cone, L. T., Henggeler, S. W., Fucci, B. R., Blaske, D. M. and R.A. Williams (1995). "Multisystemic treatment of serious juvenile offenders: long-term prevention of criminality and violence." Journal of Consulting and Clinical Psychology, 63, 569–578.
- Botvin, G.J. and K.W. Griffin (2005). "Prevention Science, Drug Abuse Prevention, and Life Skills Training: Comments on the State of the Science", Journal of Experimental Criminology, (1): 63-78.
- C.A.C. International (2005). Rapport final d'évaluation du programme Alternative Suspension.
- Costenbader, V. and M. Samia (1997). "School Suspension: A study with secondary school students," Journal of School Psychology, (36): 59-82.
- Doley, S. and N. Gagnon (2012). iR3 – Stay Real, Final Evaluation Report, Submitted to Public Safety, Kwantlen Polytechnic University. PS Senior Evaluation Advisor: Alison Warner.
- EPIS Center (2013). http://www.episcenter.psu.edu/ebp/lifeskills.
- Ference Weicker & Company Ltd. (2013). Evaluation of the Velocity Adventure Program (v.2)-St. John's, Newfoundland: Final Report. Submitted to PS by Beth Garner. Senior Evaluation Advisor: Giselle Rosario.
- Fishbein, D. and D. Boylan (1998). "Enough is Enough". Neighborhood Prevention Initiatives in the Washington/Baltimore Region, Washington/Baltimore HIDTA, 1998 Prevention Report.
- Fishbein, D. and D. Boylan (1997). "Enough is Enough". Neighborhood Prevention Initiatives in the Washington/Baltimore Region, Washington/Baltimore HIDTA, 1997 Prevention Report.
- Goss Gilroy Inc. Management Consultants (2013). Towards No Drug Abuse (TND) Impact Evaluation, 2012-2013 Annual Report Submitted to Public Safety. PS Senior Evaluation Advisor: Donna Smith-Moncrieffe.
- Harry Cummings and Associates Inc. (2012). Evaluation of the Multisystemic Therapy Program, Toronto, Ontario, Interim Evaluation Report – 2012, Submitted to PS, September 17, 2012. PS Senior Research Advisor: Cameron McIntosh.
- Henggeler, S.W., Schoenwald, S.K., Borduin, C.M., Rowland, M.D. and P.B. Cunningham (2009). Multisystemic Therapy for Antisocial behavior in Children and Adolescents (2nd ed.), New York, NY: The Guilford Press.
- Laliberté, D. (2013). Results of Crime Prevention Programs in Canada, NCPC, 36th Biennial congress of the Société de criminologie du Québec, Mont-Tremblant.
- Mackie, A., Burrows, J. and R. Hubbard (2003). Evaluation of the Youth Inclusion Programme. London: Youth Justice Board.
- Mackie, A., Hubbard, R. and J. Burrows (2008). Evaluation of the Youth Inclusion Program: Phase 2. London: Youth Justice Board.
- MacKillop, J., Ryabchenko, K.A. and S.A. Lisman (2006). "Life Skills Training Outcomes and Potential Mechanisms in a Community Implementation: A Preliminary Investigation", Substance Use & Misuse, (41): 1921-1935.
- Morrison, G. M., Anthony, S., Storino, M. and C. Dillon (2001). "An examination of the disciplinary histories and the individual and educational characteristics of students who participate in an in-school suspension program," Education and Treatment of Children, (24): 276-293.
- NRG Research Group (2013). Youth Inclusion Program Evaluation: Second Annual Report. Submitted to NCPC by Nathalie Gagnon and Lesley Duncan. Senior Evaluation Advisor, PS: Danièle Laliberté.
- Prochaska, J. and C. DiClemente (1983) "Stages and processes of self-change in smoking: toward an integrative model of change," Journal of Consulting and Clinical Psychology, (5): 390-395.
- Public Safety Canada (2013a). Velocity Adventure Program, Crime Prevention in Action - CPA-26.
- Public Safety Canada (2013b). Results from the Multisystemic Therapy Program, Building the Evidence, Evaluation Summaries, ES-2013-34. Senior Research Advisor, PS: Cameron McIntosh.
http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/rslts-mltsystmc-thrpy-prgrm/index-eng.aspx - Public Safety Canada (2013c). Results from the Youth Inclusion Program – Atlantic Canada, Building the Evidence, Evaluation Summaries, ES-2013-34. Senior Evaluation Advisor, PS: Danièle Laliberté.
http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/rslts-yth-nclsn-prgrm/index-eng.aspx - Public Safety Canada (2013d). Risk and protective factors http://www.publicsafety.gc.ca/cnt/cntrng-crm/crm-prvntn/fndng-prgrms/rsk-fctrs-eng.aspx
- Public Safety Canada (2013e). Prevention Intervention Toronto (PIT). Building the Evidence, Evaluation Summaries. Senior Evaluation Advisor, PS: Donna Smith-Moncrieffe. http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/prvntn-ntrvntn-trnt/index-eng.aspx
- Public Safety Canada (2013f). Results of Alternative Suspension (AS) Program, Building the Evidence, Evaluation Summaries. Senior Evaluation Advisor, PS: Danièle Laliberté. In press.
- Public Safety Canada (2013g). Results of the Toward No Drugs Program, Building the Evidence, Evaluation Summaries. Senior Evaluation Advisor, PS: Donna Smith-Moncrieffe. http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/rslts-twrds-ndrg-bs/index-eng.aspx
- Public Safety Canada (2012). Interim Report: Synthesis of Impact Evaluation Studies (2009-2012).
- Public Safety Canada (2011). Prevention of Youth Gang Violence in Canada: Approaches and Strategies.
- Public Safety Canada (2008). "Family-Based Risk and Protective Factors and their Effects on Juvenile Delinquency: What Do We Know?", Building the evidence – Youth at risk, 2008 YR01
- R.A. Malatest & Associates Ltd (2013a). Leadership and Resiliency Program, 1st Annual Inter-site Evaluation of the Leadership and Resiliency Program, March 2013. NCPC. Senior Evaluation Advisor, PS: Giselle Rosario.
- R.A. Malatest & Associates Ltd (2013b). Multi-site Evaluation of the Alternative Suspension Program, First Annual Evaluation Report. Submitted to PS by Trudy Johnson. Senior Evaluation Advisor, PS: Danièle Laliberté.
- R.A. Malatest & Associates Ltd (2013c). Evaluation of the Life Skills Training Program: Final Report. Submitted to PS by Eleanor Hamaluk. Senior Evaluation Advisor, PS: Giselle Rosario
- Smith-Moncrieffe, D. (2013). "Innovative Synthesis Methodologies," Presentation at the Canadian Evaluation Society National Chapter Meeting, Ottawa Convention Centre, February 21, 2013.
- Smith-Moncrieffe, D., Lauzon, C. and D. Jobin (2008). ''Building Quality from Within: Just- in-Time Evaluations at the NCPC,'' Actualités Justice Report, vol. 23, nos. 2-3.
- Social Development Research Group:
http://www.sdrg.org; Breaking the Cycle: http://www.canadiantraininginstitute.com/breaking-the-cycle/; "Healthy Relationships Curriculum" developed by Men for Change: http://www.m4c.ns.ca/index.html; Oakland Men's Project: http://www.paulkivel.com/resources/articles/item/70-the-oaklands-men-project. - Sussman, S., Dent, C.W. and A.W. Stacy (2002). "Project Towards No Drug Abuse: A Review of the Finding and Future Directions," American Journal of Health Behavior, 26 (5): 354-365.
- Tarolla, S.M., E.F. Wagner, J. Rabinowitz, and J.G. Tubman (2002). "Understanding and Treating Juvenile Offenders: A Review of Current Knowledge and Future Directions." Aggression and Violent Behavior 7(2):125-43.
- Welsh, B. (2007). "Evidence-Based Crime Prevention: Scientific Basis, Trends, Results and Implications." Public Safety Canada.
- West, T. and J. Crompton (2001). "Programs that Work: a review of the Impact of Adventure Programs on At-Risk Youth," Journal of Park and Recreation Administration, 19 (2):113-140.
- Wilson, S. J. and M.W. Lipsey (2000). "Wilderness challenge programs for delinquent youth: A meta-analysis of outcome evaluations." Evaluation and Program Planning, 23, 1-12.
- Wortley, N.S., Tanner, J., Brooks, M. and S. Somerton (2012). 6740-C1 City of Toronto Project. Prevention Intervention Toronto (PIT) Program, Youth Gang Prevention Fund, Final Report, June 30, 2012. Vol. I: Research Findings, Centre for Criminology & Socio-Legal Studies, University of Toronto. Submitted to Public Safety. Senior Evaluation Advisor, PS: Donna Smith-Moncrieffe.
- Yessine, A. (2011). "Risk Factors for Delinquency among Canadian Youth: Current knowledge and future directions," Research Report: 2011-05, Public Safety Canada.
End Notes
1 For more information, consult PS publications (Yessine, 2011; Public Safety Canada, 2008) and PS web site for a more thorough description of the risk and protective factors associated with crime and delinquency.
2 Organizations such as Substance Abuse and Mental Health Services Administration (SAMHSA) and Blueprints for Healthy Youth Development recommend intervention programs that have been evaluated with this level of rigour. These are model and promising programs that have already been evaluated and shown to be effective in preventing and reducing crime.
3 The current thematic paper focuses on 12 to 17 year olds whereas another upcoming paper will target 6 to 11 year olds.
4 Evaluation results of PSI-MTL/GDR, YIP-Québec and LRP are not available at this time.
5 The majority of funded projects worked with youth in this age range although a few accepted individuals up to the age of 30.
6 There are currently 16 service sites in the following communities and locations: British-Columbia (Abbotsford, Chilliwack and Surrey), Alberta (North Edmonton and South Edmonton), Saskatchewan (2 sites in Regina and 1 in Moose Jaw), Manitoba (Winnipeg and Seven Oaks), New Brunswick (Moncton), Nova Scotia (Dartmouth and Glace Bay), Newfoundland and Labrador (St. John's, Corner Brook and Grand Falls-Windsor).
7 Social Development Research Group:
http://www.sdrg.org; Breaking the Cycle: http://www.canadiantraininginstitute.com/breaking-the-cycle/; “Healthy Relationships Curriculum” developed by Men for Change: http://www.m4c.ns.ca/index.html; Oakland Men's Project: http://www.paulkivel.com/resources/articles/item/70-the-oaklands-men-project
8 According to the Eurogang criteria, youth belong to a gang or troublesome group of friends if they regularly hang out with a group of three or more people; AND they have been part of this group for more than one month; AND their group has engaged in at least one illegal activity; AND their group engaged in illegal activities on two or more occasions (Wortley et al, 2012).
9 Only AS participants that quit very early in the program were considered for inclusion in the comparison group.
10 For more details on methodology and interim findings, consult Public Safety Canada publications (2012, 2013a, b, c, d, e and f) and Laliberté (2013).
11 To ensure this paper could be used for a lay audience, the data was converted to user friendly classifications. However, we could have used the Comprehensive Meta-Analysis Software (CMA) that would have converted to effect sizes all the percentages, F and d coefficients to ensure the data was standardized. This is another valid method to synthesize different types of data to make strong conclusions about trends in the data.
12 Smith-Moncrieffe (2013) for a description of innovative synthesis methodologies applied at Public Safety Canada.
- Date modified: