ARCHIVE - An Interim Evaluation of the National Crime Prevention Centre Youth Gang Prevention Fund Program

Adobe Acrobat version (PDF 524KB)

Table of Contents

• Executive summary
  • Background
  • Conclusions
  • Recommendations
• Introduction
  • Background
  • Objectives of the Youth Gang Prevention Fund
  • Purpose of the Interim Evaluation
  • The YGPF Logic Model
• Evaluation Framework
  • Data Collection Methods
    • Document Review
    • Literature Review
    • Budget Data Review
    • File Review
    • Key Informant Interviews
  • Report Structure
• Evaluation Findings
  • Relevance
    • Continued Relevance of the YGPF’s Objectives
  • Program Design and Delivery
    • Level and Allocation of Resources
    • Processes, Procedures and Administration
    • Relation between Actual and Intended Project Characteristics
    • Contribution to Knowledge Regarding the Gang Phenomenon in Canada
  • Success
    • Youth Program Participation
    • Impact on Risk and Protective Factors
    • The State of Canadian Gang Literature and NCPC’s Contribution
    • Unintended Impacts
  • Cost-Efficiency
    • Costs Associated with Project Implementation
    • Stakeholder Perceptions of Cost-efficiency
• Conclusions
  • Relevance
  • Program Design and Delivery
  • Success
  • Progress toward meeting longer-term objectives
• Recommendations
• References
• Appendix A – Evaluation Framework
• Appendix B – Additional Tables
• Appendix C – List of Project-Specific Tables
• Appendix D – File review template
• Appendix E – Interview guides
  • Interview Guide for Provincial Representatives
  • Interview Guide for NCPC Staff
  • Interview Guide for YGPF Project Contacts
  • Interview Guide for Crime Prevention Researchers and Practitioners
  • Interview Guide for NCPC Management
• Appendix F – Review of recent CAnadian gang literature

Executive Summary

Background

Overview of the program

The YGPF provides time-limited funding for anti-gang initiatives in communities where youth gangs are an existing or emerging threat. It supports the development and implementation of tailored interventions aimed at youth who are in gangs or at risk for joining gangs. Treasury Board approved $33.6 million over five years for YGPF. Seventeen projects across Canada have received funding to date, and one project is in development. These projects have been approved to date for approximately $23.6 million of contributions from the YGPF.

Purpose of the evaluation

NRG Research Group was contracted by the NCPC to conduct an interim evaluation of the YGPF. The overarching goal of the evaluation is to determine if, and to what extent, the YGPF is being implemented as intended, and if it is making progress towards its long term outcomes. The key evaluation issues pertain to relevance, design and delivery, success (immediate outcomes), and lessons learned. These were assessed using multiple indicators and data sources.

Evaluation method

After reviewing the evaluation framework in the ARAF (which highlights the key questions to address in relation to program relevance, design and delivery, success (immediate outcomes), and cost-efficiency), reviewing relevant documentation, and consulting with NCPC, NRG prepared a revised evaluation framework. The evaluation framework was approved by the NCPC Evaluation Unit.

To enhance the reliability and validity of the evaluation findings, multiple data sources were used when available. The data collection methods included reviews of relevant documents, relevant literature, budget data, NCPC files, and key informant interviews.

Conclusions

Relevance

Canada's youth gang problem has escalated and continues to escalate

In 2009, three years after it began, the objectives of the YGPF remain highly relevant and more important than ever before. The youth gang problem appears to be more severe than ever before. Canada's crime statistics indicate that youth are increasingly involved in crime and that their rate of violent offending and homicide is higher than ever before. Crime prevention researchers also attest that youth gang activity is a significant and growing problem, and has been for the past 10 years or more. These facts point to the need for continued service provision to at-risk youth and youth involved in gangs.

Youth are using the services offered through the YGPF

The number of youth that are being served by YGPF-funded projects also indicates that it continues to be relevant. Projects funded by the YGPF have provided service to almost 1000 youth since the program began, and many of the projects are at or near their capacity. The demand for services is high, considering that capacity was reached quite quickly.

YGPF is not duplicated by other levels of government

The YGPF plays a unique role in the social program arena, and its role is not duplicated elsewhere. The YGPF is a program with a wide and longer-term scope, and its efforts to fund youth gang prevention projects are not duplicated by other levels of government. There are also few other gang prevention initiatives that have started since 2006, reinforcing the fact that the YGPF is leading the way in gang prevention in Canada, and that its efforts are not duplicated elsewhere. Many of the youth gang prevention programs that are funded by the YGPF would probably cease to operate without NCPC funding, or would have significantly reduced scope of activities.

YGPF is in the scope of broad government objectives

The YGPF continues to be relevant to broader government objectives. The YGPF's objectives are in line with Public Safety Canada's objectives as stated in its 2009-2010 Report on Plans and Priorities, were congruent with the Government of Canada's priorities stated 2006 and 2007, were mentioned in the Performance: The Government of Canada's Contribution reports in 2005-2006 and 2007, and are also in accordance with the RCMP's Strategic Priorities.

Finally, the YGPF is relevant and important to its stakeholders and project managers, who mostly espouse positive views of the federal government's role in youth gang prevention through the YGPF. There is strong support of the government's involvement because it is the only organization that can provide long-term funding for such projects.

Program Design and Delivery

The selection process was effective in identifying appropriate projects to fund

The selection involving the provincial representatives was very effective, since the provinces have more familiarity with the areas and programs in need of funds. The drawbacks about the selection process related to the fact that it excluded smaller communities, that funds were not allocated per jurisdiction, and that the "first come first served" selection model lead to projects seeking approval before they were ready.

NCPC's roles and responsibilities were defined, but not communicated

The roles and responsibilities of the NCPC were well defined, but the roles and responsibilities were not adequately communicated to the funded projects, which were not aware of the NCPC's definitions concerning its own role. The communication problems affected program officers, also. Their roles were not clear, forcing them to make up roles as the program progressed.

The application and approval process was too lengthy and harmed service delivery

The application and approval process was onerous and in need of streamlining. Information about the approval process obtained from the file review indicates that more than three-quarters of projects had approval processes that took more than four months, and one-third took more than six months. The accountability process should be restructured – the bureaucracy of the process was seen as inhibiting programs' capacity to deliver services. The resources that were initially provided to funding applications were not clear, and resulted in delays from the review committee. Further guidance and better templates would have helped project contacts and program officers.

After project approval fund transfers also took too long

After the lengthy approval period, there were further delays in transferring funds to the projects. These delays negatively influenced morale in recipient communities, and created some pessimism about the NCPC and federal government's commitment to the projects. The volume of reporting required from projects is also excessive (quarterly accounting and activity reports, amounting to eight reports annually), and consumes resources that could be spent on service delivery.

Tools and resources to support projects were planned but not delivered successfully

While NCPC managers were generally confident that they had provided sufficient resources to execute their projects successfully, program officers noted that the projects did not receive enough support, that resources were probably provided to projects too late (i.e., after the program had begun), and that the training was not sufficiently specialized. This divergence of opinions between NCPC management and program officers suggests that the structures and directives for resource development were arranged by management, but that they were not followed-through with actions giving resources to the funded projects.

Project contacts had diverse experiences. In fact, there experiences were so divergent that we believe the NCPC did not have a clear process to deal with their needs. Some projects received funds for external training, while some received training from NCPC, and other received none at all. NCPC is advised to further investigate the reason for the diversity of experiences. Project contacts found the documents provided by NCPC to be helpful, and the most frequently requested resource was a conference for YGPF-funded project administrators to meet and share their experiences.

YGPF projects had the characteristics needed for funding

The projects funded by the YGPF included most of the characteristics that they were expected to have. The projects did screen and identify at-risk youth and youth involved in gangs; most projects provided some quantification to their risk assessments. The projects all addressed risk factors at multiple levels, and half addressed factors at all levels. Most projects appeared to be based on literature and/or best practices, although the file review did not provide a lot of information about the projects' foundations. The projects also evidenced a high degree of partnering with other community organizations.

NCPC produced documents to provide guidance in developing successful gang prevention and intervention projects

Our literature review identified a number of documents that were prepared by the NCPC and which would have helped to develop effective, evidence-based projects. These documents are publicly available, in the expectation that they can influence the practice of gang prevention and intervention. Success

Funded projects delivered the appropriate services to the appropriate participants

YGPF-funded projects did deliver the required services to participants. They tended to deliver service using two models – intensive intervention only, or intensive intervention plus non-intensive service provision. Because of the different models, the number of participants in projects ranged from less than 10 to more than 100. The modal categories were 10 to 20 and 20 to 30. Project annual reports did not provide sufficient information about length of involvement to make conclusive statements about the length that youth had participated in the program, but figures presented regarding drop-out rates suggest that the participants are invested and engaged in the programs.

Positive impact on participants cannot be ascertained yet, but is on track to succeed

Qualitative information provided by key informants and in the Project Annual Reports indicates that positive changes are being observed among YGPF project participants. These improvements are being observed at all levels of risk factors. Because of the limited time that most projects have been operating, there is insufficient quantitative information available to make conclusions about the success of the program to date.

YGPF has not yet positively impacted Canada's literature on gangs, which remains limited

Canadian literature with primary research on gangs and gang prevention remains limited. Since the 2002 Canadian Police Survey on Youth Gangs, there has not been any attempt to quantify the scope of the youth gang problem at the national level. While the author of that report has produced some estimates of the youth gang scope in 2007, there has been no follow-up to the 2002 study conducted, leaving a large gap in Canada's knowledge about its current gang problem.

The published research from 2003 to date includes a number of reviews of programs and international research, some research on specific ethnic gang experiences, and some investigations of the predictors of gang membership and violence. The most substantial document produced in that time that would be helpful to inform the youth gang situation came from the Government of Ontario in the form of the Roots of Violence report.

Progress toward meeting longer-term objectives

Based on our evaluation, we are confident that YGPF has systems in place to ensure that the YGPF will continue toward reaching many of its longer-term objectives as far as program delivery is concerned. We have found that many projects have been funded and are operational, and that more will be in the near future. While there is not a lot of quantitative data concerning changes in risk and protective factors at this time, there is optimism expressed qualitatively.

On the knowledge development side of the program, we have some concerns about the knowledge dissemination activities of the YGPF-funded projects. To date, only about one-third of projects have engaged in community-level knowledge dissemination, and the remaining two-thirds did not address this component of the project in their Project Annual Reports. Without knowledge dissemination, the community-level changes that are anticipated in the logic model will not occur. Also, most YGPF projects have not engaged in the creation of primary Canadian research about youth gang prevention, and so the Canadian literature on the topic remains limited.

NCPC has provided YGPF-funded projects with a template for information capturing to ensure that projects are comparable over time and between each other. We have found, however, that the projects have not used these templates consistently, resulting in information that is disjointed and difficult to compile as a whole.

Recommendations

Based on our evaluation, we have the following recommendations:

1. NCPC should organize a regular meeting between YGPF project managers.
2. Reduce the administrative burden imposed by NCPC on YGPF projects.
3. Review and clarify the roles and responsibilities of NCPC staff involved in the administration of the YGPF.

Introduction

The National Crime Prevention Centre (NCPC) is located within the Community Safety and Partnership Branch of Public Safety Canada (PS). The NCPC contributes to the federal government's leadership role in sustaining community safety. It does this, in part, through providing grant and contribution funding to community-based crime prevention projects.

One of NCPC's funding programs is the Youth Gang Prevention Fund, or YGPF, which was approved by Treasury Board on October 19, 2006. The YGPF provides time-limited funding for anti-gang initiatives in communities where youth gangs are an existing or emerging threat. It supports the development and implementation of tailored interventions aimed at youth who are in gangs or at risk for joining gangs.

The YGPF's Accountability Risk and Audit Framework (ARAF) provides details on how to monitor, evaluate, and report on results throughout the delivery of the YGPF. The ARAF specifies a set of reporting commitments, one of which is the current report – the Interim Year Three Progress Report. In February, 2009, NRG Research Group was contracted to conduct the Interim Evaluation. The purpose of this report is to assess the extent to which the YGPF is being implemented as intended. The focus of this interim report is to identify the fund's strengths and weaknesses and to make recommendations for alterations or enhancements in preparation for the Year Five Outcome-based Evaluation.

Treasury Board approved $33.6 million over five years for YGPF, including $27.36 million in vote 5 (grants and contributions) funds and $6.24 in vote 1 (operations, salaries and benefits, accommodation) funds. The development of projects under the YGPF began in 2006-07. Seventeen projects across Canada have received funding to date, and one project is in development. These projects have been approved for approximately $23.6 million of contributions from the YGPF. An additional $4.5 million has been set aside for the project currently under development. A list of funded projects, funding recipients, locations, funding start dates, total project budgets, and expected contributions from NCPC, are presented in Table 1.

Background

Most young people who become engaged in criminal activity do so only occasionally, and their activities go unnoticed by society and the justice system. But for a small group of youth, these activities escalate and snowball until the commission of criminal acts become part of their identity, and social attachments develop with groups of like-minded individuals. This group is referred to as a criminal 'youth gang.'

While only a small fraction of youths are involved in youth gangs, membership and association with a youth gang is a significant predictor of involvement in criminal activity, and youth gang members commit many times more crime compared to other youth. A study of Toronto-area youth indicates that youth gang members are more than six times as likely compared to other youth to be engaged in a variety of criminal activities (Wortley & Tanner, 2006).

More troubling is the development and growth of youth gangs in Canada. Although there is limited research on the size and scope of youth gangs in Canada over time, the results of the 2002 Canadian Police Survey on Youth Gangs (Astwood Strategy Corporation, 2003) indicate that there were more than 400 youth gangs in Canada with more than 7,000 members. More recent estimates indicate that those figures had probably increased by more than 80% by 2007 (Astwood Strategy Corporation, 2008). While there has not been a quantitative tracking study of youth gangs since 2002, anecdotal evidence supports the position that the problem has grown substantially since then. For example, 2006 marked the highest youth homicide rate that Canada has ever recorded (Dauvergne, 2007), and 2005 became known as Toronto's 'Year of the Gun' (Chettleburgh, 2007).

To intervene with the social problem of youth gangs in Canada, the Government of Canada, in 2006, moved to address the problem through "work[ing] with the provinces and territories to help communities provide hope and opportunity for our youth and end the cycle of violence that can lead to broken communities and broken lives" (Speech from The Throne, April 4 2006). Just one week after the Speech, it was announced that the 2006 Budget included "$20 million over two years for youth crime prevention with a focus on guns, gangs and drugs" (Public Safety Canada, 2006). From these funds, the Youth Gang Prevention Fund (YGPF) was created.

Crime prevention experts almost unanimously agree that early identification and intervention of youth who are at-risk of joining youth gangs is the most effective method of reducing the size and severity of the youth gang problem in Canada and around the world (see, for example, Chettleburgh, 2007; RCMP, 2006; Tanasichuk & Wormith, 2008). By identifying and screening youth for known risk and protective factors, individuals who are most at-risk can be included in programming to reduce the chance of joining a youth gang, or assist with exit from one. Prior to the implementation of the YGPF, NCPC found that few gang prevention programs in Canada focused on these risk factors, however. In part due to this lack of focused interventions among gang prevention programs in Canada, the YGPF sought to fund programs that would engage the community as a whole so that the program information could be disseminated and the results used to further develop crime prevention practices in Canada. The projects were also required to follow best practices, and record empirical observations about program success throughout delivery.

The YGPF provides funding and resources for gang prevention initiatives in communities with current or emerging youth gang problems. The programs it supports are tailored to reach (1) youth who are in gangs and/or (2) youth who are at-risk of joining gangs. Programs were selected by working with provincial and territorial governments to identify communities experiencing gang issues and appropriate community-based and municipal organizations who could develop projects. The first YGPF project was funded in May, 2007. Funding extends to March 31, 2011. As of February 2009, there are 18 projects funded or being developed, and one project under review. The total amount allocated to the fund for the five-year funding period is $33.6 million.

Objectives of the Youth Gang Prevention Fund

Through funding to community-based and municipal youth gang prevention programs, the YGPF intends to:

1. Invest in initiatives in communities where gang threats exist, targeting youth in gangs or at-risk of joining gangs, to improve community safety and prevent crime and victimization.
2. Provide funding to programs that address known risk and protective factors associated with youth violence and gang membership to reduce the risk of gun violence and gang activity.
3. Give young people alternatives to joining gangs that will help them develop pro-social skills and attitudes.
4. Support evidence-based, community-level interventions targeting gun violence and youth gang issues by addressing root causes of the problems, and contribute to the development of knowledge in the field.

Purpose of the Interim Evaluation

NRG was contracted by the National Crime Prevention Centre (NCPC) to conduct an interim evaluation of the Youth Gang Prevention Fund (YGPF). The overarching goal of the evaluation is to determine if, and to what extent, the YGPF is being implemented as intended, and if it is making progress towards its long term outcomes. The primary evaluation questions are:

1. To what extent are the objectives of the YGPF still relevant?
2. Has the YGPF been designed and delivered as planned? With respect to:
   1. The adequacy of program design, and
   2. The effectiveness and efficiency of program implementation.
3. Has the YGPF been successful in meeting its immediate outcomes (as outlined in the program logic model)?
4. Have appropriate systems and procedures been developed to ensure that the YGPF program will continue toward meeting its medium- and long-term objectives? In particular does NCPC have systems in place to collect the necessary performance information?
5. What are the recommendations and lessons learned?

The YGPF Logic Model

The YGPF logic model1 shown below begins with a set of activities leading to desired outputs. These outputs, in turn, are expected to produce desired outcomes both in the immediate term and in the long term. Given that this is an interim evaluation, we are addressing primarily the aspects of the logic model pertaining to activities, outputs and immediate outcomes; these areas are shaded in the logic model below.

Evaluation Framework

The YGPF ARAF contains within it an evaluation framework. The framework highlights the key questions to address in relation to program relevance, design and delivery, impact, and cost-efficiency. The framework also includes key indicators that will help in answering these questions and information sources that will provide data to evaluate performance on these key indicators.

After reviewing the evaluation framework in the ARAF, reviewing relevant documentation, and consulting with NCPC, NRG prepared a revised evaluation framework.

The evaluation framework, approved by the NCPC Evaluation Unit, is presented in Appendix A.

Data Collection Methods

To enhance the reliability and validity of the evaluation findings, multiple data sources were used when available. The data collection methods included:

• Review of relevant documents.
• Review of relevant literature.
• Budget data review.
• File review.
• Key informant interviews.

Document Review

The document review focused on materials related to the development, implementation and management of the YGPF. Materials included in this review included:

• Policy documents produced by NCPC and other government sources related to the issue of youth gang prevention.
• The ARAF for the YGPF.

Literature Review

The environmental scan of youth gang-related literature focused on the most current publicly available documents on the subject of youth gangs and youth gang prevention. The following sources formed the basis of the literature review:

• Academic literature.
• Government documents which were publicly available on the internet.
• Government documents which were provided by the NCPC.
• Documents mentioned specifically by key informants during their interviews.

Budget Data Review

We analyzed funding information from a variety of project management sources, including the contribution agreement and any amendments for each project, and the Grants and Contributions Information Management System (GCIMS).

File Review

Each funded project undergoes a comprehensive evaluation by a third-party contractor. These evaluation reports are a rich source of data regarding the performance of each funded project versus stated commitments. The evaluation reports were supplemented, where necessary, by additional information obtained from the Ministerial Project Summaries and Evaluation Plans. A file review template was used to compile data from the 15 reports2 that were available to NRG as of February 28, 2009. This template is available in Appendix D to this report.

Key Informant Interviews

NRG interviewed a total of 36 individuals offering unique insights and perspectives on the implementation of the fund. These individuals represent the foremost experts in research, policy, and delivery of youth gang prevention in Canada. The key informants included representatives of five different groups:

• NCPC senior managers and regional directors (9).
• YGPF Program Officers and other staff with direct contact with projects (6).
• YGPF Project Contacts—including project executive directors, program managers, and coordinators (13).
• Provincial representatives (4).
• Leading researchers and practitioners in the area of crime prevention (4).

If deemed useful by the project contacts, more than one individual from the funded project could participate in the interview at the same time. All interviews were conducted by telephone. Informants were given the opportunity to participate in English or French. Two informants completed the interview in French.

Report Structure

The report that follows is presented by the evaluation themes, and each sub-section begins with the conclusion based the information that we collected. These conclusions answer the evaluation questions delineated in our evaluation proposal. The data on which the conclusions are based are presented after the conclusions.

Evaluation Findings

The findings are organized around the central issues to be addressed in the evaluation framework—relevance, design and delivery, successes, and cost-efficiency.

Relevance

Continued Relevance of the YGPF's Objectives

The objectives of the fund remain highly relevant. It can be argued that they are more relevant today than when the fund began, given the increasing problem of gang-related crime in Canada.

The Size and Severity of the Youth Gang Problem in Canada

Available data indicate that Canada's youth gang problem has been increasing and continues to increase in severity and size over time. Canada's crime statistics also indicate that youths' rate of violent offending is higher than ever before, both as measured by assault and homicide. No quantitative, primary research about the size and scope of youth gangs in Canada has been conducted since 2002. Crime prevention researchers attest that youth gang activity is a significant and growing problem, and has been for the past 10 years or more.

While there has not been any primary research assessing the number of youth gangs and youth gang members since the 2002 Canadian Police Survey on Youth Gangs (Astwood Strategy Corporation, 2003), recent estimates suggest that the number of youth gang members in major Canadian youth gang hubs has increased by more than 80% between 2002 and 2007 (from just over 6,500 to more than 12,000; Astwood Strategy Corporation, 2008).

Crime statistics correspond to the estimated increase in the size of youth gang membership in Canada. In 2007, Canada's crime rate was down 7% from the previous year to a 30-year low, while youth crime was down only 2% (and had increased 3% in the previous year; Dauvergne, 2008). From 2006 to 2007, the overall crime rate was in its third consecutive year of decline, its 16th year of general decline, and had decreased 7% from 2006. While the rest of the crime rate has been declining since 1991, the youth crime rate remained stable after peaking in 1991. What is more, the commission of violent crime (typically common assault) by youth has been increasing since the 1980s, and has resisted the downward trend observed in other areas of crime commission and continued to increase since 2000. The youth homicide rate is also higher than ever before – while it decreased from the record-high of 85 per 100,000 in 2006 by 13%, it remained the second-highest youth homicide rate since 1961 (the first year it was recorded). In summary, while the whole of the crime rate was in decline for almost 20 years, the youth crime rate has been stable and the rate of violent offending by youth increased, including homicide.

Consistent with these statistics, crime prevention experts interviewed for the evaluation see the issue of youth involvement in gangs as a significant, and growing, problem in Canadian urban centres as well as in smaller towns. There has been a significant increase in the gang activity overall in the past 10 years, and law enforcement is now struggling to 'get on top of it.' It appears that the increase is due in part to a slow response to the problem by government and law enforcement in the mid-1990s, when much more attention was focused on biker groups at the expense of attention on emerging immigrant gangs like Calgary's F.O.B. and F.O.B. Killers, whose gang war has consumed the lives of more than two dozen people since 2002. Gangs have also almost completely infiltrated the prison system, exacerbating the problem.

Individuals working in the areas served by YGPF funded projects, who work with at-risk youth, agree with the experts on the severity of the gang problem and the increasing involvement of young people. There has been an increase in the intensity of the violence—e.g., use of hand guns as opposed to less lethal weapons—in addition to the increase in the number of young people associated with gangs.

The Number of Targeted Youth Being Reached by Funded Projects

We set out to determine if the number of participants in YGPF-funded projects increased or decreased over time to determine if it was relevant from the demand perspective. While it is too early in the YGPF to make conclusions about how the number of youth being served by projects is changing over time (only two projects have been serving participants for more than one year), the information we have found concerning capacity indicates that demand is high, as more than one-third of projects are close to capacity after operating for a short period of time.

A total of 964 youth have participated in one or more programs offered by 14 projects as of December, 2008. On average, there were 69 youth enrolled in each project. This figure includes youth who were still active at the time of the file review, but also those who were inactive, or who dropped out from the programs. While the number of youth served in each project varies, half (seven) of the projects served 21-50 youth (Table 2). Five projects (36%) served less than 20 participants. At the high end, two projects (14%) had more than 100 youth participants.

1 Includes youth who participated in intensive, non-intensive, or unspecified programs. Includes participants who were active, inactive at the time, or who have dropped out from the programs.

2 One out of fifteen projects has not started accepting youth participants.

3 One project alone had more than 500 youth participants, accounting for over half of the youth clients served by all projects.

One of the evaluation objectives was to determine if the projects are serving more or fewer youths over time. This question cannot be answered, given the length of time the funded projects have been in operation. Only two projects out of fifteen (13%; see Table 3) had been operating for more than one year as of December, 2008 (when the annual reports were produced). One project had been operating for less than 6 months, and eleven had served youth for less than one year. One project was still developing its youth referral service and had not started recruiting youth participants. Two project reports indicated that over the course of one year, the number of participants enrolled exhibited significant seasonal fluctuations.

In considering the data from the file review, we found that many projects are operating at more than 50% of their participant capacity. In most cases, this was reached in less than one year. As indicated in Table 5, nine projects (65% of all) operate at over half of their capacity, with five operating "close to full" (more than 90%). Five projects are below 50% of their capacity, including three that are below 30% of their capacity; typically these projects had been operating for just a few months and were in the process of recruiting their participants.

[*] Calculated as the current number of youth participants and the targeted number of youth participants

Size and Scope of other Gang Prevention Initiatives Implemented Since YGPF Inception

Few gang prevention initiatives have started since 2006, and any that have begun are small scale and not widely known. This reinforces the view that the YGPF is leading the way in gang prevention in Canada, and that its efforts are not duplicated elsewhere.

Our literature review did not identify any gang prevention initiatives that have started since 2006 aside from those funded by the YGPF. Literature suggests that there are about 80 anti-gang programs in Canada (RCMP, 2006). We found only one inventory of "Youth Gang Reduction Programs" and its scope included only Manitoba (Proactive Information Services, 2009). Still, knowing that there were 80 gang prevention programs in 2005 makes it clear that the YGPF's programs are a significant contribution to youth gang prevention services in Canada.

The relative lack of gang prevention programs in Canada was corroborated by key informants, who were asked to comment on what other gang prevention initiatives they were aware of that started since the inception of the YGPF. The majority of key informants, which included leading crime prevention researchers and practitioners who worked at local, provincial, and federal levels, were not able to mention specific gang prevention initiatives (and certainly not able to provide a list of programs that had started since 2006, when the YGPF began). This suggests a general lack of awareness among researchers and practitioners about other gang prevention programs at all levels.

Other key informants mentioned a few specific crime prevention and support programs such as P.A.L.S. in Durham, Winnipeg's Auto Theft Initiative, Breaking the Cycle in Toronto, and the anti-gang initiative launched by the Quebec Department of Public Safety, but focused more on the role of the YGPF in heightening awareness of the gang problem and finding effective ways to address it. These informants see the fund as showing the government's commitment to 'catch up' to the problem by using evidence-based approaches that are long-term in scope. It has also created heightened interest in organizations to apply to NCPC for money to fund prevention programs.

Is the YGPF Playing a Unique Role?

The YGPF plays a unique role in the area of gang prevention. Because of its wide and longer-term scope, and the NCPC's involvement in program development and evaluation, it also reinforces the importance of the NCPC. The YGPF may not be sufficiently distinct from the CPAF, and an argument can be made for combining the two funds to reduce the costs of administration.

After interviewing NCPC managers, provincial representatives, and crime prevention experts (and based on our literature search for other gang prevention initiatives discussed earlier) we concluded that the YGPF is playing a unique role with regard to crime prevention. Our conclusion is based on several observations. First, apart from the fund's projects and impacts, the fund is serving to draw much needed attention to the gang problem in communities across Canada. Second, the design of the fund—providing larger dollar amounts to multi-year projects adopting evidence-based approaches—is a design that will maximize the probability of producing significant, and measurable, positive outcomes. Third, the fund serves to emphasize the essential, and complementary, role that gang prevention plays in relation to enforcement and suppression initiatives (e.g., the Alberta Gang Crime Suppression strategy3). Fourth, the fund plays an important role in helping reinforce the unique role of the NCPC as a 'centre of excellence' on evidence-based practices for crime prevention. Fifth, the funding recipients' highly unique and groundbreaking projects, such as the RAGS program in Regina—the first program to focus on the exit of gang leaders.

In the process of our interim evaluation, we learned about the Crime Prevention Action Fund (CPAF). In reviewing the CPAF Goals and Priorities (see below), and in reviewing information provided by key informants, we do note there is overlap between the goals and the priorities of these two funds. While the primary purpose of our evaluation is to evaluate the uniqueness of the YGPF as a gang prevention initiative (we conclude that it is) we consider it due diligence to note that the overlap in these two funds may create opportunities for increasing operational efficiency moving forward.

Stakeholder Perceptions of the Relevance of the YGPF's Objectives

Considering the relative absence of other gang prevention initiatives, the YGPF's objectives of reducing youth gang membership and crime as a result clearly continue to be relevant. The youth gang prevention programs that are funded by the YGPF would cease to exist without that funding, or would have significantly reduced scope of activities.

Without the YGPF funding, projects would likely not exist, and would certainly not have their current scope without substantial funding from other sources. Out of 15 projects that are in operation, six receive more than 70% of their operating budget from the YGPF, and another six receive 51-70% of their budgets from the Fund. Only three projects receive over half of their funding from other sources.

Collecting supplementary information from project contacts and provincial representatives on current funding and possible sources of other non-federal funding strengthened our conclusion that most of these projects would either cease to exist or be only able to serve a reduced number of at-risk youths.

Level of Stakeholder and Public Support for Federal Government Involvement

The general public's view about the federal government's role in gang prevention has not been directly measured, but their support of crime prevention and law enforcement measures is testament to their likely support of the program. The gang prevention community of practice holds an ambivalent perspective about the federal government's role in youth gang prevention through the YGPF, but the majority hold positive views overall. There is strong support for the federal government's involvement because it is the only organization that can provide longer-term funding for such projects, but some cynicism about the political motivation of such funding exists.

The general public's support for the federal government's leadership with regard to youth gang prevention is not possible to ascertain from the data that are available. Public opinion polls indicate that the majority of Canadians support both crime suppression tactics and prevention tactics, but the support for suppression is significantly stronger (i.e., statistically significant) than the support for prevention. For example, a 2006 Ipsos Canada poll found that more than seven-in-ten Canadians Strongly agreed with "Forcing all young offenders charged with a gun crime to be treated as adults by the courts," and six-in-ten Strongly agreed with "On-going major police sweeps of communities where police believe gangs and guns are present." Only five-in-ten Strongly agreed with prevention initiatives such as "Implementing programs to support positive role-models for young people who are at risk of joining gangs" and "Implementing an employment and work training program for young people who are at risk of joining gangs."

Based on the results of a more recent poll, Angus Reid Strategies (2009) reported that just under half of Canadians said that there is a "serious national gang problem currently affecting everyone," and there is strong support for incarceration tactics – nine-in-ten Canadians supported increasing sentences for gang-related homicides, and mandatory sentences for drive-by shootings and serious drug crimes.

We asked the key informants about the kind of role they think that the federal government is playing with regard to youth gang prevention, and to contrast that against role they think the federal government should be playing.

The views of gang prevention researchers and practitioners were largely consistent. Virtually all of the informants said that the federal government has a large role to play in the youth gang issue. Several perceive the YGPF as a symbol of the federal government's leadership on the issue. They see the federal government as the only organization that can provide the level of funding support necessary to engage in intense long-term prevention work. They see the NCPC as the knowledge centre that accumulates and disseminates research and information on evidence-based practices. Furthermore, they see the federal government as the natural source for prevention initiatives and best practices, as the provinces have traditionally focused more on suppression initiatives.

Some informants suggested that the YGPF may be driven in part by political considerations, and exemplified this point by commenting on how announcements of gang prevention funds occur during election campaigns and limit funding to high profile urban centres. However, even those who have previously been sceptical of the motives behind federally funded crime prevention programs admit that the size of the fund and the fact that it funds multi-year, evidence-based projects with rigorous evaluations speaks to the sincerity of the federal government's desire to tackle the issue.

Perhaps ironically, the relatively large size of the fund, the longer term nature of the projects and the labour-intensiveness of the funded projects has created heightened expectations of the federal government with regard to prevention. A number of informants pointed to the need for the federal government to play a more sustainable role over the long term—for example, to continue to fund YGPF funded projects that show successful outcomes at the end of the initial funding period. A number of informants also mentioned that the YGPF has illustrated just how much investment is needed to address the gang issue.

Relationship between YGPF Objectives and Government-wide Priorities

The YGPF continues to be relevant to broader government objectives. The YGPF's objectives are in-line with Public Safety Canada's objectives as stated in its 2009-2010 Report on Plans and Priorities, were congruent with the Government of Canada's priorities stated 2006 and 2007, were mentioned in the Performance: The Government of Canada's Contribution reports in 2005-2006 and 2007, and are also in accordance with the RCMP's Strategic Priorities.

Public Safety Canada's 2007 Sustainable Development Strategy Report, which covers its objectives for 2007 to 2009, indicates that the NCPC will "provide national leadership on effective and cost-effective ways to prevent and reduce crime and victimization by addressing known risk factors in high-risk populations and places." In 2009-2010, Public Safety Canada's Report on Plans and Priorities further reiterated its ongoing objective as "implement[ing] evidence-based interventions in communities to prevent youth from joining gangs and support their exiting gangs" (P. 11).

The Treasury Board of Canada Secretariat included related priorities in the 2006 and 2007 Performance: The Government of Canada's Contribution reports, stating "As an increasing number of young people are becoming involved with guns, gangs, drugs, and other criminal activities in communities across the country, the federal government has also committed to adopting, in collaboration with the provinces, an effective crime prevention strategy for youths at risk. This initiative will help Canadian youths make positive life choices and ultimately help them shape their role as responsible, committed, and contributing community members." (This statement was the same in both years).

The YGPF program clearly addresses these objectives by:

1. Focusing on the prevention of youth gang membership, which is considered to be the most cost-effective way to prevent and reduce crime (Chettleburgh, 2008; RCMP, 2006; Tanasichuk & Wormith, 2008). This positive life choice will help shape these youths into contributing Canadian community members.
2. Reducing victimization by disengaging youth from youth gangs, because (a) youth in gangs are at a high risk of being victimized and (b) youth who disengage from gangs are less likely to victimize others (Wortley & Tanner, 2006).
3. Identifying and providing services to high-risk youth by screening them for empirically supported risk and protective factors, and then addressing those factors in the services provided.
4. Identifying and supporting projects in areas that are most affected by existing or emerging youth gang problems through collaborative effort between the NCPC, provinces, and municipalities.

Using Speeches from The Throne from 2006 to date, the Government of Canada's priorities related to law enforcement and crime prevention were assessed in relation to the YGPF's objectives. On April 4, 2006, the Speech included a statement that the Government would work with the provinces to help communities engage youth and "prevent criminal behaviour before it has a chance to take root." Just one week after the Speech, it was announced that the 2006 Budget included "$20 million over two years for youth crime prevention with a focus on guns, gangs and drugs" (Public Safety Canada, 2006). From these funds, the Youth Gang Prevention Fund (YGPF) was created.

On October 17, 2007, the Speech from The Throne included several passages related to law enforcement and crime prevention. The Government outlined (1) its Tackling Violent Crime bill, which sought to increase penalties for violent crime, (2) its Safer Communities strategy, which sought to increase penalties for some youth crime, and (3) its commitment to helping prevent youth criminal involvement through services targeted at high-risk youth. Based on Speeches from The Throne in 2006 and 2007, the Government's objectives were met through the YGPF. In 2008 and 2009, Speeches from The Throne have not included any mention of crime prevention or law enforcement.

In addition to its congruence with PS and the Government of Canada's priorities, the YGPF is in accordance with the RCMP's Strategic Priorities (RCMP, n.d.). The RCMP strives to work with young people to reduce their involvement in crime as victims and offenders through social development. To summarize, the RCMP seeks to "prevent youth crime by addressing its underlying causes." This objective is clearly in accordance with the YGPF's objectives.

Program Design and Delivery

Level and Allocation of Resources

Funded Projects' Ability to Meet the Needs of Targeted Youth

Projects received enough funds to operate, and the budgets that were provided were generally at the level that was requested by applicant. There were a number of cases in which unanticipated costs were encountered and not included in the project budget. There is clearly a need to provide more funding to youth gang prevention, however.

After gathering information on program funding and budgets from NCPC managers, program officers, and project contacts we concluded that the projects largely have the funds they need to operate and that the funding provided at the level requested—and in a few instances more. However, there were several categories of unanticipated costs which have created budgetary challenges and have led project contacts to submit budget amendments.

One common funding challenge relates to staff compensation. Project contacts have found it difficult to compensate staff at the level required to retain them over the long-term. This challenge has been particularly acute because many of the projects have an essential requirement for front-line workers who are men, and these men were often left their positions funded by the YGPF to obtain higher paid employment in other fields. According to the information we gathered from informants, the gap in what has been budgeted versus what would be appropriate is in the 10% to 15% range.

The number of staff hours initially assumed to be adequate for program delivery has in several programs proved to be inadequate. A number of programs underestimated the labour intensiveness of these projects and therefore underestimated the staff hours necessary to run the program. Several projects anticipated needing part-time staff and discovered, after starting the projects, that full-time staff were needed.

NCPC managers and crime prevention experts noted that, while the YGPF provide may provide adequate funds for its projects, a great deal more funding is needed to serve youth at risk for joining gangs. One manager commented that NCPC's entire grants and contributions budget could be used to fund gang prevention initiatives and it would not be 'over-funding' the issue. Crime prevention experts agree strongly with this view, as there are many proven risk and protective factors contributing to gang membership which can be addressed at many points in the development of young people.

Processes, Procedures and Administration

This section reviews data and information related to the processes, procedures, and administration of the fund. It evaluates (1) if the processes were in place to select the most promising or 'best' projects for funding, (2) if processes and procedures are efficient, (3) if responsibilities are clearly defined, and (4) if the programs have been given the appropriate tools and training from NCPC to help the meet their objectives.

Process of Selecting Target Projects for Funding

Involving the provincial representatives in the selection of projects was very effective, since the provinces have more familiarity with the areas and programs in need of funds. The drawbacks of the selection process related to the fact that it excluded smaller communities, that funds were not allocated per jurisdiction, and that the "first come first served" selection model led to projects seeking approval before they were truly ready. We also note that the requirements of projects for YGPF may not have been sufficiently flexible to allow proper community customization.

The process for selecting target projects for the YGPF was different than the process that NCPC has followed in the past. For the YGPF, potential sites were identified through consultation with the provinces, and then program officers and evaluation analysts worked closely with the organizations to develop their projects. We asked NCPC managers, program officers, and provincial representatives to evaluate the extent to which this targeted strategy was successful.

The selection process, while creating some problems for NCPC and for project contacts, was largely successful. It resulted in the identification of projects with characteristics that match the characteristics specified in the YGPF logic model. The selection process was successful for several reasons: it allowed for the appropriate targeting of limited funds, and ensured that communities with the most need, and greatest capacity to deliver (urban centres), would be served. In addition, the provinces are in a better position to identify which organizations have the capacity to take on programs that can deliver, so this maximizes the probability that the best programs will be selected. However, there needs to be a very good working relationship between the NCPC and the provinces to make this process effective. The process generally worked well in Saskatchewan, Manitoba, BC, and the Atlantic region. However, the province of Ontario did not respond to this process and told the NCPC to pick the programs.

Some drawbacks of this selection process came to light, however. One drawback is that the fund is open to being criticized for selecting certain communities over others. In fact, some individuals have criticized the fund for focusing only on major urban centres and not considering rural areas in serious need of help with youth gang issues. Also, the pre-selection of specific cities did not always lead to the selection of programs that have the capacity necessary to deliver a YGPF funded program. This was the case in Edmonton. Edmonton was targeted for funding, but no program existed in Edmonton that had the capacity to meet the terms and condition of the fund. As a result, the funding was given to a program in Calgary. Another example occurred in Quebec, where the provincial partner has not been able to finalize the project proposal for over two years.

Also, there was no notional allocation given per jurisdiction. In some cases, this created a sense that the funding would be provided on a 'first come, first served basis.' This, in turn, created competition to be among the first projects approved so funds would be available. This led to some projects being approved before they were ready on the ground to deliver.

Within the targeted selection process, the fund also needs to be flexible and sensitive to which best practices will work within a given community. The fund needs to be careful not to impose a program structure that is too rigid for the community to adapt – one of the criteria for program development is that it adapt to the context in which it operates. There were some instances in which the information we gathered from key informants indicated that the structure of the YGPF project did not map mesh well with existing initiatives in a target community.

Extent to Which Roles and Responsibilities are Clear and Appropriately Defined

The roles and responsibilities of the NCPC were well defined, but the roles and responsibilities were often not adequately communicated to the funded projects, which were not aware of the NCPC's definitions concerning its role. The communication problems affected program officers, also. Their roles were not clear, forcing them to make up roles as the program progressed. Problems with communication about roles of project officers appear to exist solely within the NCPC, as the project contacts did not express concern about the roles of program officers and thought they did an excellent job.

Roles and responsibilities are generally clearly defined. The program is coordinated and administered at the national level in Ottawa, and regional directors oversee program officers who directly work with the funded projects. NCPC Ottawa could be doing a better job in coordinating with, and communicating about, the various funded projects. NCPC should consider providing more frequent updates regarding project implementation, project status, and evaluation.

Also, while the role of the NCPC Review Committee4 (which reviewed the project proposals before funding was approved) is clear, we are not completely confident that this committee can serve purely as an unbiased review body. There was some concern expressed by informants that there could be some repercussions if certain projects are rejected.

Early on in the program there were some challenges to program delivery due to a lack of clarity of the roles and responsibilities of program officers. One problem was that program officers did not have program guidelines that defined policies, procedures, and roles for program officers. This, at times, led to confusion about the responsibilities of the program officers. Program officers noted that at the beginning of the project development process, individuals from research and evaluation and the program officers were trying to 'figure out' and 'negotiate' their responsibilities. The lack of role clarity had a negative impact on communication with the YGPF project managers, who did not necessarily see the difference in roles between the program officer and evaluation analyst, and would sometimes direct questions at the incorrect parties as a result.

Program officers would likely have been more helpful early in the approval process if they had a role on the external review committee. Program officers could have expedited the approval process by participating in the committee meeting and answering questions right then and there, as opposed to responding in writing and waiting for a response. Several program officers noted that it was clear that the external review committee did not always read what the program officer had given to them because the same question would be asked twice by the committee, or the answer to the committee's question was clearly indicated.

For effective delivery of projects of this size and complexity, the role of the program officer, and his or her effective relationship with the project contact is critical. If the relationship between the project contact and officer is not consistently strong, or if the officer is replaced, this heightens the risk that the projects will not deliver. A key reason that the program officer plays such a critical role is that the administrative burden placed on the projects by NCPC is substantial. This combined with the level of human resources necessary to implement and sustain the projects can create stress for those involved at the ground level.

Extent to Which Processes and Procedures are Efficient

The YGPF application process is onerous and in need of streamlining. The approval process is also too lengthy and the PS accountability process should be restructured – the bureaucracy of the process needlessly harms programs. Information about the approval process obtained from the file review indicate that more than three-quarters of projects had approval processes that took more than four months, and one-third took more than six months. The resources that were initially provided to funding applications were not clear, and resulted in delays from the review committee. Further guidance and better templates would have been avoided at least some of the delays.

After the lengthy approval period, there were further delays in transferring funds to the projects. These delays negatively impacted morale in recipient communities, and at times created pessimism about the NCPC and federal government's commitment to the projects. The volume of reporting required from projects is also excessive (quarterly accounting and activity reports, amounting to eight reports annually), and consumes resources that could be spent on service delivery.

The file review information provides evidence of a lengthy approval process. For eleven out of fourteen projects with data about the approval processing time, the time between submitting their YGPF proposal and approval was at least 4 months, and five took more than 6 months (Table 6). This was after the projects had spent some time in consultation with the program officer and evaluation analyst, developing the logic model for the program, establishing what a realistic budget might be, and so on. To put this in comparison, the average of 24 weeks was actually about one month faster than the average for the approval of projects funded by the National Crime Prevention Strategy between 2005 and 2007 (Prairie Research Associates, 2007).

[*] Between the proposal submission date and the approval date

There were a number of administrative processes of the fund that inhibited progress. The key administrative problems related to the project approval process, the templates and forms to be used for reporting, and the frequency and number of reports required.

For the most part, the information we collected from key informants indicated the recognition of an unnecessarily lengthy approval process. NCPC managers recognized that the application and approval process is too onerous and needs to be streamlined. While managers concede that significant accountability structures need to be put into place—these are, after all, multi-year and relatively high dollar value projects—they believe the accountability process within PS should be rethought. There was a sense that the department has become too risk-averse and the many approval processes in place are needlessly slowing the movement of dollars to help youth at risk of joining gangs or wishing to exit.

One manager related the example of the budget for a project being rejected because the budget included a line for food and refreshments for the youth in the project. Evidently, under the terms and conditions of the fund, the project did not have 'hospitality paying rights,' so the budget was rejected and the project was delayed further.

We compared the length of time needed to approve YGPF projects to the length of time needed to approve projects similar in scope but funded by other sources. For example, consider Alberta's Safe Communities Innovation Fund. This $60 million three-year initiative funds evidence-based projects with evaluation components, and was announced in January 2009. In less than four months from the program's announcement, thirty projects were approved for funding.

The long approval process appears to be rooted in the external review committee, with Public Safety Canada's accountability requirements, and the lack of clear direction to project contacts regarding how to complete the application in a manner that will minimize the time needed for approval. 

With regards to the NCPC review committee, the specific issue that program officers mentioned was the repeated requests for clarification of the same issues. Program officers submitted proposals to the NCPC review committee and then the committee, after a period of time that varied between weeks and months, responded with a series of questions and requests for further information. Program officers said that there were occasions where it was unclear if the committee fully read the proposal, or prior submissions, because the same questions were asked on multiple occasions and the information requested had previously been provided.

Also, the templates provided for preparing budgets, for example, were not clear or in some cases were not provided at all. What would have been very helpful are guidelines on how to interpret the terms and conditions, as well as budget templates that are clear and well organized.

There were also delays in getting the funds released to the projects after they were approved. This delay caused a number of negative consequences for the projects and the communities within which they operate. One consequence was that a number of projects moved forward and hired staff, but then did not receive the money from the fund to pay the staff on time. This created financial strain for the organizations, which often had to get bridge financing or seek larger lines of credit.

Another consequence of the funding delays was pessimism in the community regarding the delayed project itself. When project managers 'announced' the project in the community there was often a great deal of interest. However, the delay in getting the program going created some ill-will among community stakeholders toward the program funders and funding recipients.

The delay also caused interest and motivation in the community to wane. Many target organizations or projects are accustomed to working in an environment in which things happen more quickly. When there is an important emerging issue like youth gangs, there is the sense that one has to 'strike while the iron is hot.' Otherwise, organizations that are possible targets for funding may become reluctant to apply or will seek resources elsewhere.

Extent to Which Appropriate Tools, Resources and Training are Available to Support Projects

While NCPC managers were generally confident that they had provided sufficient tools, resources, and training to support their projects successfully, program officers noted that the projects did not receive enough support, that resources were probably provided to projects too late (i.e., after the program had begun), and that the training was not sufficiently specialized. This divergence of opinion between NCPC management and program officers suggests that the structures and directives for resource development were arranged by management, but that they were not followed-through with actions giving resources to the funded projects.

Project contacts had diverse experiences. In fact, their experiences were so divergent that we suspect the NCPC did not have a clear process to deal with their needs. Some projects received funds for external training, while some received training from NCPC, and other received none at all. NCPC should further investigate the reason for the diversity of experiences. Project contacts found the documents provided by NCPC to be helpful, and the most frequently requested resource was a conference for YGPF-funded project administrators to meet and share their experiences.

In addition to tools developed specifically for YGPF projects, our literature review identified a number of documents that were prepared by the NCPC on the topic of youth gangs, which would help to develop effective, evidence-based projects. These documents are publicly available, and thus are included in a subsequent section concerning public contributions.

In addition to selecting projects and providing funding, a key activity of NCPC, as outlined in the logic model, was to "provide support, resources, monitoring and assistance to projects to maximize the likelihood of results." We asked NCPC managers, program officers and project contacts to describe the kind of tools, resources and training that have been offered to the projects to help them meet their objectives.

Managers mentioned a number of ways in which the YGPF has helped the projects. Help was given early in the project development phase, when projects were given assistance with developing logic models (11 of 17 projects received logic model development sessions), and were provided evaluation guidelines and reporting templates. The 2007 Knowledge to Practice Seminar was also mentioned. This brought together gang experts, NCPC staff, and representatives from projects in the cities that were going to be receiving funds under the YGPF in an attempt to share best practices and build up a network among the projects.

The 'Community of Practice' was also mentioned specifically as an activity that will help projects achieve their objectives by senior NCPC management. This activity involves bringing the program officers of the projects together on a regular basis to discuss lessons learned and best practices which can then be shared with project managers.

In discussing the provision of tools, training, and resources among program officers, we learned that there was probably more support the fund could have provided. Evidently, the documents, forms, and templates were provided after the projects were developed. There were also areas that were not addressed directly that should have been—e.g., how to ensure worker safety, and obtaining liability insurance coverage. Several program officers create their own 'administration manual' for the project manager, a 'how to' guide for the project manager including instruction lists for reporting, templates, and forms.

Some of the training offered to project managers was, in retrospect, probably too broad, and needs to reflect better the kinds of skills needed to deal with the target population in question, such as training to deal with suicide intervention, ethnicity sensitivity, and safely handling volatile individuals.

The experiences of project contacts with regard to training and tools have been diverse. Some said that they have not received any training or tools from NCPC. Some said that they have received funds for training from NCPC as part of the fund, but that NCPC has had no further involvement in training. Others mentioned specific training or resources that have been provided. The two-day workshop on evaluation was mentioned by a few project contacts. Some project contacts also mentioned that they have received training funds which they have used to attend either the Youth Justice conference in Toronto or the National Gang Crime Research Center Training Conference in Chicago.

Several project contacts suggested that NCPC organize a conference to bring together all of the funded projects at a central location. The purpose of this conference would be for each project to present to the larger group what they are doing, what is working well, and what some of the main challenges have been. Funded projects could share information and benefit from experiences and 'lessons learned' from others. The information exchange could be continued after the conference through the establishment of a network of project contacts.

The knowledge transfer opportunities of such an event are obvious. However, this type of conference could also serve as an important source of motivation part way through the project. Many projects' administrators were exhausted by the approval and start up process. There projects are now up and running and they are beginning to see results. The projects are very proud of the work they are doing, and giving them an opportunity to present their unique projects in a forum of like-minded professionals would likely invigorate and validate their efforts.

Relation between Actual and Intended Project Characteristics

The file review data indicates that the projects funded by the YGPF included most of the characteristics that they were expected to have. The projects did screen and identify at-risk youth and youth involved in gangs; most projects provided some quantification to their risk assessments. The projects all addressed risk factors at multiple levels, and half addressed factors at all levels. Most projects appeared to be based on literature and/or best practices, although the file review did not provide a lot of information about the projects' foundations. The projects also evidenced a high degree of partnering with other community organizations.

The YGPF logic model specifies key characteristics that YGPF projects must demonstrate to be eligible for funding. The first characteristic that was required of YGPF projects is to "Target youth at risk of joining gangs or wishing to exit." The Project Annual Report file review captured extensive information about this and other key requirements that projects were expected to meet. As Table 7 indicates, all projects used some method to screen their participants. Two-thirds used established risk assessment tools, and one-third used more qualitative methods. The effectiveness of the risk assessments was not addressed in the Annual Reports.

Fifteen of the projects provided information about the risk factors that they used to screen their participants. Twelve of the projects provided information about the number of youth who were involved in gangs at the time of screening. We can see that of these twelve, almost half have more than 50% of participants involved in gangs at the time of screening. Other commonly used screening criteria mentioned in the Annual Reports include involvement in criminal activity, substance abuse, and school attendance. Most projects reported on these factors, and based on the high number of participants exhibiting risk factors we believe the projects demonstrated that the youth who they serve are at-risk of gang membership. More information about risk factors can be found in Appendix B.

[*] Out of fifteen projects, thirteen report data on risk factors by count, while two report data on different scales.

1 Many projects report data on each criminal activity; the risk factor "current/previous criminal activities"is defined as the conduct of any one of the criminal activities.

2 Substance abuse problems are defined as using any illegal or non-prescribed drugs more than once per month.

3 Negative peer influences include having friends who have gang involvement, and/or who conduct delinquent/criminal activities.

The second requirement of YGPF projects is that they Address multiple, known risk and protective factors at the individual, family, peer, school, and community levels. Out of the 15 projects that reported data on the risk factors that they addressed, almost half included activities to impact all five levels. The remaining projects addressed between three and four levels. Most frequently, the projects addressed risk factors at the individual level (100%), family level (93%), and school level (93%), and the majority of projects addressed six to ten risk factors.

Source: project annual reports and Ministerial project summary

The third requirement is Tailoring the approach to the local context. Based on our review of the Project Annual Reports, 13 out of 15 projects reported some form of adaptation of a program model to fit the local context. The most frequently mentioned tailoring involved ethnicity-related contextual changes, and also modifying program delivery and screening to match the youth of the service area.

The fourth requirement is Adopting an evidence-based approach. From the information included in the file review, we found that the evidence most frequently consulted during program development was the literature on gang prevention and intervention, which about half mentioned. Twelve of fifteen projects' Annual Reports mentioned at least one source of evidence that was consulted.

[*] Examples include: the Medicine Wheel approach (containing traditional circle practices) and the Circle of Courage approach (developed based upon native values and the medicine wheel).

The fifth and final requirement of YGPF-funded projects is Involving community partners to address relevant risk and protective factors. In this regard, the Project Annual Reports suggest that projects have succeeded. All organizations worked with at least three partners, and five worked with 11 or more partners.

1 These organizations are named in the annual report; they each play a somewhat important role in the programming planning and delivery. These exclude the organizations that team up with the funding recipient (i.e., organizations that work as partnerships)

2 Excluding NCPC

Contribution to Knowledge Regarding the Gang Phenomenon in Canada

The NCPC has developed numerous resources and publications related to youth gang prevention. Some of these were developed for YGPF projects with the goal of helping with funding applications, program evaluation, program development, and general information concerning youth gangs. The NCPC has taken steps to disseminate this information through websites, mailing lists, and promotional activities. Some of the YGPF-funded projects have presented at conferences in Canada and the US. At the community level, only four out of 15 YGPF-funded projects indicated that they had held or were planning to hold knowledge dissemination events.

There have also been public consultations and workshops held for crime prevention practitioners, academics, communities, and government workers. These appear to have been reduced in number since 2007, and have been geographically focused only Saskatchewan, Manitoba, and Ontario. Unfortunately, the YGPF has not yet resulted in a network of gang prevention stakeholders, but the NCPC has a plan to develop this network in 2009/2010.

Resources developed related to youth gang prevention

One of the NCPC's most important goals is to contribute to the information that is available to YGPF-funded projects to maximize their chances of success, as well as to other stakeholders who may use this information. The YGPF program logic model includes a 'knowledge stream' of activities to develop and disseminate knowledge that will improve communities' abilities to address gang issues. The logic model specifies this activity as "providing support, monitoring, and resources" to the projects, as well as disseminating "information about extent and nature of gang involvement, risk and protective factors, and how to address gang violence."

We reviewed the literature published by the NCPC (on the NCPC website) and found that they have developed numerous resources that directly relate to youth gangs, the specific risk and protective factors of youth gang involvement, and reviews of best practices in youth gang prevention. In addition to these resources, a recent focus by the NCPC on family-level risk and protective factors has provided more in-depth information to stakeholders about the interventions that they can use at that social level. The NCPC also developed a guide for funding applicants that provides extensive information about receiving funding from the NCPC. The table below briefly summarizes the documents provided by the NCPC which are directly related to the YGPF logic model.

Knowledge dissemination systems and activities

While developing resources and information related to youth gang prevention is an achievement in and of itself, the NCPC is also responsible for developing systems and engaging in activities to disseminate knowledge about gang prevention to stakeholders and the general public. Through documents obtained from the NCPC, we have been able to understand the activities that NCPC has engaged in related to dissemination of information about youth gangs. Based on these activities, it is clear that NCPC is developing a community of people to receive information, and that there is an interest in the subject matter.

Only four out of fifteen Project Annual Reports reported provided information about the knowledge dissemination activities that they had engaged in or were planning to engage in. Almost three-quarters of YGPF-funded projected did not indicate that they planned to participate in community-level knowledge dissemination.

Knowledge network creation

The creation of a knowledge network of researchers and practitioners in the field of youth gang prevention, one of the anticipated outputs of the knowledge activities, has not yet occurred. NCPC intends to create a network of YGPF funding recipients, and host a conference in 2009/2010 for them to meet one another. They are also sifting through the distribution list (seen in the previous table) to identify additional members of this network. NCPC intends to collect continual information from this network to determine how it can improve.

Public consultations and presentations

Another intended activity in the YGPF logic model is providing public consultations and training sessions. Based on the NCPC's documentation, it appears that five large-scale public sessions have been held. The sessions are generally presentations done for practitioners, although one workshop was held in 2007. There is a relative lack of frequency recently in these consultations – while there were three held in 2007, only one was held in 2008, and one to date in 2009. Also, there is a concentration of these sessions on the Prairie Provinces and Southern Ontario – there are no sessions in BC or Alberta, or East of Ontario. The sessions are outlined in the table below.

NCPC managers and project officers also mentioned the recent presentation made by RAGS—the Regina Anti-Gang Strategy—as a good example of how the fund is contributing to knowledge transfer. RAGS was presented at several conferences, including one in Toronto. They also mentioned the National Gang Crime Research Center Training Conference in Chicago where two YGPF projects were presented.

Project contacts mentioned that they have been invited to conferences, or were given opportunities to present their projects to the broader community and to various stakeholder groups. A number of projects contacts mentioned that they view the evaluator that they are working with as integral to sharing with a broader audience the outcomes of their programs.

All key informant groups expressed optimism that the fund will produce a body of knowledge that will help communities deal more effectively with the problem of youth gangs.

Success

Youth Program Participation

YGPF-funded projects did deliver the required services to participants. They tended to deliver service using two models – intensive intervention only, or intensive intervention plus non-intensive service provision. Because of the different models, the number of participants in projects ranged from less than 10 to more than 100. The modal categories were 10 to 20 and 20 to 30. Project annual reports did not provide sufficient information about length of involvement to make conclusive statements about the length that youth had participated in the program, but figures presented regarding drop-out rates suggest that the participants are invested and engaged in the programs.

Generally, the YGPF-funded projects provide services at two levels – intensive intervention only, which seven (of thirteen) provide, and both intensive and non-intensive service provision, which five provide to participants. Intensive intervention programs involve one-on-one interaction with youth participants on an ongoing basis for a period of time (usually in the form of case management), while non-intensive (contact) programs have activities of lower intensity, and do not track youth participants consistently. We have focused on the number of participants receiving intensive services, because this level of engagement is what is necessary to achieve the YGPF outcomes. Of the 13 projects that provided data on the number of participants receiving intensive service, one includes fewer than 10 participants by the time its Annual Report was completed, two have more than 50 participants, and eight had reached 10 to 30 participants.

1 Projects that specify the number of youth at risk or involved in gangs, of all their participants.

2 Projects that report the percentage of youth at risk or involved in gangs, of the participants who filled out the baseline survey.

As seen previously the projects mostly used quantitative screening methods to confirm that the potential participants exhibited known risk factors. Based on these screening criteria, it appears that the projects are recruiting the right participants. The length of involvement for participants in projects is challenging to ascertain based on the Project Annual Reports. No standard convention for reporting length of participation was observed, so projects reported involvement in months, or weeks, or not at all (half of them report no data about length of involvement). While we sought to determine how long participants had been involved in activities addressing specific risk factors, there was clearly insufficient information to address the question.

While the data that we had been looking for about length of involvement was not presented in the Project Annual Reports, we did find some projects that provided important information about the drop-out rates. Eight projects reported data on drop-outs, and four had rates of less than 10%, and another three had rates under 20%. In our view, this is a telling figure that indicates that the participants were invested and engaged with the projects.

1 Some projects report the length of involvement of youth participants by months, while others report the data by weeks. It is difficult to combine different measurements, as it is unclear if the number of weeks represents the overall length of enrolment of the youth in the program(s), or the total amount of intervention the youth has received.

2 Including the project that does not have any youth participants yet

Impact on Risk and Protective Factors

Qualitative information provided by key informants and in the Project Annual Reports indicate that positive changes are being observed among YGPF project participants. These improvements are being observed at all levels of risk factors. Because of the limited time that most projects have been operating, there is insufficient quantitative information available to make conclusions about the success of the program to date.

According to the theory underlying YPGF, the immediate outcome of full engagement of youth in programs will lead to an impact on the risk and protective factors associated with gang involvement. The file review also indicated that it is too early in the program to detect substantial changes in participants and communities. Of the nine projects that provided information on the intervals at which they would be measuring changes in outcomes, only four had been in operation for 6 months or longer. While this did not entirely preclude conclusions about the impacts that projects have had, it imposed limitations. Of 14 Project Annual Reports, six could not provide any results, six provided qualitative information about their outcomes, and three provided quantitative information.

1 Between when the baseline survey was administered and when the Annual Report was produced (December 2008)

2 Through interviews/focus groups with staff and key informants

3 Through comparing risk factors between baseline survey and mid-test

The qualitative outcome findings most often included successes at all levels of risk factors. The most frequently mentioned observations were related to the self-confidence of participants and improvements in school attendance. The quantitative findings available were less positive – two out of three projects that reported quantitative analyses said that no reliable improvements were observed. Because of the limited data available and the limited time periods of data collection, these findings must be interpreted cautiously, however.

1 Caution is needed when interpreting results from the qualitative findings, as findings are usually derived from a very limited number of youth participants

2 Categories are not mutually exclusive

3 Caution is needed when interpreting results from the quantitative findings, as the number of youth participants who completed both the baseline survey and the mid test is very limited (ranging from 7 to 27 youth)

To get a sense of potential intermediate impacts of these projects, we asked project officers and project contacts if they had observed any changes in the communities or youths who are participating. Project officers and project contacts generally share the view that it is too early to detect any 'street level' change in any objective way. However, they do point to a number of promising signs.

Some of these promising signs include:

• An observed reduction in guns and drugs in Regina;
• Families of the youths becoming interested in the project;
• Families becoming more open and wanting to change;
• Youths becoming more interested in their studies;
• Better attendance at school;
• Stronger relationships between schools and community organizations;
• More intermixing of youths from different gangs;
• Fewer youths observed on the streets or in parks at night in Vancouver;
• More interest among youth in developing employable skills;
• Youths beginning to rebuild their reputations in the community; and
• Positive feedback from employers regarding youth who have gone through the program.

The State of Canadian Gang Literature and NCPC's Contribution

Canadian literature with primary research on gangs and gang prevention remains limited. Since the 2002 Canadian Police Survey on Youth Gangs, there has not been any quantification of the scope of the youth gang problem at the national level. While the author of that report has produced some estimates of the youth gang scope in 2007, there has been no follow-up to the 2002 study conducted, leaving a large gap in Canada's knowledge about its current gang problem.

The published research from 2003 to date includes a number of reviews of programs and international research, some research on specific ethnic gang experiences, and some investigations of the predictors of gang membership and violence. The most substantial document produced in that time that would be helpful to inform the youth gang situation came from the Government of Ontario in the form of the Roots of Violence report.

We found 28 publicly available articles related to youth gangs in Canada. When compared to the eight documents that NCPC has prepared, we can see that NCPC has provided a substantial contribution to the overall publications in the field.

An immediate outcome of the 'knowledge stream' of the YGPF is to "Increase in Canadian body of knowledge related to gangs, including knowledge of the risk and protective factors linked to gang involvement ." The table in Appendix E provides a summary of gang-related publications that have been published since 2003 and that are based on Canadian data (i.e., Canadian authors commenting on the American gang phenomenon are not included, only primary Canadian research) or that provide implications for the Canadian context. The time period was selected so as to provide a perspective on the recent publications, which reflects the state of information as it stands today and in recent history (and even including some years before the YGPF began). Also, 2003 marked the year when the seminal report Results of the 2002 Canadian Police Survey on Youth Gangs was released.

The documents reviewed were assessed for Canadian content – many articles written by Canadians summarize research from other countries, and our objective was to assess of the state of "Canadian knowledge," which requires research on the Canadian youth gang experience. As such, publications by Canadian authors based on data from gangs in other countries are not included. Documents provided by the NCPC were used in the review, as well as other publicly available internet resources. To review academic literature, we used EBSCO Host and searched through the following databases: PsycARTICLES, PsycBOOKS, PsycINFO, SocINDEX, and Public Administration Abstracts. The search terms were broad to ensure that any relevant literature was captured – "Canada AND (Gang OR Gangs)."

The articles and documents from the academic literature search that are included in the table are the few from the 31 results (31 articles among hundreds of thousands, which included the search terms in the author affiliations, abstracts, article titles, and publication titles) which were related to the state of gangs in Canada (i.e., theoretical reviews, racial profiling, organized crime, etc. were excluded). The articles that are not included in the table below were not related to the state of Canadian gangs, or mentioned gangs only in passing on another topic entirely. The table does not include documents available through the NCPC that are listed in the previous section.

To summarize the results, there are 28 articles in the Appendix E table. When compared to the table of NCPC contributions to literature, eight of which we would consider comparable to the 28 other resources, we can see that NCPC makes a substantial and significant contribution to the Canadian Youth Gang literature.

Unintended Impacts

The YGPF has served to increase awareness of the youth gang issue in Canada. It also defined a broad strategy for crime prevention (identifying high-risk populations for interventions) has given NCPC an identity and a unique role. The YGPF has received strong support from communities in which projects operate, and has facilitated relationships among community groups.

We asked NCPC managers, program officers, provincial representatives, and project contacts whether or not they have observed any unintended consequences or impacts—either positive or negative—as a result of the YGPF. On the positive side, managers noted that the fund has served to heighten awareness of the youth gang issue. Managers also thought that the fund validated and helped define a broader strategy—to focus on targeted interventions with specific at-risk populations and having rigorous evaluations to determine outcomes. Managers also expressed that the fund has brought NCPC together as an organization and has validated its unique role.

Program officers mentioned several unanticipated positive impacts. They indicated that they have been surprised by the level of support that the police have given to the approach taken by the YGPF. They have also been heartened by the positive impact that some of the projects are having on the families of youth who are participating. These families are becoming 'more open' and less likely to deny that their children are at serious risk.

Project contacts did point to a number of positive, unexpected consequences of the fund. Some project contacts were initially worried that the evaluation would pose an unnecessary administrative burden. However, they have found this not to be the case and view the evaluation as an important way of validating their project's impact. Other positive consequences have included:

• Better working relationships between organizations and schools;
• The development of employable skills;
• High level of support from the community;
• Strengthening of relationships between community organizations; and
• The development of positive relationships between front line workers and the youth being served.

An unexpected consequence of the increased funding to gang prevention initiatives by the various levels of government has been the increased labelling of many programs as gang-related initiatives, regardless of their activities. One informant mentioned attending a presentation about a second language program for children that was labelled as a gang prevention initiative by the presenter. Another informant mentioned that there are number of initiatives that are being called prevention initiatives but are essentially enforcement initiatives—e.g., hiring more police officers.

Cost-Efficiency

Costs Associated with Project Implementation

YGPF-funded projects ranged in cost from less than $100,000 to more than $300,000 to date1. The cost per project varied as a function of project activities, and time in operation. Similarly, the costs per participant differed significantly. While some projects cost in the one to two thousand dollar range per participant, some projects cost more than $10,000 to deliver to each participant. Differences in costs per participant were most commonly due to time in operation – the longer a program is in operation, the longer it serves each participant and thus the higher the cost per youth (as more services are added over time). The intensity of service delivery also significantly impacted cost per participant – intensive intervention-only projects have a higher cost per participant compared to those with intensive and non-intensive service delivery3.

Source: project annual reports

1 Project cost covers 9 months for most of the projects (thirteen out of fifteen), and 12 months or longer for the remaining two projects

2 Project cost does not include soft cost or evaluation cost, as specified in the reports.

3 Youth participants include participants of the intensive programs and the contact (non-intensive) programs. It is difficult to calculate cost per intensive participant or per non-intensive participant, as most projects do not separate the cost of the intensive programs and the contact programs.

Stakeholder Perceptions of Cost-efficiency

It is too early in the YGPF program to make conclusions about the overall cost-effectiveness of the fund. To date, inefficiencies have been observed in the approval processing time and the accounting and reporting requirements placed on projects. The requirements in these areas seem to have outpaced their returns by reducing service capacity at the project level.

We asked NCPC managers and program officers to provide suggestions regarding how the program could be changed to operate more cost-efficiently. A number of managers and officers indicated that it was too early to evaluate the fund from the perspective of cost-efficiency. Those who did make suggestions said that the program officers and projects would be well-served by receiving additional assistance from those who specialize in the areas of finance and accounting. They also suggested that the approval process needs to be shortened and more streamlined, and that the administrative burdens of the fund need to be lessened. According to these key informants, the fund is at the point where its excessive focus on financial accountability is at the point of diminishing returns and is taking away the projects' capacities to maximize the time they are spending working with youth at risk for joining gangs.

Conclusions

Relevance

Canada's youth gang problem has escalated and continues to escalate

In 2009, three years after it began, the objectives of the YGPF remain highly relevant and more important than ever before. The youth gang problem appears to be more severe than ever before. Canada's crime statistics indicate that youth are increasingly involved in crime and that their rate of violent offending and homicide is higher than ever before. Crime prevention researchers also attest that youth gang activity is a significant and growing problem, and has been for the past 10 years or more. These facts point to the need for continued service provision to at-risk youth and youth involved in gangs.

Youth are using the services offered through the YGPF

The number of youth that are being served by YGPF-funded projects also indicates that it continues to be relevant. Projects funded by the YGPF have provided service to almost 1000 youth since the program began, and many of the projects are at or near their capacity. The demand for services is high, considering that capacity was reached quite quickly.

YGPF is not duplicated by other levels of government

The YGPF plays a unique role in the social program arena, and its role is not duplicated elsewhere. The YGPF is a program with a wide and longer-term scope, and its efforts to fund youth gang prevention projects are not duplicated by other levels of government. There are also few other gang prevention initiatives that have started since 2006, reinforcing the fact that the YGPF is leading the way in gang prevention in Canada, and that its efforts are not duplicated elsewhere. Many of the youth gang prevention programs that are funded by the YGPF would probably cease to operate without NCPC funding, or would have significantly reduced scope of activities.

YGPF is in the scope of broad government objectives

The YGPF continues to be relevant to broader government objectives. The YGPF's objectives are in line with Public Safety Canada's objectives as stated in its 2009-2010 Report on Plans and Priorities, were congruent with the Government of Canada's priorities stated 2006 and 2007, were mentioned in the Performance: The Government of Canada's Contribution reports in 2005-2006 and 2007, and are also in accordance with the RCMP's Strategic Priorities.

Finally, the YGPF is relevant and important to its stakeholders and project managers, who mostly espouse positive views of the federal government's role in youth gang prevention through the YGPF. There is strong support of the government's involvement because it is the only organization that can provide long-term funding for such projects.

Program Design and Delivery

The selection process was effective in identifying appropriate projects to fund

The selection involving the provincial representatives was very effective, since the provinces have more familiarity with the areas and programs in need of funds. The drawbacks about the selection process related to the fact that it excluded smaller communities, that funds were not allocated per jurisdiction, and that the "first come first served" selection model lead to projects seeking approval before they were ready.

NCPC's roles and responsibilities were defined, but not communicated

The roles and responsibilities of the NCPC were well defined, but the roles and responsibilities were not adequately communicated to the funded projects, which were not aware of the NCPC's definitions concerning its own role. The communication problems affected program officers, also. Their roles were not clear, forcing them to make up roles as the program progressed.

The application and approval process was too lengthy and harmed service delivery

The application and approval process was onerous and in need of streamlining. Information about the approval process obtained from the file review indicates that more than three-quarters of projects had approval processes that took more than four months, and one-third took more than six months. The accountability process should be restructured – the bureaucracy of the process was seen as inhibiting programs' capacity to deliver services. The resources that were initially provided to funding applications were not clear, and resulted in delays from the review committee. Further guidance and better templates would have helped project contacts and program officers.

After project approval fund transfers also took too long

After the lengthy approval period, there were further delays in transferring funds to the projects. These delays negatively influenced morale in recipient communities, and created some pessimism about the NCPC and federal government's commitment to the projects. The volume of reporting required from projects is also excessive (quarterly accounting and activity reports, amounting to eight reports annually), and consumes resources that could be spent on service delivery.

Tools and resources to support projects were planned but not delivered successfully

While NCPC managers were generally confident that they had provided sufficient resources to execute their projects successfully, program officers noted that the projects did not receive enough support, that resources were probably provided to projects too late (i.e., after the program had begun), and that the training was not sufficiently specialized. This divergence of opinions between NCPC management and program officers suggests that the structures and directives for resource development were arranged by management, but that they were not followed-through with actions giving resources to the funded projects.

Project contacts had diverse experiences. In fact, there experiences were so divergent that we believe the NCPC did not have a clear process to deal with their needs. Some projects received funds for external training, while some received training from NCPC, and other received none at all. NCPC is advised to further investigate the reason for the diversity of experiences. Project contacts found the documents provided by NCPC to be helpful, and the most frequently requested resource was a conference for YGPF-funded project administrators to meet and share their experiences.

YGPF projects had the characteristics needed for funding

The projects funded by the YGPF included most of the characteristics that they were expected to have. The projects did screen and identify at-risk youth and youth involved in gangs; most projects provided some quantification to their risk assessments. The projects all addressed risk factors at multiple levels, and half addressed factors at all levels. Most projects appeared to be based on literature and/or best practices, although the file review did not provide a lot of information about the projects' foundations. The projects also evidenced a high degree of partnering with other community organizations.

NCPC produced documents to provide guidance in developing successful gang prevention and intervention projects

Our literature review identified a number of documents that were prepared by the NCPC for, which would have helped to develop effective, evidence-based projects. These documents are publicly available, in the expectation that they can influence the practice of gang prevention and intervention.

Success

Funded projects delivered the appropriate services to the appropriate participants

YGPF-funded projects did deliver the required services to participants. They tended to deliver service using two models – intensive intervention only, or intensive intervention plus non-intensive service provision. Because of the different models, the number of participants in projects ranged from less than 10 to more than 100. The modal categories were 10 to 20 and 20 to 30. Project annual reports did not provide sufficient information about length of involvement to make conclusive statements about the length that youth had participated in the program, but figures presented regarding drop-out rates suggest that the participants are invested and engaged in the programs.

Positive impact on participants cannot be ascertained yet, but is on track to succeed

Qualitative information provided by key informants and in the Project Annual Reports indicates that positive changes are being observed among YGPF project participants. These improvements are being observed at all levels of risk factors. Because of the limited time that most projects have been operating, there is insufficient quantitative information available to make conclusions about the success of the program to date.

YGPF has not yet positively impacted Canada's literature on gangs, which remains limited

Canadian literature with primary research on gangs and gang prevention remains limited. Since the 2002 Canadian Police Survey on Youth Gangs, there has not been any attempt to quantify the scope of the youth gang problem at the national level. While the author of that report has produced some estimates of the youth gang scope in 2007, there has been no follow-up to the 2002 study conducted, leaving a large gap in Canada's knowledge about its current gang problem.

The published research from 2003 to date includes a number of reviews of programs and international research, some research on specific ethnic gang experiences, and some investigations of the predictors of gang membership and violence. The most substantial document produced in that time that would be helpful to inform the youth gang situation came from the Government of Ontario in the form of the Roots of Violence report.

Progress toward meeting longer-term objectives

Based on our evaluation, we are confident that YGPF has systems in place to ensure that the YGPF will continue toward reaching many of its longer-term objectives as far as program delivery is concerned. We have found that many projects have been funded and are operational, and that more will be in the near future. While there is not a lot of quantitative data concerning changes in risk and protective factors at this time, there is optimism expressed qualitatively.

On the knowledge development side of the program, we have some concerns about the knowledge dissemination activities of the YGPF-funded projects. To date, only about one-third of projects have engaged in community-level knowledge dissemination, and the remaining two-thirds did not address this component of the project in their Project Annual Reports. Without knowledge dissemination, the community-level changes that are anticipated in the logic model will not occur. Also, most YGPF projects have not engaged in the creation of primary Canadian research about youth gang prevention, and so the Canadian literature on the topic remains limited.

NCPC has provided YGPF-funded projects with a template for information capturing to ensure that projects are comparable over time and between each other. We have found, however, that the projects have not used these templates consistently, resulting in information that is disjointed and difficult to compile as a whole.

Recommendations

Based on our evaluation, we have the following recommendations:

1. NCPC should organize a regular meeting between YGPF project managers.
2. Reduce the administrative burden imposed by NCPC on YGPF projects.
3. Review and clarify the roles and responsibilities of NCPC staff involved in the administration of the YGPF.

References

Angus Reid Strategies. (2009). Canadians Clearly Support Ideas to Deal with Gang Violence. Retrieved from http://www.angus-reid.com/uppdf/2009.03.05_Gangs.pdf.

Astwood Strategy Corporation. (2003). Results of the 2002 Canadian Police Survey on Youth Gangs. Public Safety and Emergency Preparedness Canada..

Astwood Strategy Corporation. (2008). Now is the Time to Act: Youth Gang Prevention in Ottawa. Retrieved from http://crimepreventionottawa.ca/uploads/files/Now-is-the-time-to-Act%20_FINAL-REPORT.pdf.

Canadian Research Institute for Law and the Family. (2005). Youth Gangs in Canada: A Preliminary Review of Programs and Services. Public Safety Canada. Retrieved from http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/ddrsng-prblms/index-eng.aspx.

Canadian Training Institute. (2003). Excerpt on youth gangs. Youth Justice in Canada: A Resource Manual, 45-50. Retrieved from http://www.cantraining.org/docs/CanadianReferences/Youth%20Justice%20in%20Canada%20-%20Gang%20Excerpt.pdf.

Chettleburgh, M. (2007). Young Thugs: Inside the Dangerous World of Canadian Street Gangs. Toronto: Harper Collins.

Correctional Services of Canada. (2003). A Profile of Women Gang Members in Canada. Correctional Services of Canada. Retrieved from http://www.cscscc.gc.ca/text/rsrch/reports/r138/r138_e.pdf.

Correctional Services of Canada. (2004). Street Gangs: A Review of Theory, Interventions, and Implications for Corrections. Correctional Services of Canada. Retrieved from http://www.csc-scc.gc.ca/text/rsrch/reports/r161/r161_e.pdf.

Correctional Services of Canada. (2005). Selected Annotated Bibliography: Evaluations of Gang Intervention Programs. Correctional Services of Canada. Retrieved from http://www.csc-scc.gc.ca/text/rsrch/briefs/b36/b36_e.pdf.

Criminal Intelligence Service Canada (2006). 2006 Annual Report on Organized Crime in Canada. Ottawa: CISC, 2006. Retrieved from http://www.cisc.gc.ca/annual_reports/annual_report_2006/document/annual_report_2006_e.pdf.

Dauvergne, M. (2008). Crimestatistics in Canada, 2007. Juristat, 28(7). Retrieved from www.statcan.ca, catalogue no. 85-002-X.

De Laco, G. A. (2007). Juvenile street gang members and ethnic identity in Montreal, Canada (Quebec). Unpublished dissertation.

Erickson, K., LaBoucane-Benson, P., & Grekul, J. (2007). The Community Solution to Gang Violence: A Collaborative Community Process and Evaluation Framework. Public Safety Canada. Retrieved from http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/cmmnt-sltn-gng-2007/index-eng.aspx.

Erickson, P. G., & Butters, J. E. (2006). Youth, Weapons and Violence in Toronto and Montreal. A Final Report prepared for Public Safety and Emergency Preparedness, Canada, under the direction of Project Manager, Rebecca Jesseman. University of Toronto, Canada.

Erickson, P. G., Butters, J. E., Korf, D. J., Harrison, L. D., & Cousineau, M. (2006). Weapons and violence among male delinquents: An international comparative study. International Annals of Criminology, 44 (1/2), 125-151.

European Forum for Urban Safety (2007). Guidance on Local Safety Audits: A Compendium of International Practice. Public Safety Canada. Retrieved from http://www.ffsu.org/fileadmin/efus/secutopics/EFUS_Safety_Audit_e_WEB.pdf.

Evans, D. G., & Sawdon, J. (2004). The development of a gang exit strategy: The youth ambassador's leadership and employment project. Corrections Today, 66(6), 78-81.

Federation of Saskatchewan Indian Nations. (2003). Alter-natives to Non-Violence, Aboriginal Youth Gangs Exploration: A Community Development Process (Draft report). Health and Social Development. Retrieved from http://www.cantraining.org/BTC/docs/CanadianReferences/Aboriginal%20Youth%20Gangs.pdf.

Gatti, U., Tremblay, R. E., Vitaro, F., & McDuff, P. (2005). Youth gangs, delinquency and drug use: A test of the selection, facilitation, and enhancement hypotheses. Journal of Child Psychology and Psychiatry, 46(11), 1178-1190.

Government of Ontario. (2008). The Review of the Roots of Youth Violence. Retrieved from http://www.children.gov.on.ca/htdocs/English/topics/youthandthelaw/roots/index.aspx.

Grekul, J., & LaBoucane-Benson, P. (2007). An Investigation into the Formation and Recruitment Processes of Aboriginal Gangs in Western Canada. Public Safety Canada. Retrieved from http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/brgnl-gngs-nvstgtn-2006/index-eng.aspx.

Grekul, J. & LaBoucane-Benson, P. (2008). Aboriginal gangs and their (Dis)placement: Contextualizing recruitment, membership, and status. Canadian Journal of Criminology and Criminal Justice, 50(1), 59-82.

Ipsos Canada. (2006). Crime in Canada: Canadians Speak Up on What Should be Done. Retrieved from Ipsos Canada website: www.ipsos.ca.

Kelly, K., & Tullio, C. (2005). The linkages between street gangs and organized crime: The Canadian experience. Journal of Gang Research, 13(1), 17-31.

Kwok, S. (2008). Exploring the experiences of Asian youth in the criminal justice system in Canada. Journal of Gang Research, 15(3), 1-17.

Lafontaine, T., Ferguson, M., & Wormith, J. S. (2005). Street Gangs: A Review of the Empirical Literature on Community and Corrections-Based Prevention, Intervention and Suppression Strategies. Saskatchewan Corrections, Public Safety and Policing. Retrieved from http://www.cpsp.gov.sk.ca/GangReportforCPSJune30_05.pdf.

National Crime Prevention Centre. (2007). A Blueprint for Effective Crime Prevention. Public Safety Canada. .

National Crime Prevention Centre. (2007). Addressing Youth Gang Problems: An Overview of Programs and Practices. Public Safety Canada. Retrieved from http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/ddrsng-prblms/index-eng.aspx.

National Crime Prevention Centre. (2007). First Nations Youth and Restorative Healing Project. Public Safety Canada. Retrieved from http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/2007-es-05/index-eng.aspx.

National Crime Prevention Centre. (2007). Youth Gang Involvement: What Are the Risk Factors? Public Safety Canada. Retrieved from http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/yth-gng-nvlvmnt/index-eng.aspx.

National Crime Prevention Centre. (2007). Youth Gangs in Canada: What Do We Know? Public Safety Canada. Retrieved from http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/yth-gng-nvlvmnt/index-eng.aspx.

National Crime Prevention Centre. (2008). Crime Prevention Action Fund – Application Guide. Public Safety Canada. Retrieved from http://www.publicsafety.gc.ca/cnt/cntrng-crm/crm-prvntn/fndng-prgrms/crm-prvntn-ctn-fnd-eng.aspx.

National Crime Prevention Centre. (2008). Family-Based Programs for Preventing and Reducing Juvenile Crime. Public Safety Canada. Retrieved from http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/wht-knw/index-eng.pdf.

National Crime Prevention Centre. (2008). Family-Based Risk and Protective Factors and their Effects on Juvenile Delinquency: What Do We Know? Public Safety Canada.

National Crime Prevention Centre. (2008). Promising and Model Crime Prevention Programs. Public Safety Canada. Retrieved from http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/prmsng-mdl-vlm2/index-eng.pdf.

Prairie Research Associates. (2007). Interim Evaluation of the Renewed National Crime Prevention Strategy Final Report Technical Appendices Volume II. Unpublished document received from NCPC.

Proactive Information Services. (2009). Winnipeg Youth Gang Prevention Fund Projects Draft Literature Review. Unpublished document received from NCPC.

Public Safety Canada. (2006). Canada's new government ends tattoo program for convicted criminals.

Public Safety Canada. (2007). Sustainable Development Strategy 2007-2009. Retrieved from http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/sstnbl-dvlpmnt-ssmnt-2013-14/index-eng.aspx.

Royal Canadian Mounted Police. (2003). Direct and Indirect Impacts of Organized Crime on Youth, as Offenders and Victims. Community Contract and Aboriginal Policing Service, Research and Evaluation Branch, Royal Canadian Mounted Police. Retrieved from http://www.rcmp-grc.gc.ca/pubs/ccaps-spcca/oc-co-youth-jeune-eng.htm.

Royal Canadian Mounted Police. (2006). Feature Focus: Youth Gangs and Guns. Retrieved from http://www.rcmp-grc.gc.ca/pubs/yg-ja/index-eng.htm.

Royal Canadian Mounted Police. (n.d.). Strategic Priority: Youth. Retrieved from http://www.rcmp-grc.gc.ca/youth-jeune/index-eng.htm.

Singh, H., & Plecas, D. (2006). Differences Between South Asian and Traditional Minority Gangs. Paper presented at the annual meeting of the American Society of Criminology (ASC), Los Angeles Convention Center, Los Angeles, CA; November 1, 2006.

Tanasichuk, C. L., & Wormith, J. S. (2008). Process and Outcome Evaluation of the Saskatoon Gang Strategy: Literature Review. Saskatchewan Corrections, Public Safety, and Policing. Retrieved from http://www.cpsp.gov.sk.ca/GangReportforCPSJune30_05.pdf.

Tita, G. E., Troshynski, E., & Graves, M. (2007). Strategies for Reducing Gun Violence: The Role of Gangs, Drugs and Firearm Accessibility. Public Safety Canada: National Crime Prevention Centre. Retrieved from http://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/rdcng-gn-vlnc/index-eng.aspx.

Wortley, S., & Tanner, J. (2006). Immigration, social disadvantage and urban youth gangs: Results of a Toronto-area survey. Canadian Journal of Urban Research, 15(2), Supplement pages 18-37.

Appendix A – Evaluation Framework

Method

Section 5.2 (Pp. 27-36) of the ARAF document outlines the proposed evaluation plan for the YGPF. The evaluation is conceptualized around the following themes:

•  Relevance.
•  Program Design and Delivery.
•  Success.
•  Cost-effectiveness and alternatives.

Interwoven with these themes are the Expenditure Review Committee (ERC) questions. They are:

• ERC1. Public interest – Does the program area or activity continue to serve the public interest?
• ERC2. Role of government – Is there a legitimate and necessary role for government in this program or activity?
• ERC3. Federalism – Is the current role of the federal government appropriate, or is the program a candidate for realignment with the provinces?
• ERC4. Partnership – What activities or programs should or could be transferred in whole or in part to the private/voluntary sector?
• ERC5. Value-for-money – Are Canadians getting value for their tax dollars?
• ERC6. Efficiency – If the program or activity continues, how could its efficiency be improved?
• ERC7. Affordability – Is the resultant package of programs and activities affordable? If not, what programs or activities would be abandoned?

Following the themes, Table 4 (ARAF Pp. 30-36) lists the evaluation questions, recommended indicators, and data sources. In this section, each question is listed and, in turn, the indicators and sources of information we will obtain and use to answer the question.

Relevance

R1 Issue. To what extent do the objectives of the YGPF continue to be relevant?

R1.1 Indicator. Increase or decrease in size and severity of youth gang problems in Canada.

R1.2 Indicator. Increase or decrease in number of targeted youth being reached by funded programs

R1.3 Indicator. Size and scope of other gang prevention initiatives implemented since inception of YGPF

R1.4 Indicator. Level of public and stakeholder support for the federal leadership role in the area of youth gang prevention

R1.5 Indicator. Extent to which stakeholders perceive that the objectives of the YGPF continue to be relevant

R2 Issue. To what extent are the objectives of the Youth Gang Prevention Fund consistent with Government-wide priorities?

R2.1 Indicator. Extent to which YGPF is in line with and contributes to Public Safety Canada Strategic Outcomes, as well as broader government-wide priorities?

Program Design and Delivery

P1 Issue. To what extent is the Youth Gang Prevention Fund appropriately resourced to achieve its objectives?

P1.1 Indicator. Extent to which funded projects are able to meet needs of targeted youth

P1.2 Indicator. Extent to which NCPC is able to meet demands for funding from other affected communities

P1.3 Indicator. Perceptions of stakeholders as to the appropriateness of the current allocation of resources

P2 Issue. Are effective, efficient, and standardized processes/resources available to support the Youth Gang Prevention Fund objectives?

P2.1 Indicator. Extent to which roles and responsibilities for YGPF program administration are clearly and appropriately defined, and effective procedures are in place

P2.2 Indicator. Extent to which appropriate YGPF tools/resources/documents are available to support communities

P2.3 Indicator. Extent to which training is adequate to support staff

P3 Issue5. Have the activities of the YGPF successfully produced the implementation of the desired type of projects?

P4 Issue. Have the activities of the YGPF resulted in knowledge creation and dissemination?

Success

S1 Issue. To what extent has the YGPF achieved its short-term outcomes with regards to youth program participation?

S2 Issue. To what extent has the YGPF had a positive impact on the risk and protective factors related to youth gangs?

S3 Issue. Has the YGPF led to an increase in the Canadian body of knowledge related to gangs, including risk and protective factors linked to gang involvement? Has this information increased stakeholders knowledge?

S3.1 Indicator. Existence of current and comprehensive information on the nature of gang phenomenon in Canada

S9 Issue (Not in ARAF). Has the YGPF improved communities' access to resources and knowledge about gang prevention and intervention? Have they used these effectively?

S9.1 Indicator. Existence of activities to empower communities

S9.2 Indicator. Improved community opportunity structures to reduce gangs' presence

S8 Issue. Are there any unintended impacts of the YGPF?

S8.1 Indicator. Unintended impacts observed by managers, stakeholders, and funding recipients

S8.2 Indicator. Evidence of possible displacement or diffusion effects of community-level interventions

Cost-effectiveness and alternatives

C1 Issue. Was the YGPF implemented in a cost-effective manner?

C1.1 Indicator. Costs associated with the overall and independent activities and outputs of the YGPF

C1.2 Indicator. Extent to which program processes and procedures are designed and implemented efficiently

Appendix B – ADDITIONAL TABLES

1 Of all projects that reported the age of the youth participants, two reported a 13-15 age category of their youth participants. It is unknown if these youth participants are under 14 years or above.

2 8 to 10 projects had the measurement of the factor as the percentage of the youth participants having the characteristics. Other two projects had the measurement on a scale.

Source: Ministerial Project Summary of each project

1 Including the project that does not have any youth participants yet

Appendix C – List of project-specific tables

Project specific tables will be presented in a separate excel document. The following lists tables and matrixes to be included in the excel spreadsheet:

List of project-specific tables

Table 1–Total project budget, funding from NCPC, and other partners, total amount paid to date

Table 2–Start dates (proposal submission, funding approval, accepting youth participants, administrating baseline survey/pretest)

Table 3–Current level of capacity (the number of youth reached vs. the number of youth targeted)

Table 4–Number, Area of risk factors projects work to address

Table 5–Risk assessment approach used (to screen youth participants)

Table 6–Demographics of youth participants (respondents of the baseline survey/pretest)

Table 7–Risk factors of youth participants (respondents of the baseline survey/pretest)

Table 8–Fitting local context

Table 9–Evidence consulted to design project activities

Table 10–Project partners (team partner, close partners, and the total number of partners involved)

Table 11–Average involvement of youth participants, definition of minimum dosage/expected involvement level, and percentage of youth participants over the minimum dosage

Table 12– Qualitative findings of immediate outcomes of the youth participants

Table 13– Knowledge sharing plan and community level effects

Table 14– Project cost and cost per youth participant

List of project-specific Matrixes

Matrix 1 –Services provided by each project

Matrix 2 –Risk factors of each project (aiming to address)

Matrix 3– Partners of each project, sectors represented and their functions/contributions

Appendix D – File review template

Evaluation of the National Crime Prevention Centre (NCPC) Youth Gang Prevention Fund Program (YGPF)

File Review Template

Key documents:

• Annual Project Evaluation Reports (AR)
• Ministerial Project Summary (MPS)
• GCIMS
• (possible) Project Proposal (PP)
• (possible) Evaluation Plan (EP)

Section 1. Background information

1. (GCIMS) NCPC file number: ____________

2. (GCIMS) Project name: ________________

3. (GCIMS) NCPC client name: ___________

4. (GCIMS) Province: ____________________

5. (GCIMS) City: ________________________

6. (GCIMS) Date proposal submitted: _______________________________________ (DD/MM/YY)

7. (AR) Start date of accepting youth participants: ____________________________ (DD/MM/YY)

8. (AR) Start date of evaluation activities: ___________________________________ (DD/MM/YY)

9. (AR) Start date of pre-test of youth participants:_____________________________ (DD/MM/YY)

10. (AR) Start date of mid-test of youth participants:____________________________ (DD/MM/YY)

11. (AR) Duration of the service delivery to-date: _________________________________ (months)

12. (AR) Period of time evaluated: ______________________________________________ (months)

Section 2. Input/Resources

13. (MPS/GCIMS) Total budget of the project: ___________________________________ (Cnd$)

14. (MPS/GCIMS) Total YGPF funding approved: ___________________________________ (Cnd$)

15. (GCIMS) Total current and projected funding from NCPC: ______________________ (Cnd$)

16. (GCIMS) Other funding partners and expected contributions (Cnd$)

17. (GCIMS/AR) Contributions made by other partners, including in-kind contribution (Cnd$)

18. (AR) Number of Full-time-equivalent (FET) staff: _______OR Number of staff: _________

19. (AR) Number of volunteers: ____________________________________________________

Section 3. Activities and outputs

20. (AR) Programs/services delivered* (Check all that apply)

Note: * excluding project setup and ongoing management activities

1. Mentoring

2. Life skills training/personal development (including leadership training, goal setting, etc)

3. Counselling/therapy/stress, anger, aggression management (for participants)

4. Parenting skills training

5. Family support and counselling (for families of participants)

6. Education activities (e.g. credit recover, tutoring, homework clubs, alternative school classes)

7. Social and communication skills training

8. Substance abuse treatment

9. Sports activities

10. Arts activities

11. Other recreational/leisure activities/summer camp

12. Community service or volunteer work

13. Cultural activities/traditional learning (e.g. storytelling, ceremonies, feasts)

14. Employment skills training/employment support/opportunities

15. Case management

16. Referring participants to service providers, and/or provide momentary or in-kind support

17. Relationship-building with an adult

18. Referring parents to skill training/counselling

19. Advocacy with service providers where youth/family needs are not met

20. Providing a safe, youth-friendly space for youth social activities

21. Housing support

22. Basic needs support (e.g., food, personal care items)

23. Community education/awareness workshop/forum

24. Outreach/recruitment in general

25. Youth & gang awareness/education workshop

26. Income support

27. Court support

28. Family strengthening (for parents and children)

29. Team work among peers

95. Other, specify

99. Can't determine/Not stated

21. (AR/MPS/EP) What risk or protective factors did the project address? (CHECK ALL THAT APPLY)

95. Other, specify

97. None of the above

99. Can't determine/Not stated

22. (AR) Did the project adopt a rigorous risk assessment model/tool to determine youth of the greatest risk for the participant intake?

1. Yes, specify the model/tool used

2. No, specify the approach used

99. Can't determine/Not stated

23. (MPS,AR) Was the project tailored to local context through... (CHECK ALL THAT APPLY)

1. Consulting First Nations groups, local ethnic groups, immigrant organizations to deliver programs in a culturally sensitive way/environment to youth participants

2. Adapting aspects of an existing model by serving a particular group specific to the locality (e.g., serving a different age group, choosing a different cut-off score in the risk assessment model)

3. Making other adaptations to an existing model such as adding or changing the types of activities, or the intensity or duration of the activities

4. Creating its own model that incorporates specific activities to meet the needs of the target population (e.g. trauma counseling for refugee youth, cultural components for Aboriginal youth)

5. Modifying an existing model by adding or changing risk-assessment questions/indicators specific to locality (e.g., adding gang-involvement questions to delinquency questions)

6. Adapting an existing model by incorporating the opinions and judgment from other sources (such as caregivers, professional staff), if so, specify the source(s) of judgment______

95. Other, specify _________

97. None of the above

99. Can't determine/Not stated

24. (MPS, PP) What evidence was consulted to design the program activities and the delivery mode?

1. Current crime/victimization statistics related to the issue being addressed

2. Findings from community needs assessment (e.g., statistics on risk factors, interviews with stakeholders, data on service provision and gaps)

3. Particular incident(s) in the community that has generated concern

4. The literature on gang prevention and intervention, specify if stated

5. The literature on crime prevention more generally, specify if stated

95. Other, specify ____

97. None of the above

99. Can't determine/Not stated

25. (AR) Does the lead organizer of the project team up with any other organization(s) in managing the project, if yes, specify: ____________________________________

26. (AR) Number of (closely-involved) partners cited, excluding the organizations teamed up as project co-organizer: ____________________________________

27. (AR) Total number of partners involved (at various levels): ____________________________

28. (AR) Partners involved (CHECK ALL THAT APPLY)

1. Police/law enforcement

2. Local schools/school boards

3. Non-profit organizations/advocacy organizations/ citizen groups

4. Local media

5. Provincial agencies

6. Municipal government

7. Local business

8. Professional associations/service club/sport leagues

9. Religious institution

10. Academic institution/research

11. Aboriginal organizations, tribes or government

12. Federal government

13. Hospital/health authority

95. Other, specify

97. None of the above

99. Can't determine/Not stated

29. (AR/MPS) Sectors the closely-involved partners represented (CHECK ALL THAT APPLY)

1. Aboriginal agencies or organizations (other than Tribal or Band Councils)

2. Aboriginal - Tribal or Band Council

3. Arts and culture

4. Business Associations (e.g. Chamber of Commerce, Business Improvement Associations, etc.)

5. Community, social or voluntary services (e.g. youth, family services)

6. Community coalition or network

7. Corrections (e.g. probation, juvenile detention centres, parole officers, prisons)

8. Education

9. Vocation/employment/skill training

10. For profit organization

11. Government – local, municipal

12. Government – provincial or territorial

13. Government – Aboriginal (other than Band or Tribal Council)

14. Government – federal

15. Health

16. Housing services

17. Justice (e.g. courts, prosecution services)

18. Police/law enforcement

19. Professional Associations (e.g., Teachers' Association)

20. Religious/faith

21. Service Club (e.g. Rotary, Lions )

22. Non-profit organizations/advocacy organizations/ citizen groups

23. Recreational services

24. Academic institution/research

25. Library

26. Labor and immigration

27. Community development

95. Other, specify

99. Can't determine/Not stated

30. (AR/MPS) Functions of contribution from the closely-involved partners

1. Make referrals to program

2. Provide staff to deliver some of the program activities

3. Accept referrals from program (i.e., organization gives priority or guarantees access to project participants, provides regular updates, engages in joint planning)

4. Contribute financially to the program

5. Provide in-kind contributions (e.g. space for program activities, tickets or transportation for recreational outings)

6. Member of advisory committee/provide advisory or consulting services

7. Provide training for program staff

8. Help with project organization/coordination between service providers/reaching service providers

9. Information sharing/contribution to research and/or knowledge accumulation

95. Other, specify

99. Can't determine/Not stated

31. (AR/MPS) Was any of the partners not involved to the level as expected? If yes, which one(s)?

Section 4. Immediate outcomes

32. (AR) What was the number of the target youth participants per year? How many youth were referred to the project? How many of them were accepted? And how many drop outs?

33. If different intervention dosages applied, how many youth participated in intensive programs (with high intervention level) _________________________, and how many participated in the contact programs (with low intervention level) ______________________________________

34. (AR) Duration of participation among youth participants

35. (AR) How many educational programs or workshops were held for the following groups?

36. (AR) How many attendees for each workshop?

37. (AR) What experimental design does the evaluation use or propose to use? _______________

38. (AR) Does the evaluation involve a control group, if so, which one?

1. Yes, specify the control group, add notes if necessary
2. No          

39. (AR) Does the report/evaluation contain any analysis regarding whether the risk assessment approach used helped to reach the youth at the greatest risk?

1. Yes, if so, specify the conclusion drawn by the evaluator_______________________
2. No          

40. (AR) Does the report contain any analysis of change over time?

1. Yes          [CONTINUE]
2. No           [SKIP TO SECTION 5]

38. (AR) Does the report contain results from the mid or post test of the youth participants? [CHECK ONE ONLY]

1. Yes, mid test (usually six months after the pre-test)
2. Yes, post test (usually twelve months after the pre-test)        
3. Yes, follow-up test (usually six to twelve months after the post-test)     
4. No

39. (AR) If the report only contains qualitative analysis, briefly describe the findings regarding the change. _______________________ [SKIP TO SECTION 5]

40. (AR) Among youth-in-gangs participants, was there a reported attitude change in the following areas? (Yes/No)

41. (AR) Among at-risk-youth participants, was there a change in awareness and perceptions regarding the following? (Yes/No)

42. (AR) Describe the change on risk factors, if possible. ___________

43. (AR) Have there been any changes at the community level as a result of the programs implemented? (Yes/No)

44. (AR) Have there been any positive changes among at-risk-youth participants on the following risk and protective factors? (Yes/No)

45. (AR) Have there been any positive changes among youth-in-gangs participants on the following risk and protective factors? (Yes/No)

46. (AR) Have there been any positive changes at the community level on the known risk and protective factors? (Yes/No)

47. (AR) Is there evidence of diffusing effects on the community level, as a result of the programs implemented? If yes, what effects are they?

Section 5. Financial Data

48. (AR/GCIMS/BD) What is the total cost of delivering the programs/services to date, and what's the total cost of the project, including the set-up cost? What's the cost per participant of delivering the programs/services to date, and what's the total cost per participant?

49. (AR) If workshops or educational programs were offered to groups other than youth, what is the cost per attendee on average?

Section 6. Challenges and Lessons Learned

50. (AR) Did the annual project evaluation report identify any implementation challenges, issues that needed to be addressed? If so, what are they?

51. (AR) Is there any problem with reviewing this annual project report? Examples are incomplete information/information gap, unclear reporting format of the analysis, etc. If so, describe.

52. (AR) Have evaluators identified any areas where the projects could have been executed more efficiently?

Section 7. Youth Participant Profile

53. (AR) Demographics of the youth participants on the baseline survey (% if stated; otherwise, Yes/No/NS)

Base: ______________________      n= ________________

Note: 1 Examples include poor school attendance, suspensions or expulsions, failure, poor attitude to school

Note: 2 Examples include having friends in gangs, and having friends who have previous criminal offenses, or were involved in delinquent activity

Appendix E – interview guides

Key documents:

• Provincial Representatives Interview Guide
• NCPC Staff Interview Guide
• YGPF Project Contact Interview Guide
• Crime Prevention Researcher and Practitioner Interview Guide
• NCPC Management Interview Guide

1. Provincial Representatives Interview Guide

Interview Guide for Provincial Representatives

Introduction

Hello, my name is _______, from NRG Research Group. We have been contracted by the NCPC to conduct the formative evaluation of the Youth Gang Prevention Fund program. Do you have some time to provide some insight into some of the questions that we will be addressing in the evaluation report?  

The reason I am contacting you to talk with for this evaluation is because of your role with respect to crime prevention at the provincial level. and your involvement with the National Crime Prevention Centre and its work.  Through consultation with staff at the NCPC, we identified you as a person who could speak to the state of gang prevention and gang crime in Canada.

Do you have some time now to answer some questions about the program?  [Reschedule as necessary].

Please remember all of the feedback you provide me will be strictly confidential and will be used only for the purposes of conducting this evaluation.  If there are any questions that your role or experience does not enable you to answer, just let me know and we will move on. I would like to record this interview so that I don't have to take notes as we talk. Is that okay?

Background

• Could you tell me about your experience and current role in the field of crime prevention?
• More specifically, what has been your role with respect to YGPF?

 Were you involved in the discussions leading to the selection of specific program sites?

 When did you become involved with YGPF?

 Do you have much current involvement with the funded projects?

Have you been involved in reviewing  new requests for funding for gang prevention and intervention projects?

Design and Delivery

If you have been involved in the selection of projects:

The process for identifying projects for funding under YGPF was different from that followed by NCPC in the past, in that potential sites were identified through consultation with the provinces, rather than through soliciting proposals from communities. This was followed by a project development phase whereby Program Officers and Evaluation Analysts worked closely with the recommended organizations to develop their projects.

• In your view, what were the strengths and weaknesses of this approach?  What has been learned?
• In your view, has the YGPF been designed so as to meet its objectives?
• Is it being delivered in a way that will meet its objectives?

What about the program makes it an effective approach?

Are there things about the approach or program that are ineffective or not likely to work?

 Relevance

• I'd now like to ask you about the continued relevance of the YGPF program.  From your perspective, is the YGPF playing a unique role, filling a unique need? How so?

Are the goals of the program still relevant?

Is it serving the public interest?

Does it fit well with other initiatives aimed at addressing the gang problem?

• The federal government currently provides funding to almost 20 projects through the YGPF. What would happen to these programs if the federal government did not play its current role?

Could the programs be maintained?

What would be the consequence for the youth currently involved in the programs?

Are there are other resources in the area?

What about at the provincial level?

• Are you aware of other gang prevention initiatives that have started since the inception of the YGPF?

Which ones are they?

Whom do they serve?

What kinds of activities are they engaged in?

What geographic area do they cover?

Do they complement or potentially duplicate what YGPF is trying to accomplish?

Can you direct me to a source of information about them?

• How do you view the federal government's role in the area of youth gang prevention?

Is it taking a leadership role?

Should it be taking a larger role/ smaller role? A different role?

• And what about integration of provincial and federal initiatives?  To what extent does your province have goals in common with NCPC in the area of gang prevention and intervention?  
• To what extent does the partnership with NCPC enable your jurisdiction to achieve its goals with respect to addressing youth gang problems in communities?

Cost-effectiveness/Alternatives

• In your opinion, are there more effective ways of achieving the objectives of YGPF? 

Those are the questions I have for you today with regards to the YGPF program evaluation. Do you have any additional comments that might be helpful for me with regards to the evaluation?

Thank you very much for your time today.   

2. NCPC Staff Interview Guide

Interview Guide for NCPC Staff

Introduction

Hello, my name is _______, from NRG Research Group. We have been contracted by the NCPC to conduct the formative evaluation of the Youth Gang Prevention Fund program. As an NCPC staff member who has been very involved in YGPF, your views and experiences will be very useful in helping us determine if the program is being delivered as intended. Do you have some time now to answer some questions about the program?  

Please remember all of the feedback you provide me will be strictly confidential and will be used only for the purposes of conducting this evaluation.  If there are any questions that your your role or experience does not enable you to answer, just let me know and we will move on. I would like to record this interview so that I don't have to take notes as we talk. Is that okay?

Background

I'd like to begin by asking you some questions about your role in the YGPF program. Could you tell me how long you have been involved in the program, and what your role has been in that time?

Program Design and Delivery

My first set of questions deal with issues around program design and delivery.

• Let me begin by asking about program funding. Was funding to projects at the level that was requested?

If not, was it more or less?

Was the amount adequate to serve the youth in the program?

• Were any projects rejected for funding, or discouraged from formally applying?

Which projects were they?

Why were these projects rejected or not considered?

The process for identifying projects for funding under YGPF was different from that followed by NCPC in the past. For the YGPF, potential sites were identified through consultation with the provinces, and then Program Officers and Evaluation Analysts worked closely with the organizations to develop their projects.

• What is your view on this approach?

Are there advantages to it?

Are there weaknesses to it?

What has been learned from this process?

• Do you feel that the roles and responsibilities for NCPC staff in the administration of the YGPF are clearly and appropriately defined?

What needs to be clarified or changed?

• Are the procedures and processes employed in administering the YGPF effective and efficient?

Are there changes that need to be made that would increase effectiveness or efficiency?

• What about providing the appropriate tools, resources or training to help funded projects in the communities achieve their objectives? Can you provide some examples of what you or others in NCPC  have done in this area? 

Is this something the program is doing well?

Are there improvements that could be made? What kind?

What resources or tools are of particular value?

Relevance

• Are you aware of other gang prevention initiatives have started since the inception of the YGPF?

Which ones are you aware of?

Who do they serve?

What kinds of activities are they engaged in?

What geographic area do they cover?

Do they complement or potentially duplicate what YGPF is trying to accomplish?

• Can you direct me to any sources of  information about them?

Success

The YGPF has two outcome streams – one stream is related to knowledge development and dissemination. The YGPF is expected to contribute to the development of increased knowledge concerning gang prevention in Canada.

• Are you aware of any ways in which YGPF has contributed to increasing knowledge of how to deal with youth gangs? (For example, publications, conference presentations, media interviews, etc produced or delivered by the project or those associated with the project such as the evaluator, staff, participants)

Has the program been up and running long enough to produce these kinds of outcomes?

Do you expect that it will produce these kinds of outcomes?

The second outcome stream is related to changes in individual participants and communities resulting from the funded projects.

• For the project(s) you are working with, have you seen any evidence of early outcomes, either for individual participants or for communities? 

• Have the project(s) you are working with faced any challenges in implementation? 

What are they?

Are these challenges being resolved?

Is there anything NCPC could be doing to assist?

Cost effectiveness

• In your opinion, are there more effective ways of achieving the objectives of the YGPF? 

• Can you think of any ways in which YGPF could be delivered more efficiently?

Unintended impacts

• Has the YGPF had any unexpected impacts to date, either positive or negative?

Those are the questions I have for you today with regards to the YGPF program evaluation. Do you have any additional comments that might be helpful for me with regards to the evaluation?

Thank you very much for your time today.  

3. YGPF Project Contact Interview Guide

Interview Guide for YGPF Project Contacts

Introduction

Hello, my name is _______, from NRG Research Group. We have been contracted by the NCPC to conduct the formative evaluation of the Youth Gang Prevention Fund program. As someone who manages/coordinates a program funded under the YGPF your views and experiences will be very useful in helping us determine if the program is being delivered as intended. Do you have some time now to answer some questions about the program? 

Please remember all of the feedback you provide me will be strictly confidential and will be used only for the purposes of conducting this evaluation. .  If there are any questions that your role or experience does not enable you to answer, just let me know and we will move on I would like to record this interview so that I don't have to take notes as we talk. Is that okay?

Background

I'd like to begin by asking you some questions about your role in administering the PROGRAM NAME. How long have you been managing/directing that program? What has been your role in that time?

Program Design and Delivery

My first set of questions deal with issues around program design and delivery.

• Let me begin by asking about program funding and resources. Was funding to the project at the level that was requested?

If not, was it more or less?

Is the amount adequate to serve the youths in the program?

If inadequate, what kind of problems or issues has this created?

• What additional funding or resources are needed to improve the program?

What would you do with these additional resources?

• Do you feel that the procedures and processes employed by NCPC in the administration of YGPF are effective?

What needs improvement?

What is working well?

• What about tools, resources or training provided by the YGPF to help funded projects achieve their objectives? Is this something the program is providing you?

What tools or resources have been made available to you?

Have you used them?

What resources or tools are of particular value?

Are there improvements that could be made? What kind?

• Now let me ask you a bit about your program and how it relates to the characteristics of the kinds of programs that the YGPF is intended to fund. So, how does the program:

Target youth at greatest risk of joining gangs or wishing to exit?

Address multiple, known risk and protective factors at the individual, family, peer, school and community levels?

Tailor its approach to the local community?

Adopt an evidence-based approach to both what to do and how to do it?

Include partnerships with sectors whose participation is necessary to impact all relevant risk and protective factors?

Relevance

• The federal government currently provides funding to your program through the YGPF. What would happen to your program if the federal government did not play its current role in the project?

Could the program be maintained?

What would be the consequence for the youth currently involved in the program?

Are there are other resources in the area?

• How has gang-related crime changed in your community since the inception of the YGPF in 2006?

• Are you aware of other gang prevention initiatives have started since the inception of the YGPF?

Which ones are they?

Who do these initiatives serve?

What kinds of activities are they engaged in?

What geographic area do they cover?

Do they complement or potentially duplicate what YGPF is trying to accomplish?

• Where would you recommend looking to find out more about these initiatives?

• How do you view the federal government's role in the area of youth gang prevention?

Is it taking a leadership role?

Should it be taking a larger role/ smaller role? A different role?

Success

The YGPF has two outcome streams – one stream is related to knowledge development and dissemination. The YGPF is expected to contribute to the development of increased knowledge concerning gang prevention in Canada.

• Are you aware of any ways in which YGPF has contributed to increasing knowledge of how to deal with youth gangs? (For example, publications, conference presentations, media interviews, etc produced or delivered by your project or those associated with your project such as the evaluator, staff, or program participants)

• Has the program been up and running long enough to produce these kinds of outcomes?

Do you expect that it will produce these kinds of outcomes?

The second outcome stream is related to changes in individual participants and communities resulting from the funded projects.

• How about impact at the local, community level.  Have you seen changes in your local community as a result of your program?

Unintended impacts

• Has the YGPF had any unexpected impacts to date, either positive or negative?

Those are the questions I have for you today with regards to the YGPF program evaluation. Do you have any additional comments that might be helpful for me with regards to the evaluation?

Thank you so much for your time today.

4. Crime Prevention Researcher and Practitioner Interview Guide

Interview Guide for Crime Prevention Researchers and Practitioners

Introduction

Hello, my name is _______, from NRG Research Group. We have been contracted by the NCPC to conduct the formative evaluation of the Youth Gang Prevention Fund program.

The reason I am contacting you to talk with for this evaluation is because of your expertise in crime prevention. Through consultation with staff at the NCPC, we identified you as a person who could speak to the state of gang prevention and gang crime in Canada.

Do you have some time to provide some insight into some of the questions that we will be addressing in the evaluation report? [Reschedule as necessary].

Please remember all of the feedback you provide me will be strictly confidential and will be used only for the purposes of conducting this evaluation.  If there are any questions that your role or experience does not enable you to answer, just let me know and we will move on. I would like to record this interview so that I don't have to take notes as we talk. Is that okay?

Background

Could you tell me about your experience and current role in the field of crime prevention?

In the area of youth gangs in particular?

Program Awareness

• Are you familiar with the NCPC's Youth Gang Prevention Fund?

• If you are aware of the YGPF, what are your impressions of this program?

Program Description: The YGPF is a federal government program, administered through the NCPC, that provides funding to community-based projects that seek to prevent and reduce youth gang membership and thus contribute to a reduction in gang-related crime. It was announced in 2006, and is currently funding close to 20 projects across Canada. There are specific project requirements, including the need to target interventions to youth in gangs or at high risk of joining gangs, to address known risk and protective factors in multiple domains, and to be evidence-based.

• In your opinion, does the YGPF have the characteristics of a well designed gang prevention /intervention program?

• What characteristics should this type of program have in order for it to be successful?

Need for Youth Gang Prevention Programs

• Do you think there is a need for such a program? That is, is gang-related crime a serious issue in Canada?

• How has gang-related crime changed in Canada since the inception of the YGPF in 2006?

• Is there a need for prevention or social intervention in addressing this issue?

Effectiveness of the YGPF Approach and Comparison to other Programs

• What about the program makes it an effective approach?

Are there things about the approach or program that are ineffective or not likely to work?

Do you have any suggestions about how things might be done differently to improve the chances of it being effective?

• Does it fit well with other initiatives aimed at addressing the gang problem?

How does it fit with other initiatives?

Is there duplication? Where?

• Are you aware of other gang prevention initiatives have started since the inception of the YGPF? If so, can you name them for me, and direct me to a source of information about them?

• What about how this kind of program compares to gang prevention strategies in other countries?

 Where does the YGPF stand?

Is it innovative, leading edge in what it is trying to accomplish?

Or is it in line with approaches being used elsewhere?

• Could the YGPF benefit from what is being done in other countries? How?

• Have other gang prevention strategies been shown to be effective? What can the YGPF learn from these programs/strategies?

Role of the Federal Government

• Do you have any views regarding the role of the federal government in particular in the area of youth gang prevention?

Is it taking a leadership role?

Should it be taking a larger role/ smaller role? A different role?

Those are the questions I have for you today with regards to the YGPF program evaluation. Do you have any additional comments that might be helpful for me with regards to the evaluation?

Thank you very much for your time today.

5. NCPC Management Interview Guide

Interview Guide for NCPC Management

Introduction

Hello, my name is _______, from NRG Research Group. We have been contracted by the NCPC to conduct the formative evaluation of the Youth Gang Prevention Fund program. As one of the directors or managers of the NCPC your views and experiences will be very useful in helping us determine if the program is being delivered as intended. Do you have some time now to answer some questions about the program?  

Please remember all of the feedback you provide me will be strictly confidential and will be used only for the purposes of conducting this evaluation.  If there are any questions that your role or experience does not enable you to answer, just let me know and we will move on. I would like to record this interview so that I don't have to take notes as we talk. Is that okay?

Background

I'd like to begin by asking you some questions about your role in the YGPF program. Could you tell me how long you have been involved in the program, and what your role has been in that time?

Program Design and Delivery

My first set of questions deal with issues around program design and delivery.  Let me begin by asking about  the process for identifying projects for funding under YGPF.

This process was different from that followed by NCPC in the past. For the YGPF, potential sites were identified through consultation with the provinces, and then Program Officers and Evaluation Analysts worked closely with the organizations to develop their projects.

• What is your view on this approach?

• Are there advantages to it? Weaknesses?

• What has been learned from this process?

I would also like to ask about program resourcing.  And by resourcing, I'm not only referring to the amount of money allocated for projects but how resources are allocated between project funding and PS funding, including  NCPC's organizational structure and staffing and departmental supports in areas of Human Resources, Finance, and Contracting Services.

Are there issues related to amount of resources, their allocation or both? What are these issues?

Should resources be reallocated?

How would you allocate additional resources to the program?

How much more is needed? 

Now let's turn to roles, responsibilities, and administration. 

• Are the roles and responsibilities of the NCPC in administering the YGPF clearly and appropriately defined?

What, if anything, do you feed should be changed?  

What is working well?

• What procedures should be followed to ensure effective administration of the funding program?

• In your view, are the appropriate procedures being followed? If not, what should be changed?

• Are you aware of any issues related to program administration that should be addressed?

What are these administrative issues?

How should they be addressed?

•  What about providing the appropriate tools, resources or training to help funded projects in the communities achieve their objectives? Is this something the program is doing well?

Are there improvements that could be made? What kind?

What resources or tools are of particular value?

Relevance

I'd now like to ask you about the continued relevance of the YGPF program. 

• From your perspective, is the YGPF playing a unique role, filling a unique need? How so?

Are the goals of the program still relevant?

Is it serving the public interest?

Does the program fit well with other initiatives aimed at addressing the gang problem?

How do you see the role of the federal government compared to that of provincial governments with respect to gang prevention?

• Are you aware of any other gang prevention initiatives have started since the inception of the YGPF?

Which ones are they?

Whom do these initiatives serve?

What kinds of activities are they engaged in?

What geographic area do they cover?

Do they complement or potentially duplicate what YGPF is trying to accomplish?

Can you direct me to a source of information about them?

Success

The YGPF has two outcome streams – one stream is related to knowledge development and dissemination. The YGPF is expected to contribute to the development of increased knowledge concerning gang prevention in Canada.

• Are you aware of any ways in which YGPF has contributed, either directly or indirectly,  to increasing knowledge of how to deal with youth gangs? (For example, publications, conference presentations by NCPC , its projects, or those affiliated with YGPF, etc)

Has the program been up and running long enough to produce these kinds of outcomes?

Do you expect that it will produce these kinds of outcomes?

Cost effectiveness

• In your opinion, are there more effective ways of achieving the objectives of the YGPF? 

• Can you think of any ways in which YGPF could be delivered more efficiently?

Unintended impacts

• Has the YGPF had any unexpected impacts to date, either positive or negative?

Those are the questions I have for you today with regards to the YGPF program evaluation. Do you have any additional comments that might be helpful for me with regards to the evaluation?

Thank you very much for your time today.

Appendix F – Review of recent Canadian gang literature

[*] Unfortunately, we did not have access to the full texts of these sources. The information presented is based on the abstracts.

Footnotes

1 The YGPF logic model was revised by NCPC in December 2006 to focus more on specific intended outcomes.

2 Annual report is not available for two projects, as evaluation framework is being developed

3 This initiative is mentioned in the Alberta Justice Business Plan 2009-2012. It is a "strategy based on awareness, prevention, intervention and enforcement to reduce gang activity and related violence in the province."

4 The NCPC Review Committee was composed of staff from various NCPC sections (e.g., Evaluations, Research, Programs). The Committee reviewed project proposals, which were prepared by Program Officers and Evaluation Analysts, and had to approve proposals before they could be recommended to the Minister for funding.

5 This issue is not identified in the ARAF Evaluation Issues and Questions. NRG has added it as we think it is a very important piece of the evaluation.

6 If 'yes,' the article focused specifically on the issue of youth gangs and did not include information about youth gangs with a focus on a different topic.

7 If 'yes,' the article used primary data collected in Canada with Canadian participants, or included recommendations and implications targeted to the Canadian context.

8 The synopsis was downloaded from http://ontarioguns.blogspot.com/2008/01/youth-weapons-and-violence-in-toronto.html.

Date modified:

2015-12-15