Evaluation of the National Crime Prevention Strategy

Table of contents

Executive Summary

Created in 1998, the National Crime Prevention Strategy (NCPS) is Public Safety Canada’s (PS) policy framework for the implementation of crime prevention interventions in Canada. It provides funding to selected projects that contribute to preventing and reducing crime and to increasing knowledge about what is effective in the prevention of crime.

From 2008 to 2019, the NCPS has provided funding to 650 projects in communities across Canada, representing multi-year investments of approximately $500 million. The Strategy is implemented through four funding programs, three of which are included in this evaluation: the Youth Gang Prevention Fund (YGPF); the Crime Prevention Action Fund (CPAF); and the Northern and Indigenous Crime Prevention Fund (NICPF). Due to its unique nature, the Security Infrastructure Program will be evaluated separately in the 2025-26 fiscal year.

What we examined

The purpose of the evaluation was to assess the relevance, effectiveness (achievement of outcomes) and efficiency of three of the funding programs under the NCPS: the YGPF, the CPAF and the NICPF. The evaluation covered the period from fiscal year 2018-19 to 2021-22 and was conducted in accordance with the Treasury Board Policy on Results and the Directive on Results.

What we found

Recommendations

The Senior Assistant Deputy Minister, Crime Prevention Branch, and the Assistant Deputy Minister, Emergency Management and Programs Branch should:

Background

Created in 1998, the National Crime Prevention Strategy (NCPS) is Public Safety Canada’s (PS) policy framework for the implementation of crime prevention interventions in Canada. It provides funding to selected projects that contribute to preventing and reducing crime and to increasing knowledge about what is effective in the prevention of crime.

The Strategy is based on the premise that well-designed interventions can positively influence behaviours that lead to crime, especially among youth; and that crimes can be reduced or prevented by addressing risk factors that can lead to offences.

The Strategy contains three key elements:

The NCPS is administered by PS in partnership with provinces and territories (PTs). PS, through the NCPS, provides time-limited funding in the form of grants (maximum of 36 months) and contributions (maximum of 60 months) to organizations to address priority crime issues.

In 2008, the NCPS shifted to an evidence-based approach to testing crime prevention interventions which involves funding model, promising or innovative interventions. Project evaluations have been used, since the early days of the program, to measure the effectiveness of projects in achieving their expected outcomes. To do so, a selected group of funded projects receive additional resources, including financial support for hiring an external evaluator, to undertake process and impact evaluations of projects. This contributes to the development of a knowledge base of what is effective in preventing crime within the Canadian context.

From 2008 to 2019, the NCPS has provided funding to 650 projects in communities across Canada, representing multi-year investments of approximately $500 million.

The Strategy is implemented through four funding programs, three of which are included in this evaluation:

Youth Gang Prevention Fund (YGPF)

The YGPF provides time-limited funding for community initiatives that prevent at-risk youth from joining gangs, provide exit strategies for youth who belong to gangs, and offer support to youth so they do not re-join gangs.

Its objectives are to reduce serious youth violence and youth gang threats in communities by:

Crime Prevention Action Fund (CPAF)

The CPAF provides time-limited grant and contribution funding that supports evidence-based crime prevention initiatives in communities that address known risk and protective factors associated with crime among vulnerable groups of the population, especially children and youth aged 6-24 years, and chronic offenders.

It has three main objectives:

Northern and Indigenous Crime Prevention Fund (NICPF)

The NICPF (previously the Northern and Aboriginal Crime Prevention Fund) assists communities experiencing multiple risk factors and other challenges that affect their ability to respond to crime issues, such as remote geographical location and limited capacity. It achieves this by providing time-limited funding to support culturally sensitive initiatives that foster the development and implementation of crime prevention approaches in Indigenous communities, both on-and off-reserve and in the North. In addition, it assists in building the knowledge and capacity required to develop or adapt culturally sensitive, effective ways to prevent crime.

To affect positive changes in risk and protective factors and foster crime prevention in Northern and Indigenous communities, the NICPF supports:

Security Infrastructure Program (SIP)

The Communities at Risk: SIP was created in 2007, in response to concerns raised by a number of communities across Canada regarding their vulnerability to hate-motivated crime. The SIP provides time-limited financial assistance in the form of matched funds (up to 50% of a project’s cost) to private, non-profit organizations linked to a community as risk of hate-motivated crime, to make security improvements to their community gathering spaces.

Please note that, due to its unique nature, this program will be evaluated separately in the 2025-26 fiscal year.

Projects

Through the funding programs, the NCPS funds a variety of projects. Three project descriptions are provided below as examples of the projects that are funded:

Surrey Anti-Gang Family Empowerment (SAFE) Program

Funded for $7.5M through the YGPF, this program, led by the City of Surrey, works to keep children and youth out of gangs while building positive life skills and increasing connections with family, school and community. SAFE is made up of 11 programs delivered by 10 partner agencies, including not-for-profit, law enforcement, local governments and universities.

Project Venture

Funded for $1.8M through the NICPF*, this program, led by the Naskapi Nation of Kawawachikamach, uses an evidence-based model first developed by an Indigenous community. This program uses traditional values to assist youth in developing a positive self-concept combined with effective social skills, a community service ethic as well ass decision-making and problem-solving skills.

Open New Tab: A comprehensive initiative to address cyberviolence in the lives of Nova Scotia’s young people

Funded for $1.9M through the CPAF, and delivered by the YWCA Halifax, this program delivers anti-cyberbullying programming to diverse youth ages 9-17 in schools and community spaces in urban and rural locations. The program uses a preventative model to prevent cyber violence using a gender lens.

*The NICPF did not receive dedicated funding until 2022-23. Prior to that, NICPF projects were funded under CPAF.

Engagement Purpose and Methodology

The purpose of this evaluation was to assess the relevance, effectiveness (achievement of outcomes) and efficiency of three of the funding programs under the NCPS: the Crime Prevention Action Fund, the Northern and Indigenous Crime Prevention Fund and the Youth Gang Prevention Fund. The evaluation covered the period from fiscal year 2018-19 to 2021-22 and was conducted in accordance with the Treasury Board Policy on Results and the Directive on Results.

Interviews

Forty-one key informant interviews were conducted with individuals from PS; Federal, Provincial, Territorial (FPT) Working Group members; external evaluators; and funding recipients.

Literature and Program Document Review

Program documents and literature (e.g. government reports, articles, academic research) were reviewed.

Performance and Financial Data

Available performance data was reviewed and program financial data was analysed.

Limitations

Due to changes in project reporting formats, there was limited performance data available (compiled Annual Performance Report (APR)) data was only available for 2020-21 and 2021-22). To mitigate this challenge, the evaluation team made use of impact evaluations and evidence provided during interviews.

Findings

Continued Need

Finding: The NCPS is aligned with the mandate of PS and adjusts to align with federal priorities related to crime prevention.

Mandate and priorities

Crime prevention and reduction is a key part of the mandate of PS with recent mandate letters for the Minister of Public Safety including measures to keep communities safe from gun and gang violence. Additionally, the 2019 mandate letter committed PS and its portfolio partners to collaborate with Women and Gender Equity (WAGE) to continue to build on the Strategy to Prevent and Address Gender-Based Violence. PS received funding over five years for work to support the Strategy which included funding for the NCPS to enhance and develop a number of community-based projects aimed at preventing bullying and cyberbullying.

Further, in 2021, the mandate letter included continuing to combat systemic racism and discrimination in the criminal justice system with a focus on addressing the overrepresentation of Black and racialized Canadians as well as Indigenous peoples.

Crime prevention is also a shared priority for the Federal, Provincial, Territorial Ministers of Justice and Public Safety. All levels of government are concerned with the mounting pressure on the criminal justice system and the costs associated with traditional crime control measures.  The government is committed to funding community-led services that “stop crime before it starts through the empowerment of at-risk individuals”.

During the time period covered by the evaluation, the NCPS launched several Calls for Applications. The various priorities for these Calls demonstrate that the NCPS adjusts to align with the priorities of the federal government as they relate to crime prevention.

2018 NCPS Call for Applications’ priorities were direct intervention and strengthening community readiness projects among Indigenous and vulnerable populations to prevent or reduce the impacts of youth gangs, youth violence, bullying and cyberbullying on communities.

These priorities continued to be relevant in 2021 and beyond and complemented work being undertaken across the federal government. They also demonstrated an agility on behalf of the government in responding to the call to address the needs of Black and Indigenous peoples.

The 2021 CPAF Targeted Approach supported the implementation of interventions that incorporate a multi-sectoral partnerships approach, a key component of Community Safety and Well-being, to enhance protective factors and reduce risk factors impacting vulnerable populations. This approach demonstrated a focus on populations overrepresented in the Criminal Justice System.

As of 2022-23, the CPAF funding priorities continue to focus on multi-sectoral community-based interventions and support the implementation of bullying and cyberbullying intervention projects that include gender-specific programming. These funding priorities also include the dissemination of knowledge and the development of tools and resources to address crime prevention issues amplified by the COVID-19 pandemic including an increased focus on the risk factors related to mental health, family violence and substance misuse.

Finding: While risk and protective factors have not significantly changed since the inception of the program, criminal activity has evolved. The NCPS addresses ongoing risk factors and attempts to adapt to changes in criminal activity.

Risk factors associated with crime

Risk and protective factors help to explain why individuals or groups are more likely to become involved in crime. Risk factors are the negative influences in the lives of an individual or community and include such things as poverty, family instability, trauma, intergenerational issues and social isolation. Protective factors are positive influences that can improve the lives of individuals or the safety of a community and include familial support, access to proper healthcare and financial security.

Crime within Canada varies throughout different regions, making it difficult to address through a standardized approach. The NCPS provides capacity to support relevant interventions and tools for crime prevention by funding local, community-led interventions to reduce risk factors and increase protective factors in each region or unique community.

In the current setting of criminal activity, key informants discussed an increase in certain types of crime and the severity of crime occurring within Canada. These changes were particularly present in Indigenous and Black communities, as well as among youth. There was a growing reported occurrence of hate crimes, cyberbullying, gang activity, domestic violence and sexual violence. Despite the increase in certain types of crime, the underlying risk factors that lead to various types of criminal activity remain relatively static, which explains the consistency in risk factors over decades.

The NCPS was described by key informants as willing, through its objectives, to take a preventative approach towards crime by increasing protective factors through innovative programs and directing programming to youth with high risk factors before they become involved in crime.

However, given the rate at which the trends related to crime are changing, there was concern among key informants that the NCPS may not be able to keep pace. Emerging technologies have allowed for new approaches to committing crime, particularly online and the gradual process of the NCPS may miss these dynamic trends within its objectives.

Finding: The NCPS funding programs target communities and populations with elevated risks as well as addressing newer trends in crime. This has included focusing on Black and Indigenous youth and communities, activities related to cyberbullying and additional gender-specific programming.

Communities/populations with elevated risks

Through their funding programs, the NCPS attempts to address the prevention of crime and understand the risks associated with crime in communities to better address the issues affecting these groups. These groups, with elevated risks associated with crime include:

The array of funding programs under the NCPS address various groups with elevated risks. For instance, the NICPF attempts to minimize Indigenous youth involvement in anti-social behaviours and criminal activity by providing opportunities to communities to find new and culturally relevant ways to address their crime issues.

The CPAF 2021 cyberbullying targeted approach included preventing and addressing bullying and cyberbullying through intervention projects that provided information on healthy relationships, gender-based intervention and case management. The 2021 targeted approach ensured that it incorporated gender-specific programming for priority populations, including girls and 2SLGBTQI+ youth.

Finding: From the key informants’ perspective, the NCPS programs are reaching priority populations, but barriers remain.

Priority populations

During the time period covered by the evaluation, the NCPS funded 89 projects all of which targeted at-risk populations.

In 2020-21, 42% of projects that completed an Annual Performance Report (APR) (n=26), had members of the Black community as their primary participants. This increased to 58% of projects in 2021-22.

In 2020-21, 38% of projects that completed an APR had Indigenous primary participants and in 2021-22, 35% had Indigenous primary participants.

From a gender perspective, 18 of the 26 (69%) projects in 2020-21 and 24 of the 31 (77%) projects in 2021-22 reported having collected information on primary participants based on gender. Both years included more male primary participants than female. Non-binary, transgender, and gender-fluid people made up an even smaller number of participants.

Many of the projects also collected information on the age of primary participants. The largest group of primary participants in 2020-21 and 2021-22 were youth aged 12 to 15, followed by youth 16 to 17 and children aged 9 to 11.

While not all projects are achieving their anticipated participation rate, key informants feel that for the most part, the NCPS programs are reaching priority populations.

However, key informants expressed some concerns about the challenges reaching priority populations that may not be addressed through the NCPS.

Difficulties ranged from access to those in rural areas, to addressing the mistrust that some communities or individuals have with government or authority. In some cases, this meant that communities may not apply for funding, and in others it meant that projects spent additional efforts to reach at-risk youth. As well, across Indigenous communities there may be a wide range of capacity requirements to manage projects, and a variety of needs that must be addressed. Accordingly, existing program funding may be insufficient to address these challenges and capacity gaps. While some of these challenges may be beyond the scope of NCPS, the program should, at a minimum, ensure geographic considerations are made in selecting recipients.

PS has conducted research on how best to work with Indigenous communities and in 2021 published the results of a research study that examined the unique implementation issues for crime prevention programs aiming to serve Indigenous populations. PS is currently implementing lessons learned with its approach to the NICPF. Evaluations of Indigenous projects, funded through the 2018 call, are currently underway and it is expected that these evaluations will lead to additional lessons about working with Indigenous communities.

Finding: Other crime prevention funding programs exist at both PS and in other departments. While these programs are considered complementary, some duplication and overlap occurs.

Crime prevention programs

The majority of key informants were aware of other federal crime prevention programs that had similar objectives to the NCPS.

At PS these programs include the Building Safer Communities Fund (BSCF) and the Gun and Gang Violence Action Fund (GGVAF). While overlaps exist between the GGVAF, BSCF and the NCPS, including similar objectives in providing funding for prevention, intervention and data collection, there are some differences. The GGVAF funding is allocated to PTs who can distribute it to partners within their jurisdiction. The GGVAF has an emphasis on law enforcement and a scope of activities which include enhancing prosecution capacity and providing training for enforcement, prevention and other professionals. The BSCF is a program aimed at helping municipalities and Indigenous communities prevent gun and gang violence. Recipients may further distribute funding to organizations who, where possible, should incorporate crime prevention models that have been proven to work in the Canadian context.

In other federal departments, the main programs noted by key informants included Justice Canada’s Youth Justice Fund and WAGE’s Federal Gender Based Violence Strategy. As mentioned previously, the NCPS received supplementary funding from the GBV Strategy to address bullying and cyberbullying.

As the risk and protective factors for crime are broad, and there are many entry points to criminal activity, the federal response needs to be varied and all encompassing. Some key informants suggested it is important to ensure that “each program is addressing or filling a gap that another program has”.

Finding: Overall, projects were implemented as planned though changes were made to respond to the COVID-19 pandemic. While projects achieved successes, they also faced challenges.

Effectiveness

Project implementation

Key informants felt that NCPS funded projects were generally implemented as planned though the majority of projects made adjustments due to a variety of factors including challenges in implementing the project according to the model, resourcing issues and changes to project scope.

There were many successes for the project participants, particularly youth and families, identified by interviewees and case studies. The connections created between youth and project staff were identified as an important factor in reaching and helping youth. This was attributed to having positive role models and trusting relationships with the workers.

Some participants in funded projects were reported to have returned to school, improved school attendance, finished school, shown improved behaviour at school or had a general increased attachment to school. Success at school is a protective factor that can decrease the likelihood that individuals become engaged in and/or victims of crime.

Other successes ranged from helping decrease criminal or violent behaviour among the participants, to transferring Indigenous cultural knowledge to the younger generation.

Some interviewees spoke to the importance of having positive Indigenous role models as well as Elders. More than one project involved an Indigenous skateboarding group visiting the community and sharing positive messaging; attending youth were excited to smudge their skateboards.

Some projects were seen to have had a positive impact on the wider community, and sometimes other communities. Some examples given included how Indigenous youth wanted to help their communities after participating in the project, and how the project had been able to reach out to youth in other communities. One project has four communities as part of their project and they run a youth program that often has youth from four to five Indigenous communities participating.

“If a kid needs support, we support them” – Funding recipient

The community was also identified as an important actor to make positive changes to prevent crime and violence, including through increasing community pride and becoming part of the discussion on crime.

Challenges

In interviews, key informants identified challenges related to resources for the funded projects. These included staff and volunteer recruitment and retention, the cost of implementing pre-existing models, and training staff to ensure fidelity to those models, as well as the time limited nature of funding.

Data from APRs for 2020-21 and 2021-22 supported key informants’ statements. In 2020-21, 42% of projects reported that difficulties recruiting and retaining staff had a very significant or significant impact on the delivery of their project. In 2021-22, 29% of projects reported similar difficulties.

Challenges identified by interviewees were also at the community level. While collaboration with community organizations was beneficial, there were challenges. It was recognized that establishing effective partnerships takes time, intentionality, and effort and that there wasn’t always the opportunity to consistently do so. In 2020‑21, 65% of projects reported that a lack of engagement with relevant parties/working with community partners had an impact on the delivery of their project, in 2021-22, 58% of projects reported the same.

COVID-19

The pandemic impacted many aspects of NCPS funded projects and resulted in difficulties forming connections between staff and clients which in turn affected participation rates for projects. Early research on the impacts of the pandemic also indicates that rates of family violence increased and the pandemic created challenging conditions for mental health which key informants supported. One key informant spoke about mental health issues and the changing social landscapes, with some families never leaving their house and not accessing services.

In 2020-21, 60% of projects reported COVID-19 challenges had either a very significant impact (24%), significant impact (32%) or minor impact (4%) on the delivery of their project. Ninety four percent of projects reported COVID-19 challenges on project delivery in 2021-22.

Despite these challenges, some key informants spoke to positive outcomes as a result of the pandemic which included the adaptation by projects to continue to meet the needs of clients.

“We pivoted service-wise. We got stronger. Partners had to find creative ways to work with populations. Virtual parent groups [were] created and there was more collaboration. We had three new groups created as a response to COVID.” – Funding recipient

“We had anticipated doing the therapy…in person. Instead we had to switch to virtual platforms. We weren’t sure how this type of work would occur online but we did it in a careful and planned way. It was a success and has allowed us to expand this program range a bit more but engagement online is more difficult.” – Funding recipient

Finding: NCPS funding equipped communities to address risk factors that lead to crime in their priority populations.

Addressing risk factors

Most key informants agreed that communities were better equipped to address factors leading to crime after having received the support of the NCPS. 

NCPS funding allows projects to provide or enhance protective factors, including pro-social activities, providing youth with increased interactions with workers and mentors, and supporting increased attachment to education. Some projects were able to reduce risk factors that lead to crime such as alcohol misuse and were able to provide healthier activities for children so they were not as heavily influenced by gangs.

Other projects noted there was an inability to provide support/services to all the target population, whether because some contributing factors were not being addressed or that they were unable to reach some populations because of risk to workers or societal inequities. Some of these challenges may be beyond the reach of the NCPS.

While the NCPS funding allowed projects to provide more support and services to target populations, it was considered by some to be insufficient to solidify lasting change.

One of the targets for the NCPS, as reported in the departmental results framework, is that 75% of projects will demonstrate that participants experienced positive changes in risk and protective factors related to offending. This target was met in 2019-20 but not in 2020-21. There are concerns with this measure, in that it is only reported by projects that complete impact evaluations. In 2021-22, 15 projects collected data for the impact evaluations, out of 32 projects that received funding for the evaluations. The remaining impact evaluations were either in the planning stage, or had data collection delayed as a result of the COVID-19 pandemic.

While only select projects complete impact evaluations, all projects are expected to complete annual reports. Data on outcomes is collected via self reporting in the APRs. While self-reported data is not without issues, including some uncertainty around the accuracy of reported numbers, it is important to collect and leverage NCPS project data. APR data demonstrated that significant or very significant improvements occurred for participants for a number of risk factors, with increases between 2020-21 and 2021-22. These are the two years with consistent APR data. In 2020-21, 26 projects completed an APR while in 2021-22, 31 projects did the same.

The following changes were noted:

Finding: The NCPS publishes policy and knowledge resources related to best practices in crime prevention. Awareness of these resources could be improved as many external key informants made limited use of the resources to inform their decision making.

Knowledge resources and best practices

The NCPS aims to increase the existing knowledge base of crime prevention initiatives in Canada. This is done through support for research, as well as the publishing of relevant reports and the summaries of project level impact evaluations.

A key element of this is the Crime Prevention Inventory (CPI). The CPI is a repository of information on crime prevention projects that have been funded by PTs, Non-Governmental Organizations or through the NCPS. The intent is to promote best practices in crime prevention by providing evidence-based information. As of June 1, 2023, the CPI had 216 resources available.

From page views (19,585 in 2021-22) and feedback, it is clear key informants are accessing the CPI. In the words of one FPT Working Group on Crime Prevention member, the CPI is “the one thing that really sticks with us”.

Along with the CPI, the PS website offers resources and descriptions of best practices for crime prevention, such as the importance of cultural elements; the role of the family as both a risk factor and a protective factor; and, the key role of partnerships and key informants.

The number of publications has varied over the past four years, but there have been a total of 22 reports and 27 project process and impact evaluations. For impact evaluations, the summaries are published.

While resources have been published, the percentage of key informants who reported consulting PS policy and knowledge resources to inform their decision making was above the target of 70% in 2020-21 and 2021-22. This indicator is reported in the Departmental Results Report.

Awareness of the resources also varied amongst the key informants consulted as part of the evaluation. They may know reports are available online, and may even have looked them up, but they could not recall specifics. Key informants were more likely to attribute their own strong knowledge of best practices in crime prevention to their personal experience and involvement with social agencies, than to the available NCPS resources.

A few of the key informants used the NCPS policy and knowledge resources as background research and to help inform their work, including making holistic decisions and connecting with target populations. In one instance, a key informant used the resources to create presentations for the community they were involved with. Another mentioned looking at what is happening in crime prevention programming at the provincial and federal levels to inform their work.

As several key informants were not aware of the resources offered, they did not benefit from them or utilize them to support decision making. One key informant felt that the policy and knowledge resources were abstract, and would be more beneficial if they provided guidance on implementation.

PS has previously made use of a variety of methods to increase the awareness of NCPS policy and knowledge resources. This has included:

While some of these methods are ongoing, key informants acknowledge that there has been less engagement in the past five years, and that PS is not as active in knowledge dissemination as it once was. One key informant stated that increasing awareness of NCPS resources is perhaps one of PS’ weakest areas.

4.3 Efficiency

Funding

Finding: Funding requests for the NCPS programs have consistently exceeded available funding.

Through the CPAF, YGPF and NICPF, the government invests over $40 million annually in community-based crime prevention efforts. Despite this, there is a significant gap between the amount of funding requested, recommended and available.

For the 2018 Call for Applications for the three funding programs, over 550 applications were received; 400 of these were reviewed. The cumulative funding request exceeded $1 billion. At the time the available funding was $26M over two years. As funding is provided for multi-year projects, the available amount varies. As part of the review process, the 39 projects that were recommended totalled $89.7M with a cost of $18.5M in the first year.

This pattern was seen again in the 2021 CPAF call for applications. One hundred and fifty applications were deemed eligible for funding, with an estimated cost of over $464M. Despite this, only 36 projects were funded.

PS staff acknowledged that the number of proposals received for each call always exceeds the availability of the funding. As well, key informants mentioned that there has been a trend for the NCPS to fund fewer projects, but these projects were of a larger scale. This requires more long-term funding and resources. It was felt that there is also a need to fund smaller projects in order to have a balance of projects which have differing approaches, reach different populations and may be more sustainable. PS attempted to address this by funding a component approach in 2018.

Finding: The NCPS is co-managed within PS. Governance issues exist, including communication challenges which may impede program efficiency.

Governance

The NCPS is managed by both the Crime Prevention Branch (CPB) and the Emergency Management and Programs Branch (EMPB). Crime prevention policy, project and research responsibilities fall within CPB. EMPB leads program implementation including the administration of funding and the collection of project level reporting.

Some PS staff perceived the governance structure to be siloed, mentioning a lack of communication, issues around roles and responsibilities and limited connection between the teams. These issues were also noted in the 2018 Evaluation.

While some PS staff felt the NCPS could benefit from having a stronger governance structure that incorporates programs, policy and research under one management structure, not all felt that way.

Despite these issues, key informants spoke positively of the new system put in place for the 2018 Call for Applications which involved members of the program, policy, research and project evaluation groups actively participating in the decision-making process and in the development of the materials related to the call. This process was employed again for the 2021 calls.

Another key concern was staff turnover resulting in a loss of institutional knowledge. Project recipients felt that they had to provide basic information to program staff, and that there was a lack of knowledge of the projects in general and the overall NCPS. PS is working on improving communication and developing a knowledge transfer plan for maintaining institutional knowledge. Despite this challenge, funding recipients reported good relationships with PS regional staff, who kept in contact with them and answered questions as needed.  

Finding: Information management and technological (IM/IT) infrastructure is dated and ineffective. Limited project level information is analyzed or utilized by PS, leading to inefficiencies.

Information management and use

Within PS, it was felt that the existing IM/IT infrastructure is dated thus creating barriers. Project information is supposed to be saved and referenced in the Public Safety Information Management System (PSIMS). Not all project information was up-to-date, accurate or complete in PSIMS, and no oversight of the information was apparent. When project data is entered by program staff, it is not clear how or if this information is leveraged or utilized other than to check off the provision of information as a Contribution Agreement requirement.

PSIMS is not considered user friendly, and it is difficult to input and transfer data within the system.  PS has attempted to modernize the application and reporting process with an online survey tool, called Snap Survey. While an improvement in some aspects, SNAP tools are not compatible with PSIMS adding another layer of complexity for program staff. It is important to note that PSIMS is not just for NCPS, but rather is the system for the majority of Grants and Contribution Programs in PS. Work is underway to develop a replacement system for PSIMS.

Funding recipients also had mixed reviews about the reporting requirements and the systems used. While some appreciated the online SNAP tools, that replaced Word or Excel documents, others described them as not user-friendly. Funding recipients are required to complete financial tracking as well as activity reporting. The financial reporting is done using Excel spreadsheets, and one respondent felt they were “super intimidating”. This is of particular concern for smaller organizations who may not have sufficient resources to dedicate to these reporting requirements.

Along with financial tracking, all projects are required to complete activity reports, as well as annual reporting. The APR was standardized in 2020-21 with minor changes made in 2021-22. It replaced the Performance Monitoring and Assessment Report (PMAR) and Annual Activity Report (AAR).

While the standardization of reporting can be beneficial, the changes between the PMAR, AAR and APR meant that projects had to adjust their own data collection mid-way through their activities in order to meet the reporting requirement. In some cases, this was not possible, so the validity of the self-reported data in the APR cannot be confirmed.

There was no evidence of analysis of the performance data done by PS of the APR, or PMAR. There was also no evidence of the sharing of performance data between EMPB and CPB. This has resulted in limited project level information being available for use in policy or research decisions. Several PS staff spoke to concerns regarding project data with one stating “the information isn’t accessible and getting information requires more steps”. Challenges with project level performance data were also raised in the 2018 evaluation.

An original cornerstone of the NCPS funding programs was the inclusion of project level impact evaluations supported by external evaluators and funded via grants and contributions. During the time period of this evaluation, 31 projects were involved in completing these evaluations and 21 evaluations are currently in progress. Additional funding is provided to projects for these evaluations.

As these evaluations are conducted by external evaluators, PS staff has limited input in the development or conduct of the work. While the evaluations are to be built around the outcome areas listed in the Contribution Agreements, there is no consistency in terms of data collection or analysis. Summaries of the evaluations are available through the CPI.

Various concerns were raised regarding evaluations, with the sentiment that some organizations feel pressured into evaluations. Some projects raised additional challenges related to evaluations including problems setting up the evaluation, gathering data and the level of effort required.

From the perspective of key informants, the NCPS does not capitalize on the lessons learned from project implementation and evaluations; projects generate a lot of information; and, at least one key informant feels it’s a mistake to primarily rely on impact evaluations.

Finding: PS lacks information on whether projects continue following the end of NCPS funding. Key informants stated that sustainability is less feasible for smaller organizations.

Sustainability

NCPS provides time limited funding for a maximum of 60 months. The majority of projects that receive funding under the NCPS may not continue after funding ends. A sustainability study was conducted in 2017 of 95 NCPS projects of which 44 were thought to have been sustained beyond the NCPS funding, either in whole or in part. Of the 44 projects identified, 20 agreed to participate in the study. Of the sample of 20, it was determined that 18 projects were sustained fully or in part, and 2 were not sustained. No further research has been done to examine project sustainability.

Several key informants, both PS and recipients, were not aware if projects have continued or will continue following the end of PS funding. For some recipients, it was too early in their project to know if it would continue however, some were hopeful they would be able to continue when funding came to an end.

A few recipients stated their project would continue after NCPS funding came to an end, with one indicating that a sustainability plan for their project had been in place from inception.

A couple of key informants noted that sustainability is not possible for smaller organizations while larger, more well-established organizations have a greater chance of continuing.

Some PS staff suggested that efforts to increase sustainability should be undertaken, such as increasing the emphasis on sustainability from the beginning of the project by incorporating a requirement for a sustainability plan into applications and/or establishing a one-time grant for a transitional period.

Conclusions

Crime prevention and reduction remains a priority for the Government of Canada as well as provincial and territorial governments. The NCPS, and its funding programs, is aligned with this priority and adjusts to align with changing federal priorities related to crime prevention. The program has evolved to include a focus on cyberbullying, and has focused on addressing the needs of Black and Indigenous peoples.

The NCPS continues to address ongoing risk factors related to crime and attempts to adapt to changes in criminal activity.

Communities and populations with elevated risk remain a focus for the NCPS funding programs. These communities include Indigenous Peoples, Black Canadians and children who are victims of cyberbullying. While the NCPS is reaching priority populations, barriers continue to exist in reaching certain populations.

Despite the COVID-19 pandemic, projects were implemented as planned though most made adjustments as a result of various factors. Projects experienced successes and challenges; while some projects had success creating connections between youth and project staff, others experienced challenges with staff recruitment and retention. The pandemic created challenges which affected participation rates though some key informants experienced positive outcomes as a result of the pandemic.

The NCPS provides funding which equips communities to address risk factors that lead to crime by enhancing protective factors while also making efforts to reduce risk factors. While funding allowed key informants to provide support and services to at-risk populations, it was considered by some to be insufficient to make a lasting change.

The NCPS publishes policy and knowledge resources related to best practices in crime prevention, however awareness of these varies. PS has previously used a variety of methods to increase the awareness of these resources however, not all efforts are ongoing leading to less engagement with these resources.

The management of the NCPS is split between two branches which some within PS find challenging with issues concerning communication and clarity of roles and responsibilities persisting.

Additionally, the technological infrastructure used for the program is dated and not all project information is up to date or complete. Projects are required to complete annual reports however, it does not appear that PS is making sufficient use of the data and key informants do not feel the NCPS is capitalizing on the lessons learned from project implementation.   

Concerns were raised over the sustainability of projects with focused concern over whether smaller organizations could sustain their projects beyond NCPS funding.  While some suggestions were provided on how to increase sustainability for projects, additional financial resources would be required to put in place some of the suggested solutions.

Recommendations

The Senior Assistant Deputy Minister, Crime Prevention Branch, and the Assistant Deputy Minister, Emergency Management and Programs Branch should:

  1. Ensure that project level performance data is systematically collected, analyzed and leveraged for decisions regarding policy and research directions.
  2. Develop and implement information management practices, and tools, to assist in the maintenance of corporate knowledge and lessen the burden on projects.
  3. Increase the dissemination of policy and knowledge resources and make efforts to improve the accessibility of existing policy and knowledge resources.
  4. Consider options to further support project sustainability.

Management Action Plan

Recommendation

1. Ensure that project level performance data is systematically collected, analyzed and leveraged for decisions regarding policy and research decisions.

Action(s) Planned

1.1 Continue to ensure the collection of all Annual Performance Reports (APR) from each recipient and work on new strategies to ensure reports are received within set timelines.
Planned completion date: April 2024
1.2 Continue to collect Activity Reports (AR) from recipients throughout the year to ensure project deliverables are being met. The ARs are available for specific project analysis when required.
Planned completion date: April 2024
1.3 CPB to access and review reporting and performance information to conduct evaluations of overarching intervention models.
Planned completion date: April 2024
1.4 Resources permitting, assign a resource to support business process and reporting template updates to sustain the efforts of the time-limited dedicated resource assigned in 2023 to redesign all EMPB APRs (including those under the NCPS). This effort is aimed to increase standardization and increase GBA Plus disaggregated data collection to better inform program implementation and policy decisions while simplifying reporting for recipients.
Planned completion date: April 2024
1.5 Continue increasing communication and collaboration between Programs, Policy, PDU and Research teams by maintaining the Data Collection and Reporting Working Group created in January 2023 to review and improve data collection, analysis and reporting processes. This working group provides a collaborative forum to better inform policy and research decisions in order to enhance the development of evidence-based crime prevention programs.
Planned completion date: April 2024

Recommendation

2. Ensure that project level performance data is systematically collected, analyzed and leveraged for decisions regarding policy and research decisions.

Action(s) Planned

2.1 PSIMS Issues

Following EMPB’s Program Data Unit (PDU)’s development of a new business intelligence tool (PSIMS search tool) to mitigate gaps in the native PSIMS search tool, address issues of user-friendliness, and facilitate access to administrative data for Program Delivery Staff (PDS); PDU will continue improvements including:

2.1.1 Contribute to the development of IM specific training for Program Deliver Staff (PDS). This would include training on SNAP/Qualtrics.
Planned completion date: October 2023
2.1.2 Implement a streamlined data request process and InfoCentral page for facilitating access to data and PDU’s services.
Planned completion date: March 2024
2.1.3 Contribute to departmental initiative to replace PSIMS.
Planned completion date: October 2023

2.2 SNAP Issues

2.2.1 PDU to replace SNAP with Qualtrics in two areas of data collection, namely application intake and performance reporting (APRs).
Planned completion date: November 2023
2.2.2 Identify digital solutions that are more efficient and transparent for program staff, by leveraging Qualtrics’s system integration capabilities and automation through APIs.
Planned completion date: September 2025
2.2.3 Monitor and incorporate recipient feedback provided through reporting and application templates with respect to the tool itself and continuously improve our tools based on that feedback.
Planned completion date: Ongoing

2.3 Reporting burden and lack of sharing and analysis.

2.3.1 Through PDU’s participation in the Data Collection and Reporting Working Group:
  1. Aim to reduce the length of the APRs and standardize them to ensure it only collects data that will be analyzed and contribute to decision-making;
  2. Explore the feasibility of digitizing the activity report templates as well, to make them easier for recipients to complete;
  3. Support the WG to improve data sharing internally and externally by identifying the audiences and their data interests.

Planned completion date: April 2024

2.3.2 Develop a user friendly and automated Project Development Tracker tool to facilitate the data collection and dissemination of information. The aim is to resolve the current issue of disparate and unstandardized excel based trackers.
Planned completion date: January 2024

Recommendation

3. Increase the dissemination of policy and knowledge resources and make efforts to improve the accessibility of existing policy and knowledge resources.

Action(s) Planned

3.1 As policy documents are produced by PS, ensure they are disseminated and discussed with members of the FPT Working Group on Crime Prevention.
Planned completion date: January 2024
3.2 In the last year and a half, the Research Division has increased the dissemination of knowledge resources on ‘what works’ in Crime Prevention to internal and external stakeholders and this will continue. It has also funded Statistics Canada for key statistics on Cyberbullying and infographics that are shared on its website. Moving forward, connections will be made with Public Safety Communications to explore opportunities for updating resources on the Public Safety Canada website. Sharing of research findings with internal and external partners will continue.
Planned completion date: September 2023
3.3 The Research Division, Crime Prevention Branch, is developing a SharePoint site to foster a community of practice where program funding recipients can access information and materials to support their performance measurement for evaluation purposes.
Planned completion date: May 2024

Recommendation

4. Consider options to further support project sustainability.

Action(s) Planned

4.1 Crime Prevention Policy will consider options to further support project sustainability within the existing policy authority by:
  • Raising awareness with other partners (FPT, OGD) of successful models;
  • Exploring capacity building activities that could be encouraged among funding recipients; and,
  • Exploring opportunities for providing core funding through the NCPS.

Planned completion date: September 2024

4.2 Continue with Emergency Management and Programs Branch, the inclusion and assessment methodology for Sustainability Plans as part of the NCPS application process.
Planned completion date: April 2025
4.3 The Research Division, in consultation with policy and program representatives, will draft a research project plan to examine factors that affect project sustainability. This could become a research project under the Research Division’s annual research plan, resources permitting.
Planned completion date: September 2024
Date modified: