ARCHIVE - 2007-2008 Interim evaluation of the renew national crime prevention strategy

Adobe Acrobat version (PDF 696KB)

Table of contents

• Executive summary
• 1.0 Introduction
  • 1.1 Outline of the report
• 2.0 Overview of the renewed NCPS
  • 2.1 Context and chronological overview
  • 2.2 Objectives
  • 2.3 Components
  • 2.4 Organization
  • 2.5 Expected results and logic model
  • 2.6 Recent developments
• 3.0 Methodology
  • 3.1 Evaluation framework
  • 3.2 Data collection methods
• 4.0 Evaluation findings
  • 4.1 Design and delivery
  • 4.2 Impact
  • 4.3 Effectiveness and efficiency
  • 4.4 Relevance
• 5.0 Conclusions and recommendations
• Appendix A: Evaluation Framework
• Appendix B: Data Collection Instruments

[ * ] - In accordance with the Privacy and Access to Information Acts, some information may have been severed from the original reports.

Executive summary

The National Crime Prevention Centre (NCPC) contributes to the federal government’s role in sustaining community safety. It is responsible for administering the National Crime Prevention Strategy (NCPS), through which the federal government provides grant and contribution funding to community-based crime prevention projects; develops local, provincial/territorial, national, and international partnerships; and develops and transfers knowledge to further the practice of crime prevention.

The NCPS was renewed in 2005/2006. The renewed Strategy consists of three components. Under the community action component, NCPC provides funding and technical support to crime prevention initiatives across Canada through three funding programs: the Crime Prevention Action Fund (CPAF); the Policing, Corrections and Communities Fund (PCCF); and the Research and Knowledge Development Fund (RKDF). In addition, NCPC works to develop multi-sector partnerships to strengthen crime prevention efforts through the partnership component of the Strategy, and under the knowledge component, it gathers, interprets and exchanges knowledge about evidence-based crime prevention.

The Accountability, Risk and Audit Framework (ARAF) for the renewal included an evaluation plan and reporting requirements, including a requirement for an Interim Year Two Progress Evaluation. Public Safety Canada (PSC) engaged PRA Inc. to conduct the interim evaluation.

The evaluation, which examined issues of relevance, design and delivery, impact, and cost-effectiveness/alternatives, was charged with the challenging task of evaluating a moving target. When the evaluation began in February 2007, the NCPS was not the same Strategy as it was in April 2005 when it first received renewal funding. But the challenges facing the evaluation are insignificant in comparison to those that NCPC has confronted. Since April 2005, the organization has experienced a 27% reduction in FTEs, an internal reorganization including elimination and downgrading of many positions, a department-wide “pause” in funding between July 2006 and December 2006 that resulted in $22 million being lapsed and reductions of $3.6 million in 2006-2007 and $10.5 million in 2007-2008 as part of the government-wide fiscal responsibility framework. Most recently, NCPC has been given a new, more focused mandate that, like the events of the last two years, will affect the delivery of the NCPS. This interim evaluation has attempted to take these contextual issues into account in drawing conclusions and providing recommendations.

Methodology

To enhance the reliability and validity of the findings, multiple data sources were used wherever possible to respond to the evaluation questions. Data collection methods included:

• a review of program documents
• a review of Canadian and international literature on crime prevention
• a review of administrative data contained in NCPC’s Grants and Contributions Information Management System (GCIMS)
• a review of program files (n=112)
• a review of performance measurement data
• key informant interviews (n=34)
• a survey of funded organizations (n=164).

The findings, conclusions and recommendations of the evaluation follow below.

Findings, conclusions and recommendations

Design and delivery

Under the community action component of the NCPS, NCPC provides funding and technical support to crime prevention initiatives across Canada. With the renewal of the NCPS, NCPC streamlined its program platform by reducing the number of funding programs and introducing various changes to their design and delivery. These changes included funding a higher proportion of projects through contribution agreements; emphasizing multi-year, strategic, comprehensive crime prevention projects; increasing the targeting of NCPS funding; and emphasizing evidence-based crime prevention projects with strong research and evaluation components.

The evaluation found widespread (though not unanimous) agreement among key informants that these changes represent an appropriate direction for the Strategy and for NCPC. Furthermore, there is some evidence that NCPC has begun to fund projects that reflect these changes. For projects funded since April 2005, the ratio of contribution agreements to grants is 63:37, in comparison to 26:74 in the pre-renewal period, and the average NCPC contribution per CPAF project is almost three times more than it was for projects funded under the Community Mobilization Program (CMP), the predecessor to CPAF.

On the other hand, the available data on project duration are inconclusive. Although CPAF projects were, on average, longer than CMP projects, median project duration in both cases was one year. Similarly, the evaluation found little change in the comprehensiveness or strategic nature of funded projects pre- and post-renewal (using number of other revenue sources as an indicator), and because of a lack of comparable pre-renewal data, it could not assess whether projects funded since the renewal are more targeted in comparison to the pre-renewal period. In future, data on projects funded in the first two years of the renewal should be used as a baseline against which progress in these areas can be assessed.

Recommendation

1. In order to measure progress in implementing the changes to funding program design and delivery that were introduced with the renewal of the NCPS, NCPC should use the data on projects funded in the first two years of the renewal as a baseline for comparison.

NCPC has taken several steps to emphasize evidence-based crime prevention projects with strong research and evaluation components. However, NCPC key informants acknowledged that compiling and communicating easy to use, current information on the state of evidence in a variety of crime prevention areas is an area of weakness for the organization. Indeed, results from the survey of funded organizations indicate that applicants require more assistance from NCPC staff in integrating evidence-based crime prevention strategies into their proposals. In this context, it is important to note that neither the CPAF Overview nor the Applicants’ Guide mentions the need to consult the evidence base when it describes the steps involved in developing a proposal, and the CPAF Application Form does not specifically require applicants to describe the evidence base of their proposed interventions. Therefore, some applicants may not be aware that this is an expectation for all proposals. As a further consequence, the extent to which funded interventions are based on available evidence on effective crime prevention cannot be systematically ascertained.

Moreover, it is clear that there are differing interpretations among NCPC personnel of the term “evidence-based crime prevention projects,” with some interpreting the term narrowly to mean projects that have been tested against very rigorous evaluation standards, and others interpreting it more loosely to mean that the available evidence was consulted when designing the proposed intervention. These different perspectives within NCPC itself suggest a need to clarify NCPC’s understanding of the term.

Recommendation

2. NCPC should clearly define what it means by “evidence-based,” in order to ensure a common understanding of the term among NCPC staff.

Recommendation

3. To fulfill its commitment to evidence-based practice, NCPC should, first, inform funding applicants of the requirement to integrate the evidence base into their proposed interventions; and second, develop a means of tracking the extent to which successful applicants integrate evidence into their projects.

The evaluation found that the technical assistance provided by NCPC staff is essential to the development of strong funding proposals. Moreover, the need for technical assistance has grown considerably as a result of the renewed Strategy’s more stringent demands for project planning and accountability, while cuts to NCPC staffing have effectively limited the amount of technical assistance that can be provided. While projects surveyed found the technical assistance they received to be useful, the evaluation found clear evidence that increased efforts to assist applicants may be required in some areas, particularly in identifying risk and protective factors, evaluation planning, and integrating evidence-based crime prevention strategies. Furthermore, many key informants expressed concern that the current proposal requirements have placed NCPS funding beyond the reach of many high-needs and high-risk communities (particularly Aboriginal and/or Northern communities), where the capacity to prepare sophisticated proposals and administer complex, comprehensive prevention projects remains a major challenge.

Recommendation

4. NCPC should ensure that sufficient resources (i.e. , staff, tools, and resources) are available to ensure that funding applicants receive adequate technical assistance during proposal development, particularly in identifying risk and protective factors, evaluation planning, and integrating evidence-based crime prevention strategies.

Recommendation

5. Given that crime and victimization are pressing issues for many Aboriginal and Northern communities, and given that one of NCPC’s priorities under its new mandate is to foster prevention specifically in Aboriginal communities, NCPC should consider either simplifying proposal requirements or modifying the proposal development process for applicants that meet the program’s funding criteria but have language or other barriers that may prevent them from putting forth successful proposals. In addition, NCPC should consider providing additional assistance with certain aspects of project management, such as financial reporting, to successful applicants that lack the capacity to manage projects effectively on their own.

Under NCPC’s repositioning, the process by which it solicits applications for funding will shift from generic calls for proposals (which have for the most part continued, to date, under the renewed Strategy despite a commitment to increased targeting and the application of this principle at the proposal review stage) to targeted solicitations that explicitly identify the priorities, types of interventions, and particular communities that will be funded. By reducing the volume of proposals received, targeted solicitations are widely expected to improve efficiencies and ultimately produce more effective projects that better contribute to achieving objectives.

The major weaknesses in NCPC’s proposal process are, at present, lengthy proposal review and approval times. For projects funded since the renewal, average processing time is 199 days (just over half a year). While the “pause” is partly responsible for these delays, a major contributing factor is the complex process of review and approval that has emerged since the renewal. Whereas in the past, the delegation of authority to approve projects for funding was dependent on the funding amount (with authority to approve some funding residing at the NCPC and Branch level), PSC now requires all projects recommended for funding by NCPC and the regional Joint Management Committees (JMCs) to be scrutinized and approved by the Minister to ensure compliance with departmental priorities. The approval process now consists of three stages – program review, JMC or external review, and Ministerial review – and at least 15 distinct steps. In addition to creating lengthy delays, this process has jeopardized the viability of some projects and created resentment among NCPC’s provincial/territorial partners who believe that their priorities, as reflected in the decisions of the JMCs, are not being respected. In order to improve efficiencies and repair goodwill, therefore, efforts should be made to streamline the proposal review and approval process and to clearly articulate the roles and responsibilities of all parties involved.

Recommendation

6. In the interest of improving efficiencies, NCPC, together with PSC representatives, should investigate ways of rationalizing the proposal review and approval process, including the possibility of re-delegating the authority to approve project funding to the Branch and Directorate levels.

Recommendation

7. The roles and responsibilities of all parties in the proposal review and approval process and, in particular, the roles and responsibilities of the JMCs should be clearly articulated.

Recommendation

8. While the lengthy processing times experienced since the renewal of the NCPS may be an aberration resulting from unusual circumstances, NCPC and/or PSC should consider implementing and adhering to a standard processing time for funding proposals. The same recommendation was made by the Blue Ribbon Panel on Grant and Contribution Programs for federal departments in general. A reasonable target could be identified by examining processing times of similar funding programs within the federal government, such as Status of Women Canada’s Women’s Program, which also takes a social development approach.

The partnership component of the NCPS is based on the recognition that multi-sector partnerships and coalitions of governmental and non-governmental stakeholders will strengthen crime prevention efforts. NCPC has had a mixed record in implementing this component of the Strategy. Its unstable situation since renewal has had an impact on its partnership activity, and its relationship with the provinces and territories, in particular, has suffered. Among NCPC’s provincial/territorial partners, there is a strong belief that the organization has failed to consult them regarding the renewal and has failed to keep them fully abreast of developments over the past two years (although we do report evidence from minutes that show that NCPC provided information on the renewal at federal/provincial/territorial meetings). In addition, the provinces and territories object to what they perceive as the federal government’s unilateral decision-making since the renewal. Many believe that their priorities are not being respected in funding decisions, that the federal government has set priorities for the NCPS that they do not share (e.g. , youth gangs, guns, and drugs are not pressing issues in all jurisdictions) and that the federal government has begun to move away from crime prevention through social development (CPSD) in favour of other approaches. While these tensions are beginning to ease, many provincial/territorial key informants emphasized that the NCPS is a bilateral strategy and that there is a need for genuine collaboration between the federal and provincial/territorial governments in its delivery.

Recommendation

9. NCPC should continue to work at repairing its relationship with the provinces and territories. Efforts should be made to ensure a common vision of the NCPS, as well as a common understanding of the respective roles and responsibilities of the partners.

In contrast to the strained relationship with the provinces and territories, NCPC has had some success at horizontal policy work within the federal government. NCPC works closely with other units within PSC on joint research and policy planning, and has signed important new agreements with Justice Canada, Statistics Canada, and Heritage Canada, among other departments. Many external partners envision NCPC as the lead organization responsible for coordinating the federal role in crime prevention. Such a role, furthermore, is strongly supported by evidence from the literature on effective crime prevention strategies, which points to the need for integrated action by multiple stakeholders to solve social problems.

Recommendation

10. To enhance the federal government’s response to crime and victimization, NCPC should work to establish itself as the lead agency responsible for coordinating the federal role in crime prevention.

In addition, NCPC’s long-standing relationships with external partners, such as the Royal Canadian Mounted Police (RCMP) and the Canadian Association of Chiefs of Police (CACP), have been strengthened, and key informants reported that strong partnerships have developed in some regions. All external partners believe that their organization has goals in common with the NCPS (although some are worried that CPSD is no longer a focus of the Strategy) and that their partnership with NCPC influences their policy development in positive ways. Some, such as the RCMP and the CACP, have incorporated CPSD into their official policy.

Finally, under the partnership component of the NCPS, NCPC is expected to facilitate the formation of partnerships among other organizations. NCPC personnel observed that this has occurred simply as a function of the requirement that all funding applicants identify partners and other funding sources in their proposals. Both GCIMS and file review data confirm that projects funded since the renewal involve multiple partners from a wide range of organizational sectors – nine partners each, on average, for CPAF and PCCF projects. Project partners played diverse roles and came from a diverse range of organizational sectors, including community, social and voluntary services; criminal justice, corrections and police; education; health; and provincial/territorial governments. Based on the available data, however, this does not appear to represent any change over the pre-renewal period, which is perhaps not unexpected given NCPC’s historic emphasis on partnerships.

Under the knowledge component of the NCPS, NCPC committed to improve efforts to synthesize best practices and lessons learned and to disseminate this information to the crime prevention community and the general public. Some of this work is occurring through projects funded through the RKDF. To date, 12 projects have been funded under this component, although only one (the planning phase of a demonstration project) has yet been completed. NCPC also supports this component of the Strategy by publishing on its website and in hard copy materials specifically devoted to disseminating knowledge on evidence-based practices, providing technical assistance to projects, developing guidance materials for funding applicants and funded projects, and participating in a wide variety of workshops, conferences, training sessions, presentations, and other activities related to the production and dissemination of knowledge. A knowledge function composed of five FTEs within the Research and Knowledge Unit has been created to implement and coordinate knowledge activities.

Nevertheless, the evidence from this evaluation suggests that the knowledge component is an area of weakness for NCPC. Among key informants, only a few of NCPC’s external and provincial/territorial partners had ever used or heard of NCPC knowledge products, and many of those who had used them found them impractical, too limited or superficial to be of real value, or culturally or linguistically inappropriate in their jurisdictions. On the other hand, many of NCPC’s partners noted that the NCPC’s research and knowledge capacity is its greatest strength, and suggested that it focus on becoming a centre of excellence for evidence-based crime prevention. NCPC key informants view the synthesis and dissemination of evidence-based knowledge as a major challenge, particularly given current staffing levels, and suggested that part of the solution lies in improved linkages between the organization’s Programs and Policy, Research and Evaluation divisions.

Recommendation

11. NCPC should continue to develop its knowledge component, with a view to becoming a recognized source for evidence-based crime prevention information in Canada.

The resource reductions that NCPC has experienced over the past two years have profoundly hampered its ability to implement and achieve the objectives of the renewed NCPS. The pause severely curtailed the number of projects that were funded and delayed others; in total, some 150 projects were affected. Because NCPC did not communicate the changing program parameters to communities in a timely fashion, both NCPC and the provinces and territories, as delivery agents of the Strategy, lost credibility with community organizations. At the same time, commitments to increased use of contribution agreements, increased targeting of funding, integration of evidence-based interventions, and an enhanced knowledge function were made even though staff capacity to implement these changes was reduced. As a consequence, NCPC has suffered from high turnover in staff and senior leadership since the renewal, further diminishing its ability to deliver on the mandate of the renewed NCPS.

Many key informants observed that these developments forced NCPC to rethink its mandate and to emerge better positioned to achieve the objectives of the renewed NCPS. Nevertheless, there remain ongoing issues with respect to organizational structure, delivery structure, and resources. First, the current organizational structure was created in response to external imperatives, without any strategic analysis of the structure and resources necessary to deliver on the renewed NCPS mandate. There is evidence, for example, that staffing at the regional level may be inadequate, particularly given increased demands for project planning and accountability. There is also evidence that, although the relationship between the Programs and Policy, Research and Evaluation Divisions has greatly improved in recent months as a result of NCPC’s new senior leadership, the organization still lacks a mechanism for integrating policy, research, and evaluation with program delivery, especially at the regional level. As a result, the regions, which are responsible for recommending projects for funding, have no mechanism for systematically accessing information on best practices and evidence-based crime prevention. An organizational analysis examining such questions as staffing levels and allocation among the national and regional offices, the role of different NCPC units at each stage of the project cycle from solicitation to evaluation, and the structures that need to be in place to enable NCPC to fulfill the two core activities of its mandate since the repositioning (supporting targeted interventions and building and sharing practical knowledge), would help to identify any necessary modifications to the current structure.

Recommendation

12. NCPC should undertake a strategic analysis of its organizational structure in order to identify any modifications that may be necessary to enable it to deliver on the mandate of the renewed NCPS. The analysis should examine questions such as staffing levels and allocation of staff among the national and regional offices; the role of the various NCPC units at each stage of the project cycle from solicitation to evaluation; and the structures that need to be in place to enable the organization to fulfill its mandate.

Second, while tensions between the provinces and territories are beginning to ease, the strained relationship has by no means been completely repaired. In addition to ongoing uncertainty over federal and provincial/territorial roles and responsibilities in the NCPS, there is also uncertainty about the roles and responsibilities of the Federal/Provincial/Territorial (FPT) Working Group. Indeed, it is not clear if the Working Group has a policy or an operational focus, and its membership is quite diverse in rank and background. The resulting mixture has not produced a particularly cohesive or effective group. Key informants agreed that there is a need to refine and refocus the mandate of the Working Group to ensure that it can be an effective agent in delivering the renewed NCPS.

Recommendation

13. Together with its provincial/territorial partners, NCPC should refine and focus the mandate of the FPT Working Group and clearly articulate its roles and responsibilities with respect to the renewed NCPS. The membership of the Working Group should be composed of individuals of similar background and rank within their organizations. Ideally, members should have decision-making authority within their own organizations.

Under the renewed NCPS, NCPC committed to implementing an enhanced performance measurement system and has taken several steps toward implementing this system, including developing standard application forms and reporting templates for each funding program, requiring all funded projects to provide final reports and conduct evaluations, providing evaluation training and developing evaluation resource materials for NCPC staff and funding recipients, enhancing the electronic GCIMS system, and developing performance measurement templates to track activity under the partnership and knowledge components of the NCPS.

At present, the main challenge for performance measurement is related to GCIMS, which is intended to capture information collected by the standard forms used by the funding programs. While development and data entry have been underway since 2004/2005, the database is not yet capable of generating reports or data to support timely project management, performance monitoring, or impact assessment. NCPC has developed an Action Plan to address the main obstacles to full implementation of GCIMS, namely inconsistent data entry and incomplete development of the database. However, from an evaluation perspective, there are two additional issues with respect to GCIMS. First, there is evidence of poorly defined fields and attributes within the database, leaving substantial opportunity for subjective judgment in data entry that will ultimately affect the quality and validity of the data (although it is true that these issues are currently being addressed by NCPC). Second, although GCIMS is an information management system, it is nevertheless capturing primarily qualitative data; the task at hand is to design a database capable of ensuring that these qualitative data can be quantified, and that they will be meaningful once quantified. At the same time, NCPC should not rely solely on a database solution for performance measurement and evaluation purposes and overlook the value of other qualitative approaches, such as in-depth case studies, especially for illustrating program impact. Based on the evaluator’s limited exposure to GCIMS, further work on GCIMS would benefit from the expertise of specialists in database development, performance measurement, and evaluation.

Recommendation

14. In order to ensure that GCIMS will ultimately be capable of generating useful, valid, and reliable data, NCPC should consider involving additional expertise in database development, performance measurement, and evaluation to assist with its development and implementation.

Recommendation

15. Rather than relying solely on a database solution that focuses on capturing and summarizing the information collected on the standard forms, NCPC should consider a diversified performance measurement strategy that also includes objective qualitative methods, such as in-depth case studies, to illuminate outcomes achieved.

Impact

Given the relative infancy of the renewed NCPS, impact is difficult to discern at this time. Only about 20% of projects funded since April 2005 have concluded and GCIMS does not yet capture information on project outcomes. Consequently, data on impact is very limited.

Nevertheless, the evaluation found some preliminary indications of positive impacts, particularly in the area of increased local capacity to prevent crime and victimization. About 70% of funded organizations, for example, agreed that NCPC funding has improved their organization’s and their community’s capacity to prevent crime and victimization. Among key informants, many agreed that the objective of increased local capacity has been accomplished or that substantial progress has been made in this direction, particularly since increased community capacity has been a focus of the NCPS for some years, not only since renewal. However, while some key informants believe that the time has come for the NCPS to focus on more concrete outcomes, others believe that increasing local capacity should continue to be a goal of the Strategy, given that many communities (particularly Aboriginal communities) continue to struggle to plan and implement crime prevention initiatives.

There is also some evidence of improved comprehensive partnerships under the renewed NCPS. GCIMS and file review data show that funded projects involve multiple partners (nine each, on average) from a wide range of organizational sectors including, most commonly, community, social and voluntary services; criminal justice, corrections and police; education; health; and provincial/territorial governments. Moreover, 78% of funded organizations agreed that their organization has developed partnerships with organizations whose involvement was necessary to achieving their project objectives as a result of NCPC funding or technical assistance, and 59% agreed that NCPC funding helped their organization to leverage other sources of funding.

As for NCPC’s own partnership activity, positive developments include numerous new horizontal partnerships with other federal departments and a strengthening of existing partnerships with the RCMP and the CACP, both of which have incorporated CPSD into their official policy. However, in the evaluator’s opinion, the current expected outcomes pertaining to the partnership component of the renewed NCPS are more accurately described as “outputs,” since they seem to imply that simply establishing partnerships is, by itself, sufficient to achieve crime prevention objectives. That being said, it should be acknowledged that the revised logic model for the NCPS, currently being drafted, appears to address this issue.

Recommendation

16. In revising the logic model for the NCPS, NCPC should take care to ensure that all expected outcomes articulate what is expected to change as a result of NCPS activities and outputs and do not merely reflect those activities or outputs.

Finally, the evaluation found very preliminary evidence of progress in integrating evidence-based crime prevention into policies and practices. Key informants believe that there is greater awareness among community organizations of evidence-based crime prevention, and indeed 62% of funded organizations agreed that their organization has a better understanding of evidence-based crime prevention strategies as a result of NCPC funding and technical assistance. As mentioned earlier, this component of the renewed NCPS is an acknowledged area of weakness, and one that will require greater effort in the future. Projects funded through the RKDF should help to improve knowledge production and dissemination.

Effectiveness/alternatives

Like program impact, it is somewhat early to assess whether there are more effective ways of achieving the objectives of the renewed NCPS. That being said, the evaluation found considerable support in the literature for various features of the NCPS, including involvement of national government in crime prevention; a whole government approach to crime prevention that engages multiple stakeholders to deliver integrated solutions to problems; and evidence-based crime prevention. Among key informants, there is also general agreement that these are desirable features of an effective crime prevention strategy. That being said, some key informants expressed caveats. Provincial/territorial representatives, for example, cautioned that a strong federal role should not mean that provincial/territorial interests and priorities are subsumed or ignored. Other key informants argued that emphasis on evidence-based interventions should not mean forgoing innovative but untested initiatives, particularly where there is a dearth of evidence on “what works” (as is the case for Aboriginal populations who are judged to be particularly high risk and therefore most in need of intervention). Indeed, funding and evaluating previously untested crime prevention interventions is arguably one important way in which NCPC might fulfill its commitment to producing evidence-based knowledge.

Recommendation

17. While there is considerable support in the international literature for implementing evidence-based approaches to crime prevention, NCPC should ensure that innovative interventions continue to be funded, particularly for communities or populations where evidence may be lacking. Furthermore, such interventions should be evaluated in order to contribute to the evidence base.

The literature review also highlighted the importance of effective implementation to successful crime prevention initiatives. As has been emphasized throughout this report, effective implementation of the renewed Strategy has been hampered by significant reductions to NCPC resources over the past two years. Given the context, it should not come as a surprise that evidence of impact is limited at this stage. However, based on the literature it appears that the basic elements are in place for success in the future, provided that the outstanding implementation issues identified in this evaluation are addressed. Among NCPC key informants there is considerable agreement that if NCPC implements its new mandate, focusing on a more limited number of targeted interventions, both the effectiveness and the efficiency of the renewed NCPS will improve dramatically.

Relevance

The evaluation found the renewed NCPS to be closely aligned with the strategic objectives of PSC as well as federal priorities. Among key informants as well as funded organizations, there is a broad consensus that the federal government should continue to play a role in crime prevention and that the NCPS is still relevant to crime prevention in Canada. In fact, many stakeholders believe that if the NCPS did not exist, the consequences for crime prevention would be devastating. Two-thirds of survey respondents said that in the absence of the NCPS, their own crime prevention work would be severely limited or altogether curtailed, and about one-fifth said the outcomes they have achieved would not have been realized. A similar proportion predicted an increase in crime and victimization in the absence of the Strategy.

Despite general agreement on the relevance of the NCPS, some key informants are unconvinced that the changes that have been introduced to ensure alignment with departmental and federal priorities are appropriate. In particular, they object to what they perceive to be a reduced emphasis on community capacity building, a focus on selected issues that are not relevant in all communities or jurisdictions, and a perceived move away from CPSD. These key informants believe that to be truly relevant, as well as effective in the long term, the NCPS must permit communities and jurisdictions to focus on local and provincial/territorial issues and priorities.

Recommendation

18. NCPC should clarify for all stakeholders both whether and the extent to which community capacity building, locally identified issues, and CPSD continue to be elements of the renewed NCPS.

1.0 Introduction

The National Crime Prevention Centre (NCPC) contributes to the federal government’s role in sustaining community safety. It is responsible for administering the National Crime Prevention Strategy (NCPS), through which the federal government provides grant and contribution funding to community-based crime prevention projects; develops local, provincial/territorial, national, and international partnerships; and develops and transfers knowledge to further the practice of crime prevention.

The NCPS was renewed in 2005/2006. The Accountability, Risk and Audit Framework (ARAF) for the renewal included an evaluation plan and reporting requirements, including a requirement for an Interim Year Two Progress Evaluation. Public Safety Canada (PSC) engaged PRA Inc. to conduct the interim evaluation. The evaluation examined issues of relevance, design and delivery, impact, and cost-effectiveness/alternatives.

1.1 Outline of the report

This report is divided into several sections. Section 2 provides an overview of the renewed NCPS while Section 3 describes the methodology used to complete the evaluation. Section 4 provides the evaluation findings, and Section 5 concludes and provides recommendations.

Two appendices follow the main body of the report. Appendix A contains the evaluation framework and Appendix B the data collection instruments. In addition, we have provided detailed technical results from the data collection methods used in this evaluation in a separate volume of appendices (Volume II).

2.0 Overview of the renewed NCPS

This section of the report provides a description of the context for the renewed NCPS, and describes the Strategy’s objectives, components, and expected results.

2.1 Context and chronological overview

The National Crime Prevention Strategy has evolved substantially since its inception in 1994. In the Safe Home/Safe Streets policy set out in Towards Safer Communities,1 the federal government made a commitment to a comprehensive approach to crime that involved five related initiatives, one of which was the National Strategy for Community Safety and Crime Prevention. At the heart of the Government’s approach at that time was the belief that traditional measures to deal with crime – law enforcement, the courts, and incarceration – must be balanced with strong crime prevention efforts which recognize that the communities that are most resistant to crime are those that actively participate in addressing the social and economic factors that can lead to crime.

The National Strategy provided a framework for federal efforts in this area, facilitated cooperation with provincial and territorial governments, and emphasized community-level action to prevent crime. The most visible part of Phase I of the Strategy was the creation of the National Crime Prevention Council. The Council, which consisted of volunteers and the support of a professional Secretariat, was charged with focusing on crime prevention across Canada and addressing crime in a comprehensive manner by involving a broad range of partners and sectors.

Phase II of the National Strategy, launched in 1998, consisted of the Safer Communities Initiative (consisting of five funding programs), a Promotion and Public Education Program, and the National Crime Prevention Centre (NCPC), which was located within the Department of Justice and which was responsible for implementing the Strategy in partnership with the Solicitor General of Canada. 2 Like its predecessor, Phase II was based upon a crime prevention through social development (CPSD) approach. Through its funding programs, the National Strategy provided grant and contribution funding to projects at the community level, where resources, needs, and problems are best understood and can be most effectively addressed. Phase II had an initial investment of $32 million per year. In 2001, Phase II was expanded with an additional investment of $145 million over four years (2001–2005). During the expansion period, the Strategy was renamed the National Crime Prevention Strategy (NCPS), and responsibility for it was transferred from the Department of Justice to the Department of Public Safety and Emergency Preparedness Canada (PSEPC, now Public Safety Canada or PSC).

[   *   ]

2.2 Objectives

According to the ARAF, the objectives of the renewed NCPS are:

• To increase sustainable community action
• To develop and share knowledge of crime prevention strategies
• To coordinate multilevel support for crime prevention efforts.

In addition, each of the funding programs under the renewed NCPS has its own set of objectives, which are included in the descriptions of the individual programs below.

2.3 Components

The renewed NCPS consists of three components: community action, partnerships, and knowledge.

2.3.1 Community action

Under the community action component, NCPC provides funding and technical support to crime prevention initiatives across Canada. NCPC launched its new funding program platform, consisting of the CPAF, the PCCF, and the RKDF, in autumn 2005. Despite the recent changes to its funding programs described above, these three original programs are the subject of the current interim evaluation and are described below.

Crime Prevention Action Fund

The CPAF provides time-limited grant and contribution funding that supports crime prevention initiatives at the community level. 3 The program is delivered regionally in partnership with the provincial and territorial governments, and nationally for projects that involve more than one jurisdiction. The objectives of the CPAF are to:

• Enhance the capacity of diverse communities to decrease risk factors and increase protective factors related to crime and victimization
• Promote cross-sector collaboration in crime prevention, with particular emphasis on engaging the voluntary and private sectors and participating in multi-partnership initiatives
• Develop, disseminate and transfer information, tools and resources, including best practices and evidence of what works in crime prevention
• Support the development of comprehensive community initiatives to address crime and victimization issues.

Eligible organizations include not-for-profit volunteer organizations, groups, and professional organizations and associations; community-based coalitions and networks; Aboriginal communities including Band and Tribal Councils; provincial, municipal, and Aboriginal police services; and provincial, municipal, and regional governments. The maximum available funding per recipient for an individual project is $100,000 per year for up to three consecutive years for grants and $200,000 per year for up to five consecutive years for contributions. CPAF does not fund 100% of project costs.

NCPC has developed a detailed Applicant’s Guide that covers topics such as preparing a project budget, project planning and evaluation, sustainability for crime prevention, partnerships, and risk and protective factors. The Guide is intended to assist applicants to CPAF as well as applicants to PCCF in preparing a strong proposal for funding.

Policing, Corrections and Communities Fund

The PCCF funds projects where community partners work together to prevent crime, primarily through social development. 4 The goals of the PCCF are to:

• Develop partnerships, tools and knowledge of CPSD within police and other law enforcement organizations, community corrections services, community-based organizations, and Aboriginal communities
• Develop and implement demonstration projects targeting high-risk communities and at-risk groups, including children, youth and their families, and Aboriginal people
• Support families of Aboriginal offenders to prevent the cycle of institutionalization of Aboriginal youth and to promote Aboriginal community mobilization and wellness
• Develop, implement and evaluate comprehensive models of crime prevention in the community that include efforts to support the reintegration of offenders in order to prevent further criminal behaviour and victimization
• Strengthen partnerships between community corrections and other crime prevention initiatives within communities.

Eligible organizations are similar to those eligible under CPAF. The maximum available funding under PCCF is $200,000 per year per project for up to three years. According to the PCCF Overview, the PCCF does not normally fund 100% of project costs. As previously mentioned, when the NCPC experienced budget cuts in 2006, the PCCF was eliminated as a separate funding program. No new projects have been funded through this program since October 2006.

Research and Knowledge Development Fund

The RKDF is delivered through the national NCPC office. 5 Broadly speaking, the RKDF is intended to enhance public awareness, understanding, and practice of crime prevention, and to strengthen linkages among NCPC, academics, academic institutions, and crime prevention researchers and practitioners. More specifically, the objectives of the RKDF are to:

• Identify, develop, support and evaluate promising community-based crime prevention models of potential Canada-wide significance
• Support research, including community-based action research, and other activities related to the compilation of evidence on crime prevention and victimization, with particular emphasis on social development approaches
• Promote and facilitate linkages with crime prevention researchers, practitioners, and communities
• Support and promote the advancement of effective approaches to crime prevention at the international level.

The RKDF supports three types of projects: research projects, demonstration projects with associated evaluations, and knowledge transfer projects. Research projects help advance basic crime prevention knowledge, policy and/or practice; demonstration projects test the theory underlying a program and determine its effectiveness; and knowledge transfer projects ensure the widespread distribution of practical crime prevention information and knowledge, generated by research and demonstration projects, in usable formats.

Eligible organizations include some of the same types of organizations as those eligible under CPAF, but also Canadian educational institutions and boards of education; networks of academics; and academic institutions and centres of excellence, among others. The RKDF provides both grant and contribution funding (although no grants have been awarded to date) and, like CPAF, does not fund 100% of project costs. The maximum grant per recipient per project is $100,000 per year, while the maximum contribution per recipient per project is $500,000 per year.

Technical support

NCPC personnel provide technical support to applicants and funded projects under all three funding programs. The types of technical support provided include:

• provision of general information about NCPC or NCPC funding programs
• assistance with proposal development, such as developing a project budget, logic model, evaluation plan, or work plan; referrals to other potential funding sources; and assistance in finding project partners and other sources of funding
• assistance to recommended projects, including providing feedback on proposals or explaining the results of the proposal review process
• assistance to funded projects in areas such as cash flow review, review of project reports, provision of project management or delivery advice, and conflict resolution
• assistance with networking, such as connecting organizations with similar groups and providing ideas or assistance to promote project activities or results across communities.

Through the provision of technical support, NCPC hopes to improve local community capacity to prevent crime and victimization and help projects move toward an evidence-based approach.

2.3.2 Partnerships

The partnership component is based on recognition that multi-sector partnerships and coalitions of governmental and non-governmental stakeholders will strengthen crime prevention efforts. According to the ARAF, the NCPS “provides a framework for increased cooperation on crime prevention via partnership arrangements signed with each province and territory” and works with key partners such as the Royal Canadian Mounted Police (RCMP), the Canadian Association of Chiefs of Police (CACP), and other federal departments such as Justice and Health, among others, to “promote shared, collaborative undertakings.” As a result of such partnerships, crime prevention initiatives are expected to have greater impact and sustainability over time. In addition, partnerships are expected to help embed CPSD approaches in the policies and practices of organizations that have an impact on community safety.

2.3.3 Knowledge

Under the knowledge component, the NCPS is responsible for gathering, interpreting, and exchanging knowledge about best practices in crime prevention and evidence-based crime prevention. Knowledge transfer activities are expected to increase community capacity to address local crime and victimization issues, to inform policy and programming decisions at all levels of government, and to increase knowledge on integrating CPSD approaches into policies and practices.

2.4 Organization

The following bodies have responsibilities related to the management and delivery of the NCPC elements of the renewed NCPS. 6

National NCPC office – The National NCPC office consists of the Executive Director’s Office, the Programs Division, and the Policy, Research and Evaluation Division. The National office is responsible for the overall program and policy direction of the NCPC elements of the renewed NCPS, and delivers the PCCF and the RKDF as well as CPAF projects that involve more than one jurisdiction. The National office also provides functional guidance to the NCPC regional offices and is responsible for performance monitoring, measurement, and evaluation of the renewed Strategy.

Regional NCPC offices – The regional NCPC offices are the regional arms of the NCPS. They deliver the CPAF in collaboration with the provinces and territories (e.g. , soliciting, developing, and recommending projects for funding), and also play a role in managing RKDF and YGPF projects, developing and maintaining relationships with regional stakeholders, liaising with other federal government representatives, and providing regional input into crime prevention policy. There are six regional offices: Northern, British Columbia, Prairies, Ontario, Quebec, and Atlantic, with staff located in 16 cities. The six Regional Managers report to the Director of Programs.

Joint Management Committees – Since its inception, the NCPS has been administered by the NCPC and co-managed with the provinces and territories through Joint Management Committees (JMCs). JMCs are comprised of NCPS officials, provincial/territorial representatives and community representatives, and are responsible for the strategic implementation of the NCPS in their respective jurisdictions. As such, one of their main responsibilities is to review submitted proposals and select projects for funding that meet identified provincial/territorial needs within the parameters of the NCPS.

Federal/Provincial/Territorial (FPT) Working Group on Community Safety and Crime Prevention – The FPT Working Group was established as a forum for collaboration and coordination of community safety and crime prevention activities overseen by the NCPS. The FPT Working Group currently consists of one representative from each of the provinces and territories, one representative from the RCMP, and three representatives from NCPC: the Director of Programs, the Director of Policy, Research and Evaluation, and a Policy Advisor from the Policy, Research and Evaluation Division.

2.5 Expected results and logic model

The ARAF for the renewed NCPS identifies three immediate outcomes, three intermediate outcomes, and one ultimate outcome that are all expected to result from the Strategy. The immediate outcomes are:

• Improved leadership/facilitator/educator role in crime prevention
• Increased comprehensive and strategic partnerships
• Improved knowledge of effective crime prevention approaches

The intermediate outcomes are:

• Increased local capacity to prevent crime and victimization
• Improved comprehensive partnerships in effective crime prevention
• Improved integration of evidence-based crime prevention into policies and practices.

Ultimately, the renewed Strategy is expected to lead to enhanced community safety. The logic model on the next page is reproduced from the ARAF and depicts the activities, outputs, and outcomes associated with the Strategy.

2.6 Recent developments

NCPC continues to take action on the commitments it made to Treasury Board in its September 2005 Management Action Plan and March 2006 Management Plan. Beginning in the fall of 2006, the organization began a “repositioning” exercise that will have implications for the delivery for the NCPS. The repositioning is being driven by several factors, including an increased need to focus and target efforts; the need to better collaborate with and better integrate crime prevention with enforcement, corrections, and other relevant interventions; the need to ensure a balanced and targeted approach in response to priority and specific crime issues affecting particular groups and places (where a balanced approach includes CPSD as well as situational crime prevention and secondary and tertiary prevention); and the requirement to obtain measurable results by affecting the risk factors specific to priority crime and victimization issues. 7

In response to these drivers, NCPC recently proposed to adopt a renewed mission and core activities that provide a sharper focus on crime prevention and reduction. The Minister of Public Safety announced the new, more focused approach for NCPC on June 8, 2007. 9 To accomplish this mission, NCPC will concentrate on two core activities: supporting targeted interventions, and building and sharing practical knowledge. All NCPC activity will be based on principles that, in order to be effective, crime prevention initiatives must be:

• Integrated with the activities of other programs and services
• Based on knowledge of known risk and protective factors, and evidence-based practices
• Focused on specific priorities. NCPC’s new priorities are to address early risk factors among vulnerable families and children and youth at risk; to respond to priority crime issues (youth gangs, drug-related crime); to prevent recidivism among high-risk groups; and to foster prevention in Aboriginal communities.
• Able to provide measurable results.

The NCPS logic model is currently being revised in order to align it with the organization’s new directions.

3.0 Methodology

This section of the report provides a detailed description of the methodology used to complete the evaluation.

3.1 Evaluation framework

PRA, in consultation with an Evaluation Advisory Committee composed of three NCPC representatives and one PSC representative, developed an evaluation framework to guide the research. The framework (Appendix A) covers the standard issues of relevance, design and delivery, impact, and cost-effectiveness/alternatives required by Treasury Board for formative or interim evaluations. However, it replaces some of the standard questions with others better suited to the particular context in which this evaluation is taking place. As described earlier, since the renewal of the NCPS in 2005, NCPC has experienced an internal reorganization, cuts to funding, and a departmental-wide “pause” in funding. In addition, NCPC has recently been given a new, more focused mandate as a result of its repositioning exercise. One of the key issues for this evaluation, therefore, is to determine the impact of these recent events and changes on the implementation of the renewed NCPS and on NCPC’s ability to meet the objectives of the renewed Strategy. The evaluation framework includes questions and indicators intended to capture this information.

3.2 Data collection methods

To enhance the reliability and validity of the findings, multiple data sources were used wherever possible to respond to the evaluation questions. Data collection methods included:

1. document review
2. literature review
3. GCIMS data review
4. file review
5. review of performance measurement data
6. key informant interviews
7. survey of funding recipients.

Each of these methods is described below. Data collection instruments are in Appendix B.

3.2.1 Document review

The document review focused on materials produced as a result of the development, implementation, and management of the renewed NCPS. The document review assisted in developing the program profile and helped to answer many of the evaluation questions, as outlined in the evaluation framework. Materials reviewed as part of this task included:

1. policy documents – for context, historical perspective, and current policy directions
2. departmental documents, such as reports on plans and priorities
3. NCPC submissions to Treasury Board
4. NCPC responses to Treasury Board directives, particularly the March 2006 Management Plan and September 2005 Management Action Plan
5. documentation describing organizational and delivery structure
6. the ARAF for the renewal of the NCPS
7. the ARAF for the Youth Gang Prevention Fund
8. funding program guides and guidelines, application forms, and reporting templates
9. minutes and reports from meetings of the CACP, the FPT Working Group on Community Safety and Crime Prevention, and regional JMCs for the NCPS
10. documentation pertaining to NCPC’s GCIMS system (e.g. , user manual and implementation work plans)
11. previously completed evaluation reports.

3.2.2 Literature review

PRA examined the national and international literature on crime prevention. An important component of the literature review was to examine how other countries (namely, the U.K. , Australia, and the United States) have organized their crime prevention activities. This added to our knowledge of the role that national governments can play in crime prevention, thus helping to determine whether the NCPS is relevant. In addition, the literature review examined the available evidence on best practices in crime prevention. This helped us to determine if the new approach is a valid one and if there are more effective ways of achieving the objectives of the renewed NCPS. The literature review is in Volume II.

3.2.3 GCIMS data analysis

We analyzed data contained in NCPC’s Grants and Contributions Information Management System (GCIMS) for projects funded under the new funding programs between April 1, 2005 and February 26, 2007 and, for comparison purposes, projects funded under the Community Mobilization Program between April 2005 and October 2005.

An electronic data extract was provided to PRA by NCPC in Excel format and was imported into SPSS for purposes of analysis. Because GCIMS is still in development, the analysis was limited to variables for which data entry has been reasonably consistent across the country, and for which GCIMS could produce valid and reliable data. Detailed results from the analysis of GCIMS data are in Volume II.

3.2.4 File review

In light of the fact that GCIMS is still in development, we supplemented the analysis of GCIMS data with a detailed review of a sample of projects funded under the new funding programs since April 1, 2005 and up until a cutoff date of February 26, 2007. We reviewed a systematic sample of 25% of CPAF projects (to ensure an equal percentage of projects from each region) and, in light of their small number, 100% of projects funded through the PCCF and the RKDF. Overall, 32% of all funded projects (n=112) were included in the file review. Table 1 below summarizes our sampling methodology for this task.

Separate file review templates were developed for each of the funding programs, based on the typical contents of the files. The file review templates for CPAF and PCCF consist primarily of closed-ended questions based on the standard application forms and final report templates for these programs. These results were data entered and analyzed quantitatively using SPSS, a statistical software package commonly used in social science research. By contrast, standardized forms do not exist for the RKDF program, so the RKDF template consists mainly of open-ended questions. For this reason, and because of the very small number of projects funded through the RKDF to date (n=12), the results of the RKDF file review were analyzed qualitatively. Detailed tabular results from the review of CPAF and PCCF files are appended in Volume II.

3.2.5 Review of performance measurement data

We conducted a review and analysis of performance measurement data provided by NCPC. NCPC provided us with activity logs enumerating the following activities: memoranda of understanding, letters of agreement, partnership arrangements, and networks; committees on which NCPC staff participated in some form; policy/program delivery tools and resources; workshops, conferences, internal/external training and presentations; and technical assistance. We extracted from these logs an extensive activity list that is included in Volume II and integrated the main findings into the body of the report.

3.2.6 Key informant interviews

We interviewed a total of 34 individuals offering particular insights into various aspects of the renewed NCPS. Potential key informants were identified by NCPC and included representatives of three groups: NCPC personnel, external partners, and provincial/territorial representatives. Provincial/territorial representatives were drawn from the co-chairs of the JMCs or FPT Working Group members, while external partners included representatives from other federal departments and non-governmental organizations with which NCPC has or has had partnerships. Within NCPC, potential interviewees were selected to represent the functional areas addressed by the evaluation questions (e.g. , senior management, program delivery, policy, research and knowledge, evaluation) and as much as possible to reflect different regional perspectives, although not every region was represented. Where possible, individuals were selected who had “worn another hat” in the organization and could thus provide information about more than one area. A separate interview guide was developed for each of the three groups of key informants.

Before being contacted, all key informants except for NCPC personnel received a letter from PRA describing the purpose and nature of the evaluation and inviting them to participate (NCPC personnel received an email notice from NCPC’s evaluation representative instead of a mailed letter). Approximately one week after the introductory letter was circulated, we contacted key informants by telephone to schedule interviews. We completed the majority of the interviews by telephone in the preferred official language of key informants.

3.2.7 Survey of funding recipients

The survey of funding recipients provided important information on the NCPS from those who are its primary, intended beneficiaries. We used a census approach to the survey of funding recipients. That is, all organizations that had received funding under the new funding programs of the renewed NCPS between April 1, 2005 and February 26, 2007 were sent the survey (n=342). The survey sample was provided to PRA by NCPC in Excel format, and contained names, contact information, and language preferences.

We used a mail survey to distribute the questionnaire. The questionnaire was accompanied by an introductory letter explaining the purpose of the evaluation and the importance of the survey. Respondents had the option of returning the survey to PRA by mail in a self-addressed, postage-paid envelope included with the mailed questionnaire, or by toll-free fax. We conducted several follow-ups by telephone to secure a high response rate.

Twelve surveys were returned to PRA because of invalid addresses, leaving a final sample of 330. In total, we received 164 completed surveys (115 in English and 49 in French), resulting in a 50% response rate. This is an excellent response rate for a survey of grant and contribution program beneficiaries. The survey results were data entered and analyzed using SPSS. Detailed tabular results from the survey are found in Volume II.

4.0 Evaluation findings

This section of the report summarizes the findings from all data collection methods used in this evaluation.

4.1 Design and delivery

4.1.1 Funding programs

With the renewal of the NCPS, NCPC streamlined its program platform by reducing the number of funding programs and introducing various changes to the design and delivery of these programs with the overall goal of improving accountability, risk management, and objectives achievement. These changes included funding a higher proportion of projects through contribution agreements; emphasizing multi-year, strategic, comprehensive crime prevention projects; increasing the targeting of NCPS funding; and emphasizing evidence-based crime prevention projects with strong research and evaluation components.

Among key informants, there is a broad consensus that these changes – which will continue to be pursued as a result of NCPC’s repositioning – represent an appropriate direction for the Strategy and for NCPC. Furthermore, there is evidence that NCPC has begun to fund projects that reflect the changes implemented with the renewal.

Higher ratio of contribution agreements to grants

NCPC has successfully achieved a higher ratio of contribution agreements to grants since the renewal. GCIMS data show that of the 349 projects that received NCPS funding under the new funding programs, 63% were funded through contribution agreements, while 37% were funded through grants. This ratio represents a significant shift in NCPC’s program platform, which prior to April 2005 consisted primarily of grants (74% grants compared to 26% contributions). 10

Emphasis on multi-year, strategic, comprehensive crime prevention projects

Recognizing that its previous practice of funding many small, short-term projects through the Community Mobilization Program (CMP) may not be the most effective way of achieving NCPS objectives, NCPC committed to funding larger, longer-term, more strategic projects under the renewed Strategy. The available evidence suggests that NCPC has begun to move in this direction. GCIMS data show that under CPAF, the successor to the CMP, the average NCPC contribution per project was $91,155. This compares with $33,005 committed per project under the CMP (note that under the CMP, funding was capped at $50,000 per project). 11 Thus, the average contribution under CPAF represents an almost threefold increase over the CMP. Notwithstanding this increase, more than half of CPAF projects are relatively small, receiving $50,000 or less in NCPC funding.

With regard to project duration, the available data are inconclusive, although there appears to be a trend toward longer projects since the renewal. GCIMS data show that most projects funded as part of the renewal (58%) are relatively short-term initiatives – 365 days or less – although 42% are more than 365 days in duration. Among CPAF projects specifically, 60% were one year or less in duration, which compares favourably with 90% of projects funded under the CMP between April 2005 and October 2005. Similarly, CPAF files were, on average, 432 days in duration, whereas CMP projects were, on average, 320 days in duration. However, there was no difference in median project duration among CPAF and CMP projects: 365 days and 364 days, respectively.

The evaluation had difficulty in assessing whether projects funded under the renewal are more comprehensive or strategic than those funded in the pre-renewal period. Using number of project partners and number of revenue sources (other than NCPC) as measures, there is evidence that funded projects involve multiple partners from a wide range of organizational sectors. File review data, for example, indicate that CPAF and PCCF projects averaged nine partners (the median was eight). Project partners played diverse roles and came from a diverse range of organizational sectors, including, most commonly, community, social and voluntary services; criminal justice, corrections, and police; education; health; and provincial/territorial governments. Similarly, GCIMS data show that half of funded projects have five or more sources of revenue other than NCPC; the median was five other sources12 On the other hand, among CMP projects funded between April 2005 and October 2005, the median number of other revenue sources was also five. This result is perhaps not unexpected, given the emphasis that NCPC has historically placed on partnerships.

Increased targeting of NCPS funding, to focus on priority groups and issues and a more limited number of communities experiencing higher rates of crime

Increased targeting of funding is a cornerstone of the renewed NCPS. With the renewal, NCPC committed to focusing on four priority groups: Aboriginal people, youth, children, and women and girls. It is not clear what priority issues, if any, it committed to addressing at the time of renewal. As shown in Table 2, it appears that NCPC has increased its focus on Aboriginal people and on women and girls in comparison to the pre-renewal period, whereas the proportion of projects focusing on youth and children has declined slightly. However, there are significant issues of comparability between the pre- and post-renewal data, and these findings should be treated with caution. 13

NCPC also committed to focusing on ten cities experiencing higher rates of crime (Regina, Saskatoon, Abbotsford, Winnipeg, Vancouver, Edmonton, Victoria, Halifax, Thunder Bay and Montreal). 15 This list overlaps to some extent with the list of ten cities experiencing high crime rates. However, some high crime communities, including Regina, Saskatoon, Abbotsford, and Edmonton, are notably absent from it. A comparison with pre-renewal data found no clear evidence of greater geographic focus, although there has been a small reduction in the total number of cities with funded projects, and one high crime city (Winnipeg) has experienced an increase in the number of funded projects.

Finally, based on the review of CPAF and PCCF files, 84% of projects cited some evidence of need, including findings from a community needs assessment (44%), current crime or victimization statistics (39%), or a particular incident or incidents in their community (13%).

Emphasis on evidence-based crime prevention projects with strong research and evaluation components

Since the renewal, NCPC has taken several steps to emphasize evidence-based crime prevention projects with strong research and evaluation components. With respect to ensuring strong evaluations, NCPC requires all funded projects to include evaluation plans in their applications and to carry out evaluations as a condition of receiving funding, and provides technical assistance during proposal development to strengthen evaluation plans. NCPC’s Evaluation Unit has also developed (though not yet fully implemented) an Evaluation Strategy for CPAF projects that proposes to prioritize projects that are most likely to produce higher level results, and provide support, guidance, and monitoring of their evaluations during the funding period. 16 In addition, for RKDF demonstration projects, which are funded in two phases (development and implementation), NCPC requires evaluators to work with applicants during the project development phase to ensure their evaluability, and requires a satisfactory evaluation plan as a condition of phase II funding.

With respect to research, NCPC provides technical assistance during proposal development to ensure that projects are based on available evidence about effective crime prevention strategies. Moreover, the RKDF Guidelines specifically ask for applicants to consult the existing literature when developing their proposals, and the internal and external review forms for RKDF contain specific questions for the reviewer to assess the degree to which projects are based on evidence. However, neither the CPAF Overview nor the Applicants’ Guide mentions the need to consult the evidence base when it describes the steps involved in developing a proposal, and the CPAF Application Form does not specifically require applicants to describe the evidence base of their proposed interventions. Therefore, some CPAF applicants may not be aware that this is an expectation for all proposals. As a further consequence, the extent to which funded interventions are based on available evidence on effective crime prevention (if at all) cannot be systematically ascertained.

Finally, based on the key informant interviews, there appears to be differing interpretations among NCPC personnel of the term “evidence-based crime prevention projects.” Some interviewees interpret the term quite narrowly to mean projects that have been tested against very rigorous evaluation standards, while others interpret it more loosely to mean that the available evidence was consulted when designing the proposed intervention.

4.1.2 Proposal process

Along with changes to the design of its funding programs, the process by which NCPC solicits, develops, assesses, and approves funding proposals has also changed. Some of the changes in process were implemented by NCPC, whereas others were required by PSC. The evaluation found some weaknesses and inefficiencies in the current proposal process, which are described below. However, key informants are hopeful that the process will become more efficient and will better contribute to achieving NCPS objectives as NCPC moves forward with its new mandate.

Proposal solicitation and development

To date, NCPC has for the most part continued to issue generic calls for proposals under the renewed Strategy, albeit with significant differences in approach and timing across the regions. 17 This practice has continued despite growing emphasis on increased targeting of NCPS funding and the application of this principle at the proposal assessment stage. Many key informants reported that the funding criteria and program parameters have continued to evolve since the renewal, and that these changes have not been clearly communicated to community organizations or to the provinces and territories. As a result, key informants and survey respondents reported that many community organizations that applied for NCPS funding in good faith, believing that they understood and complied with the funding criteria and would receive funding, were eventually informed that these criteria had changed and that they were no longer eligible. (In fact, survey results indicate that 60% of funded organizations understand the criteria used to assess proposals for funding, suggesting room for improvement in this regard. ) In short, NCPC experienced large volumes of proposals being submitted and requiring review, many of which did not meet the funding criteria.

Proposal development has also reportedly become more onerous and time-consuming since the renewal, both for community organizations and for NCPC staff. Key informants attributed this change primarily to NCPC’s shift to contribution agreements, which involve more stringent demands for project planning and accountability. In some cases, according to key informants, proposal development (as well as project management, including financial reporting) has become too complicated a process, especially for some Aboriginal and Northern communities where significant issues of capacity and language remain. 18 These key informants argued that there is a need to simplify the process or to consider alternative approaches to application that do not involve the same level and complexity of paperwork. One suggestion was introducing simplified application forms followed by teleconferences with applicants, involving translators if necessary, to establish and confirm project details. Based on the survey, a small majority of funded organizations (57%) believe that the amount of paperwork associated with developing an acceptable proposal is reasonable.

With the repositioning announced on June 8, 2007, NCPC plans to introduce more targeted and more frequent solicitations that explicitly identify the priorities, types of interventions, and particular communities that will be funded. NCPC key informants believe that targeted solicitations will reduce the volume of proposals received, allowing NCPC staff to focus their efforts on a smaller number of promising proposals and thereby improve the efficiency of the process. Ultimately, NCPC key informants believe that targeted solicitations will produce stronger, more effective crime prevention projects that better contribute to achieving NCPS objectives. Some observed, however, that targeted solicitations will come at a price: NCPC will no longer have an “ear to the ground” in communities across the country. As a result, they fear it may forego funding worthwhile or innovative crime prevention projects that respond to the unique needs of specific communities.

Proposal assessment and approval

In its March 2006 Management Plan, NCPC committed to streamlining the proposal approval process, along with various other program-related processes, in the interests of achieving greater efficiencies and lower operating costs. According to key informants, however, the contrary has occurred: proposal review has become more protracted. These delays have reportedly had detrimental consequences for many community organizations, including the loss of important project partners or the loss of volunteers and project staff. In some cases, the viability of projects has been compromised. Among funded organizations that were surveyed, only 28% believe that proposals are approved in a timely fashion, whereas 56% disagree. Overall, survey respondents’ most frequent suggestion for improving the proposal process was to expedite funding approval (34%).

While the evaluation lacked appropriate comparison data on application processing time, GCIMS data (summarized in Table 3) show that about half of projects were processed in 182 days (6 months) or less, while a similar proportion were processed in 183 days or more. For a small percentage of projects, application processing took more than one year. On average, processing time was 199 days (just over half a year). The median was slightly lower: 188 days.

One of the factors responsible for these processing times is likely the departmental-wide “pause” in funding between July 2006 and December 2006, during which period no projects were approved and no funds were released. However, there is general agreement among NCPC key informants that the main contributing factor is the complex process of proposal review that has been instituted since the renewal. Whereas in the past, the delegation of authority to approve projects for funding was dependent on the funding amount (with authority to approve some funding residing at the NCPC and Branch level), PSC now requires all projects recommended for funding by NCPC and the regional JMCs to be scrutinized and approved by the Minister to ensure that departmental priorities are being met. The approval process now consists of three stages: program review, Joint Management Committee review (for regional CPAF projects) or external review (for RKDF and national CPAF projects), and Ministerial review. In all, there are 15 distinct steps for projects received by the regional offices and 16 distinct steps for national projects received at headquarters. 19

While this level of involvement in the approval process is perfectly within the Minister’s purview and not unknown among federal grants and contributions programs, it does not represent normal procedure. 21 As will be discussed further in other sections of this report, provincial/territorial representatives emphasized that the NCPS is a partnership between the federal and provincial/territorial governments and expressed concern about what they perceive as unilateral decision-making by the federal government since the renewal of the Strategy.

Finally, the need to streamline the proposal solicitation, development, review and approval process is particularly pertinent given the recent report of the Blue Ribbon Panel on Grant and Contribution Programs. The Report recommended, among other things, that Treasury Board Secretariat and federal departments, first, simplify the grant and contribution application process by reducing the number of steps involved, making greater use of common forms, and simplifying information requirements; and second, publish clear service standards that are directly related to the administration of the grant or contribution, such as speed of decision on applications, clarity of criteria and reporting requirements, and timeliness of payments. 22

4.1.3 Technical assistance

NCPC personnel offer various forms of technical assistance to community-based organizations. In addition to program officers providing support directly to applicants during proposal development process and to successful projects throughout the project lifecycle, NCPC has developed numerous guidance materials to assist community organizations with project development and evaluation planning. These materials include three major project evaluation resources developed since April 2005. 23 It has also produced various internal guidance materials, primarily related to project planning, evaluation, and contribution agreements, to assist NCPC staff in fulfilling their role with respect to proposal development and project management. In addition, performance measurement data show that NCPC has organized several major workshops and conferences on these topics and has given presentations and training on project and evaluation planning in communities.

A large majority of the funded organizations surveyed have benefited from some form of technical assistance, including, most often, general information about NCPC, NCPC funding programs, or CPSD (82% of all survey respondents); review and feedback on their draft application (77%); and assistance with developing their project idea (71%).Table 4 provides more information.

Among funded organizations that received assistance from NCPC staff, almost three-quarters found the assistance very useful, while about one-quarter said the assistance was somewhat useful. Organizations were also largely positive when asked for their level of agreement with a series of specific statements about the technical assistance they received. Two-thirds agreed that the assistance they received helped their organization to clarify the goals and objectives of their project, and the same proportion agreed that the assistance and/or training helped them to prepare a good proposal. Slightly fewer (60%) agreed that NCPC assistance helped them to identify their project’s expected outcomes. Approximately half agreed that NCPC assistance helped them to ensure that their project was based on available evidence about effective crime prevention strategies; to develop a realistic evaluation plan; and to identify the risk and protective factors that their project would address. See Table 5 for the details.

Based on these data, it is clear that NCPC technical assistance is essential to the development of strong funding proposals. However, increased efforts to assist applicants may be required in some areas, particularly in identifying risk and protective factors, evaluation planning, and integrating evidence-based crime prevention strategies. Several NCPC key informants acknowledged that compiling evidence on best practices and evidence-based interventions, and making this information accessible to the community, continues to be an area of weakness for the organization. These key informants noted that NCPC must continue to work on bridging its Programs and Policy, Research and Evaluation Divisions, in order ensure that the available information is understood by all staff, appropriately synthesized, and disseminated to the community.

Finally, several NCPC key informants reported that although the need for technical assistance has grown as a result of the renewed Strategy’s more stringent demands for project planning and accountability, cuts to NCPC staffing have effectively limited the amount of technical assistance that can be provided. Resource issues are discussed in more detail in Section 4.1.6.

4.1.4 Partnerships

The partnership component of the renewed NCPS is based on the recognition that multi-sector partnerships and coalitions of governmental and non-governmental stakeholders will strengthen crime prevention efforts. Under this component, NCPC is expected to seek new partnerships and strengthen its existing ones, facilitate the formation of partnerships among other organizations, and coordinate crime prevention efforts among multiple stakeholders.

NCPC has had a mixed record on these fronts. Some key informants reported that the uncertain situation in which NCPC has found itself since the renewal, characterized by spending restrictions, staffing cuts, shifting government priorities, and changing leadership and management within the organization, has negatively affected its partnership activity. As will be described in more detail below, NCPC’s relationship with its provincial/territorial partners, in particular, has suffered. On the other hand, there is also evidence of positive developments, including an increase in horizontal policy work within the federal government; the development of strong partnerships in some regions: a greater acceptance of CPSD among major partners, such as police and provincial/territorial policy makers; and the involvement of multiple partners from diverse sectors - including health, education, social services, and corrections, as well as justice and law enforcement - in the crime prevention projects submitted for NCPS funding consideration.

NCPC’s partnerships with the provinces and territories

Since the inception of the NCPS in 1994, the provinces and territories have been NCPC’s most important partners. It is clear from key informant interviews and minutes of the FPT Working Group that these relationships have become strained and weakened since the renewal. Among NCPC’s provincial/territorial partners there is a strong belief that the organization failed to consult them regarding the renewal and failed to keep them fully abreast of developments over the past two years. 24 These accusations were partly acknowledged as true by NCPC key informants who observed that NCPC itself often lacked the information necessary to provide full disclosure to its provincial/territorial partners. That being said, minutes of the FPT Working Group indicate that beginning in June 2005 NCPC representatives provided regular updates to committee members on the status of the renewal and subsequent developments. In any event, the perceived lack of transparent and timely communication has fostered considerable suspicion and mistrust among the provinces and territories, which has been compounded by a lack of continuity in NCPC’s senior leadership over the past two years.

At root, the provinces and territories object to what they perceive as the federal government’s unilateral decision-making since the renewal. As noted earlier in this report, many provincial/territorial representatives believe that provincial/territorial crime prevention priorities, as reflected in the recommendations of the JMCs, have not been respected in funding decisions. Indeed, some reported that their governments’ goals and priorities are not necessarily the same as those of the federal government. For example, several observed that NCPC has recently focused on issues that are irrelevant in their jurisdictions (e.g. , youth gangs, guns, and drugs are not pressing issues for the Atlantic provinces). Others expressed concern about what they perceive as NCPC’s move away from CPSD in favour of other approaches. 25

In short, among provincial/territorial key informants there was some concern that NCPC, in responding to the priorities of the current federal government, has unilaterally begun to move away from what the FPT Working Group has been working to accomplish for the past decade. Observing that the NCPS is a bilateral strategy between the federal and provincial/territorial governments, these key informants emphasized the need for genuine collaboration on the NCPS among the Strategy’s partners.

It now appears that some of these tensions are beginning to be resolved. Most provinces and territories appear to support the repositioning of NCPC, and the FPT Working Group provided input into the Blueprint for Effective Crime Prevention, announced June 8, 2007. NCPC and each of the provinces and territories are currently negotiating bilateral Framework Agreements outlining the terms of reference for implementing funding programs under the NCPS. As of May 11, 2007, most frameworks were under review by the provinces’ and territories’ respective legal services. 26

NCPC’s partnerships with other external organizations

In contrast to the strained relationship with the provinces and territories that has characterized the renewal period, NCPC has had some success at establishing new partnerships, both within the federal government and with other external partners. Based on a review of NCPC’s performance measurement data and key informant interviews, the most important of the agreements established since the renewal include:

• A partnership with Justice Canada and other federal departments on the National Anti-Drug Strategy, announced in the 2007 Budget. NCPC has an MOU with Justice Canada for $50,000 for the National Conference for Drug Treatment Programs. In addition, the two departments collaborated on a Memorandum to Cabinet seeking authorization to expand their youth crime prevention initiatives to specifically address issues of youth involvement in guns, gangs, and drugs. This was achieved through the creation of the Youth Gang Prevention Fund at NCPC and by enhancing the Youth Justice Renewal Fund at Justice Canada.
• The Labrador Innu Comprehensive Healing Strategy, which coordinates federal responses for Innu in Davis Inlet/Natuashish. Partners include three federal departments, two band councils, and two departments of the Newfoundland and Labrador government.
• Partnership agreements with Statistics Canada, one involving geocoding of crime statistics in certain high crime urban centres, and another involving special studies on youth crime and Aboriginals. NCPC has also signed an MOU with Statistics Canada for the Canadian component of the International Youth Survey. Other partners in this agreement include Youth Justice, Human Resources and Social Development Canada, and Health Canada.
• An MOU for a Strategic Framework for Collaboration with Heritage Canada to deal with risk factors contributing to youth marginalization and youth violence and criminality in ethnocultural/racial communities.
• An MOU with the Correctional Service of Canada for a project targeted at children with incarcerated parents who may be at high risk for involvement with the justice system. The project involved Big Brothers and Big Sisters of Canada, social services agencies, and schools.

NCPC also works closely with other units within PSC, including the Aboriginal Corrections Policy Unit and the Aboriginal Policing Directorate, to conduct joint research and policy planning as well as fund projects. In addition, NCPC has a number of long-standing relationships with external partners that preceded the renewal and have continued since that time. For example:

• NCPC partners closely with the RCMP in several ways. An RCMP representative sits on the FPT Working Group, while an NCPC representative sits on the RCMP strategic planning committee for youth. In addition, representatives from the RCMP and NCPC meet regularly and collaborate on specific projects, and the RCMP has received NCPS funding for specific projects.
• NCPC partners with the Canadian Association of Chiefs of Police (CACP) through the latter’s Crime Prevention Committee (CPC). The CPC and NCPC collaborated to develop a work plan whose overall objective was the long-term survival of the concept of crime prevention through social development and the integration of this concept as a philosophy into police services across Canada. Most recently, NCPC provided funding to CACP for the Crime Prevention Coalition, a two-year project that culminated in a major conference in 2007 that brought together approximately 50 organizations from a wide variety of sectors. The purpose of the conference was to discuss crime prevention through social development and sustainability of CPSD approaches and interventions.

Finally, analysis of NCPC’s performance measurement data reveals that NCPC staff are involved in a broad range of committees. Most of this committee work is being done by regional staff operating at the municipal and provincial levels. The committees are all clearly linked to the goals and mandate of NCPC and represent a good cross-section of groups that seek to reduce crime and disorder in Canadian communities. However, it appears that some regions are more involved in this activity than others, which may be a function of staffing issues or diligence at reporting these activities.

All of NCPC’s external partners who were interviewed reported that their organization has goals in common with the NCPS. They cited, in particular, crime prevention, social development, at-risk youth, and evidence-based knowledge, although, like some of the provinces and territories, some worried that NCPC has recently started moving away from CPSD. Almost all external partners believe that their partnership with NCPC helps their organization to achieve its goals and that NCPC influences their policy development process in a positive way; in fact, several, including the CACP and the RCMP, have incorporated CPSD into their overall policy as a result of their involvement with NCPC. Several singled out NCPC’s research capacity and knowledge resources as being one of its greatest assets and a great benefit to their organization. It was suggested that NCPC should become a centre of excellence for evidence-based crime prevention information, as well as the lead department responsible for coordinating the federal role in crime prevention.

While most external partners reported that their relationship with NCPC has been strengthened since the renewal, some reported that the relationship has weakened. In a few cases, proposed partnerships and projects did not move forward, either because of the pause, changing priorities at NCPC, high staff turnover and changing leadership at NCPC, or other reasons unknown to key informants. Several key informants observed that NCPC’s new senior management brings considerable subject matter and management expertise to the organization. They were hopeful that greater stability within the NCPC’s senior leadership would facilitate relationship-building and partnerships in the future.

Facilitating partnerships among other organizations

Finally, under the partnership component of the NCPS, NCPC is expected to facilitate the formation of partnerships among other organizations. NCPC personnel observed that this has occurred simply as a function of the requirement that all funding applicants identify partners and other funding sources in their proposals. As was reported earlier, both GCIMS and file review data confirm that projects funded since the renewal involve multiple partners from a wide range of organizational sectors – nine partners each, on average, for CPAF and PCCF projects. Project partners played diverse roles and came from a diverse range of organizational sectors, including community, social and voluntary services; criminal justice, corrections and police; education; health; and provincial/territorial governments. However, based on the available data and given NCPC’s historic emphasis on partnerships, it is not clear that this represents any change over the pre-renewal period.

4.1.5 Knowledge

Under the third component of the renewed NCPS, NCPC committed to improved efforts to synthesize best practices and lessons learned and to disseminate this information to the crime prevention community and to the general public. Some of NCPC’s knowledge work is occurring through projects funded through the RKDF, the objectives of which include developing and evaluating promising community-based crime prevention models and supporting research and other activities related to the compilation of evidence on crime prevention and victimization. To date, 12 projects have been funded under the RKDF. Of the 12, only one has been completed, and this is phase one of a demonstration project, which consists of developing the project and evaluation model to submit for a second phase of funding. Table 6 below provides a brief overview of RKDF projects.

In addition to supporting knowledge activities through projects funded under the RKDF, there are also other indications that NCPC is engaging in knowledge synthesis and dissemination activities. For example:

• Following renewal, NCPC created a Research and Knowledge Unit to enhance research and formalize the knowledge development function, signaling that it is taking this commitment seriously. There are five FTEs assigned to work in the knowledge area with the unit. Two (Knowledge Transfer Officers) are new positions while the others have been brought together from other parts of the organization (i.e. , GCIMS, resource distribution area) to work under the knowledge umbrella.
• NCPC’s website lists 26 publications specifically devoted to disseminating knowledge on evidence-based practices, 13 of which were published after the renewal. 27
• The Research and Knowledge Unit has instituted formal procedures for planning and producing publications via a Publications Plan, which is now in its second year. In addition, for each publication produced, a Distribution Plan is prepared. The Distribution Plan identifies potential users of the information and draws upon a web-based distribution list that allows Canadians to register to receive NCPC publications. The distribution list, which also includes names of individuals and organizations compiled by NCPC staff across the country, now contains 1,941 entries and can be sorted by topic or by geographic area to permit targeted distribution of materials.
• NCPC staff are responsible for ensuring that funding applicants incorporate evidence-based interventions into their proposals by providing technical assistance during proposal development. Among funded organizations surveyed, just over half (54%) reported that the technical assistance they received from NCPC staff helped them to ensure that their project was based on available evidence about effective crime prevention strategies. While this is certainly encouraging, there is clearly also room for improvement.
• NCPC’s performance measurement data show that the organization has undertaken numerous activities related to the production and dissemination of knowledge since the renewal, including organizing several major workshops and conferences, attending a wide variety of conferences dealing with crime and crime prevention and giving presentations at several of these conferences, giving numerous presentations and training in communities, and developing guidance materials for funding recipients as well as internal guidance materials. 28

Among key informants, only a minority of NCPC’s external and provincial/territorial partners had ever used NCPC knowledge products, and some were altogether unaware of any such products. Key informants who had used the products were split between those who found them useful and a larger group who did not. The latter group said the products were impractical, too limited or superficial to be of real value, or culturally or linguistically inappropriate in their jurisdiction. NCPC key informants, for their part, view the synthesis and dissemination of evidence-based knowledge as a major challenge and an area of weakness for NCPC, and suggested that part of the solution lies in improved linkages between the organization’s Programs and Policy, Research and Evaluation divisions. External partners, for their part, suggested that NCPC become more aggressive in marketing its knowledge products by, for example, systematically distributing these products through channels available to its partners.

4.1.6 Organizational structure, delivery structure, and available resources

NCPC has faced a challenging task of implementing the renewed NCPS in a context of significantly reduced resources. Since April 2005, the organization has experienced a 27% reduction in FTEs, an internal reorganization including elimination and downgrading of many positions, a Department-wide “pause” in funding between July 2006 and December 2006 that resulted in $22 million being lapsed, and reductions of $3.6 million in 2006-2007 and $10.5 million in 2007-2008 as part of the government-wide fiscal responsibility framework.

These reductions to NCPC’s resources have profoundly hampered its ability to implement and achieve the objectives of the renewed Strategy. The pause and subsequent budget cuts severely curtailed the number of projects that were funded and delayed others. Given that funding projects is a primary means by which the Strategy is expected to achieve results, this had obvious consequences for the achievement of its objectives. In total, some 150 projects were affected by the “pause.”30 Others were funded, but experienced major delays in receiving approval and then payments. As a consequence, according to key informants and funded organizations that were surveyed, both NCPC and the provinces and territories, as the delivery agents of the Strategy, lost credibility with community organizations.

Staffing cuts and internal reorganization have also impeded NCPC’s ability to implement the renewed Strategy. The Strategy’s commitments to increased use of contribution agreements, increased targeting of funding, integration of evidence-based interventions, and an enhanced knowledge function were made at the same time as staff capacity to implement these commitments was reduced. NCPC has suffered from extremely high turnover in staff and senior leadership since the renewal, and morale has been a major problem among remaining staff. 31 Key informants believe that this reflects staff frustration at the limited resources with which the organization was expected to implement the renewal’s ambitious vision.

In short, resources decreased, while expectations did not; on the contrary, expectations increased. Yet, several key informants saw a positive side to these developments: they forced NCPC to rethink its mandate, and the outcomes have been NCPC’s repositioning and its new mandate. There is considerable optimism among key informants that the new mandate will better enable NCPC, in collaboration with its partners, to achieve the objectives of the Strategy. Nevertheless, there remain several issues and challenges related to organizational structure, delivery structure, and available resources.

NCPC’s current organizational structure was produced without strategic analysis

First, key informants pointed out that NCPC’s current organizational structure was created, somewhat haphazardly, in response to external (i.e. , Treasury Board) imperatives. In other words, NCPC’s staffing cuts and internal reorganization were accomplished without any strategic analysis of the structure necessary to deliver on the renewed NCPS mandate. Key informants believe there is a need to review the organizational structure to ensure that the necessary structure is in place. They recommended that the review consider such questions as staff allocation among the national and regional offices, the role of different NCPC units at each stage of the project cycle from solicitation to evaluation, and the structures that need to be in place to better enable NCPC to fulfill its policy functions.

Staffing may be inadequate, particularly at the regional level

Many NCPC key informants believe that the objectives of the renewed NCPS are unattainable with current staff resources, particularly current resources at the regional level. As has been emphasized throughout this report, the renewed Strategy, with its focus on targeted, evidence-based interventions administered through contribution agreements, brought greatly increased demands for project planning and accountability. However, the subsequent staffing cuts diminished NCPC’s capacity in the regions to provide technical assistance to community organizations; to manage the proposal process; and to manage contribution agreements throughout the project lifecycle, precisely at the time when this capacity was needed most. To address this problem, key informants suggested that NCPC examine the allocation of staff among the national and regional offices to ensure an appropriate level and distribution, and if appropriate, hire additional staff or adjust the ratio of staff at headquarters to those in the regions.

Regional program delivery is not integrated with NCPC’s policy function

Historically, according to key informants, the Programs and Policy, Research and Evaluation (PRE) divisions of NCPC have not collaborated well, a situation that has reportedly improved dramatically since the current Directors were hired. The two divisions now have a close relationship, and worked together, for example, to develop the Youth Gang Prevention Fund. That being said, several key informants observed that NCPC continues to lack a mechanism for integrating PRE with program delivery and particularly with regional program delivery. As a result, the regions, which are responsible for making decisions on what projects to fund, have no way of systematically accessing information on best practices in crime prevention and evidence-based crime prevention. Several key informants emphasized the need for a policy function at the regional level. It was suggested that part of the solution might be to reintroduce the Regional Director position (downgraded to Regional Manager as part of NCPC’s internal reorganization) and require these individuals to report directly to the Executive Director of NCPC, rather than to the Director of Programs as Regional Managers presently do.

The role of the FPT Working Group is not clear

The strained relationship between NCPC and its provincial/territorial partners and the reasons for it have been described at length elsewhere in this report. While these tensions are beginning to be resolved, there continues to be some uncertainty around the roles and responsibilities of the FPT Working Group. Minutes of the FPT Working Group indicate that the committee typically covers a wide range of topics at each meeting, some of which are operational in focus while others are policy-oriented. Furthermore, key informants pointed out that the members of the Working Group are quite diverse in rank and background (e.g. , policy or programs). According to one key informant, the resulting mixture does not produce a particularly cohesive, focused, or effective Working Group. On the other hand, another interviewee argued that it is the lack of a clear definition of the Working Group’s role that has produced its varied membership. In any case, key informants suggested that there is a need to refine and refocus the mandate of the FPT Working Group to ensure that it can be an effective agent in delivering the renewed NCPS.

Departmental supports to NCPC are lacking

NCPC key informants reported that Departmental supports to NCPC are severely lacking in a number of areas. The Department’s Human Resources unit, which itself is suffering from high staff turnover, has reportedly been incapable of fulfilling NCPC’s staffing needs in a timely fashion with the consequence that key NCPC positions have been vacant for long periods of time. Similarly, key informants reported that seemingly routine administrative functions, such as reimbursement of travel expenses and pay increases, require inordinately long periods of time to execute and that information technology support, which is provided out of headquarters, is not accessible to the regions after 2:00 p.m. Pacific Time. Finally, PSC appears to be unusually risk averse and unwilling to give any decision-making latitude to NCPC, as evidenced by the high level of scrutiny applied to proposals at the Departmental and Ministerial levels.

4.1.7 Performance measurement

The ARAF for the Renewal of the National Crime Prevention Strategy (dated May 30, 2005) sets out a performance measurement and audit strategy for the NCPS. The strategy includes an evaluation framework and a reporting plan that includes annual roll-up reports for internal use, internal and recipient audit reports, and progress and outcome evaluations in years two and five, respectively. With renewal of the NCPS, NCPC committed to the development and implementation of an enhanced performance measurement system to assist in meeting these reporting requirements. The organization has taken several steps toward implementing this system, including:

• developing standard application forms and reporting templates for each funding program
• requiring all funded projects to provide final reports and conduct evaluations
• providing evaluation training and developing evaluation resource materials for NCPC staff and for funding recipients
• enhancing the electronic Grants and Contributions Information Management System (GCIMS)
• developing performance measurement templates to track activity under the partnership and knowledge components of the NCPS.

Based on the results from the survey of funded organizations, it appears that NCPC has had considerable success in communicating its performance measurement requirements to funding recipients and that these have been generally well-received. About 70% of survey respondents believe that NCPC requirements for evaluation and final reporting are clear, that NCPC resource materials provide useful information for preparing a project proposal, and that the requirement to identify clear project objectives and outcomes has helped them to develop a stronger project. Slightly fewer believe that their organization is in a better position to demonstrate the results of their initiative(s) as a result of NCPC requirements for project planning and evaluation. See Table 7 for the details.

Furthermore, over half of funded organizations (52%) reported that they face no challenges in complying with the requirements of the performance measurement strategy. The main reported challenge, cited by 12% of survey respondents, was related to human resources. These respondents reported that they lacked the time, staff, or internal expertise within their organizations to comply with NCPC’s performance measurement requirements, or that performance measurement interfered with service delivery or their day-to-day operations. Other challenges mentioned included lack of documentation or clarity about the performance measurement requirements; difficulties in evaluating impact after a short time period; lack of funding for evaluation activities; difficulties in measuring outcomes related to social development; and difficulties in tracking and maintaining the necessary data. Although these results are encouraging, they should be treated with caution, since only 42% of organizations surveyed had actually concluded the project for which they received funding, so some organizations may not yet have proceeded to the final reporting and evaluation stages when they responded to these questions. As more projects are completed, final reports and evaluation reports should be examined to determine if projects are experiencing any difficulties in complying with NCPC’s performance measurement requirements.

At present, the GCIMS database, which is the source for project-based performance measurement information, faces significant challenges. GCIMS is intended to hold information on each proposal submitted for consideration under NCPC’s funding programs. Its purpose is threefold: to track the status of applications and funded projects; to facilitate reporting and responding to requests for information; and to identify projects for further analysis in order to articulate best practices, lessons learned, and the like. 32 While data entry into GCIMS began in the 2004/2005 fiscal year, the database is not yet capable of generating data or reports to support timely project management, performance monitoring, or impact assessment.

There are currently three main issues related to GCIMS. 33 First, data entry is not consistent across the country, and as a result, reports and other information generated through GCIMS are not reliable. Several reasons for the inconsistency in data entry are documented. One is NCPC’s move from the Department of Justice to PSC, which interrupted the initial roll-out of GCIMS. Another is the limited ability of GCIMS to produce certain reports (such as Ministerial Packages) due to incomplete development of the database and lack of information technology capacity at the regional level. Because of these limitations, program staff must enter information in other formats in order to produce the necessary documents, which has led to skepticism about the value of GCIMS and reduced the incentive to enter data into the system. NCPC key informants confirmed that many regions continue to use their own databases to generate reports and that GCIMS data entry “falls off the table” because of staffing limitations and the perceived redundancy of the database.

• Second, GCIMS has not yet been fully developed, so NCPC staff cannot retrieve the range of information that they require for reporting purposes. For example, the section on Performance Management, where information on expected and actual outcomes of projects is to be recorded, is still under development.
• Third, once GCIMS is fully developed, roll-out will require resources and planning to ensure that all staff receive the training and support they need and are using the system consistently, and to ensure that project data entered in an earlier version of GCIMS are entered into the revised version.

NCPC has developed an Action Plan for GCIMS in which it sets out a detailed strategy for addressing these issues, and according to the most recent update of the Plan (May 31, 2007), progress is being made on all fronts. 34 However, based on PRA’s own limited exposure to GCIMS (we received a limited data extract in Microsoft Excel format), there are two additional issues with GCIMS that NCPC should address. First, even if data entry is done consistently and the database is fully implemented, GCIMS may not be capable of producing valid and reliable data. PRA saw evidence of poorly defined fields and attributes (e.g. , vague or non-existent definitions, overlapping or non-mutually exclusive attributes), new attributes having been added on an ad hoc basis, and similar information being captured in slightly different ways by more than one field, leading to uncertainty over which field contains accurate information. In short, within GCIMS there is currently considerable room for subjective judgment when entering data, which will have a negative impact on the validity and reliability of the data. However, the most recent information available to PRA indicates that NCPC is in the process of addressing these issues.

Second, although GCIMS is an information management system, it is nevertheless capturing primarily qualitative data (one obvious exception is project financial information). The task for NCPC is to design a database capable of ensuring that these qualitative data can be quantified, and that they will be meaningful once quantified. At the same time, as several key informants also pointed out, NCPC should not rely solely on a database solution for performance measurement and evaluation purposes and overlook the value of other qualitative approaches, such as in-depth case studies, to describe and illuminate what the Strategy has achieved.

It should be noted that, at present, the GCIMS Working Group is comprised of five representatives from Policy, Research and Evaluation (four from the Knowledge Unit and one from Evaluation), six representatives from Programs (four Program Officers, one administrative person, and one from Grants and Contributions Management), and one departmental IT representative. The expertise of additional specialists in database development, performance monitoring, and evaluation would be helpful in the process of developing a database that will meet the needs of evaluation.

The evaluation identified two other main challenges for the performance measurement strategy. First, the performance measurement templates currently used to track partnership and knowledge activity are in Microsoft Word, and as a result, these data are time-consuming to record and to analyze; it would be considerably more efficient in the long term to develop a simple Microsoft Access database to capture and analyze this information. Second, since current funding is probably inadequate to support formal, rigorous evaluations of every funded project, NCPC should develop criteria to determine which projects should be formally evaluated.

4.2 Impact

The evaluation examined the extent to which the Strategy has achieved results in three areas corresponding to its community action, partnership, and knowledge components: increased local capacity to prevent crime and victimization, improved comprehensive partnerships in effective crime prevention, and improved integration of evidence-based crime prevention into policies and practices. For any program at the interim evaluation stage outcomes tend to be limited or very preliminary. In the case of the renewed NCPS, impact is particularly difficult to discern at this interim evaluation stage, since most projects funded since the renewal have not yet concluded. 35 GCIMS does not yet track outcome data, and of the projects reviewed in the file review, only 18 had submitted a final report. Outcome information was tracked for these 18 projects and can be found in Volume II, but this information is too limited to be included here.

Finally, as described earlier in this report, the developments that followed the renewal had a detrimental impact on NCPC’s ability to implement the Strategy and, by extension, to achieve results. Nevertheless, there are some early indications of positive results in each of the three areas, based primarily on the survey of funded organizations and the key informant interviews.

Increased local capacity to prevent crime and victimization

Results from the survey of funded organizations provide evidence that the renewed NCPS has made an impact in this regard. As shown in Table 8, survey respondents overwhelmingly agreed that NCPC funding has enabled their organization to undertake initiatives it otherwise would not have been able to pursue (93%). Moreover, three-quarters agreed that their capacity to prevent crime and victimization has improved as a result of NCPC funding or technical assistance, while 70% agreed that their community’s capacity to prevent crime and victimization has improved.

Furthermore, among funded organizations whose project had been completed (42% of survey respondents or n=69), commonly reported outcomes included increased awareness within the community (38%) and mobilization of the community to take action on or participate in crime prevention (22%). Table 9 provides a complete summary of reported outcomes.

Many key informants agreed that the objective to increase local capacity to prevent crime and victimization has been accomplished or that significant progress has been made in this direction, particularly since increased community capacity has been a focus of the NCPS for some years, not only since renewal. For these key informants, the time has come to focus on more concrete outcomes, such as crime reduction and reduction of known risk factors, which is precisely what NCPC intends to do under its new mandate. Others, however, believe that increasing local capacity should continue to be a goal of the NCPS, given that many communities continue to struggle to plan and implement crime prevention initiatives.

Improved comprehensive partnerships

This expected outcome pertains both to partnerships established by funded projects and NCPC’s own partnerships. As reported earlier, there is clear evidence that funded projects involve multiple partners from a wide range of organizational sectors. GCIMS data show that half of funded projects have five or more sources of revenue other than NCPC. Similarly, file review data indicate that CPAF and PCCF projects averaged nine partners each, and partners came from a diverse range of organizational sectors, including, most commonly, community, social and voluntary services; criminal justice, corrections and police; education; health; and provincial/territorial governments. Among funded organizations who participated in the survey, as shown in Table 8, 78% agreed that, as a result of NCPC funding or technical assistance, their organization has developed partnerships with organizations whose involvement was necessary for achieving their project objectives, and 59% agreed that NCPC funding helped their organization to leverage other sources of funding. Among those whose project had been completed, 28% reported that new or strengthened partnerships had resulted (see Table 9).

NCPC’s own partnership activities were described at length in Section 4.1.4 of this report. To briefly summarize, although NCPC’s unstable situation since the renewal has affected its partnership activity, there is evidence of positive developments, including several new horizontal partnerships with other federal departments and a strengthening of existing partnerships with the RCMP and the CACP. Virtually all of NCPC’s external partners reported that their organizations have goals in common with the NCPS and that the partnership has helped them to achieve their goals and has influenced their policy development. Some organizations, such as the RCMP and the CACP, have incorporated CPSD into their official policy as a result of their involvement with NCPC.

While this issue was not raised by any stakeholders involved in the evaluation, it is worthwhile to note that from an evaluation perspective, “improved comprehensive partnerships” is questionable as an outcome. More accurately, this phrase describes an output of an activity. A true outcome would attempt to capture what has been achieved as a result of improved partnerships, since partnerships for their own sake are clearly of little value. It should be noted that the new logic model currently being drafted for the NCPS has eliminated “improved partnerships” as an outcome, and instead identifies “strategic partnerships” as an output. 36

Improved integration of evidence-based crime prevention into policies and practices

To date, evidence of progress in integrating evidence-based crime prevention into policies and practices is limited to the key informant interviews and the survey of funded organizations; although 12 RKDF projects have been funded under the renewed strategy, to date only one of these projects had been completed. Key informants believe that there is greater awareness among community organizations of evidence-based crime prevention and better understanding of its importance. However, they also observed that most community organizations are small and do not have the financial or human resources to devote to researching evidence-based crime prevention. Among survey respondents, a majority (62%) agreed that their organization has a better understanding of evidence-based crime prevention strategies as a result of NCPC funding or technical assistance, while 54% agreed that NCPC technical assistance helped them ensure that evidence-based crime prevention was integrated into their project. As mentioned several times in this report, NCPC key informants acknowledged that this component of the renewed NCPS has been an area of weakness and one that will require greater effort in the future.

Unintended impacts

The evaluation found little evidence of unintended impacts of the renewed Strategy. Many key informants reiterated that NCPC’s ability to implement the Strategy has been constrained by budgetary and staffing cuts, internal reorganization, and the pause in funding, and that any unintended impacts that have occurred over the past two years are a result of these factors, rather than a result of the renewed NCPS itself. These unintended impacts have been primarily negative and have been described in detail throughout this report. To briefly summarize, they include cumbersome administrative processes resulting in lengthy delays in proposal approval and payments to funded organizations; inability by NCPC to clearly communicate its evolving situation to its provincial/territorial partners and to community organizations; the development of strained relationships with the provinces and territories and with community organizations; and high turnover and low morale among NCPC personnel.

A small number of key informants identified positive unintended impacts. Several observed that the events that occurred since the renewal have forced NCPC to examine and rethink its mandate, resulting in the newly repositioned and (they believe) more effective organization. In addition, instability at the federal level over the past two years has reportedly prompted one jurisdiction to develop a provincial crime prevention policy of its own, which it had previously lacked.

4.3 Effectiveness and efficiency

The NCPS renewal in April 2005 and the repositioning announced in June 2007 have set the future course for NCPC and consequently for the Strategy. Given the relative infancy of these changes, and the fact that further refinements are possible as NCPC implements its new mandate, it is somewhat early to assess whether there are more effective ways of achieving the objectives of the Strategy. That being said, it is possible to consider whether NCPC’s new direction is likely to be successful, and to note areas in which NCPC might consider changing as it moves ahead, based on what can be learned from the situation in other countries. As will be discussed in this section, there is considerable support in the literature for involvement of the national government in crime prevention; for a whole government approach to crime prevention; and for the need for evidence-based crime prevention. In a point that is particularly pertinent to the renewed NCPS, the literature review also highlighted the critical nature of effective implementation and the role that poor implementation has played in the failure of many prevention initiatives. 37

The role of national government in crime prevention

Most crime prevention experts who have considered the issue have made a strong case for the involvement of the national government in crime prevention. For example, commenting on the system in the United States, Irvin Waller has recommended that “it is important to get a permanent agency such as those set up in Scandinavian countries, Australian states, Belgium, Canada, and France. Unless the agency has access to permanent and significant funds…it cannot fulfill its mission.”42 In fact, NCPC is used as a positive example by Welsh and Farrington who recommend that the United States adopt a similar national body. Thus, the literature strongly suggests that central government must play a strong role in crime prevention. All of the experts whose work we reviewed agreed with this conclusion.

Among key informants, there is widespread support for a national role in crime prevention. However, significant differences of opinion were evident when it came to the precise nature of this role. One provincial/territorial representative, for example, summed up the essence of the debate by questioning whether it is the provinces’ and territories’ role in the NCPS to contribute to federal crime prevention objectives, or whether, conversely, it is the role of the federal government to support the provinces and territories in achieving their objectives. Indeed, many provincial/territorial key informants insisted that the NCPS must continue to accommodate provincial/territorial crime prevention priorities, while others argued that the funding component of the Strategy could be more effectively (and more efficiently) delivered by the provinces and territories. In addition, key informants from all three groups (NCPC, the provinces and territories, and external partners) recommended that the federal role in the NCPS should be to focus on the Strategy’s knowledge and partnership components. These key informants envision NCPC not primarily as a funding body, but rather as a national centre of excellence in research, knowledge development, and crime prevention expertise, and as the main body leading and coordinating crime prevention work in Canada. This latter point is discussed in more detail below.

The whole government approach

One of the key components of the NCPC repositioning is strengthening partnerships to maximize synergy and avoid duplication. An important dimension of this component is working with other federal departments and with other levels of government. This corresponds with the whole government approach, which involves trying to break down the silos (or “Cylinders of Excellence” as they are now sometimes called) between departments and between levels of government. As Homel has described, the approach is “an example of a more general shift in public administration away from a command and control mode of governance toward governance through multiple stakeholders working together to deliver integrated solutions to social problems across sectors and tiers of government.”43 In the literature review, this trend was also strongly supported, particularly in England.

Among key informants interviewed for this evaluation, some believe that in the past NCPC had not done sufficient work to provide leadership within federal departments. They argued that NCPC should take the lead in ensuring that the whole range of relevant departments, particularly those with public safety and social development mandates, work in a coordinated fashion on the prevention of crime. Crime is a very complex issue and does not correspond with the boundaries of individual departments. Individuals and communities at high risk of crime have a wide variety of problems and needs that are not addressed by any single government department. For example, at-risk children have income needs, health problems, educational difficulties, problems with parents and role models, psychological difficulties, and potential problems finding employment as they grow older to name just a few. These needs, along with the problems created by their involvement in the youth justice system, cut across many different departments and levels of governments and cannot be solved by the efforts of any one part of government.

Since the renewal, NCPC has made some attempts to expand this role, such as the agreement that has been signed with Heritage Canada and various partnership arrangements with other federal departments. However, partnership activities have been constrained by events since the renewal, and there is considerable room for further expansion of these activities.

Evidence-based crime prevention

Virtually all of the agencies promoting crime prevention, including the British government, Australian federal and state governments, the United States National Crime Prevention Council, as well as most academic experts, support the evidence-based approach to crime prevention. While it is necessary to leave some room for innovation and for programs that suit particular local circumstances, there is a growing consensus that crime prevention should be based upon the best evidence about what is likely to be effective. The United Nations Guidelines for the Prevention of Crime (2002) also support the need for countries to adopt policies involving evidence-based crime prevention.

Among key informants, there is considerable support for emphasizing evidence-based crime prevention interventions, although some worried that in doing so, the NCPS may be forgoing potentially effective, even if untested, crime prevention interventions. For example, several pointed out that in fact there is a dearth of evidence on “what works” for certain populations and communities, such as Aboriginal populations, which are judged to be particularly high-risk and therefore most in need of intervention. They emphasized an ongoing need to support innovative, community-based crime prevention projects, particularly in areas where evidence may be lacking.

In this context, it bears repeating that NCPC personnel appear to hold different interpretations of the term “evidence-based crime prevention projects,” with some interpreting it narrowly to mean projects that have been tested against very rigorous evaluation standards, and others interpreting it more loosely to mean that the best available evidence was consulted when designing the proposed intervention. These different perspectives within NCPC itself suggest a need to clarify the organization’s understanding of the term.

Implementation issues

While project and program implementation is often taken for granted, it is often very problematic. Much of the literature identifies implementation failure as the major cause of the failure of many crime prevention initiatives, including, most notably, the United Kingdom’s Crime Reduction Program (CRP), the failure of which was attributed primarily to the central government’s failure to become actively involved in helping with local implementation. Indeed, one of the most consistent messages in the literature is that without implementation support from central governments, it is unlikely that programs delivered at the local level will succeed. While budget and staffing allocations will inevitably limit the support that can be provided, the literature suggests that a significant investment in effective program planning, management, and evaluation is critical to the future success of the NCPS. Similarly, many NCPC key informants argued that, despite designing a renewed NCPS and despite redefining its own mandate, NCPC has yet to conduct a strategic analysis of the organizational structure and resources necessary to effectively implement either the changes promised by the renewal, or more recently, its own new directions.

To summarize, evidence from the literature suggests that as a national strategy involving a strong federal role and a focus on horizontal partnerships and evidence-based crime prevention, the renewed NCPS possesses a structure that will enable it to have an impact on crime in Canada.

4.3.1 Potential efficiencies

Among NCPC key informants, there is considerable agreement that if NCPC implements its new mandate, focusing on a more limited number of targeted interventions, efficiency of delivery will improve dramatically. Based on evidence from the key informant interviews and the other data collection methods used in this evaluation, potential efficiencies may also be achieved by:

• introducing a targeted, rather than an open, proposal solicitation process, in order to reduce the number of proposals submitted, particularly given the time and resources that NCPC staff currently spend on assisting applicants to develop acceptable proposals
• revisiting the proposal review process in order to identify possible efficiencies (e.g. , re-delegating decision-making authority to the JMCs and NCPC)
• developing a consensus vision for the NCPS among the federal and provincial/territorial partners to the Strategy
• clearly delineating the respective roles and responsibilities of the Strategy’s partners, as well as those of the FPT Working Group
• ensuring full support for the NCPS and for NCPC among PSC’s senior leadership, in order to secure appropriate departmental administrative and human resources support
• conducting an organizational review of NCPC to ensure that the organization is appropriately staffed to deliver on its mandate and the mandate of the renewed NCPS, including the knowledge and partnership components as well as the funding programs
• completing implementation of a fully functional GCIMS database that will provide adequate and useful information for assisting in timely project management, ongoing performance monitoring and program adjustment, and impact assessment.

4.4 Relevance

Beginning with the renewal of the NCPS in April 2005, NCPC has made concerted efforts to ensure that the Strategy aligns with the current strategic objectives and priorities of PSC and the federal government more generally. The renewed Strategy contributes to PSC’s strategic outcome of “enhancing public safety, security and emergency preparedness of Canadians in an open society,”and to the departmental priority of “enhancing community safety and security.” PSC’s Report on Plans and Priorities (RPP) for 2007/2008, released on March 29, 2007, describes the Department’s focus on known crime risk factors, priority crime issues, and the dissemination of knowledge regarding crime prevention best practices,44 all of which are also priorities for the renewed NCPS. The RPP states that:

Reducing crime and increasing personal safety is one of the federal government’s priorities and a priority for Canadian citizens. The prevention of crime and re-offending is key to meeting this priority. The Department is tackling this challenge by taking steps to implement evidence-based and cost-effective interventions targeted at prevailing crime issues in communities most at risk. 45

In addition, the Youth Gang Prevention Fund, which was introduced in Fall 2006 and was not part of this evaluation, is mentioned in the RPP as one of the Department’s activities in support of its key initiative to “assess and support local crime prevention measures addressing, in particular, gang-related crime and drug and alcohol related crime.”Finally, the recently announced repositioning of NCPC was in large part a response to the need to further align the organization with the objectives and priorities of the Department and the federal government.

Among key informants, there is broad consensus that the federal government should continue to play a role in crime prevention and that the NCPS is still relevant to crime prevention in Canada. In fact, many believe that if the NCPS did not exist, the consequences for crime prevention would be devastating. However, not all key informants are convinced that the renewed Strategy is appropriately designed, and they criticized precisely the changes that have been introduced to ensure alignment with federal objectives and priorities. In particular, they object to what they perceive as the move away from crime prevention through social development and community capacity building and the greater focus on selected issues that are not relevant in all communities or jurisdictions. These key informants believe that to be truly relevant across Canada, the NCPS must permit communities and jurisdictions to focus on local and provincial/territorial issues and priorities.

Both NCPC and the NCPS remain extremely relevant from the perspective of funded organizations. The vast majority of those who participated in the survey agreed that the NCPC is an important resource for organizations working in the field of crime prevention (97%) and that the NCPS is still relevant to crime prevention in Canada (94%). Survey respondents also predicted detrimental consequences for their own organizations and for crime prevention in Canada if the NCPS did not exist. Two-thirds of respondents said that in the absence of the Strategy, their own organization’s crime prevention work would be severely limited or altogether curtailed, and about one-fifth said that the outcomes they have managed to achieve through NCPC-funded projects would not have been realized. One-third argued that Canada would suffer a reduction in crime prevention activity, and one-quarter said that progressive solutions to crime and victimization would be compromised. About one-fifth predicted an increase in crime and victimization in the absence of the NCPS.

5.0 Conclusions and recommendations

This interim evaluation of the renewed NCPS was charged with the challenging task of evaluating a moving target. When the evaluation began in February 2007, the NCPS was not the same Strategy as it was in April 2005 when it first received renewal funding. But the challenges facing the evaluation are insignificant in comparison to those that NCPC has confronted. Since April 2005, the organization has experienced a 27% reduction in FTEs, an internal reorganization including elimination and downgrading of many positions, a department-wide “pause” in funding between July 2006 and December 2006 that resulted in $22 million being lapsed and reductions of $3.6 million in 2006-2007 and $10.5 million in 2007-2008 as part of the government-wide fiscal responsibility framework. Most recently, NCPC has been given a new, more focused mandate (the repositioning recently announced by the Minister of Public Safety) that, like the events of the last two years, will affect the delivery of the NCPS.

This interim evaluation has attempted to take these contextual issues into account in drawing conclusions and providing recommendations.

Design and delivery

Community action component: funding programs

Under the community action component of the NCPS, NCPC provides funding and technical support to crime prevention initiatives across Canada. With the renewal of the NCPS, NCPC streamlined its program platform by reducing the number of funding programs and introducing various changes to their design and delivery. These changes included funding a higher proportion of projects through contribution agreements; emphasizing multi-year, strategic, comprehensive crime prevention projects; increasing the targeting of NCPS funding; and emphasizing evidence-based crime prevention projects with strong research and evaluation components.

The evaluation found widespread (though not unanimous) agreement among key informants that these changes represent an appropriate direction for the Strategy and for NCPC. Furthermore, there is some evidence that NCPC has begun to fund projects that reflect these changes. For projects funded since April 2005, the ratio of contribution agreements to grants is 63:37, in comparison to 26:74 in the pre-renewal period, and the average NCPC contribution per CPAF project is almost three times more than it was for projects funded under the CMP, the predecessor to CPAF.

On the other hand, the available data on project duration are inconclusive. Although CPAF projects were, on average, longer than CMP projects, median project duration in both cases was one year. Similarly, the evaluation found little change in the comprehensiveness or strategic nature of funded projects pre- and post-renewal (using number of other revenue sources as an indicator), and because of a lack of comparable pre-renewal data, it could not assess whether projects funded since the renewal are more targeted in comparison to the pre-renewal period. In future, data on projects funded in the first two years of the renewal should be used as a baseline against which progress in these areas can be assessed.

Recommendation

1. In order to measure progress in implementing the changes to funding program design and delivery that were introduced with the renewal of the NCPS, NCPC should use the data on projects funded in the first two years of the renewal as a baseline for comparison.

NCPC has taken several steps to emphasize evidence-based crime prevention projects with strong research and evaluation components. However, NCPC key informants acknowledged that compiling and communicating easy to use, current information on the state of evidence in a variety of crime prevention areas is an area of weakness for the organization. Indeed, results from the survey of funded organizations indicate that applicants require more assistance from NCPC staff in integrating evidence-based crime prevention strategies into their proposals. In this context, it is important to note that neither the CPAF Overview nor the Applicants’ Guide mentions the need to consult the evidence base when it describes the steps involved in developing a proposal, and the CPAF Application Form does not specifically require applicants to describe the evidence base of their proposed interventions. Therefore, some applicants may not be aware that this is an expectation for all proposals. As a further consequence, the extent to which funded interventions are based on available evidence on effective crime prevention cannot be systematically ascertained.

Moreover, it is clear that there are differing interpretations among NCPC personnel of the term “evidence-based crime prevention projects,” with some interpreting the term narrowly to mean projects that have been tested against very rigorous evaluation standards, and others interpreting it more loosely to mean that the available evidence was consulted when designing the proposed intervention. These different perspectives within NCPC itself suggest a need to clarify NCPC’s understanding of the term.

Recommendation

2. NCPC should clearly define what it means by “evidence-based,” in order to ensure a common understanding of the term among NCPC staff.

Recommendation

3. To fulfill its commitment to evidence-based practice, NCPC should, first, inform funding applicants of the requirement to integrate the evidence base into their proposed interventions; and second, develop a means of tracking the extent to which successful applicants integrate evidence into their projects.

The evaluation found that the technical assistance provided by NCPC staff is essential to the development of strong funding proposals. Moreover, the need for technical assistance has grown considerably as a result of the renewed Strategy’s more stringent demands for project planning and accountability, while cuts to NCPC staffing have effectively limited the amount of technical assistance that can be provided. While projects surveyed found the technical assistance they received to be useful, the evaluation found clear evidence that increased efforts to assist applicants may be required in some areas, particularly in identifying risk and protective factors, evaluation planning, and integrating evidence-based crime prevention strategies. Furthermore, many key informants expressed concern that the current proposal requirements have placed NCPS funding beyond the reach of many high-needs and high-risk communities (particularly Aboriginal and/or Northern communities), where the capacity to prepare sophisticated proposals and administer complex, comprehensive prevention projects remains a major challenge.

Recommendation

4. NCPC should ensure that sufficient resources (i.e. , staff, tools, and resources) are available to ensure that funding applicants receive adequate technical assistance during proposal development, particularly in identifying risk and protective factors, evaluation planning, and integrating evidence-based crime prevention strategies.

Recommendation

5. Given that crime and victimization are pressing issues for many Aboriginal and Northern communities, and given that one of NCPC’s priorities under its new mandate is to foster prevention specifically in Aboriginal communities, NCPC should consider either simplifying proposal requirements or modifying the proposal development process for applicants that meet the program’s funding criteria but have language or other barriers that may prevent them from putting forth successful proposals. In addition, NCPC should consider providing additional assistance with certain aspects of project management, such as financial reporting, to successful applicants that lack the capacity to manage projects effectively on their own.

Under NCPC’s repositioning, the process by which it solicits applications for funding will shift from generic calls for proposals (which have for the most part continued, to date, under the renewed Strategy despite a commitment to increased targeting and the application of this principle at the proposal review stage) to targeted solicitations that explicitly identify the priorities, types of interventions, and particular communities that will be funded. By reducing the volume of proposals received, targeted solicitations are widely expected to improve efficiencies and ultimately produce more effective projects that better contribute to achieving objectives.

The major weaknesses in NCPC’s proposal process are, at present, lengthy proposal review and approval times. For projects funded since the renewal, average processing time is 199 days (just over half a year). While the “pause” is partly responsible for these delays, a major contributing factor is the complex process of review and approval that has emerged since the renewal. Whereas in the past, the delegation of authority to approve projects for funding was dependent on the funding amount (with authority to approve some funding residing at the NCPC and Branch level), PSC now requires all projects recommended for funding by NCPC and the regional JMCs to be scrutinized and approved by the Minister to ensure compliance with departmental priorities.

The approval process now consists of three stages – program review, JMC or external review, and Ministerial review – and at least 15 distinct steps. In addition to creating lengthy delays, this process has jeopardized the viability of some projects and created resentment among NCPC’s provincial/territorial partners who believe that their priorities, as reflected in the decisions of the JMCs, are not being respected. In order to improve efficiencies and repair goodwill, therefore, efforts should be made to streamline the proposal review and approval process and to clearly articulate the roles and responsibilities of all parties involved.

Recommendation

6. In the interest of improving efficiencies, NCPC, together with PSC representatives, should investigate ways of rationalizing the proposal review and approval process, including the possibility of re-delegating the authority to approve project funding to the Branch and Directorate levels.

Recommendation

7. The roles and responsibilities of all parties in the proposal review and approval process and, in particular, the roles and responsibilities of the JMCs should be clearly articulated.

Recommendation

8. While the lengthy processing times experienced since the renewal of the NCPS may be an aberration resulting from unusual circumstances, NCPC and/or PSC should consider implementing and adhering to a standard processing time for funding proposals. The same recommendation was made by the Blue Ribbon Panel on Grant and Contribution Programs for federal departments in general. A reasonable target could be identified by examining processing times of similar funding programs within the federal government, such as Status of Women Canada’s Women’s Program, which also takes a social development approach.

Partnership component

The partnership component of the NCPS is based on the recognition that multi-sector partnerships and coalitions of governmental and non-governmental stakeholders will strengthen crime prevention efforts. NCPC has had a mixed record in implementing this component of the Strategy. Its unstable situation since renewal has had an impact on its partnership activity, and its relationship with the provinces and territories, in particular, has suffered. Among NCPC’s provincial/territorial partners, there is a strong belief that the organization has failed to consult them regarding the renewal and has failed to keep them fully abreast of developments over the past two years (although we do report evidence from minutes that show that NCPC provided information on the renewal at FPT meetings). In addition, the provinces and territories object to what they perceive as the federal government’s unilateral decision-making since the renewal. Many believe that their priorities are not being respected in funding decisions, that the federal government has set priorities for the NCPS that they do not share (e.g. , youth gangs, guns, and drugs are not pressing issues in all jurisdictions) and that the federal government has begun to move away from CPSD in favour of other approaches. While these tensions are beginning to ease, many provincial/territorial key informants emphasized that the NCPS is a bilateral strategy and that there is a need for genuine collaboration between the federal and provincial/territorial governments in its delivery.

Recommendation

9. NCPC should continue to work at repairing its relationship with the provinces and territories. Efforts should be made to ensure a common vision of the NCPS, as well as a common understanding of the respective roles and responsibilities of the partners.

In contrast to the strained relationship with the provinces and territories, NCPC has had some success at horizontal policy work within the federal government. NCPC works closely with other units within PSC on joint research and policy planning, and has signed important new agreements with Justice Canada, Statistics Canada, and Heritage Canada, among other departments. Many external partners envision NCPC as the lead organization responsible for coordinating the federal role in crime prevention. Such a role, furthermore, is strongly supported by evidence from the literature on effective crime prevention strategies, which points to the need for integrated action by multiple stakeholders to solve social problems.

Recommendation

10. To enhance the federal government’s response to crime and victimization, NCPC should work to establish itself as the lead agency responsible for coordinating the federal role in crime prevention.

In addition, NCPC’s long-standing relationships with external partners, such as the RCMP and the CACP, have been strengthened, and key informants reported that strong partnerships have developed in some regions. All external partners believe that their organization has goals in common with the NCPS (although some are worried that CPSD is no longer a focus of the Strategy) and that their partnership with NCPC influences their policy development in positive ways. Some, such as the RCMP and the CACP, have incorporated CPSD into their official policy.

Finally, under the partnership component of the NCPS, NCPC is expected to facilitate the formation of partnerships among other organizations. NCPC personnel observed that this has occurred simply as a function of the requirement that all funding applicants identify partners and other funding sources in their proposals. Both GCIMS and file review data confirm that projects funded since the renewal involve multiple partners from a wide range of organizational sectors – nine partners each, on average, for CPAF and PCCF projects. Project partners played diverse roles and came from a diverse range of organizational sectors, including community, social and voluntary services; criminal justice, corrections and police; education; health; and provincial/territorial governments. Based on the available data, however, this does not appear to represent any change over the pre-renewal period, which is perhaps not unexpected given NCPC’s historic emphasis on partnerships.

Knowledge component

Under the knowledge component of the NCPS, NCPC committed to improve efforts to synthesize best practices and lessons learned and to disseminate this information to the crime prevention community and the general public. Some of this work is occurring through projects funded through the RKDF. To date, 12 projects have been funded under this component, although only one (the planning phase of a demonstration project) has yet been completed. NCPC also supports this component of the Strategy by publishing on its website and in hard copy materials specifically devoted to disseminating knowledge on evidence-based practices, providing technical assistance to projects, developing guidance materials for funding applicants and funded projects, and participating in a wide variety of workshops, conferences, training sessions, presentations, and other activities related to the production and dissemination of knowledge. A knowledge function composed of five FTEs within the Research and Knowledge Unit has been created to implement and coordinate knowledge activities.

Nevertheless, the evidence from this evaluation suggests that the knowledge component is an area of weakness for NCPC. Among key informants, only a few of NCPC’s external and provincial/territorial partners had ever used or heard of NCPC knowledge products, and many of those who had used them found them impractical, too limited or superficial to be of real value, or culturally or linguistically inappropriate in their jurisdictions. On the other hand, many of NCPC’s partners noted that the NCPC’s research and knowledge capacity is its greatest strength, and suggested that it focus on becoming a centre of excellence for evidence-based crime prevention. NCPC key informants view the synthesis and dissemination of evidence-based knowledge as a major challenge, particularly given current staffing levels, and suggested that part of the solution lies in improved linkages between the organization’s Programs and Policy, Research and Evaluation divisions.

Recommendation

11. NCPC should continue to develop its knowledge component, with a view to becoming a recognized source for evidence-based crime prevention information in Canada.

Organization and resources

The resource reductions that NCPC has experienced over the past two years have profoundly hampered its ability to implement and achieve the objectives of the renewed NCPS. The pause severely curtailed the number of projects that were funded and delayed others; in total, some 150 projects were affected. Because NCPC did not communicate the changing program parameters to communities in a timely fashion, both NCPC and the provinces and territories, as delivery agents of the Strategy, lost credibility with community organizations. At the same time, commitments to increased use of contribution agreements, increased targeting of funding, integration of evidence-based interventions, and an enhanced knowledge function were made even though staff capacity to implement these changes was reduced. As a consequence, NCPC has suffered from high turnover in staff and senior leadership since the renewal, further diminishing its ability to deliver on the mandate of the renewed NCPS.

Many key informants observed that these developments forced NCPC to rethink its mandate and to emerge better positioned to achieve the objectives of the renewed NCPS. Nevertheless, there remain ongoing issues with respect to organizational structure, delivery structure, and resources. First, the current organizational structure was created in response to external imperatives, without any strategic analysis of the structure and resources necessary to deliver on the renewed NCPS mandate. There is evidence, for example, that staffing at the regional level may be inadequate, particularly given increased demands for project planning and accountability. There is also evidence that, although the relationship between the Programs and Policy, Research and Evaluation Divisions has greatly improved in recent months as a result of NCPC’s new senior leadership, the organization still lacks a mechanism for integrating policy, research, and evaluation with program delivery, especially at the regional level. As a result, the regions, which are responsible for recommending projects for funding, have no mechanism for systematically accessing information on best practices and evidence-based crime prevention. An organizational analysis examining such questions as staffing levels and allocation among the national and regional offices, the role of different NCPC units at each stage of the project cycle from solicitation to evaluation, and the structures that need to be in place to enable NCPC to fulfill the two core activities of its mandate since the repositioning (supporting targeted interventions and building and sharing practical knowledge), would help to identify any necessary modifications to the current structure.

Recommendation

12. NCPC should undertake a strategic analysis of its organizational structure in order to identify any modifications that may be necessary to enable it to deliver on the mandate of the renewed NCPS. The analysis should examine questions such as staffing levels and allocation of staff among the national and regional offices; the role of the various NCPC units at each stage of the project cycle from solicitation to evaluation; and the structures that need to be in place to enable the organization to fulfill its mandate.

Second, while tensions between the provinces and territories are beginning to ease, the strained relationship has by no means been completely repaired. In addition to ongoing uncertainty over federal and provincial/territorial roles and responsibilities in the NCPS, there is also uncertainty about the roles and responsibilities of the FPT Working Group. Indeed, it is not clear if the Working Group has a policy or an operational focus, and its membership is quite diverse in rank and background. The resulting mixture has not produced a particularly cohesive or effective group. Key informants agreed that there is a need to refine and refocus the mandate of the Working Group to ensure that it can be an effective agent in delivering the renewed NCPS.

Recommendation

13. Together with its provincial/territorial partners, NCPC should refine and focus the mandate of the FPT Working Group and clearly articulate its roles and responsibilities with respect to the renewed NCPS. The membership of the Working Group should be composed of individuals of similar background and rank within their organizations. Ideally, members should have decision-making authority within their own organizations.

Performance measurement

Under the renewed NCPS, NCPC committed to implementing an enhanced performance measurement system and has taken several steps toward implementing this system, including developing standard application forms and reporting templates for each funding program, requiring all funded projects to provide final reports and conduct evaluations, providing evaluation training and developing evaluation resource materials for NCPC staff and funding recipients, enhancing the electronic GCIMS system, and developing performance measurement templates to track activity under the partnership and knowledge components of the NCPS.

At present, the main challenge for performance measurement is related to GCIMS, which is intended to capture information collected by the standard forms used by the funding programs. While development and data entry have been underway since 2004/2005, the database is not yet capable of generating reports or data to support timely project management, performance monitoring, or impact assessment. NCPC has developed an Action Plan to address the main obstacles to full implementation of GCIMS, namely inconsistent data entry and incomplete development of the database. However, from an evaluation perspective, there are two additional issues with respect to GCIMS. First, there is evidence of poorly defined fields and attributes within the database, leaving substantial opportunity for subjective judgment in data entry that will ultimately affect the quality and validity of the data (although it is true that these issues are currently being addressed by NCPC). Second, although GCIMS is an information management system, it is nevertheless capturing primarily qualitative data; the task at hand is to design a database capable of ensuring that these qualitative data can be quantified, and that they will be meaningful once quantified. At the same time, NCPC should not rely solely on a database solution for performance measurement and evaluation purposes and overlook the value of other qualitative approaches, such as in-depth case studies, especially for illustrating program impact. Based on the evaluator’s limited exposure to GCIMS, further work on GCIMS would benefit from the expertise of specialists in database development, performance measurement, and evaluation.

Recommendation

14. In order to ensure that GCIMS will ultimately be capable of generating useful, valid, and reliable data, NCPC should consider involving additional expertise in database development, performance measurement, and evaluation to assist with its development and implementation.

Recommendation

15. Rather than relying solely on a database solution that focuses on capturing and summarizing the information collected on the standard forms, NCPC should consider a diversified performance measurement strategy that also includes objective qualitative methods, such as in-depth case studies, to illuminate outcomes achieved.

Impact

Given the relative infancy of the renewed NCPS, impact is difficult to discern at this time. Only about 20% of projects funded since April 2005 have concluded and GCIMS does not yet capture information on project outcomes. Consequently, data on impact is very limited.

Nevertheless, the evaluation found some preliminary indications of positive impacts, particularly in the area of increased local capacity to prevent crime and victimization. About 70% of funded organizations, for example, agreed that NCPC funding has improved their organization’s and their community’s capacity to prevent crime and victimization. Among key informants, many agreed that the objective of increased local capacity has been accomplished or that substantial progress has been made in this direction, particularly since increased community capacity has been a focus of the NCPS for some years, not only since renewal. However, while some key informants believe that the time has come for the NCPS to focus on more concrete outcomes, others believe that increasing local capacity should continue to be a goal of the Strategy, given that many communities (particularly Aboriginal communities) continue to struggle to plan and implement crime prevention initiatives.

There is also some evidence of improved comprehensive partnerships under the renewed NCPS. GCIMS and file review data show that funded projects involve multiple partners (nine each, on average) from a wide range of organizational sectors including, most commonly, community, social and voluntary services; criminal justice, corrections and police; education; health; and provincial/territorial governments. Moreover, 78% of funded organizations agreed that their organization has developed partnerships with organizations whose involvement was necessary to achieving their project objectives as a result of NCPC funding or technical assistance, and 59% agreed that NCPC funding helped their organization to leverage other sources of funding.

As for NCPC’s own partnership activity, positive developments include numerous new horizontal partnerships with other federal departments and a strengthening of existing partnerships with the RCMP and the CACP, both of which have incorporated CPSD into their official policy. However, in the evaluator’s opinion, the current expected outcomes pertaining to the partnership component of the renewed NCPS are more accurately described as “outputs,” since they seem to imply that simply establishing partnerships is, by itself, sufficient to achieve crime prevention objectives. That being said, it should be acknowledged that the revised logic model for the NCPS, currently being drafted, appears to address this issue.

Recommendation

16. In revising the logic model for the NCPS, NCPC should take care to ensure that all expected outcomes articulate what is expected to change as a result of NCPS activities and outputs and do not merely reflect those activities or outputs.

Finally, the evaluation found very preliminary evidence of progress in integrating evidence-based crime prevention into policies and practices. Key informants believe that there is greater awareness among community organizations of evidence-based crime prevention, and indeed 62% of funded organizations agreed that their organization has a better understanding of evidence-based crime prevention strategies as a result of NCPC funding and technical assistance. As mentioned earlier, this component of the renewed NCPS is an acknowledged area of weakness, and one that will require greater effort in the future. Projects funded through the RKDF should help to improve knowledge production and dissemination.

Effectiveness/alternatives

Like program impact, it is somewhat early to assess whether there are more effective ways of achieving the objectives of the renewed NCPS. That being said, the evaluation found considerable support in the literature for various features of the NCPS, including involvement of national government in crime prevention; a whole government approach to crime prevention that engages multiple stakeholders to deliver integrated solutions to problems; and evidence-based crime prevention. Among key informants, there is also general agreement that these are desirable features of an effective crime prevention strategy. That being said, some key informants expressed caveats. Provincial/territorial representatives, for example, cautioned that a strong federal role should not mean that provincial/territorial interests and priorities are subsumed or ignored. Other key informants argued that emphasis on evidence-based interventions should not mean forgoing innovative but untested initiatives, particularly where there is a dearth of evidence on “what works” (as is the case for Aboriginal populations who are judged to be particularly high risk and therefore most in need of intervention). Indeed, funding and evaluating previously untested crime prevention interventions is arguably one important way in which NCPC might fulfill its commitment to producing evidence-based knowledge.

Recommendation

17. While there is considerable support in the international literature for implementing evidence-based approaches to crime prevention, NCPC should ensure that innovative interventions continue to be funded, particularly for communities or populations where evidence may be lacking. Furthermore, such interventions should be evaluated in order to contribute to the evidence base.

The literature review also highlighted the importance of effective implementation to successful crime prevention initiatives. As has been emphasized throughout this report, effective implementation of the renewed Strategy has been hampered by significant reductions to NCPC resources over the past two years. Given the context, it should not come as a surprise that evidence of impact is limited at this stage. However, based on the literature it appears that the basic elements are in place for success in the future, provided that the outstanding implementation issues identified in this evaluation are addressed. Among NCPC key informants there is considerable agreement that if NCPC implements its new mandate, focusing on a more limited number of targeted interventions, both the effectiveness and the efficiency of the renewed NCPS will improve dramatically.

Relevance

The evaluation found the renewed NCPS to be closely aligned with the strategic objectives of PSC as well as federal priorities. Among key informants as well as funded organizations, there is a broad consensus that the federal government should continue to play a role in crime prevention and that the NCPS is still relevant to crime prevention in Canada. In fact, many stakeholders believe that if the NCPS did not exist, the consequences for crime prevention would be devastating. Two-thirds of survey respondents said that in the absence of the NCPS, their own crime prevention work would be severely limited or altogether curtailed, and about one-fifth said the outcomes they have achieved would not have been realized. A similar proportion predicted an increase in crime and victimization in the absence of the Strategy.

Despite general agreement on the relevance of the NCPS, some key informants are unconvinced that the changes that have been introduced to ensure alignment with departmental and federal priorities are appropriate. In particular, they object to what they perceive to be a reduced emphasis on community capacity building, a focus on selected issues that are not relevant in all communities or jurisdictions, and a perceived move away from CPSD. These key informants believe that to be truly relevant, as well as effective in the long term, the NCPS must permit communities and jurisdictions to focus on local and provincial/territorial issues and priorities.

Recommendation

18. NCPC should clarify for all stakeholders both whether and the extent to which community capacity building, locally identified issues, and CPSD continue to be elements of the renewed NCPS.

Appendix A: Evaluation Framework

Appendix B: Data Collection Instruments

Evaluation of the renewed National Crime Prevention Strategy File Review Template for CPAF

1. NCPC file number: __________________________

2. Fiscal year: _____________________________

3. Funding type:    ±₁ Grant    ±₂ Contribution    ±₈ Can't determine

4. Duration of initiative:    # of months: __________    ±₈ Can't determine

5. CPAF goals to be addressed by project. (AF Q14)

• ±₁ Enhance the capacities of diverse communities to decrease risk factors and increase protective factors related to crime and victimization
• ±₂ Promote cross-sectoral collaboration in crime prevention, with particular emphasis on engaging the voluntary and private sectors and participating in horizontal initiatives
• ±₃ Develop, disseminate and transfer information, tools and resources, including best practices and evidence of what works in crime prevention
• ±₄ Support the development of comprehensive community initiatives to address crime and victimization issues
• ±₈ Can’t determine/no CPAF goals identified

6. Crime/victimization issue(s) to be addressed. (AF Q16)

• ±₀₁ Abuse (elder, emotional, psychological, or any other type if not otherwise specified)
• ±₀₂ Bullying
• ±₀₃ Child abuse
• ±₀₄ Drug/alcohol related crime
• ±₀₅ Family violence
• ±₀₆ Fear of crime
• ±₀₇ Gang criminal activity
• ±₀₈ Gun crime
• ±₀₉ Hate/bias crime
• ±₁₀ Property crime
• ±₁₁ Sexual exploitation
• ±₁₂ Violence
• ±₆₆ Other (specify): ____________________________
• ±₈₈ Can’t determine/no issues identified

7. What evidence was cited to indicate a need for the project? (AF Q17)

• ±₀₁ Current crime/victimization statistics related to the issue being addressed
• ±₀₂ Findings from community needs assessment (e.g. , statistics on risk factors, interviews with stakeholders, data on service provision and gaps)
• ±₀₃ Particular incident(s) in the community that has generated concern
• ±₆₆ Other (specify): ___________________________________________
• ±₈₈ Can’t determine/no evidence cited

8. Primary risk/protective factors to be addressed. (AF Q18)

• ±₈₈ Can’t determine/no factors identified

Individual level

• ±₀₁ Alcohol and drug abuse
• ±₀₂ Aggression
• ±₀₃ Anti-social attitudes
• ±₀₄ Desire for rewards
• ±₀₅ Early/precocious sexual activity
• ±₀₆ Lack of life/employment skills
• ±₆₆ Other individual (specify):

Peer group

• ±₀₁ Alcohol/drug abuse among peers
• ±₀₂ Association with/commitment to delinquent peers
• ±₆₆ Other peer (specify):

Family

• ±₀₁ Alcohol/drug abuse in family
• ±₀₂ Family disorganization
• ±₀₃ Abuse/neglect
• ±₀₄ Delinquent/criminal behaviour in family
• ±₀₅ Parenting practices
• ±₆₆ Other family (specify):

School

• ±₀₁ Poor school performance
• ±₀₂ Low educational aspirations
• ±₀₃ Learning difficulties
• ±₀₄ Negative labelling
• ±₀₅ Negative school climate
• ±₆₆ Other school (specify):

Community

• ±₀₁ Availability of drugs
• ±₀₂ Availability of firearms
• ±₀₃ Feeling unsafe in neighbourhood
• ±₀₄ Poverty
• ±₀₅ Social disorganization (low social control, mobility, visible crime and neglect)
• ±₆₆ Other community (specify): ____________________________

9. Stated overall goals of project. (AF Q28)    ±₈ Can’t determine/none identified

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

10. Project activities. (AF Q29)

• ±₀₁ Community mobilization and partnership development (includes activities aimed at bringing stakeholders together to learn about, plan, and take action on a problem)
• ±₀₂ Social-recreational activities
• ±₀₃ Training (not of beneficiaries but of those who will work with them, e.g. , teachers, police, etc. )
• ±₀₄ Resource development (e.g. , production of videos, books, manuals, websites, CDs, etc. )
• ±₀₅ Research/evaluation
• ±₀₆ Social intervention (includes counselling, workshops, outreach, support groups for the target group)
• ±₀₇ Public awareness and education (broad-based marketing campaigns)
• ±₀₈ Arts programming (bringing target group members together to produce and perform a play, develop a TV show, art exhibit, etc)
• ±₀₉ Program development (would apply to a phase one project, where a project is planning a program, developing a curriculum, etc. )
• ±₁₀ Cultural activities
• ±₆₆ Other (specify):
• ±₈₈ Can’t determine/activities not identified

11. Planned outcomes of project. (AF Q29)    ±₈ Can’t determine/planned outcomes not identified

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

12. Number of project partners. (AF Q31)    # of partners __________    ±₈ Can't determine

13. Roles of project partners. (AF Q31)

• # of partners with communication role: _________
• # of partners with cooperation role:____________
• # of partners with coordination role: ___________
• # of partners with collaboration role: ___________

• ±₈ Can't determine partner roles

14. Types of partners. (AF Q31)

• ±₀₁ Aboriginal agency or organization
• ±₀₂ Aboriginal government or Tribal/Band Council
• ±₀₃ Arts and Culture
• ±₀₄ Business
• ±₀₅ Community, social or voluntary services
• ±₀₆ Criminal justice, Corrections, Police
• ±₀₇ Education
• ±₀₈ Employment
• ±₀₉ Government – local, municipal
• ±₁₀ Government – provincial, territorial
• ±₁₁ Government – federal
• ±₁₂ Housing Services
• ±₁₃ Health
• ±₁₄ Parks and Recreation
• ±₁₅ Religious/faith
• ±₁₆ Service Club
• ±₆₆ Other (specify):
• ±₈₈ Can't determine types of partners

15. (For contributions only) Does the file include any of the following interim deliverables?

• ±₁ Evaluation plan
• ±₂ Activity report(s)
• ±₀ No interim deliverables
• ±₆₆ Other (specify):

16. Has a final report been submitted (using either the Final Report Template or another format)?

• ±₁ Yes
• ±₀ No (Go to Q29)
• ±₈ Can't determine (Go to Q29)

17. Was the Final Report Template used?

• ±₁ Yes
• ±₀ No
• ±₈ Can't determine

18. Actual project outcomes. (FR Q13)    ±₈ Can't determine/no actual outcomes reported

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

19. Did the project report unexpected outcomes? (FR Q15)

• ±₁ Yes, positive
• ±₂ Yes, negative
• ±₀ No
• ±₈ Can't determine

20. If yes, describe any unexpected outcomes. (FR Q15)

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

21. Did the project experience any challenges? (FR Q16)

• ±₁ Yes
• ±₀ No
• ±₈ Can't determine

22. If yes, describe the challenges experienced. (FR Q16)

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

23. Products, tools or resources produced. (FR)

• ±₀₁ Brochure, pamphlet, flyer or poster
• ±₀₂ CD-ROM, DVD, video, film
• ±₀₃ Coalition, network or other group
• ±₀₄ Community action plan or strategic plan
• ±₀₅ Conference/symposium/workshop
• ±₀₆ Curriculum
• ±₀₇ Database
• ±₀₈ Manual or guide
• ±₀₉ Internet tool, resource or website
• ±₁₀ Report, study, conference proceedings or paper
• ±₁₁ RV or radio ad, promo
• ±₆₆ Other (specify):
• ±₈₈ Can't determine

24. What lessons learned did the project identify? (FR Q19)    ±₈ Can’t determine/none identified

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

25. Reported contribution to CPAF objectives. (FR Q20-23)

1. Positive impact on risk and protective factors related to crime and victimization.
   • ±₁ To no degree
   • ±₂ To some degree
   • ±₃ To a great degree
   • ±₇ Not applicable
   • ±₈ Can't determine
2. Promoted cross-sector collaboration in crime prevention.
   • ±₁ To no degree
   • ±₂ To some degree
   • ±₃ To a great degree
   • ±₇ Not applicable
   • ±₈ Can't determine
3. Supported development, transfer, and dissemination of information, etc.
   • ±₁ To no degree
   • ±₂ To some degree
   • ±₃ To a great degree
   • ±₇ Not applicable
   • ±₈ Can't determine
4. Supported development of/part of comprehensive community initiative.
   • ±₁ Yes
   • ±₀ No
   • ±₈ Can't determine

26. Were the proposed partnerships sustained throughout the project? (compare AF Q31 to FR Q24)

• ±₁ Yes
• ±₀ No
• ±₈ Can't determine

27. Does the FR indicate plans to continue project activities/achieve results in other ways? (FR Q27)

• ±₁ Yes
• ±₀ No
• ±₈ Can't determine

28. Does the FR indicate that other funding sources/ways to cover costs have been found? (FR Q28)

• ±₁ Yes
• ±₀ No
• ±₈ Can't determine

29. Was a Final Evaluation Report submitted?

• ±₁ Yes
• ±₀ No
• ±₈ Can't determine

30. Based on the file, did the project receive any of the following types of assistance from NCPC staff? (This information will be contained in emails, letters, and other correspondence between the project and NCPC. )

• ±₀₁ General information about NCPC, NCPC funding programs, or crime prevention through social development
• ±₀₂ Referral to/provision of relevant research, resources, documents and tools (e.g. , Applicant’s Guide. , websites, research reports, NCPC products)
• ±₀₃ Assistance with developing project idea
• ±₀₄ Assistance with developing project budget
• ±₀₅ Assistance with developing project evaluation plan
• ±₀₆ Assistance with developing project work plan
• ±₀₇ Feedback on/suggestions for improving other aspects of draft application
• ±₀₈ Referrals/references to other potential funding sources
• ±₀₉ Assistance with finding partners for project
• ±₁₀ Project management/delivery advice
• ±₁₁ Conflict resolution
• ±₁₂ Connecting organization with other groups with similar ideas
• ±₁₃ Promoting project activities/results across communities
• ±₆₆ Other (specify):
• ±₈₈ Can't determine

31. Based on the file, did the project experience any of the following difficulties?

• ±₀₀ No difficulties
• ±₀₁ Difficulty developing an acceptable proposal (e.g. , more than two drafts)
• ±₀₂ Difficulty meeting final reporting requirements (e.g. , more than two drafts)
• ±₀₃ Difficulty meeting project evaluation requirements
• ±₀₄ Difficulty meeting timelines for interim/final reports
• ±₆₆ Other (specify):__________________________

Please explain any difficulties experienced by the project.

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

32. Note any difficulties with reviewing the file.

• ±₀₀ No difficulties
• ±₀₁ Missing documents
• ±₀₂ Inconsistent information among documents
• ±₀₃ Some information could not be located because file was transferred from CMP to CPAF
• ±₆₆ Other (specify):_____________________

Please explain any difficulties with reviewing the file.

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

Evaluation of the renewed National Crime Prevention Strategy File Review Template for PCCF

1. NCPC file number: __________________________

2. Fiscal year: _____________________________

3. Funding type:    ±₁ Grant    ±₂ Contribution    ±₈ Can't determine

4. Duration of initiative:    # of months: ________________    ±₈ Can't determine

5. Crime/victimization issue(s) to be addressed. (P)

• ±₀₁ Abuse (elder, emotional, psychological, or any other type if not otherwise specified)
• ±₀₂ Bullying
• ±₀₃ Child abuse
• ±₀₄ Drug/alcohol related crime
• ±₀₅ Family violence
• ±₀₆ Fear of crime
• ±₀₇ Gang criminal activity
• ±₀₈ Gun crime
• ±₀₉ Hate/bias crime
• ±₁₀ Property crime
• ±₁₁ Sexual exploitation
• ±₁₂ Violence
• ±₆₆ Other (specify): ___________________
• ±₈₈ Can’t determine/no issues identified

6. What evidence was cited to indicate a need for the project? (P)

• ±₀₁ Current crime/victimization statistics related to the issue being addressed
• ±₀₂ Findings from community needs assessment (e.g. , statistics on risk factors, interviews with stakeholders, data on service provision and gaps)
• ±₃₃ Particular incident(s) in the community that has generated concern
• ±₆₆ Other (specify): ____________________________
• ±₈₈ Can’t determine/no evidence cited

7. Primary risk/protective factors to be addressed. (P)

• ±₈₈ Can’t determine/no factors identified

Individual level

• ±₀₁ Alcohol and drug abuse
• ±₀₂ Aggression
• ±₀₃ Anti-social attitudes
• ±₀₄ Desire for rewards
• ±₀₅ Early/precocious sexual activity
• ±₀₆ Lack of life/employment skills
• ±₆₆ Other individual (specify):

Peer group

• ±₀₁ Alcohol/drug abuse among peers
• ±₀₂ Association with/commitment to delinquent peers
• ±₆₆ Other peer (specify):

Family

• ±₀₁ Alcohol/drug abuse in family
• ±₀₂ Family disorganization
• ±₀₃ Abuse/neglect
• ±₀₄ Delinquent/criminal behaviour in family
• ±₀₅ Parenting practices
• ±₆₆ Other family (specify):

School

• ±₀₁ Poor school performance
• ±₀₂ Low educational aspirations
• ±₀₃ Learning difficulties
• ±₀₄ Negative labelling
• ±₀₅ Negative school climate
• ±₆₆ Other school (specify):

Community

• ±₀₁ Availability of drugs
• ±₀₂ Availability of firearms
• ±₀₃ Feeling unsafe in neighbourhood
• ±₀₄ Poverty
• ±₀₅ Social disorganization (low social control, mobility, visible crime and neglect)
• ±₆₆ Other school (specify): ____________________________________

8. Stated overall goals of project. (P)    ±₈ Can’t determine/none identified

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

9. Project activities. (P)

• ±₀₁ Community mobilization and partnership development (includes activities aimed at bringing stakeholders together to learn about, plan, and take action on a problem)
• ±₀₂ Social-recreational activities
• ±₀₃ Training (not of beneficiaries but of those who will work with them, e.g. , teachers, police, etc. )
• ±₀₄ Resource development (e.g. , production of videos, books, manuals, websites, CDs, etc. )
• ±₀₅ Research/evaluation
• ±₀₆ Social intervention (includes counselling, workshops, outreach, support groups for the target group)
• ±₀₇ Public awareness and education (broad-based marketing campaigns)
• ±₀₈ Arts programming (bringing target group members together to produce and perform a play, develop a TV show, art exhibit, etc)
• ±₀₉ Program development (would apply to a phase one project, where a project is planning a program, developing a curriculum, etc. )
• ±₁₀ Cultural activities
• ±₆₆ Other (specify): _______________________________
• ±₈₈ Can’t determine/activities not identified

10. Expected results of project. (P)    ±₈ Can’t determine/expected results not identified

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

11. Number of project partners (P):    # of partners ________________    ±₈ Can't determine

12. Types of partners. (P)

• ±₀₁ Aboriginal agency or organization
• ±₀₂ Aboriginal government or Tribal/Band Council
• ±₀₃ Arts and Culture
• ±₀₄ Business
• ±₀₅ Community, social or voluntary services
• ±₀₆ Criminal justice, Corrections, Police
• ±₀₇ Education
• ±₀₈ Employment
• ±₀₉ Government – local, municipal
• ±₁₀ Government – provincial, territorial
• ±₁₁ Government – federal
• ±₁₂ Housing Services
• ±₁₃ Health
• ±₁₄ Parks and Recreation
• ±₁₅ Religious/faith
• ±₁₆ Service Club
• ±₆₆ Other (specify):
• ±₈₈ Can't determine types of partners

13. (For contributions only) Does the file include any of the following interim deliverables?

• ±₁ Evaluation plan
• ±₂ Activity report(s)
• ±₀ No interim deliverables
• ±₆₆ Other (specify):

14. Has a final report been submitted?

• ±₁ Yes
• ±₀ No (Go to Q26)
• ±₈ Can't determine (Go to Q26)

15. Actual project outcomes. (FR)    ±₈ Can't determine/no actual outcomes reported

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

16. Did the project report unexpected outcomes? (FR)

• ±₁ Yes, positive
• ±₂ Yes, negative
• ±₀ No
• ±₈ Can't determine

17. If yes, describe any unexpected outcomes. (FR)

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

18. Did the project experience any challenges? (FR)

• ±₁ Yes
• ±₀ No
• ±₈ Can't determine

19. If yes, describe the challenges experienced. (FR)

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

20. Products, tools or resources produced. (FR)

• ±₀₁ Brochure, pamphlet, flyer or poster
• ±₀₂ CD-ROM, DVD, video, film
• ±₀₃ Coalition, network or other group
• ±₀₄ Community action plan or strategic plan
• ±₀₅ Conference/symposium/workshop
• ±₀₆ Curriculum
• ±₀₇ Database
• ±₀₈ Manual or guide
• ±₀₉ Internet tool, resource or website
• ±₁₀ Report, study, conference proceedings or paper
• ±₁₁ RV or radio ad, promo
• ±₆₆ Other (specify):
• ±₈₈ Can't determine

21. What lessons learned did the project identify? (FR)    ±₈ Can’t determine/none identified

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

22. Reported contribution to PCCF objectives (FR)

1. Develop partnerships, tools and knowledge of Crime Prevention through Social Development, etc.
   • ±₁ To no degree
   • ±₂ To some degree
   • ±₃ To a great degree
   • ±₇ Not applicable
   • ±₈ Can't determine
2. Developing and implementing a demonstration project(s) targeting at-risk communities/groups, etc.
   • ±₁ Yes
   • ±₀ No
   • ±₈ Can't determine
3. Support families of Aboriginal offenders to prevent cycle of institutionalization, etc.
   • ±₁ To no degree
   • ±₂ To some degree
   • ±₃ To a great degree
   • ±₇ Not applicable
   • ±₈ Can't determine
4. Involve developing, implementing and evaluating comprehensive models of crime prevention, etc.
   • ±₁ Yes
   • ±₀ No
   • ±₈ Can't determine
5. Strengthen partnerships between community corrections and other crime prevention initiatives, etc.
   • ±₁ Yes
   • ±₀ No
   • ±₃ To a great degree
   • ±₇ Not applicable
   • ±₈ Can't determine

23. Were the proposed partnerships sustained throughout the project? (compare P and FR)

• ±₁ Yes
• ±₀ No
• ±₈ Can't determine

24. Does the FR indicate plans to continue project activities/achieve results in other ways? (FR)

• ±₁ Yes
• ±₀ No
• ±₈ Can't determine

25. Does the FR indicate that other funding sources/ways to cover costs have been found? (FR)

• ±₁ Yes
• ±₀ No
• ±₈ Can't determine

26. Was a Final Evaluation Report submitted?

• ±₁ Yes
• ±₀ No
• ±₈ Can't determine

27. Based on the file, did the project receive any of the following types of assistance from NCPC, APD, or ACPU staff? (This information will be contained in emails, letters, and other correspondence between the project and NCPC. )

• ±₀₁ General information about NCPC, NCPC funding programs, or crime prevention through social development
• ±₀₂ Referral to/provision of relevant research, resources, documents and tools (e.g. , Applicant’s Guide. , websites, research reports, NCPC products)
• ±₀₃ Assistance with developing project idea
• ±₀₄ Assistance with developing project budget
• ±₀₅ Assistance with developing project evaluation plan
• ±₀₆ Assistance with developing project work plan
• ±₀₇ Feedback on/suggestions for improving other aspects of draft application
• ±₀₈ Referrals/references to other potential funding sources
• ±₀₉ Assistance with finding partners for project
• ±₁₀ Project management/delivery advice
• ±₁₁ Conflict resolution
• ±₁₂ Connecting organization with other groups with similar ideas
• ±₁₃ Promoting project activities/results across communities
• ±₆₆ Other (specify):
• ±₈₈ Can't determine

28. Based on the file, did the project experience any of the following difficulties?

• ±₀₀ No difficulties
• ±₀₁ Difficulty developing an acceptable proposal (e.g. , more than 2 drafts)
• ±₀₂ Difficulty meeting final reporting requirements (e.g. , numerous drafts)
• ±₀₃ Difficulty meeting project evaluation requirements
• ±₀₄ Difficulty meeting timelines for interim/final reports
• ±₆₆ Other (specify):

Please explain any difficulties experienced by the project.

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

29. Note any difficulties with reviewing the file.

• ±₀₀ No difficulties
• ±₀₁ Missing documents
• ±₀₂ Inconsistent information among documents
• ±₀₃ File transferred from another funding program; some information difficult to locate
• ±₆₆ Other (specify):

Please explain any difficulties with reviewing the file.

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

Evaluation of the renewed National Crime Prevention Strategy File Review Template for RKDF

1. NCPC file number: __________________________

2. Fiscal year: _____________________________

3. Duration of initiative:    # of months: ______________    ±₈ Can't determine

4. Type of project. (AF)

• ±₁ Research project
• ±₂ Demonstration project
• ±₃ Knowledge transfer project
• ±₈ Can’t determine

5. If demonstration project, check type of funding (AF):

• ±₁ Developmental funding
• ±₂ Implementation funding
• ±₈ Can’t determine

6. Crime/victimization issue(s) to be addressed. (P, MPS):

• ±₀₁ Abuse (elder, emotional, psychological, or any other type if not otherwise specified)
• ±₀₂ Bullying
• ±₀₃ Child abuse
• ±₀₄ Drug/alcohol related crime
• ±₀₅ Family violence
• ±₀₆ Fear of crime
• ±₀₇ Gang criminal activity
• ±₀₈ Gun crime
• ±₀₉ Hate/bias crime
• ±₁₀ Property crime
• ±₁₁ Sexual exploitation
• ±₁₂ Violence
• ±₆₆ Other (specify):
• ±₈₈ Can’t determine/no issues identified

7. How do reviewers assess the importance/significance of these issues? (RAF)

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

8. What evidence or rationale is given to demonstrate a need for the project? (P, MPS)

• ±₀₁ Current crime/victimization statistics related to the issue being addressed
• ±₀₂ Findings from community needs assessment (e.g. , statistics on risk factors, interviews with stakeholders, data on service provision and gaps)
• ±₀₃ Particular incident(s) in the community that has generated concern
• ±₆₆ Other (specify): ____________________________
• ±₈₈ Can’t determine/no evidence cited

9. How do reviewers assess the justification provided by the project? (RAF)

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

10. How does the project expect to contribute to the knowledge base on crime prevention/inform crime prevention policy and practice? (P, MPS)

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

11. How do reviewers rate the project’s likely contribution? (RAF)

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

12. What are the stated overall goals/objectives of the project? (P, MPS)

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

13. What are the planned outcomes of the project? (P, MPS)

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

14. Based on the file, did the project receive any of the following types of assistance from NCPC staff? (This information will be contained in emails, letters, and other correspondence between the project and NCPC. )

• ±₀₁ General information about NCPC, NCPC funding programs, or crime prevention through social development
• ±₀₂ Referral to/provision of relevant research, resources, documents and tools (e.g. , Applicant’s Guide. , websites, research reports, NCPC products)
• ±₀₃ Assistance with developing project idea
• ±₀₄ Assistance with developing project budget
• ±₀₅ Assistance with developing project evaluation plan
• ±₀₆ Assistance with developing project work plan
• ±₀₇ Feedback on/suggestions for improving other aspects of draft application
• ±₀₈ Referrals/references to other potential funding sources
• ±₀₉ Assistance with finding partners for project
• ±₁₀ Project management/delivery advice
• ±₁₁ Conflict resolution
• ±₁₂ Connecting organization with other groups with similar ideas
• ±₁₃ Promoting project activities/results across communities
• ±₆₆ Other (specify):
• ±₈₈ Can't determine

15. Based on the file, did the project experience any of the following difficulties?

• ±₀₀ No difficulties
• ±₀₁ Difficulty developing an acceptable proposal (e.g. , more than two drafts)
• ±₀₂ Difficulty meeting interim reporting requirements (e.g. , more than two drafts)
• ±₀₂ Difficulty meeting timelines for interim reports
• ±₆₆ Other (specify):

Please explain any difficulties experienced by the project.

16. Note any difficulties with reviewing the file.

• ±₀₀ No difficulties
• ±₀₁ Missing documents
• ±₀₂ Inconsistent information among documents
• ±₀₃ File transferred from another funding program; some information difficult to locate
• ±₆₆ Other (specify):

Please explain any difficulties with reviewing the file.

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

Interim Evaluation of the Renewed National Crime Prevention Strategy Interview Guide for NCPC Personnel

Background

1. Please tell me about your involvement with the National Crime Prevention Centre and the NCPS. (Probe: What is your role within NCPC? How long have you been involved with the organization? With crime prevention policy and programming more broadly?)

Design and delivery

2. In your opinion, are the new funding programs designed in a way that helps the NCPS achieve its objectives? Are they delivered in a way that helps the NCPS achieve its objectives? Please explain.

3. What changes have you observed to the proposal solicitation, development, assessment, and approval process since the renewal of the NCPS? From your perspective, will these changes make this process more efficient? Why or why not? Will they improve NCPC’s ability to meet its objectives? Why or why not?

4. Is there evidence that the technical assistance provided by NCPC staff helps projects to move toward an evidence-based approach? Can you provide some specific examples?

5. Overall, how active has NCPC been at developing and promoting partnerships? Can you give some specific examples of how NCPC has:

• sought new partnerships?
• facilitated the formation of new partnerships among other organizations?
• strengthened its existing partnerships?
• coordinated crime prevention efforts among stakeholders?

6. Is the organizational and delivery structure of the renewed NCPS appropriate? (Probe: the relationship between Programs and Policy, Research and Evaluation within NCPC; the regional delivery structure; the respective roles and responsibilities of the federal government and the provinces/territories). If not, what changes are required?

7. Is NCPC facing any resource challenges that are affecting its ability to achieve its objectives? (Probe: available funding, ratio of grants and contributions funds to operating funds, organizational structure/staffing, Departmental supports in areas of Human Resources, Finances, Contracting Services) Please describe any challenges you are aware of, and their impact. Do you have any documentation that you could share with us?

8. Since April 2005, NCPC has experienced an internal reorganization, cuts to funding, and a PSEPC-wide “pause” in funding. What impact have these events had on its activities and its ability to implement the renewed NCPS? Do you have any documentation that you could share with us?

9. In your opinion, does the new performance measurement strategy:

• provide timely project management information?
• assist NCPC in ongoing performance monitoring and program adjustment?
• capture adequate and useful information for assessing program impact?

10. What challenges remain with respect to performance measurement? What changes are necessary to enhance NCPC’s ability to report on results?

11. NCPC is currently undertaking a “repositioning” exercise. What is being repositioned? What are the purpose and main elements? What concrete steps is NCPC taking to implement the new approach?

12. Have NCPC’s stakeholders been made aware of the new approach? Based on what you have observed so far, do they support it?

Impact

13. The expected outcomes of the renewed NCPS are:

• increased local capacity to prevent crime and victimization
• improved comprehensive and strategic partnerships in crime prevention
• improved integration of evidence-based crime prevention into policies and practices

Can you describe some examples of changes in these areas?

14. Has the renewed NCPS had any unexpected impacts, either positive or negative?

Cost-effectiveness/Alternatives

15. In your opinion, are there more effective ways of achieving the objectives of the renewed NCPS? What evidence is there that these alternatives are more effective?

16. If the renewed NCPS continues, how could its efficiency be improved?

17. Do you have any other comments?

Interim Evaluation of the Renewed National Crime Prevention Strategy Interview Guide for NCPC Partners

Background

1. Please describe your current position and responsibilities. How are you involved with the National Crime Prevention Centre and the NCPS? How long have you been involved? How familiar are you with its goals and objectives?

Design and delivery

2. Please describe your organization’s relationship with the NCPC. What is the purpose of the relationship? Is it formal or informal, i.e. , is there a partnership agreement in place? How often is your organization in contact with NCPC, and for what reasons (e.g. , information-sharing, policy discussion, sharing of resources, etc. )?

3. To what extent does your organization have goals in common with the NCPS? To what extent does the partnership enable your organization to achieve its goals? To achieve broader crime prevention goals?

4. To what extent has the relationship made a difference in your policy development process (e.g. , adding a “crime prevention lens” to the analysis, broadening the range of organizations consulted, creating opportunities for joint ventures)?

5. Do you have any documentation about your organization’s partnership with NCPC, or its accomplishments, that you would be willing to share with us (e.g. , agreements, communiqués, minutes, reports, etc. )?

6. As you may know, the National Crime Prevention Strategy was renewed in April 2005. In the last two years since the renewal, have you noticed any changes in the nature of your organization’s relationship with NCPC? Has the relationship strengthened or weakened? To what do you attribute these changes? How could the relationship be improved?

7. Part of NCPC’s role is to develop knowledge products on best practices and evidence-based crime prevention. Have you used any of these products? If so, which ones? How did you use them?

8. Are you aware of NCPC’s current “repositioning” exercise? If yes, how did you find out about it? What do you know about it? (Probe: What is being repositioned? What is the purpose? What are the main elements? What concrete steps is NCPC taking to implement the new approach?) Do you support the new approach? Why or why not?

Impact

9. The expected outcomes of the renewed NCPS are:

• increased local capacity to prevent crime and victimization
• improved comprehensive and strategic partnerships in crime prevention
• improved integration of evidence-based crime prevention into policies and practices

Can you describe some examples of changes in these areas?

10. Has the renewed NCPS had any unexpected impacts, either positive or negative?

Cost-effectiveness/Alternatives

11. In your opinion, are there more effective ways of achieving the objectives of the renewed NCPS? What evidence is there that these alternatives are more effective?

Relevance

12. Is the renewed NCPS still relevant to crime prevention in Canada? Why or why not? What would be the consequence for crime prevention in Canada if it did not exist?

13. Do you have any other comments?

Interim Evaluation of the Renewed National Crime Prevention Strategy Interview Guide for Provincial/Territorial Representatives

Background

1. Please tell me about your current position and your responsibilities. How are you involved with the National Crime Prevention Centre and the NCPS? Are you part of a Joint Management Committee or part of the Federal/Provincial/Territorial Working Group on Community Safety and Crime Prevention, or both? How long have you been involved?

Design and delivery

2. To what extent does your province/territory have goals in common with the NCPS? To what extent does the partnership enable your jurisdiction to achieve its goals with respect to crime prevention?

3. As you may know, the National Crime Prevention Strategy was renewed in April 2005. In the last two years since the renewal, have you noticed any changes in the nature of your relationship with NCPC? Has the relationship strengthened or weakened? To what do you attribute these changes? How could the relationship be improved?

4. Part of NCPC’s role is to develop knowledge products on best practices and evidence-based crime prevention. Have you used any of these products? If so, which ones? How did you use them?

5. Is the organizational and delivery structure of the renewed NCPS appropriate? (Probe: the regional delivery structure; the respective roles and responsibilities of the federal government and the provinces/territories). If not, what changes are required?

6. Since April 2005, NCPC has experienced an internal reorganization, cuts to funding, and a PSEPC-wide “pause” in funding. As far as you are aware, what impact have these events had on its activities and its ability to implement the renewed NCPS?

7. Are you aware of NCPC’s current “repositioning” exercise? If yes, how did you find out about it? What do you know about it? (Probe: What is being repositioned? What is the purpose? What are the main elements? What concrete steps is NCPC taking to implement the new approach?) Do you support the new approach? Why or why not?

8. Do you have any suggestions for NCPC as it undertakes the repositioning, either with respect to the substance of the changes or the process for implementing them?

Impact

9. The expected outcomes of the renewed NCPS are:

• increased local capacity to prevent crime and victimization
• improved comprehensive and strategic partnerships in crime prevention
• improved integration of evidence-based crime prevention into policies and practices

Can you describe some examples of changes in these areas?

10. Has the renewed NCPS had any unexpected impacts, either positive or negative?

Cost-effectiveness/Alternatives

11. In your opinion, are there more effective ways of achieving the objectives of the renewed NCPS? What evidence is there that these alternatives are more effective?

Relevance

12. Is the renewed NCPS still relevant to crime prevention in Canada? Why or why not? What would be the consequence for crime prevention in Canada if it did not exist?

13. Do you have any other comments?

Survey of Funded Organizations

Interim Evaluation of the National Crime Prevention Strategy

1. Which of the following categories best describes your organization? (Check one response only)

• ±₀₁ Aboriginal agency or organization (not government or Tribal/Band Council)
• ±₀₂ Aboriginal government or Tribal/Band Council
• ±₀₃ Arts and culture
• ±₀₄ Business (e.g. , Chamber of Commerce, etc. )
• ±₀₅ Community, social or voluntary services
• ±₀₆ Criminal Justice, Corrections, Police
• ±₀₇ Education
• ±₀₈ Employment
• ±₀₉ Government – local or municipal
• ±₁₀ Government – provincial or territorial
• ±₁₁ Housing services
• ±₁₂ Health
• ±₁₃ Parks and recreation
• ±₁₄ Religious/faith
• ±₁₅ Service club (e.g. , Rotary Club, etc. )
• ±₆₆ Other (specify) ___________________________________________________

2. How long has your organization been in existence? (Check one response only)

• ±₁ Less than 1 year
• ±₂ 1-2 years
• ±₃ 3-5 years
• ±₄ 6-10 years
• ±₅ 11-15 years
• ±₆ 16-20 years
• ±₇ 21 years or more

3. In what province or territory is your organization based? (Check one response only)

• ±₀₁ Alberta
• ±₀₂ British Columbia
• ±₀₃ Manitoba
• ±₀₄ New Brunswick
• ±₀₅ Newfoundland and Labrador
• ±₀₆ Northwest Territories
• ±₀₇ Nova Scotia
• ±₀₈ Nunavut
• ±₀₉ Ontario
• ±₁₀ Prince Edward Island
• ±₁₁ Quebec
• ±₁₂ Saskatchewan
• ±₁₃ Yukon

4. What is the scope of your organization? (Check one response only)

• ±₁ International
• ±₂ National
• ±₃ Interprovincial/interterritorial
• ±₄ Provincial/territorial
• ±₅ Regional
• ±₆ Municipal/local

Important instructions – please read carefully before proceeding

The remainder of this survey gathers information about the renewed National Crime Prevention Strategy. When completing the survey, please answer with reference only to projects funded through either the Crime Prevention Action Fund (CPAF), the Research and Knowledge Development Fund (RKDF), and the Policing, Communities and Corrections Fund (PCCF).

5. During the process of developing your proposal(s), during the proposal approval process, or during the delivery of your project(s), did your organization receive any of the following types of technical assistance from National Crime Prevention Centre staff?

If you answered “Yes” to any part of question 5, please answer questions 6 to 8.
Otherwise, please go directly to question 9.

6. Overall, how would you describe the assistance you received from NCPC staff?

• ±₃ Very useful
• ±₂ Somewhat useful
• ±₁ Not at all useful

7. Please explain your answer.

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

8. The next series of questions asks you to give a more detailed assessment of the assistance you received from NCPC staff. Please read each statement below and check the response that best represents your opinion.

9. We are interested in your opinions about various aspects of the proposal development and approval process. Please read each statement below and check the response that best represents your opinion.

10. Do you have any recommendations for improving the proposal development and approval processes?

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

±₀ No suggestions

11. With the implementation of its three new funding programs in April 2005, the National Crime Prevention Centre introduced an improved performance measurement strategy that focuses on measuring results achieved by funded projects. Overall, the performance measurement strategy is designed to improve NCPC’s ability to report on how well it is achieving its goal of increase community safety.

We are interested in your opinions about various aspects of the performance measurement strategy. Please read each statement below and check the response that best represents your opinion.

12. What challenges does your organization face in complying with the requirements of the performance measurement strategy? Please explain.

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

±₀ No challenges

13. What could NCPC do to make it easier for your organization to comply with the requirements of the performance measurement strategy? Please explain.

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

±₀ No suggestions

14. We are interested in the impact of the funding and technical assistance that your organization received from NCPC since the renewal of the National Crime Prevention Strategy in April 2005. Please restrict your response to the assistance your organization received through the Crime Prevention Action Fund (CPAF), the Research and Knowledge Development Fund (RKDF), or the Policing, Communities and Corrections Fund (PCCF). Please read each statement below and check the response that best represents your opinion.

15. To date, has your organization’s funded project(s) had unintended consequences, either positive or negative?

• ±₁ Yes, positive unintended consequences
• ±₂ Yes, negative unintended consequences
• ±₀ No unintended consequences (Go to Q17)
• ±₈ Don’t know (Go to Q17)

16. Please describe any positive or negative unintended consequences.

17. Has the project for which your organization received funding been completed?

±₁ Yes     ±₀ No (Go to Part D)

18. Please describe what outcomes have been achieved as a result of your funded project.

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

19. Please read each statement below and check the response that best represents your opinion.

20. What would be the consequence for your organization if the National Crime Prevention Strategy did not exist?

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

±₈ No opinion/don't know

21. What do you think would be the consequence for crime prevention in Canada if the National Crime Prevention Strategy did not exist?

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

±₈ No opinion/don't know

22. Do you have any other comments?

___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

Thank you for taking the time to complete this survey.
Please return it in the enclosed self-addressed, postage-paid envelope to:

PRA Inc.
500 – 363 Broadway
Winnipeg, Manitoba R3C 3N9

Or you can fax it back to us toll-free at:

1-800-717-5456

1 Department of Justice Canada. (1995, August). Towards Safer Communities: A Progress Report on the Safe Homes/Safet Streets Agenda. Ottawa: Department of Justice Canada.

2 National Crime Prevention Centre. (1998). Crime Prevention Partnership Program: Access Guide. Ottawa: NCPC.

3 Information about the CPAF is taken from the CPAF Application Form, the CPAF Overview on the NCPC website, and the Applicant’s Guide.

4 Information about the PCCF is taken from the PCCF Overview and Application Form.

5 Information about the RKDF is taken from the RKDF Overview, Applicant’s Guide, and Application Form

6 Information in this section is taken primarily from the ARAF and from an NCPC document entitled “Planned Staffing and Reporting Relationships.”

7 Federal/Provincial/Territorial Working Group on Community Safety and Crime Prevention. (2007, April 12). Implementing the National Crime Prevention Centre’s New Directions: A Blueprint for Change [Draft].

8 Public Safety Canada. (2007, June 8). Minister Day announces focused crime prevention approach.

9 Public Safety Canada. (2007, June 8). A Blueprint for Effective Crime Prevention.

10 NCPC. (2006, March 20). Report to the Treasury Board Secretariat: Management Plan. p. 17.

11 Ekos Research Associates Inc. (2005, April 27). Summative Evaluation of the Phase II Expansion of the National Crime Prevention Strategy. p. 12.

12 Note, however, that NCPC funding is of significant importance to projects, accounting for 57% of total project funding on average.

13 Pre-renewal data are taken from the Summative Evaluation of the Phase II Expansion of the National Crime Prevention Strategy, Appendix C, p. 1. The source of these data is NCPC’s Project Control System, which included a field called “priority group.” Post-renewal data are taken from GCIMS, which does not have a “priority group” field. Instead, data are taken from a “priority area” field. Note that in GCIMS there are also separate “target gender” and “target age” fields, which give different results than the priority area field. According to the target gender field, 10% of projects target females. According to the target age field, 44% of projects target youth and 30% of projects target children.

14 These ten cities were selected based on 2004 Statistics Canada data which showed them as experiencing the highest crime rates in Canada.

15 However, it should be noted that capital cities tend to be over-represented within GCIMS because when a provincial or national non-governmental organization is the project sponsor, the city in which that organization is located is recorded in GCIMS, rather than the location in which the project actually occurs

16 Examples of higher level results identified in the CPAF Evaluation Strategy are a reduction and/or mitigation of risk factors associated with crime and victimization, an increase in protective factors, a reduction in crime, a reduction in the fear of crime, a reduction in the cost of crime, and reduced call outs to police.

17 One exception is the Atlantic Region. It introduced a transition strategy called “Moving Forward” that approved targeted calls for proposals targeting risk factors present in the lives of youth at risk. A targeted call for proposals was issued in Prince Edward Island in early 2006. In the other Atlantic provinces, calls for proposals were deferred until after the pause and the repositioning exercise.

18 GCIMS data show that 4% of funded projects were located in Yukon, 2% in the Northwest Territories, and 1% in Nunavut.

19 Information provided by Claude Turgeon, Director of Programs, in personal communication, June 1, 2007.

20 Of the many federal grant and contribution programs that PRA has evaluated over the years, we are aware of only one other program in which the Minister approved all funding.

21 This sentiment was evident both in the interviews with provincial/territorial representatives and in the minutes of the Federal/Provincial/Territorial Working Group on Community Safety and Crime Prevention.

22 The Report of the Independent Blue Ribbon Panel on Grant and Contribution Programs. (December 2006). From Red Tape to Clear Results, p. 24-25.

www.publicsafety.gc.ca/cnt/cntrng-crm/crm-prvntn/index-

24 Provincial/territorial representatives also reported that they received contradictory information from the regional NCPC staff and those at NCPC headquarters.

25 According to NCPC, social development approaches will continue to be part of the NCPS, but will be more focused on at-risk populations, rather than being universally applied. Bob Cormier, FPT Working Group Minutes. (2007, January 30).

26 FPT Working Group Minutes. (2007, May 11).

www.publicsafety.gc.ca/cnt/rsrcs/pblctns/index-eng.aspx. June 12, 2007.

28 A complete summary of these activities can be found in Volume II.

29 Information provided by Claude Turgeon, Director of Programs, in personal communication. (2007, June).

30 According to NCPC, some of these projects have reportedly reapplied under the new funding criteria.

31 The 2005 Public Service Employee Survey showed that among federal departments, morale was lowest at PSEPC, and within PSEPC, lowest at NCPC.

32 Speaking Notes for Claude Turgeon, Director of Programs. Management Weekly Meeting. (2007, February 13).

33 Speaking Notes for Claude Turgeon, Director of Programs. Management Weekly Meeting. (2007, February 13).

34 Action Plan for GCIMS ¾ January–March 31, 2007 (updated February 14, 2007), and Action Plan for GCIMS ¾ March–October 31, 2007 (updated May 31, 2007).

35 According to NCPC, of the 349 projects funded under the renewal, 75 (21%), had concluded (i.e. , a final report had been received) by July 3, 2007.

36 NCPC. (2007, May 23). Renewed National Crime Prevention Strategy Logic Model [Draft].

37 The full literature review is contained in Volume II, Appendix A of this report.

38 Irvin Waller. (2006). Less Law, More Order. Westport, Connecticut: Praeger, 126.

39 United Nations. (2002). Guidelines for the Prevention of Crime. Economic and Social Council Resolution 2002/13, annex, adopted 24 July 2002. Vienna, Austria: United Nations.

40 Welsh, B. C. , & Farrington, D. P. (2007). Saving Children From a Life of Crime: Toward a National Strategy for Early Prevention. Victims and Offenders, 2(1), 1-20.

41 World Health Organization. (2002). World Report on Violence and Health. Geneva: World Health Organization.

42 Brandon C. Welsh and David P. Farrington. (2007). “Saving Children From a Life of Crime: Toward a National Strategy for Early Prevention.” Victims and Offenders: Journal of Evidence-Based Policies and Practices 2:1-20, p. 9.

43 Homel, P. (2006). Joining Up the Pieces: What Central Agencies Need to Do to Support Effective Local Crime Prevention. In Knutsson and Clarke (eds. ). Putting Theory to Work: Implementing Situational Prevention and Problem-Oriented Policing. (pp. 111-138). Crime Prevention Studies, 20. Monsey, NY: Criminal Justice Press, p. 115.

44 Public Safety Canada. (2007, March 19). Report on Plans and Priorities 2007-2008. p. 17.

45 Public Safety Canada. (2007, March 19). Report on Plans and Priorities 2007-2008. p. 34.

Date modified:

2015-12-15