Guide to Select Promising and Model Crime Prevention Programs - 2012

Guide to Select Promising and Model Crime Prevention Programs - 2012 PDF Version (480 KB)

Introduction

Using the most recent findings from research on offending trajectories and related risk factors, and findings from thorough studies of crime prevention programs, Public Safety Canada's National Crime Prevention Centre (NCPC) has developed a funding approach for community projects based on the use of model and promising programs. This approach has several objectives, including developing knowledge on effective programs in Canada and sharing this knowledge with policy makers and practitioners across the country. Knowledge of the programs' impact on reducing offending behaviour and the factors leading to their successful implementation are of particular importance. It is hoped that the knowledge acquired will contribute to the sustainability of interventions that demonstrate positive results, are cost-effective and realistic to implement.

With regard to the Fall 2012 Call for Letters of Intent (the Call), the NCPC has chosen to focus on selected promising and model programs that are directly related to the three priority issues for the Call.  

Letters of Intent (LOI) submitted under this Call will need to indicate which of the nine promising and model programs the applicants intend to implement.  This raises the question of how applicants choose a program that best fits their circumstances. What criteria should be taken into account? Selecting a program for potential implementation requires significant investment of time and thought.

Few studies have been conducted on the process for selection a program or on the criteria that influence or guide this decision.  Evaluation studies are largely results-based (e.g. in terms of crime reduction) and very rarely focus on implementation considerations.

This Guide was created with this in mind. Although not exhaustive, it presents some key considerations and questions designed to help people make the most informed decisions possible when selecting a program for implementation in their community.

The following figure shows the four (4) main areas that will be explored in this guide.

Figure 1. Matrix of the key steps for selecting a crime prevention program

1. Portrait and analysis of the local situation:
   • What are the problem areas, their characteristics, etc.?
2. Effective crime prevention practices:
   • Which program best matches the local situation?
3. Program implementation:
   • What are the factors for success?
4. Toward sustainability:
   • What elements encourage sustainability?

The sections that follow provide a more in-depth look at each of these four main areas.

Portrait and Analysis of the Local Situation

The first step in a crime prevention approach is the development of a local portrait/diagnostic, also called environmental scan or strategic assessment, using a variety of data collection methods.i This portrait will provide an overview of the local reality, the nature and characteristics of existing crime and delinquency, the characteristics of the populations involved, the contributing factors, the services already in place and the resources available in the community.

The importance of this local analysis cannot be underestimated. Lack of understanding of the realities can result in the selection of the wrong type of prevention program to implement. If the nature, characteristics and extent of the problems are poorly defined, the selected program might not target the right client population, or even worse, could lead to inappropriate interventions. Not only could the program, whether promising or model, fail to generate the expected results, it could be counter‑productive. Research on correctional interventions in communities has demonstrated that intervening with people who do not need support (for example, low-risk offenders) can have negative repercussions.

As indicated in the document "Funding Priorities of the National Crime Prevention Strategy - 2012", NCPC has identified three priority issues: 1) preventing violence among youth aged 12 to 17 years; 2) preventing Aboriginal youth delinquency in urban centres (aged 6‑17 years); and, 3) preventing school-based bullying. These priorities have been identified based on consultations with various groups, official statistics and literature on the main risk factors associated with offending trajectories.

All three of these priorities might be relevant in a community. For the Call, it is necessary to focus on one of the three priority areas. The work to conduct an analysis of the community context is critical to determine which priority will be the focus for the application. Sources of data and information that will help develop this portrait of the community include:  police reports, health and social service reports, educational system reports and socio-economic reports and data. There may be many sources of information to consult and it is important that information and data used to make these decisions is recent, as objective as possible and backed by sound data collection methods.

Once the main priority issue has been identified, gathering information to fully understand the nature and characteristics of the people and circumstances involved is the next step. For example, knowing such things as the age, gender and ethnicity of the youth who engage in violent behaviour is critical to choosing a program. Similarly, knowing where youth spend their time, when violent behaviour takes place and any other factors related to incidences of violence will help to select a program that has the best chances of changing the behaviours.  

Before making a decision on the program to propose for implementation, it is also important to know what services and resources are currently provided and available in the community. An inventory of existing resources and programs relevant to the issue and population being focused on is valuable from a number of perspectives. It can help to identify gaps in services currently available, can reduce duplication of work and can help to identify potential partners for a new initiative in the community. Every crime prevention program requires specific resources and it is only with a thorough understanding of what is available that an informed decision can be made about what programs are realistic in any given community context.

In summary, some key questions to guide the development of a local portrait are provided in the box above.ii By using these questions along with others people in the community will think of and examining the sources of information available to help answer them, one of the programs identified by NCPC may start to stand out as a good fit. Describing the fit between the program selected and the information collected provides the rationale for the selection and is a key component of what is required on the LOI for the Call.

Effective Crime Prevention Practices 

As part of the Call, NCPC has selected only certain programs for replication; a total of nine (9) programs (Table 1). Appendix A provides an overview of each program and a detailed description of the programs is included in the document entitled "Implementation Fact Sheets on Promising and Model Crime Prevention Programs - 2012". These programs were selected using a standardized process that analyzed the scientific literature available on the various programs as it related to the priority issues identified. This process is described in Appendix B.

*The priority "Preventing Aboriginal youth delinquency in urban centres" refers to children and youth aged 6 to 17 years living in areas with around 30,000 residents or more.

The literature identifies three (3) main categories of variables to be taken into account when selecting a program:

1. Suitability and alignment of the program;
2. Quality and adaptability of the program;
3. Organizational capacities and resources.

1. Suitability and Alignment of the Program

The suitability and alignment of the program is the first main category of factors and can be divided into three subgroups of variables:iii

1. Suitability with the lead organization;
2. Suitability with risk factors;
3. Suitability with target group.

The box below shows examples of questions in relation to these three subgroups of variables.

a) Suitability with the lead organization

Although it seems obvious, the selected program's alignment with the mission of the lead organization is often neglected.iv The lead organization is the organization taking a leadership role in applying for funding and in the implementation of the proposed program.

The more the program is in line with the organization's philosophy and organizational values, the better the chances are that the program will be accepted by staff and others in the community. Similarly, the more the program is designed to reach a clientele already known by the organization, the better the chances are that the right clientele will be reached. Another factor to consider is how the program will complement the other programs provided by the lead organization or other organizations. New programs implemented in a community should address the gaps and provide services to reach needs that are not met. This complementarity will help develop a comprehensive approach, which, in the long term, could lead to interventions that are more effective and durable.

b) Suitability with risk factors

The second alignment component looks at how well a program addresses the level and complexity of the risk factors present for the participants. A key consideration is the duration and intensity of the program. Reducing the impact of certain risk factors (e.g. substance abuse or impulsivity) requires interventions that are long enough to change attitudes or behaviours in participants. Understanding the main risk factors present and what types of interventions are suitable to address them is a main element for the selection of an appropriate program. Program requirements will thus be reviewed based on what is known of the risk factors and populations targeted.

c) Suitability with target group

The last component related to suitability and alignment is the fit with the characteristics of the target group. Every program is developed and designed to work in certain settings and with specific populations. For example, it will be different working with youth in school compared to street youth. The issue is to increase the likelihood that those targeted will participate for the full length required to achieve the expected changes. In other words, the degree of effort required to participate fully in the program for the required amount of time must correspond to the effort that the target group is able to put in.  For example, what are the chances that vulnerable youth will participate in a long-term program with regular meetings? It is also important to ask whether the time and place of meetings suit the target audience.

Another important element to consider is the adaptation of a given program to the culture and values of the target population. Except for few culturally-based programs, the majority of evaluated preventive interventions have not been developed on ethno-cultural dimensions. And when culturally appropriate programs have been evaluated, few used rigorous evaluation standards. Note also that in many cases, if a community wishes to modify the program culturally, adaptations must be introduced with the consent and under the supervision of the program's developers.

2. Quality and Adaptability of the Program

The program's quality and adaptability is the second main category of factors that is taken into account when selecting a promising or model program.

With regard to the quality of programs, those that have been selected as part of this funding call have already demonstrated their quality and have been tested and validated.

Regardless, these programs have either limited or no testing in Canada. The adaptability of the program refers to the degree of flexibility allowed in the local implementation of the program against compliance to the original program.v While the faithful replication of the main elements of the original program in the implementation and the delivery of services plays a big role in achieving the expected results of the program,vi, vii there is no clear consensus on how an evidence-based program can be modified to meet the needs of the community.

According to a report published by What Works,viii the program's suitability and adaptability should be addressed based on the acceptable types of adaptation rather than risky types of adaptation that could affect the program's potential success. The box above, which is taken from this report, gives examples of acceptable and risky adaptations. Either way, it is important to communicate with the developers of the program before making changes, regardless of their nature.

3. Organizational Resources and Capacities

Organizational resources and capacities is the third category of factors to be taken into account when selecting a promising and model program.

The analysis of organizational resources and capacities is too often underestimated. Promising and model programs vary in their complexity and in the level and type of effort and resources they require. All programs cannot be implemented everywhere, and it is important to consider organizational resources and capacities.

Implementing an evidence-based program requires investments with respect to money, time and human resources. Sometimes even material investments are needed. Even if the program has a high degree of suitability and alignment, if the lead organization and its partners lack the necessary resources and capacities, the chances of getting the expected results are limited.

Various organizational factors facilitate the implementation of a high-quality program, such as:

• Operational capacity: for example, qualified personnel, good staff retention, training and supervision, monitoring system in place;
• Financial capacity: for example, appropriate financial controls, qualified personnel to monitor and report financial information;
• Previous experience in the implementation of similar programs;
• Sound partnerships and networks in the community.

In summary, taking into account a program's alignment with the potential participants' realities and needs, with the objectives and resources of the lead organization, with the environment and community context and understanding the degree of program adaptation needed creates a solid foundation for the selection of a program.

Program Implementation

Not all programs present the same degree of complexity, and their implementation will therefore be affected. Special attention needs to be focused on a few key implementation components.

The first consideration is the recruitment and retention of participants.  The programs available to select from are prevention programs designed for at-risk populations (secondary prevention). The interventions are not designed for all youth (universal) and, therefore, a key implementation consideration is how to make sure the right clientele is reached.  Given their circumstances and histories, at‑risk youth and children are often difficult to attract to new programs. During program implementation, the systematic use of risk assessment tools is very important to ensure the right youth are being reached with the program and its interventions. The NCPC has made a number of these tools available (Tools to Identify and Assess the Risk of Offending Among Youth). This is important not only to intervene with those who need it, but also to respect their privacy and avoid further stigmatization.

In addition, because participation in these programs is voluntary, thought may need to be given to incentives in order to maintain the level of commitment necessary from participants. Such incentives might include transportation assistance or recreational activities to enhance interventions that are often more psychosocial in nature.

The second component of implementation to be looked at is staff recruitment and training. The lead organization may already have qualified staff, but more often than not, people will need to be recruited and trained for the delivery of the specific program. It is essential to ensure that staff delivering the program have the tools, skills and support they need for success.

The third essential aspect is partnerships. Whether because participants will be referred by other organizations (e.g. the police or child welfare services) or because the intervention can be delivered only in partnership with an institution (e.g. a school), implementation of a program must include partnerships with various organizations. Partnerships will lead to greater collective expertise, but also mean processes will be more cumbersome and more time consuming. For example, protocols or memorandums of understanding may be required to clearly articulate roles and responsibilities or to ensure appropriate sharing of information with consideration for privacy legislation.

The fourth key component in implementation is data collection. The NCPC funds and evaluates time-limited projects in order to build a base of Canadian knowledge on effective crime prevention practices. The minimum evaluation required for each funded project is an assessment of its implementation based primarily on the systematic and rigorous collection of data on participants, interventions, partners, costs, etc. In some cases, the NCPC will also fund a study to evaluate project impacts, especially the reduction of key behaviours (violence, bullying, crime). The lead organization will therefore have to anticipate being involved in a rigorous evaluation process. The NCPC offers the lead organizations various types of tools for performance tracking and evaluation. The performance tracking tools monitor the program's development and help identify where changes need to be made, while the tools for evaluating the implementation process and results will detail the program's results and implementation context.

Toward sustainability

While it may seem surprising, sustainability is an aspect that needs to be considered when choosing a program for implementation. Not all successful programs have the same likelihood of being incorporated into or combined with existing systems or of receiving sustainable funding. When a program is selected, consideration needs to be given to whether it could be maintained and included in the organization's structure once the demonstration funding ends. Some programs may fit into organizational structures more easily than others. The cost of the program, while not the only variable, is an important factor since it seems that programs requiring a major financial investment are more likely to be discontinued after the funding runs out.

But sustainability is not only a matter of funding:ix it also entails creating and maintaining the momentum needed for change by reorganizing and optimizing resources.

As described in a toolkit developed by the Institute for Educational Leadership,x sustainability is a multidimensional concept encompassing a variety of resources, actions and processes. It may be reflected in a number of ways:

• The institutionalization of all or part of a program;
• Momentum that mobilizes and leads to a reorganization (for example, in provided services);
• The continuation of all or components of the project as a separate entity within the larger service system, through an ongoing funding arrangement;
• The transformation of policies, governance structures, fiscal arrangements and service practices in place.

Appendix A – Overview of Promising and Model Programs Selected for the 2012 Call

For each of promising and model programs selected under this call for funding, a brief overview of the main dimensions to be taken into consideration during the selection process for the program is provided below. For more details, please refer to the Implementation Fact Sheets.

Appendix B – Selection process for promising and model programs

Under this call for funding, the NCPC adopted a rigid process for the selection of promising and model programs. The selection process was based on a list of criteria: four (4) mandatory criteria and three (3) additional criteria. In total, the NCPC reviewed and analysed 32 programs on the basis of the following criteria:

Mandatory criteria

The four mandatory criteria were developed on the basis of the key principles of evidence-based crime prevention. During analysis of the programs, as soon as one of the criteria was not met by a program, that program was eliminated.

1. At a minimum, the program must achieve results supported by scientific evaluation standards and fit the definition of a promising or model program. Based on this criterion, innovative programs were automatically excluded.
2. The program must be in keeping with one of the NCPC's priorities for 2012 and must be considered a secondary or tertiary prevention program. Based on this criterion, universal programs or programs addressing an issue that did not fit in with the NCPC's priorities were automatically excluded.
3. Technical assistance must be available from the program developers. Having access to technical assistance from the program developers is considered one of the key factors in the success of evidence-based programs. Based on this criterion, programs where it was difficult or impossible to contact the developers were excluded.
4. The program must have tools to facilitate program implementation and delivery of services/interventions (for example, implementation manual, risk assessment tools, tracking tool).

Asset criteria

When programs met the mandatory criteria, three additional criteria were assessed; the purpose of those criteria was not to draw distinctions between programs, but merely to add information:

1. The level of effort required to implement the program: The purpose of this aspect was to gain a better understanding of the program's complexity and assess some of the key components often associated with major challenges (for example, program requiring specialized staff, program being implemented over a period of several years, partnerships requiring adjustments to roles). A level-of-required-effort continuum was developed to ensure that the programs selected reflected the various levels of effort and complexity.
2. Knowledge of the implementation of this program and results in a Canadian context is already available. Because this call was specially targeted, it was preferable, but not mandatory, that the program have already demonstrated positive results in a Canadian context.
3. Program tools and other implementation and tracking material are already available in both official languages.

Bibliography

Crosse, S. et al. (2011). Prevalence and Implementation Fidelity of Research-Based Prevention Programs in Public Schools - Final Report. U.S. Department of Education.

Hastings, R. (no date). Building a Safer Canada: Effective planning for crime prevention. Institute for the Prevention of Crime – Institut pour la prévention de la criminalité, University of Ottawa, Ottawa.

Institute for Educational Leadership. Building Sustainability in Demonstration Projects for Children, Youth and Families – Toolkit Number 2 – Systems Improvement Training and Technical Assistance Project. OJJDP.

National Center for Chronic Disease Prevention and Health Promotion (Division of Adult and Community Health). A Sustainability Planning Guide for Healthy Communities. Available at http://www.cdc.gov/healthycommunitiesprogram/pdf/sustainability_guide.pdf.

National Registry of Evidence-Based Programs and Practices. Questions to Ask as You Explore the Possible Use of an Intervention. http://www.nrepp.samhsa.gov/pdfs/questions_to_ask_developers.pdf

O'Connor, C., Small, S. A. and Cooney, S. M. (2007). Program fidelity and adaptation: Meeting local needs without compromising program effectiveness. What Works, Wisconsin – Research to Practice Series, 4. Madison, WI: University of Wisconsin–Madison/Extension. Available at http://whatworks.uwex.edu/attachment/whatworks_04.pdf.

Promise Neighborhoods Institute. Evidence‑Based Practice: A Primer for Promise Neighborhoods. Available at http://www.cssp.org/community/neighborhood-investment/other-resources/Evidence-Based-Practice-A-Primer-for-Promise-Neighborhoods.pdf.

Roger Przybylski (2008). What Works: Effective Recidivism Reduction and Risk-Focused Prevention Programs. RKC Group, 2008.

Small, S. A., Cooney, S. M., Eastman, G. and O'Connor, C. (2007). Guidelines for selecting an evidence ‐based program: Balancing community needs, program quality, and organizational resources. What Works, Wisconsin – Research to Practice Series, 3. Madison, WI: University of Wisconsin–Madison/Extension. Available at http://whatworks.uwex.edu/attachment/whatworks_03.pdf.

Welsh, B. (2007). Evidence-Based Crime Prevention: Scientific Basis, Trends, Results and Implications for Canada. National Crime Prevention Centre, Public Safety Canada, Ottawa.

Endnotes

i For an example of this type of document, consult the Guide Book on Local Safety Audits; this document is available at http://efus.eu/en/topics/tools-and-methods/audits-and-evaluation/efus/654/.

ii Examples of questions presented in this box are based on the community assessment tool developed by the National Native American AIDS Prevention Center. For more information, consult the following site: http://www.nnaapc.org/resources/toolkit/module_4/module48.html.

iii This division and the examples of questions presented in this section are based on the report prepared by Small, S.A., Cooney, S.M., Eastman, G., & O'Connor, C. (2007). Guidelines for selecting an evidence ‐ based program: Balancing community needs, program quality, and organizational resources.

iv Guidelines for Selecting an Evidence-Based Program.

v Dusenbury L. et al. (2003). "A review of research on fidelity of implementation: implications for drug abuse prevention in school settings," in Health Education Research, 18 (2), 237-256; Crosse S. et al. (2011). Prevalence and Implementation Fidelity of Research-Based Prevention Programs in Public Schools - Final Report. U.S. Department of Education

vi Mihalic S. and al. 2008. Implementing the LifeSkills Training drug prevention program: factors related to implementation fidelity.

vii Roger Przybylski (2008). What Works: Effective Recidivism Reduction and Risk-Focused Prevention Programs. RKC Group, 2008

viii Wisconsin team at the University of Wisconsin–Madison.

ix National Center for Chronic Disease Prevention and Health Promotion (Division of Adult and Community Health). A Sustainability Planning Guide for Healthy Communities.

x Institute for Educational Leadership. Building Sustainability in Demonstration Projects for Children, Youth and Families – Toolkit Number 2 – Systems Improvement Training and Technical Assistance Project. OJJDP.

Date modified:

2022-08-02