ARCHIVE - Final Report 2009-2010 Evaluation of the Investments to Combat the Criminal Use of Firearms Initiative

ARCHIVE - Final Report 2009-2010 Evaluation of the Investments to Combat the Criminal Use of Firearms Initiative PDF Version (443 KB)

List of Acronyms

CBSA

Canada Border Services Agency

CID

Criminal Intelligence Directorate

CIBIN

Canadian Integrated Ballistics Identification Network

CISC

Criminal Intelligence Service Canada

FRT

Firearms Reference Table - refers to both an organization unit within the Royal Canadian Mounted Police and the product produced by this unit

IBIS

Integrated Ballistics Identification System

ICCUF

Investments to Combat the Criminal Use of Firearms

JFO

Joint Forces Operation

NJMT

National Joint Management Team

NSFTA

National Strategic Firearms Threat Assessment

NWEST

National Weapons Enforcement Support Teams

PS

Public Safety Canada

RCMP

Royal Canadian Mounted Police

RIFLO

Regional Intelligence Firearms Liaison Officer

RFIC

Regional Firearms Intelligence Coordinator

SIB

Strategic Intelligence Brief

Glossary of Terms

CBSA port-of-entry lookouts -  The subject of a lookout is a person, conveyance, or a good that has been reliably identified through confirmed information, as someone or something that is likely to or has been, involved in the commission of an offence. A lookout is the product of CBSA Intelligence Operations, based on confirmed information being placed through the five-step intelligence process.

CIBIN correlation - The Canadian Integrated Ballistics Identification Network (CIBIN) is a national network of Integrated Ballistics Identification System (IBIS) instruments that collect, analyze and correlate fired bullets and cartridge cases in a central database to generate investigative leads for police. A correlation is a comparison of the digital signature from a suspect bullet or cartridge case to other bullets and cartridge cases in the CIBIN database resulting in a ranking which describes the likelihood that the same firearm might have been used in several crimes.

CIBIN "hit" - A "hit" is a link discovered through CIBIN between firearm crimes from anywhere across Canada and those committed over time. CIBIN will link crime scenes where the same firearm has been discharged even if the firearm has not been recovered. Also, CIBIN will link seized or found firearms to the crimes where they were used.

Intelligence - Intelligence is information that has been subjected to the intelligence process of collection, evaluation, collation, analysis and dissemination. The purpose of intelligence is to provide knowledge and understanding upon which operational and strategic decisions can be made.

• Actionable intelligence - Refers to the direct link between the delivery of timely, adequate and useful intelligence and enforcement action being taken.
• Strategic intelligence - Strategic intelligence provides a comprehensive and current picture of the scope and direction of criminal activity, in order to assist management decision-making and the determination of future action.
• Tactical intelligence - Tactical intelligence supports ongoing specific criminal investigations, focusing primarily on specific targets and has a very focused and restricted scope. The primary client for tactical intelligence will often be the lead investigator and, by extension, the crown prosecutor.

Joint Forces Operation - It is a unit or formation comprising various federal, provincial and municipal law enforcement personnel working together with joint common mandates and objectives. There are many forms of joint force operations, ranging from multi-agency co-located operations to short-term ad hoc operations.

Investigative Support (also known as Enforcement Support) - It includes all types of assistance rendered to aid an investigator in support of his/her investigation. Examples of investigative support include assistance with drafting warrants, identifying or verifying firearms. Examples of analytical investigative support are link charts, timelines, geospatial and statistical analysis, etc. The role of an investigative support officer is to supply onsite expertise within the context of a specific investigation to increase the timeliness and effectiveness of the investigation.

Executive Summary

Evaluation supports accountability to Parliament and Canadians by helping the Government of Canada to credibly report on the results achieved with resources invested in programs. Evaluation supports deputy heads in managing for results by informing them about whether their programs are producing the outcomes that they were designed to achieve, at an affordable cost; and, supports policy and program improvements by helping to identify lessons learned and best practices.

What we examined

This is the 2009-2010 Evaluation of the Investments to Combat the Criminal Use of Firearms Initiative (ICCUF). For the purpose of this evaluation, the terms "guns" and "firearms" are used interchangeably.

The Investments to Combat the Criminal Use of Firearms Initiative was created in 2004-2005. Prior to its existence, several gaps in information and intelligence collection had been identified concerning the extent and patterns of smuggling and trafficking of firearms used in crime. In response, the Government launched the Initiative with an aim to enhance the national collection, analysis and sharing of firearms-related intelligence, in order to improve investigations and increase knowledge of patterns of smuggling and trafficking of illegal firearms.

This Initiative began with a budget of $50 million for five years, which is thereafter ongoing.  It operates as a horizontal initiative within the Public Safety Portfolio, and involves three partner department/agencies:

• Public Safety Canada (PS):
  • Firearms and Operational Policing Policy Division;
• Royal Canadian Mounted Police (RCMP):
  • Forensic Science and Identification Services which operates:
    • Canadian Integrated Ballistics Identification Network;
  • Firearms Investigative and Enforcement Services Directorate which includes:
    • National Weapons Enforcement Support Teams;
    • Firearms Reference Table; 1
  • Criminal Intelligence Directorate;
  • Criminal Intelligence Service Canada;2
• Canada Border Services Agency (CBSA)
  • Intelligence and Targeting Operations Directorate.

Why it is important

The Investments to Combat the Criminal Use of Firearms Initiative is relevant as it aligns with the Government of Canada public safety agenda relating to the illicit movement of firearms and gun crime. gun control and gun crime. The intended outcomes of the Initiative are aligned with the strategic outcomes and priorities of federal partner department/agencies in contributing to keeping Canadians safe and secure.

What we found

Relevance

There is a continuing need for the Initiative. Gun crime/violence is prevalent and appears to be on the rise. In addition, the Initiative responds to many of the information and intelligence gaps related to gun crime. Finally, users of the Initiative's products and services attested to their continuing need.

This Initiative is appropriate to the federal mandate as is prescribed by the legislative framework. In addition, since guns and criminals travel across domestic and international borders, a federal role avoids a "patchwork" approach to intelligence development and analysis related to illicit firearms in Canada.

Performance - Achievement of Expected Outcomes

Overall, the Initiative has achieved its expected immediate and intermediate outcomes. Based on feedback from stakeholders, the Initiative has also contributed to its longer-term ultimate outcome of the prevention of the criminal use of firearms.

Immediate Outcomes

The Initiative has improved the sharing of actionable intelligence related to gun crime. Partners of this Initiative have participated in joint force operations at various levels, ranging from longer-term, multi-agency, co-located joint force operations, to short-term, ad hoc joint force operations. However, this evaluation cannot quantify the extent of involvement of partners in these operations, as quantitative information was not available.

In addition, partners of this Initiative have produced and disseminated a range of tactical and strategic intelligence products and services to a wide audience. Stakeholders, including law enforcement officers, agreed that these products and services are useful to them and provide actionable intelligence; although intelligence from some products and services was considered more actionable (Firearms Reference Table and the Canadian Integrated Ballistics Identification Network provided by the Royal Canadian Mounted Police, intelligence provided by Canada Border Services Agency's Regional Intelligence Firearms Liaison Officers) than from others (tactical reports provided by the Criminal Intelligence Directorate of the Royal Canadian Mounted Police).

The Initiative has contributed to increased knowledge of investigative procedures through a significant number of training sessions delivered and advice provided. Interviewees agreed that the training and advice/assistance provided by the Initiative has increased their knowledge of investigative procedures; however, feedback information was largely not available from training sessions, conferences or workshops delivered by partners of the Initiative. As well, interviewee responses indicated that, for tactical advice provided by the Regional Firearms Intelligence Coordinators of the Royal Canadian Mounted Police's Criminal Intelligence Directorate, there is room for improvement.

Intermediate Outcomes

The Initiative has increased knowledge of gun crime issues, trends and threats, through the provision of Public Safety Canada's research reports and strategic intelligence products provided by Criminal Intelligence Service Canada, Royal Canadian Mounted Police.  Stakeholders believed that the Initiative has increased their knowledge of gun crime issues, trends and threats. In addition, those who work in the policy area reported that the intelligence and research generated by the Initiative have contributed to better-informed policy advice.

The Initiative has enhanced national coordination of gun crime investigations and enforcement. Documentation suggests that prior to the implementation of the Initiative, information and intelligence was collected by various agencies, but efforts were not coordinated. Under the Initiative, the Royal Canadian Mounted Police's National Weapons Enforcement Support Teams provide a one-stop shop for law enforcement officers, and Canada Border Services Agency's Regional Intelligence Firearms Liaison Officers collect and share intelligence from one regional jurisdiction to another. Interviewees overwhelmingly agreed that the Initiative has enhanced the national coordination of investigation and enforcement efforts. As well, according to National Joint Management Team representatives (this Team manages and coordinates the activities of the Initiative and is comprised of program and operational management from all three partner department/agencies), the Initiative contains essential elements of a successful national firearms enforcement strategy.

The Initiative has led to improved investigations and enforcement relating to gun crime, through: the sharing of actionable intelligence that meets the needs of the law enforcement community; increased knowledge of investigative procedures; and, the national coordination of investigations and enforcement. There has been an increase in the number of firearms seized and in the number of hits identified by Canadian Integrated Ballistics Identification Network of the Royal Canadian Mounted Police, suggesting that investigations and enforcement have improved. Finally, stakeholders attested to the fact that the Initiative has improved investigations and enforcement relating to gun crime.

Ultimate Outcomes

According to stakeholder perception, the Initiative contributed to the prevention of the criminal use of firearms. Interviewees noted that the absence of products and services provided by the Initiative would have a significant negative impact on their activities, including successful seizures, efficiency and effectiveness of investigations, and informed policy-making.

In 2008-2009, with the collaborative efforts of a multitude of partners of the Initiative, a significant number of firearms (1,200 firearms or about 18% of all firearms seized by RCMP in a year) had been prevented from entering Canada. In addition, the re-classification of these firearms has prevented future shipments of these firearms from entering Canada.

Performance - Demonstration of Efficiency and Economy

National Joint Management Team representatives indicated that they have implemented changes over the course of the Initiative for the purpose of improving efficiency, including establishing processes and procedures to facilitate access to resources. Stakeholders perceived that the Initiative is being delivered efficiently. In addition, stakeholders reported examples of cost savings to their organizations that have resulted from their use of products and services provided by the Initiative. The current approach of the Initiative was viewed as a good model providing value for money.

Overall, partners are producing the expected level of outputs and users are mostly satisfied with the products and services. However within the Royal Canadian Mounted Police, there is a strong correlation between spending over the budgeted amount allotted under the Initiative and the ability to produce desired outputs and generate a high level of user satisfaction. Units that have spent over the budgeted amount at the Royal Canadian Mounted Police include National Weapons Enforcement Support Teams, Firearms Reference Table, and Canadian Integrated Ballistics Identification Network.3 There are two units in the Royal Canadian Mounted Police that have spent less than the budgeted amount and they are: Criminal Intelligence Directorate and Criminal Intelligence Service Canada. If these two units were to eventually expend the budgeted amount for their activities, the Royal Canadian Mounted Police may not be able to balance its overall budget for the Initiative. Between 2004-2005 and 2008-2009, RCMP spent 2% less than the budgeted amount allotted under the Initiative.

The Initiative was demonstrating increased economy. The cost per seized firearm showed a decreasing trend, starting in 2007.

Recommendations

Two recommendations emerge from the conduct of this evaluation. It is recommended that:

1. Royal Canadian Mounted Police examines what improvements could be made to ensure that the tactical reports and tactical advice provided by the Regional Firearms Intelligence Coordinators of the Criminal Intelligence Directorate are as successful as the other products and services provided by the Initiative; and
2. Royal Canadian Mounted Police assesses the funding allocation of its various activities within the Initiative. Overall, spending by the Royal Canadian Mounted Police is slightly less than the budgeted amount allotted under the Initiative. Royal Canadian Mounted Police needs to address the reasons why some units spend more than the budgeted amount (National Weapons Enforcement Support Teams, Firearms Reference Table, Canadian Integrated Ballistics Identification Network) and why some units spend less than the budgeted amount (Criminal Intelligence Directorate, Criminal Intelligence Service Canada).

1. Introduction

This is the 2009-2010 Evaluation of the Investments to Combat the Criminal Use of Firearms Initiative.

This evaluation provides Canadians, parliamentarians, Ministers, and central agencies an evidence-based, neutral assessment of the relevance and performance of this federal government Initiative. It assesses the extent to which this Initiative continues to address a demonstrable need and the extent to which this Initiative aligns with federal government priorities and roles and responsibilities. It also studies the extent to which effectiveness, efficiency and economy have been achieved.

2. Profile

2.1 Background

The Investments to Combat the Criminal Use of Firearms (ICCUF) Initiative was created in 2004-2005, as part of the Government of Canada's comprehensive package announced in May 2004 aimed at enhancing the capacity of law enforcement agencies to address gun crime and smuggling, among other items.4

To have an informed national enforcement strategy to address gun crime and trafficking of firearms, the Government of Canada must first have coordinated and comprehensive national firearm intelligence gathering and analysis. However prior to ICCUF and as noted in the 1995 Firearm Smuggling Working Group report,5 several gaps in information and intelligence collection have been identified concerning the extent and patterns of smuggling and trafficking of firearms used in crime.

In response and as a first step, the Government launched ICCUF with an aim to enhance the national collection, analysis and sharing of firearms-related intelligence, in order to improve investigations and increase knowledge of patterns of smuggling and trafficking of illegal firearms.

The ICCUF began as a five-year Initiative with a budget of $50 million, which is thereafter ongoing.  It operates as a horizontal initiative within the Public Safety Portfolio, and involves three partner department/agencies:

• Public Safety Canada (PS):
  • Firearms and Operational Policing Policy Division;
• Royal Canadian Mounted Police (RCMP):
  • Forensic Science and Identification Services which operates:
    • Canadian Integrated Ballistics Identification Network;
  • Firearms Investigative and Enforcement Services Directorate which includes:
    • National Weapons Enforcement Support Teams;
    • Firearms Reference Table;
  • Criminal Intelligence Directorate;
  • Criminal Intelligence Service Canada;
• Canada Border Services Agency (CBSA)
  • Intelligence and Targeting Operations Directorate.

2.2 Program Objectives

ICCUF has the following immediate program objectives:

• Actionable intelligence is shared and meets the needs of the law enforcement community; and,
• Increased knowledge of investigative procedures.

These immediate objectives are intended to lead to the following three intermediate program objectives:

• Increased knowledge of gun crime, issues, trends and threats;
• Enhanced national coordination of investigations and enforcement; and,
• Improved investigations and enforcement relating to gun crimes.

Together these immediate and intermediate objectives are intended to enhance the national collection, analysis and sharing of firearms-related intelligence, in order to improve investigations and increase knowledge of patterns of smuggling and trafficking of illegal firearms. Indirectly, these objectives are also intended to lead to the ultimate program objective of the prevention of the criminal use of firearms, contributing to safer homes and streets in Canada.

2.3 Resources and Partner Activities

Table 1 presents the ICCUF budget for each partner during the period 2004-2005 to 2008-2009; and, it provides a summary of partner activities under this Initiative. For a more detailed description of partner activities, see Annex A.

As shown, the RCMP was allocated about 85% of the budget, CBSA 13% and PS 2%. Within the RCMP, NWEST was allocated the majority of the budget followed by CIBIN, CID, FRT, and CISC.

Note:
1) Funding distribution differs from the original allotted funding amount for CISC and NWEST. Since the ICCUF inception in 2004-2005, the responsibility for the RCMP's Firearms Tracing Centre has been shifted from CISC to NWEST. Subsequently for each year since 2004-2005, there was a transfer of $506,583 from CISC to NWEST. The above figures reflect the budgeted amounts of the two RCMP units after the transfer has taken place.

2) When ICCUF was launched in 2004-2005, an amount of $8.965 million over 5 years was allotted to the RCMP National Police Services which included the FRT and CIBIN. The above figures for FRT and CIBIN, which added up to $8.965 million over 5 years, reflected the internal RCMP funding distribution between the two units.

2.4 Program Theory

The logic model presented on the following page is a visual representation that links what the Initiative is funded to do (activities) with what the Initiative produces (outputs) and what the Initiative intends to achieve (outcomes). The logic model for ICCUF was first developed as part of the Evaluation Framework for the Initiative in August 2005,6 and later updated in the 2009 Methodology Report7 to ensure compliance with the 2009 Treasury Board of Canada Policy on Evaluation8 and to reflect changes in organizational structure of partner department/agencies.

Note: Outcome 13 "Prevention of Firearms Tragedies" was not evaluated as there is no foundational document to support that this is an intended ICCUF outcome.

3. About the evaluation

3.1 Objective and Scope

The objective of this evaluation is to provide Canadians, Parliamentarians, Ministers, central agencies and deputy heads with an evidence-based, neutral assessment of the relevance and performance (effectiveness, efficiency and economy) of the Initiative after five years of operation.

In assessing the performance of this Initiative, the focus is to assess the extent to which intended results have been achieved (i.e., outcome achievement). This evaluation will not assess the implementation and management of the Initiative as a formative evaluation was conducted in May 20079 focusing on these aspects. Annex B provides a summary of the recommendations put forward in the formative evaluation and the associated responses of partner department/agencies.

The period of this evaluation is from the launch of the Initiative in 2004-2005 to 2009-2010.10

3.2 Issues

The following questions are formulated based on, and as required by the Treasury Board of Canada Secretariat Directive on the Evaluation Function. Linkages of the questions to the core issues described in the Directive are noted:11, 12

Relevance

• 1. Is there a continuing need for the ICCUF Initiative? (core issue 1)
• 2. Is the ICCUF appropriate to the federal mandate and aligned with federal government priorities? (core issues 2 and 3)

Performance

• 3. To what extent has the Initiative achieved its outcomes? (core issue 4)
• 4. Has the management of the ICCUF Initiative contributed to the efficient use of financial and human resources? (core issue 5)
• 5. Is there another way to deliver the ICCUF Initiative that would provide better value for money? (core issue 5)

3.3 Methodology

This evaluation was conducted in accordance with the Treasury Board Secretariat Standard on Evaluation for the Government of Canada13 and resulting standards of the Evaluation Directorate of Public Safety Canada. The evaluation team employed the 2005 Evaluation Framework and the 2009 Methodology Report as guides, using the following lines of evidence: document review, interviews, consultations with the National Joint Management Team, and a review of performance and financial data.

Each of these methods is described in more detail in the following sections.

3.3.1 Review of documents

The following types of documentation were reviewed for the evaluation (Annex C):

• Corporate, accountability and policy documents: ICCUF partners' Program Activity Architectures, Departmental Performance Reports, and Reports on Plans and Priorities. Other documents included Speeches from the Throne, legislation, briefing notes and related material;
• ICCUF partner operational documents: terms of reference and guidelines for ICCUF partners and related organizations;
• Program outputs: annual and monthly reports, presentations, communiqués, etc.; and,
• Academic, research reports on the subject of firearms such as information on trends in illicit movement of firearms.

3.3.2 Interviews

A total of 55 interviews were conducted using interview guides developed for each of the interview groups,14 as described in the table below.

In order to measure the achievement of ICCUF intended outcomes, this evaluation needed to gather opinions of end-users to assess the effectiveness of ICCUF products and services. Thus, almost all interviewees (groups A to D) are direct users of ICCUF products and services.

Law enforcement personnel (group A and B) are and will remain key recipients of ICCUF products and services. Efforts were made to ensure that law enforcement interviewees (group A) were chosen to have a good representation across provinces and product and service usage. These interviewees were also heavy users of multiple ICCUF products and services15  and were able to provide substantive comments regarding the usefulness of the ICCUF products and services. (See Annex D for a breakdown of these law enforcement interviewees by products and services and provinces.). In addition, the evaluation team interviewed law enforcement personnel serving in small communities (group B),16 who may have different viewpoints as heavy multiple-product users.

Regulatory Users and Crown Attorneys (group C) typically use only one or two of the ICCUF products and services (such as NWEST training or the FRT provided by RCMP). The PS Policy Users interviewee list (group D) was composed of individuals who work in the policy area and who use the research and advice provided by PS. In addition to questions relating to the achievement of ICCUF outcomes, these individuals were asked about how the ICCUF products and services had assisted with policy and strategic planning.

The "Other" interviewee category (group E) included individuals who answered questions related to the relevance of the ICCUF.

3.3.3 National Joint Management Team Consultations

The National Joint Management Team (NJMT) manages and coordinates ICCUF efforts and is comprised of program and operational management from RCMP (CIBIN, CID, CISC, FRT, NWEST), CBSA, and PS. The NJMT has insights of the Initiative's continued relevance and performance and thus, a focus group17 was held with NJMT to garner its opinions with respect to relevance, achievement of outcomes, and efficiency and economy.

In addition, NJMT representatives use each others' products and services extensively and thus, these representatives were asked to individually complete an interview guide to gather their perspectives as users of ICCUF products and services.18 Follow-up interviews were conducted (where necessary) to clarify responses received.

3.3.4 Review of program data (quantitative) and financial data (budget and actual)

ICCUF partners completed a template that was developed and distributed during the evaluation to provide quantitative information relating to ICCUF outputs and outcomes. See Annex G for a copy of the template. Quantitative information was gathered from all partners, for the five-year period between 2004-2005 and 2008-2009. In addition (where available), quantitative information pre-dating the ICCUF Initiative was gathered for some output or outcome indicators (e.g., number of times the FRT was updated or produced, number of requests for firearms identification, number of traces).

Similarly, ICCUF partners completed a financial information template (see Annex H). This template asked the partner department/agencies to provide actual spending information for the same period between 2004-2005 and 2008-2009 and to break down this spending figure based on the ICCUF logic model.

The quantitative information provided was used to assess the achievement of expected ICCUF outcomes, and was combined with the financial information obtained to assess the economy and efficiency of the ICCUF Initiative.

3.4 Limitations of the Methodology

Challenges and/or limitations relating to the methodology of the evaluation include:

1. Interviews with law enforcement users: An Initiative such as ICCUF inevitably serves a broad spectrum of users from the law enforcement community. However, it was not feasible both technically and economically, to solicit opinions from all these users. A decision was made to select interviewees to ensure representation across all ICCUF products and services and across all provinces. As well, most of these interviewees were heavy users of multiple ICCUF products or services, and thus were able to provide substantive comments. The extent of the "reach" of ICCUF products and services in servicing occasional users was examined via some of the output indicators (e.g., number of training participants). In addition, viewpoints of law enforcement personnel serving small communities were also solicited.
2. Quantitative data: Quantitative data captured by ICCUF partners are documented either fiscally or by calendar year, and financial data are available only by fiscal year, creating inconsistencies in some efficiency calculations (comparing costs to outputs). In addition, some performance data relating to outcome indictors (i.e., firearms seizures) were not available for years prior to the implementation of ICCUF. As a result, a pre- and post-ICCUF comparison, which would have been useful to assess the impact of the ICCUF Initiative, was not possible.
3. Country comparison: A comparison among countries was originally planned to compare the cost per seizure of a firearm under the ICCUF Initiative to the cost per seizure under similar initiatives in other countries. This analysis was not possible within the cost and time constraints of the evaluation. After a preliminary review, the evaluation team could not find programs in other countries that have comparable data available (i.e., cost of a firearms-specific program and/or number of firearms seized).

3.5 Protocol

Engagement and Collaboration

An interdepartmental ICCUF Evaluation Working Group was established to support the planning and conduct of the evaluation, as well as the coordination of the review of draft reports. This collaborative arrangement was comprised of the National Joint Management Team and the evaluation representatives of Public Safety Canada, the Royal Canadian Mounted Police, and Canada Border Services Agency. Public Safety Canada Evaluation Directorate led this evaluation.

During the conduct of the evaluation, the working group assisted in the identification of key stakeholders and provided documented evidence to support the evaluation. Collaborative participation, with its multi-party effort and engagement, greatly enriched the evaluation process and enhanced the reporting product.

Approvals

Each ICCUF partner department/agency has accepted and approved the final draft evaluation report, including the combined management response and action plan. These documents were presented to the Public Safety Canada Departmental Evaluation Committee for consideration and for final approval by the Deputy Minister of Public Safety.

4. Findings

This section presents the research findings for the evaluation of the ICCUF Initiative, and the supporting evidence for those findings. The findings are grouped under three main headings: Relevance; Performance - Achievement of Expected Outcomes; and, Performance - Demonstration of Efficiency and Economy.

4.1 Relevance

4.1.1 Continuing Need for ICCUF Initiative

Extent of Problem of Gun Crime/Violence

Document review indicates that gun crime/violence is a problem in Canada, and gun violence is expected to increase in the coming years if current trends continue.19  According to Juristat, Homicide in Canada, 2008 (Statistics Canada publication in 2009), there were 611 police-reported homicides in Canada; 200 were committed with a firearm in 2008. The same source reports that the rate of firearm homicides in Canada increased 24% between 2002 and 2008, including a 5% increase in 2008. Furthermore, research20 indicates there has been an increase in the use of firearms by criminal organizations and street gangs.

Firearms incidents occurring in public places raise the issue of public safety. For example, in the 2007 National Strategic Firearms Threat Assessment published by CISC (RCMP), it was reported that in 2006, there was an increase in the number of shots-fired incidents that occurred in public places in British Columbia. In addition, events such as the Dawson College shootings, in September 2006, have brought gun crime issues to the foreground.

Information and Intelligence Gaps

As noted in the 1995 Firearm Smuggling Working Group report and the 2006 CISC's National Strategic Firearms Threat Assessment, there were challenges in collecting information and intelligence to determine the magnitude of the firearms smuggling problem. There was no national database to record seized crime guns, no mandated format for reporting, and no mandatory requirements for all seized crime guns to be submitted for a trace.

Other than difficulties encountered in information and intelligence gathering, other challenges included:

• Law enforcement officials across the country noted a lack of resources as well as challenges associated with information sharing;21
• Law enforcement Canada's Integrated Ballistics Identification System (IBIS) machines were fully occupied by requests from law enforcement agencies, and thus were not able to meet the increased demand in providing investigative leads to link firearm crimes, creating delays in investigations and prosecutions; and,
• The search for explanations and effective interventions after firearms-related homicides in 2006 revealed how little research was available in Canada to address these issues.22

The ICCUF Initiative was created to fill in many of these gaps by:

• Allocating dedicated firearms specialists across Canada (NWEST and CID of RCMP, and CBSA Regional Intelligence Firearms Liaison Officers resources) to provide investigative support in communities;
• Providing three new IBIS terminals and the Canadian Integrated Ballistics Information Network (RCMP);
• Establishing policy, research, and strategic intelligence capacities (i.e., PS and RCMP's CISC); and,
• Putting in efforts to improve the collection of consistent firearm-related data. For example, as of January 1, 2007, the National Minimum Collection Standard was adopted.

Perceptions of Continued Need for the ICCUF Initiative

Interviewees agreed that there is a continued need for the ICCUF Initiative. When asked how likely they are to continue to use the ICCUF products and services, all interviewees reported they were somewhat or very likely to continue. Furthermore, most interviewees (83%) indicated that the overall effectiveness of their jobs would be affected 'to some or a large extent' in the absence of ICCUF. They noted:

• Investigations would be less effective due to gaps in information and intelligence and a loss of knowledge and expertise;
• There would be less efficient and timely investigations; and,
• There would be an increase in their workloads if they had to do work for which they currently rely on ICCUF partners for assistance.

Users of PS research and advice noted that, in the absence of the ICCUF Initiative, they would not be able to provide timely and informed policy advice.

When asked what the impact would be if ICCUF products and services were not available, both the heavy multi-product users and users serving small communities provided the same perspective. Both groups reported that if they did not have the ICCUF expertise, investigations would take much longer to complete. Both groups indicated it would be much harder for them to identify linkages between related crimes (e.g., same firearm used in different crimes, or same person who commits different crimes).

4.1.2 Alignment to the Federal Mandate and Federal Government Priorities

Appropriateness to the federal mandate

The legislative authorities of relevance to the ICCUF Initiative include: the Customs Act, the Anti-terrorism Act, the Firearms Act, and the Criminal Code, Part III. As well, given that guns and criminals travel across domestic and international borders, a federal role avoids a "patchwork" approach to intelligence development and analysis related to firearms in Canada.

Thus far, the ICCUF Initiative is the only national firearms-related initiative of its kind. Interviewees named other similar provincial and municipal initiatives (e.g., Provincial Weapons Enforcement Unit in Ontario), but interviewees viewed the ICCUF Initiative as complementary rather than duplicative. Interviewees noted that ICCUF partners with these other provincial and municipal law enforcement units in different scenarios and capacities (sometimes co-locating at the same office or building).

Al ignment with government public safety agenda

In recent years, the Government of Canada has stated that keeping Canadians safe by "getting tough on crime" and advocating "stronger penalties for gun crimes" as government priorities. This is illustrated in a number of previous Speeches from the Throne, as indicated in the table below:

Table 3: Speeches from the Throne

Speech from the Throne March 3, 2010

"...those who commit crimes must be held to account.... Our Government acted decisively to crack down on crime and ensure the safety and security of our neighbourhoods and communities. It introduced laws mandating prison sentences for gun crimes...".

Speech from the Throne November 18, 2008

"Our Government will take tough action against crime and work with partners to improve the administration of justice. Serious offences will be met with serious penalties... Gun laws will be focused on ending smuggling and stronger penalties for gun crimes, not at criminalizing law-abiding firearms owners".

Speech from the Throne October 16, 2007

"There is no greater responsibility for a government than to protect this right to safety and security... Our Government will immediately reintroduce (these) measures with a single, comprehensive Tackling Violent Crime bill to protect Canadians and their communities from violent criminals and predators. This will include ... stricter bail and mandatory prison sentences for those who commit gun crimes".

Alignment with priorities of partner department/agencies

As noted in the 2010-2011 Reports on Plans and Priorities of partner department/agencies, the intended outcomes of ICCUF Initiative are aligned with the strategic objectives and priorities of partner department/agencies in the following ways:

Table 4: Alignment of ICCUF Objectives with Strategic Outcomes/Priorities of Partner Department/Agencies

• Public Safety Canada
  • safe and resilient Canada
  • support policing in Canada and to combat serious and organized crime
• Royal Canadian Mounted Police
  • safe home and safe communities for Canadians
  • minimize risk to public safety from firearms
• Canada Border Services Agency
  • Canada's population is safe and secure from border-related risks. CBSA prevents the movement of unlawful people and goods across the border.

4.2 Performance - Achievement of Expected Outcomes

This section of the report examines to what extent the ICCUF Initiative has achieved the intended outcomes of:23

• Actionable intelligence is shared and meets the needs of the law enforcement community;
• Increased knowledge of investigative procedures;
• Increased knowledge of gun crime issues, trends and threats;
• Enhanced national coordination of investigations and enforcement;
• Improved investigations and enforcement relating to gun crime; and,
• Prevention of the criminal use of firearms.

4.2.1 Actionable Intelligence

To assess the extent to which the ICCUF Initiative achieved this outcome, the evaluation examined the extent of the distribution and "reach" of intelligence products and services, and then if the recipients of the intelligence perceived it to be 'actionable'. In addition, the evaluation looked at ICCUF partner department/agencies' participation in joint force operations (JFOs), with the expectation that participation in JFOs would result in the sharing of actionable intelligence through the communication and cooperation gained during the operations.

Provision of Intelligence products and services

ICCUF partners provide a range of strategic and tactical intelligence products and services to a wide audience. The following is a discussion of the strategic and tactical intelligence products and services provided, with some observation on the frequency of reporting, dissemination and/or requests for services. For a more detailed discussion, see Annex I.

CBSA and RCMP (CISC and NWEST) are both involved in producing strategic intelligence products for the ICCUF Initiative. Products include: CISC's National Strategic Firearms Threat Assessment (NSFTA) and Strategic Intelligence Briefs (SIBs); CBSA Strategic Firearms Threat Assessments; and, NWEST National Joint Assessments on Illicit Guns and their Sources.

The products are produced anywhere from yearly to monthly and are disseminated to a wide audience inside and outside of the organizations that produce them. The trend in the frequency of the production of reports varied over the evaluation years as follows:

• Products with decreased production frequency:
  • CBSA Strategic Firearms Threat Assessments (monthly in 2007 to annually projected for 2010); and,
• Products with unchanged production frequency:
  • NWEST National Joint Assessments on Illicit Guns and Their Sources (produced four times per year); and,
  • CISC National Strategic Firearms Threat Assessment (NSFTA) and Strategic Intelligence Briefs (SIBs) (production of the NSFTA decreased from once per year to once every two years, but SIBs were added which offset the decrease).

CBSA and RCMP (CIBIN, CID, FRT, NWEST) both produce tactical intelligence products and services. These include:

• The FRT - an electronic database of firearms descriptions and identification and Canadian legal classifications, cross referenced to the Criminal Code, provided to police and regulatory agencies;
• Requests for CIBIN correlation - CIBIN is a network of Integrated Ballistics Identification System (IBIS) instruments that collect, analyze and correlate bullets and cartridge cases with a view to linking criminals to crimes and linking related crime scenes across Canada;
• CID tactical reports - firearms-specific intelligence gathered from the work of CID front-line intelligence officers working on enforcement activities;
• Request for NWEST firearms investigative advice and tracing;
• CBSA RIFLO monthly reports;
• CBSA Port of Entry lookouts; and,
• CBSA firearms intelligence files.

Some of the tactical products, such as the FRT and the CID tactical reports, are for distribution to a wide audience and some, such as the CBSA firearms intelligence files and CBSA RIFLO monthly reports, are for internal organizational use only. Other services offered to partners include: requests for CIBIN correlations; requests for NWEST investigative advice and tracing; and CBSA Port of Entry lookouts.

There is some indication that interest in the FRT and CIBIN/IBIS has increased over the years as demonstrated by an increase in the dissemination of the FRT and an increase in the number of CIBIN/IBIS requests. Although the evaluation team could not obtain reliable trend statistics in NWEST trace requests,24 there is evidence to show that these requests are country-wide, from all provinces and even internationally. CBSA Port of Entry lookouts fell from a high of 524 in 2005-2006 to a low of 370 in 2007-2008, before rising again over the two subsequent years to 488 in 2009-2010. The number of CBSA firearms intelligence files has been increasing over the years, from 225 files in 2005-2006 to 527 in 2009-2010.

Degree to which intelligence is actionable and useful

Most (75% of respondents) ICCUF stakeholders "somewhat or strongly agreed" that specific ICCUF intelligence products and services had provided them with actionable intelligence. However, there was some variance with the extent of agreement across products and services:

• All respondents "somewhat or strongly agreed" that FRT provided them with actionable intelligence;
• About 85% of the respondents said the same, including:
  • Intelligence provided by CBSA RIFLOs (e.g., intelligence reports and bulletins);
  • CIBIN correlation; and,
• Two-thirds of the respondents "somewhat or strongly agreed" that CID tactical reports provided actionable intelligence.

Though RCMP's CISC provides strategic intelligence and is not expected to provide actionable intelligence, half of CISC users still "somewhat agreed" that the CISC National Strategic Firearms Threat Assessments and/or Strategic Intelligence Briefs provided them with actionable intelligence.

In many cases, interviewees also reported that ICCUF investigative support products and services provided them with actionable intelligence, even though these products and services are primarily intended to provide investigative support as opposed to intelligence. This may suggest the inter-connectedness between intelligence and investigative support products and services. For example, interviewees identified the following products or services as providing actionable intelligence:

• Intelligence support to police investigations by CBSA RIFLOs;
• NWEST (RCMP) training and information sessions;
• Tracing by NWEST;
• Investigative advice or support by NWEST or the 24/7 investigative support line; and,
• NWEST expert advice and witnesses.

All interviewees indicated that the ICCUF products and services are useful to them. When asked overall how useful, to them or their organization, ICCUF products and services were, all interviewees responded either very or somewhat useful. Most law enforcement interviewees who serve at small communities (seven out of eight) rated the ICCUF products and services to be "very useful". The usefulness of the RCMP's FRT and CIBIN/IBIS is further corroborated by feedback surveys. According to the 2007 FRT client feedback survey, most FRT users considered the product an essential tool or a valuable asset. Based on the CIBIN/IBIS feedback survey results as of February 2010, a little over half of CIBIN/IBIS users found the CIBIN/IBIS helpful.

Participation in Joint Forces Operation

According to the NJMT representatives, ICCUF partners have participated in joint force operations (JFOs) at different levels as follows:

• RCMP's NWEST members, RCMP's CID Regional Firearms Intelligence Coordinators (RFIC), and CBSA RIFLOs are directly participating in provincial enforcement units or specialized weapon teams; such as the BC weapons unit, Toronto Guns and Gangs, the Ontario Provincial Police's Provincial Weapons Enforcement Unit, the Fundy Integrated Intelligence Unit in New Brunswick and the Joint Forces Firearms, Ammunition and Explosives Investigations Unit in Quebec. These are multi-agency, co-located JFOs;
• ICCUF partners participate on project-based JFOs which may range anywhere from a couple of months to a couple of years. These formalized project-based JFOs exist to address a specific problem. ICCUF members on these JFOs contribute information and intelligence, advice, resources, training and education;
• ICCUF members participate in short-term, ad hoc JFOs, which can range from a few days to a few weeks, for a specific purpose.

Quantitative information about the number of JFOs, in which ICCUF partner department/agencies participated, was not available.

4.2.2 Increased Knowledge of Investigative Procedures

To assess the extent to which the ICCUF Initiative achieved this outcome, the evaluation examined the provision of investigative support products and services (i.e., training sessions, advice and support provided by ICCUF partner department/agencies). It further examined if the recipients of these outputs believed it had increased their knowledge of investigative procedures. It should be noted that while the ICCUF logic model shows only investigative support provided by NWEST (RCMP) as contributing to increased knowledge of investigative procedures, agencies other than NWEST provide advice and expertise, which also contributes to increasing knowledge of investigative procedures.  More specifically, CBSA RIFLOs and RCMP's CID Regional Firearms Intelligence Coordinators (RFIC) also provide advice and assistance.

Provision of investigative support products and services

According to the 2007 ICCUF Formative Evaluation, the ICCUF has contributed to increased knowledge of investigative procedures as evidenced by the investigative advice provided, the number of training sessions delivered, and perceptions of interviewees at that time.

Since that time, a significant amount of investigative advice/assistance and training sessions continues to be provided and delivered. Requests for NWEST investigative assistance were significant, and increased from 3,180 in 2006-2007, to 3,445 in 2008-2009.25 It should be noted that a significant proportion of the calls for assistance received by NWEST came from users associated with smaller forces or communities; in 2009, 34% of NWEST calls for assistance were devoted to users from smaller communities.

The number of requests for assistance in the production of search warrants remained steady between 2003 and 2008, at between 150 and 250 requests, but then increased dramatically in 2009 to a total of 939 requests. While the evaluation was unable to determine the cause of the increase in 2009,26 NJMT representatives speculated that because search warrant production requires specialized knowledge, most law enforcement officers would not possess sufficient knowledge to start the process to issue one and therefore, the low number in requests for assistance. However, as these officers increase in their knowledge as a result of better information (e.g., training) and intelligence provided through ICCUF, they become better equipped in pursuing the issuance of search warrants in investigations (and thus, a higher number of requests relating to search warrants).

With regard to NWEST training, the number of training participants remained about the same between 2005-2006 and 2008-2009, at 3,300 participants each year. There was an exception in 2004-2005, when just over 4,000 participants received training. The number of training or information sessions was steady at about 155 to 170 sessions over the 2004-2005 to 2007-2008 period, but dropped to 121 sessions in 2009.

Estimates provided by the RCMP's CID indicated that there were approximately 100 training or information sessions delivered by the unit between 2005 and 2009 (this often included informal training by RFICs to stakeholders). The average number of participants per event was 30. In addition, CID estimated that over the five-year period, there were approximately 2,900 instances of tactical advice provided by RFICs to support operations; going from a low of 200 in 2005, to an estimated high of 800 in each of 2008 and 2009.

No information is available with regard to the number of instances of advice provided by CBSA RIFLOs. CBSA provides advice to police investigations through the RIFLO network and, according to CBSA program representatives, advice is provided frequently (as often as daily) and the number of instances of advice has not been tracked.

Perceptions of increased knowledge of investigative procedures

Stakeholders agreed that ICCUF products and services have increased their knowledge of investigative procedures. Overall, the majority of interviewees "somewhat or strongly agreed" that ICCUF products or services increased knowledge of investigative procedures. However, agreement varied depending on the product:

• For firearms tracing provided by NWEST (RCMP), almost three-quarters of interviewees "somewhat or strongly agreed" that this contributed to increased knowledge of investigative procedures;
• Many (about 65%) "somewhat or strongly agreed" for:
  • NWEST firearms-related expert advice and witnesses;
  • Investigative advice or support from NWEST or the 24/7 Investigative Support Line;
  • Intelligence support to police investigations by CBSA RIFLOs; and,
  • NWEST training and/or information sessions.
• Only 27% "somewhat or strongly agreed" that tactical advice from RCMP's CID RFIC increased their knowledge of investigative procedures.

Three of the ICCUF NJMT representatives who completed an interview guide for the evaluation had used NWEST training/information sessions. Two of these three individuals agreed or strongly agreed that NWEST training or information sessions had increased their organization's knowledge of investigative procedures. NJMT representatives also noted other ICCUF products and services had increased their knowledge of investigative procedures such as: NWEST expert advice and witnesses, firearms tracing, NWEST investigative advice or support, and CBSA RIFLO intelligence support to police investigations.

Feedback information was largely not available for training sessions, conferences or workshops delivered by ICCUF partners.

4.2.3 Increased Knowledge of Gun Crime Issues, Trends and Threats

To assess the extent to which the ICCUF Initiative achieved this outcome, the evaluation examined research and intelligence produced by the partners, as well as the perceptions of ICCUF stakeholders as to whether these products and services had led to increased knowledge of gun crime issues, trends and threats. In addition, the evaluation assessed whether stakeholders believe ICCUF strategic intelligence products and services had assisted with informed policy making and strategic planning. It should be noted, however, that while ICCUF products and services are expected to inform policy development, policy advice is inevitably informed by the activities of, and products from, various sources.

Provision of research and strategic intelligence products

The two main ICCUF components of research and strategic intelligence that have contributed to increased knowledge of gun crime issues, trends and threats are: PS's Firearms and Operational Policing Policy Division, which produces research reports and provides policy advice; and RCMP's CISC, which produces strategic intelligence products such as the National Strategic Firearms Threat Assessment (NSFTA) and Strategic Intelligence Briefs.

PS and RCMP's CISC have contributed to increased knowledge of gun crime issues, trends and threats through the publication of PS research reports and studies, and the production of the CISC's National Strategic Firearms Threat Assessment. Between 2006 and 2009, PS produced nine research reports,27 and is undertaking an additional three studies.28 Since 2005-2006, there have been 230 recipients of these research reports/studies. It should be noted these reports are security classified, and distribution is restricted to those with required security clearance. CISC produced the National Strategic Firearms Threat Assessment (NSFTA) for the years 2006, 2007 and 2008, with a distribution of approximately 600 reports each year. In order to more effectively meet the needs of the Canadian intelligence community, CISC, in consultation with its NJMT partners and CISC's membership, decided to produce Strategic Intelligence Briefs (SIBs) on specific firearms-related topics ([ * ]) in place of doing the NSFTA for 2009-2010.29 From now on, CISC intends to produce the NSFTA every two years, including the 2010-2011 fiscal year.

Perceptions of increased knowledge of gun crime issues, trends, and threats

Stakeholders agreed that the strategic intelligence generated by ICCUF products and services has increased their knowledge of gun crime issues, trends and threats. Most interviewees (92%) indicated that ICCUF products and services had increased their knowledge of gun crime issues, trends and threats to "some or to a large extent" by:

• Increasing the "big picture" knowledge of trends and issues of gun crime;
• Increasing the knowledge of supply, demand and distribution of firearms, including sources for supply and demand, distribution channels, ports of entry and movement; and/or,
• Acting as a source of quantitative and statistical information for educational use.

All interviewees, who provided comment on the subject, indicated the strategic intelligence products and services, have assisted with informed policy-making and strategic planning to "some or to a large extent". In addition, interviewees who work in the policy area reported that ICCUF-generated research and intelligence has increased their knowledge enabling them to provide better-informed policy advice. These interviewees noted that ICCUF partners' accumulated knowledge and up-to-date information on firearms have also helped to ensure that their policy advice and policy development are relevant. There is a full range of policy-related documents provided and the number of documents provided by PS has been steadily increasing over the years (from 75 documents in 2005-2006 to 155 documents in 2009-2010), with a reported total of 560 documents between 2005-2006 and 2009-2010.

All NJMT representatives reported that strategic intelligence generated by ICCUF products and services increased their knowledge of gun crime issues, trends and threats "to a large extent". These representatives indicated that intelligence provides the current status, environment and emerging issues relating to gun crime; and that intelligence is communicated at a national, strategic and comprehensive level.

4.2.4 Enhanced National Coordination of Investigations and Enforcement

To assess success against this outcome, the evaluation looked at indications of coordination prior to the ICCUF Initiative, and stakeholder perceptions about the level of coordination since the introduction of the ICCUF Initiative.

Coordination prior to ICCUF

Documentation suggests that in the years leading up to the ICCUF Initiative, firearms-related information and intelligence had been collected by various agencies but not in a coordinated effort. [ * ];30

Perceptions of coordination since ICCUF

Stakeholders believed that the ICCUF Initiative has enhanced coordination of investigations and enforcement relating to gun crime. Interviewees believed that the Initiative has enhanced, 'to some extent' or 'to a large extent', the coordination of gun crime investigations across law enforcement organizations (97%), within provinces (100%), across provinces (97%), and nationally (97%). Specific examples given by respondents of how ICCUF has enhanced coordination of investigations and enforcement relating to gun crime include:

• Contact points are readily identifiable;
• Practices are institutionalized; and,
• Integrated network incorporates strategic, intelligence and investigation units.

NJMT representatives also noted that the ICCUF network has increased the connectivity among partners and law enforcement agencies. Particularly, these representatives noted that national connectivity is enhanced because NWEST (RCMP) provides a "one-stop shop" for law enforcement officers; and CBSA RIFLOs collect and share intelligence from one regional jurisdiction to another and work with local law enforcement organizations.

Elements of a successful national firearms enforcement strategy

A national enforcement strategy has not yet been developed,31 however NJMT representatives suggested that the ICCUF Initiative provides a foundation for the development of a national enforcement strategy in many ways:

• It identifies best practices for investigative procedures;
• It provides a national minimum collection standard;
• It has developed standard operating procedures;
• It provides training on various aspects related to firearm investigations;
• It provides a uniform method to identify and classify firearms, through the FRT (RCMP);
• tracing is nationally standardized;
• Information and intelligence gaps have been identified and addressed;
• Partners are working closely together, which enables a better understanding of each other's standards and needs; and,
• It provides national linkage of firearms used in crime scenes through CIBIN (RCMP).

4.2.5 Improved Investigations and Enforcement Relating to Gun Crime

To assess success against this outcome, the evaluation examined the number of firearms seized as an indicator of improved investigations and enforcement; the number of CIBIN hits; and perceptions of improvements.

Number of Firearms seized

Table 5 indicates the number of firearms seized by NWEST (RCMP) during the 2005 to 2009 period.32 While these seizure figures were provided by NWEST, it should be noted that activities, products and services provided by all ICCUF partner department/agencies contributed to these seizures, as indicated by the program theory articulated in the logic model. As the table demonstrates, the number of seized firearms has been increasing since 2007. In particular, there is a sizeable increase in the number of seized firearms in 2009. It is possible that this large increase in seizures is correlated to the large increase in search warrants noted previously.

In addition to the NWEST seized firearms above, a total of 109 firearms suspected to be involved in criminal activity33 were also seized at the border by CBSA, during the 2006-2009 period.34 In addition to firearms seizures, CBSA noted that the ICCUF Initiative may result in other enforcement actions, such as detentions of guns at the port of entry, which may also serve as an indicator of improved investigations and enforcement relating to gun crime. Statistics relating to CBSA detentions were not available for this evaluation.

CIBIN Requests and Hits

The use of CIBIN (RCMP) by law enforcement partners and any resulting "hit" for bullets and cartridge cases is also an indicator of improved investigations. A CIBIN hit provides a link discovered between two events (e.g., shootings, murders) that were not previously suspected to be related. As shown in Table 6, the number of CIBIN requests and hits for both bullets and cartridge cases has increased. While the trend was increasing already prior to the introduction of the ICCUF Initiative, the ICCUF funding allowed for the purchase of new IBIS terminals that might have provided the additional resources to accommodate further increases in requests.

Perceptions of improved investigations

All interviewees agreed that ICCUF products and services have been useful and have led to more successful investigations and enforcement relating to gun crime to "some or to a large extent". They noted that ICCUF products and services have been useful in providing a complete and comprehensive investigative and enforcement service and a forum for knowledge-sharing and exchanges between the various partners and stakeholders.

NJMT representatives noted that the ICCUF network increases connectivity among partners, which, in turn, increases the efficiency and effectiveness of investigations and enforcement activities. They pointed out an increased awareness of the existence of the ICCUF Initiative at all levels allows for informed resource deployment for more effective investigative effort. They also noted that the ICCUF Initiative allows for greater synergy and efficiency as a result of the ICCUF network's accumulated skill, experience and knowledge.

4.2.6 Prevention of the Criminal Use of Firearms

The evaluation attempted to assess the number of firearms seized before and after the implementation of the ICCUF Initiative as a measure of the impact of ICCUF on the prevention of the criminal use of firearms. However, seizure information was not available from partner department/agencies (RCMP and CBSA) prior to ICCUF. As a result, the evaluation has had to rely on interviewee perceptions to assess whether or not the ICCUF Initiative has achieved its ultimate outcome of prevention of the criminal use of firearms.

Perceptions regarding the prevention of the criminal use of firearms

Interviewees and NJMT representatives believed the ICCUF Initiative has contributed to the prevention of the criminal use of firearms. When asked what the impact would be if ICCUF products and services were not available, the most frequently reported impacts were:

• A reduced number of seizures;
• Increased time and complexity of investigations;
• Less capacity to identify leads and link related crimes;
• Gaps in information and intelligence and gun crime knowledge; and,
• Less informed policy decisions.

Interviewees and NJMT representatives provided examples of how ICCUF contributes to the prevention of criminal use of firearms. According to these stakeholders, ICCUF contributes to the prevention of criminal use of firearms because:

• Successful seizures ensure that guns are no longer on the streets;
• The ICCUF network and services allow for tracking, monitoring and intercepting trafficking activities and suspects; and,
• ICCUF products and services allow officers to collect and bring the right evidence to court for successful prosecutions.

The following significant example provides a demonstration of how the collaborative efforts of ICCUF have led to the prevention of the criminal use of firearms.[ * ]35

[ * ]36.

[ * ]37.

[ * ] According to NJMT representatives, the partnerships and relationships that developed as part of the ICCUF Initiative made this enforcement action possible.

4.3 Performance - Demonstration of Efficiency and Economy

Under the 2009 Treasury Board of Canada's Policy on Evaluation, efficiency is defined as maximizing the outputs produced with a fixed level of inputs or minimizing the inputs used to produce a fixed level of outputs; and economy is defined as "minimizing the use of resources [...] to achieve expected outcomes". These elements of performance are, therefore, demonstrated when:

1. Outputs are produced at minimum cost (efficiency); and
2. Outcomes are achieved at minimum cost (economy).

4.3.1 Efficiency

In assessing efficiency of the ICCUF Initiative, the evaluation analyzed expenditures against budgets; spending variance versus satisfaction with outputs; cost per output by logic model component; and perceptions of efficiency and value for money.

Expenditures against budgets

Table 7 illustrates the total expenditures on the ICCUF Initiative over the evaluation period. Overall, ICCUF partner department/agencies spent 2% less than the ICCUF budgeted amount.

Note: Units which have spent more than the ICCUF budgeted amount (actual expenditure greater than budget) are indicated in the "% Variance" column by a "positive" percentage; while those that have spent less than the ICCUF budgeted amount are indicated by a "negative" percentage.

The expenditures by some of the partners have varied from the budget, with some partners spending less than the ICCUF budgeted amount (as indicated by a negative variance value in Table 7 above) and some spent more than the budgeted amount (as indicated by a positive variance value in Table 7 above). CBSA was on target and PS expended less than budgeted. Spending by the RCMP overall was 2% less than the ICCUF budgeted amount. Within RCMP, CID and CISC expended less than budgeted; and, FRT, NWEST, and CIBIN expended more than anticipated.

Spending variance versus user satisfaction levels

To be able to interpret reasons for the budget variance noted in Table 7, the level of success achieved and the level of user satisfaction were considered. Findings in section 4.2 were utilized to determine, a) if units have produced the desired outputs, and b) the level of user satisfaction. These results are summarized in Table 8.

As shown in Table 8, those partner department/agencies that spent amounts in excess of their ICCUF budgets all produced the desired outputs, and had high satisfaction levels with their users. PS, which under-expended its budget by 17%, also produced its expected outputs and had a high level of satisfaction among its users. CISC (RCMP), which under-expended by 42%, produced the desired outputs and had a moderate to high level of user satisfaction. For the most part, CID (RCMP) produced the desired outputs, although user level of satisfaction for this organization was somewhat lower.

The reasons provided by NJMT representatives, for under-expending on the ICCUF budget, included the late receipt of funds in the first fiscal year (2004-2005) and difficulty finding qualified personnel to fill positions. CID and CISC both expected to increase staffing levels to support the production of desired outputs for meeting the needs of the law enforcement and intelligence communities. In neither case was the desired staffing level achieved, therefore the full ICCUF budget was not expended. PS program staff indicated that their under-expending was due to delays in research conducted in 2007-2008 because of difficulty experienced in accessing expert interviewees.

Within RCMP, FRT, NWEST, and CIBIN program representatives indicated they spent more than their ICCUF budget amount because their allotted ICCUF budget was not sufficient to support their operations. Thus, a deliberate decision was made within RCMP to re-allocate its budget in-house. Between 2004-2005 and 2008-2009, FRT, NWEST, and CIBIN had funding from sources within RCMP on top of the ICCUF budgeted amount. FRT received funding from the RCMP's Canada Firearms Centre. Funding from the Centre ended in 2005-2006.39 During the 2004-2005 and 2008-2009 period, NWEST received funding from Canada Firearms Centre. CIBIN received an amount of $660,712 from RCMP to evergreen one IBIS instrument in 2008-2009. The CIBIN representative noted that the allotted ICCUF budget for CIBIN is not enough to evergreen the IBIS instruments, which will become obsolete by March 2013.

Cost per output by logic model component

Efficiency was further examined by reviewing the costs of the various categories of ICCUF outputs under the logic model components of policy and research, intelligence, investigative support and other. ICCUF partners were asked to provide the percent of their budget that was expended on producing each of the ICCUF outputs listed in the logic model. These percents were then multiplied by the actual expenditures of each partner and added together to get the combined cost per output for all partners.

Table 9 provides a summary of the total cost, over the 2004-2005 to 2008-2009 period, to produce each of the logic model outputs; the percent of actual partner cost expended on each output; and a listing of the partners that contribute to that output.

The table above demonstrates that the majority of funds were being expended on investigative support (56.9%), followed by intelligence-related outputs (38.3%), policy/research (3.6%) and other (1.3%). These expenditures are in line with expectations as NWEST was allocated the greatest percent of the budget (51%) and was heavily involved in providing investigative support. All of the partners (with the exception of PS) were involved somewhat in producing intelligence and PS, which was allocated the smallest portion of the budget (2.4%), was the primary producer of policy and research.

As discussed in relation to Table 8, partners were producing the expected level of outputs and users are, for the most part satisfied with the products and services that were being produced. The exception was with CID (RCMP) outputs. Table 9 above shows that the limited CID expenditures were spread out across tactical intelligence and investigative support. CID was provided with funds to establish a dedicated RCMP firearms intelligence collection program through placing front-line intelligence officers in major centres (six full-time equivalents), but CID had difficulty with staffing and had not been able to hire the desired number of front-line intelligence officers (CID expended 55% of budget). The intelligence officers that are available may be spread too thin with their involvement in training (15% of expenditure), providing investigative support (35% of expenditure) and producing CID tactical reports (50% of expenditure).

Improvements in Efficiency and User Perception of Efficiency

NJMT representatives indicated that they have implemented changes over the course of the Initiative for the purpose of improving efficiency. Examples provided included more efficient processes (e.g., national minimum collection standards) and the fact that the FRT is now available on the Internet, in addition to a DVD version.

As well, interviewees agreed that the use of ICCUF products and services has resulted in cost savings to their organizations. Interviewees noted the following as the primary ways that ICCUF products and services contributed to cost savings with some specific quotes from users:

• The ICCUF reduces the time and level of effort needed by law enforcement officers in conducting their investigations by providing access to expertise and specific services (i.e., law enforcement agencies save time and effort because they do not need to conduct their own traces, they can use best practices identified by ICCUF and avoid investigative methods/procedures that were unsuccessful, and ICCUF resources provide them with analytical assistance):

  "NWEST provides a very specialized services, within half-an-hour they can send us complete information to help us link cases for instance and lead to an arrest. In the past, it took two to three days to link cases and identify suspects."

• The ICCUF Initiative allows law enforcement organizations to avoid costs they may otherwise need to assume for additional staff or man-hours, or through contracting external resources (e.g., research and expertise currently provided by ICCUF):

  "ICCUF represents a significant saving in time and salary dollars for additional resources. If we didn't have it, it would take a long time to identify firearms and would require additional resources to do the work."

The current model was viewed as providing good value for money by stakeholders.  Some existing similar products and services were mentioned, but they were perceived to be complementary to ICCUF. The following reasons were given for why alternative non-federally led models would not work:

• Criminals would be in a better position to capitalize on gaps and variances in procedures between police jurisdictions;
• Intelligence would not be as comprehensive or timely;
• Efforts would be ad hoc;
• Less sharing of information;
• Loss of national picture; and,
• Loss of control over quality.

4.3.2 Economy

In assessing economy of the ICCUF Initiative, the evaluation conducted an analysis of the number of firearms seized by the RCMP as an indicator for the outcome "prevention of the criminal use of firearms". This indicator directly aligns with this outcome since the removal of a firearm, through a seizure, prevents its future criminal use. In addition, the numbers of firearms seized is quantifiable and, while the actual firearms seizures are made only by NWEST (RCMP) and CBSA, all ICCUF partners contribute to the seizures through their ICCUF activities.

Table 10 illustrates that the cost per RCMP seized firearm peaked in 2007 ($1,859) and has been declining for the last two years to $1,212 per seized firearm in 2009. The downward trend would indicate that the program is becoming more economical (producing more results for less cost) over time. It should be noted that, although firearms are seized by CBSA, the cost of these seized firearms are not included in the calculation.41 The economy calculation is therefore conservative (the actual cost per firearm seized is lower than those reported below).

^* Note that seizures statistics are reported by calendar year but financial information (cost) is reported by fiscal year.

5. Conclusions

5.1 Relevance

There is a continuing need for the ICCUF Initiative. Gun crime/violence is prevalent and appears to be on the rise. In addition, the ICCUF Initiative responds to many of the information and intelligence gaps related to gun crime. Finally, users of ICCUF products and services attested to their continuing need.

ICCUF is appropriate to the federal mandate and aligned with government priorities. As prescribed by the legislative framework and since guns and criminals travel across domestic and international borders, ICCUF is appropriate to the federal mandate. So far, ICCUF is the only national firearms-related initiative of its kind. As well, the ICCUF Initiative aligns with the Government's public safety agenda which, over the past several years, has referenced "getting tough on crime" and included specific references to gun control and gun crime. The intended outcomes of the ICCUF align to strategic objectives and priorities of the three federal partner department/agencies.

5.2 Performance - Achievement of Expected Outcomes

Overall, the ICCUF Initiative has achieved its expected immediate and intermediate outcomes. Based on feedback from stakeholders, the Initiative has also contributed to its longer-term ultimate outcome of the prevention of the criminal use of firearms.

Immediate Outcomes

The ICCUF Initiative has improved the sharing of actionable intelligence related to gun crime. ICCUF partners have participated in joint force operations at various levels, ranging from longer-term, multi-agency, co-located joint force operations, to short-term, ad hoc joint force operations. However, this evaluation cannot quantify the extent of ICCUF involvement in these operations, as quantitative information was not available.

In addition ICCUF partners have produced and disseminated a range of tactical and strategic intelligence products and services to a wide audience. ICCUF stakeholders, including law enforcement officers, agreed that these products and services are useful to them and provide actionable intelligence; although intelligence from some products and services was considered more actionable (RCMP's FRT, RCMP's CIBIN, intelligence provided by CBSA RIFLOs) than from others (tactical reports provided by the RCMP Criminal Intelligence Directorate).

The ICCUF Initiative has contributed to increased knowledge of investigative procedures through a significant number of training sessions delivered and advice provided. Interviewees agreed that the ICCUF training and advice/assistance has increased their knowledge of investigative procedures; however, feedback information was largely not available from training sessions, conferences or workshops delivered by ICCUF partners. As well, interviewee responses indicated that, for tactical advice provided by the Regional Firearms Intelligence Coordinators of the RCMP Criminal Intelligence Directorate, there is room for improvement.

Intermediate Outcomes

The ICCUF Initiative has increased knowledge of gun crime issues, trends and threats, through the provision of PS research reports and RCMP CISC's strategic intelligence products (National Strategic Firearms Threat Assessment and Strategic Intelligence Briefs). Stakeholders believed that ICCUF has increased their knowledge of gun crime issues, trends and threats. In addition, those who work in the policy area reported that ICCUF-generated intelligence and research have contributed to better-informed policy advice.

The ICCUF Initiative has enhanced national coordination of gun crime investigations and enforcement. Documentation suggests that prior to ICCUF, information and intelligence was collected by various agencies, but efforts were not coordinated. Under the ICCUF Initiative, RCMP NWEST provides a one-stop shop for law enforcement officers, and CBSA RIFLOs collect and share intelligence from one regional jurisdiction to another. Interviewees overwhelmingly agreed that ICCUF has enhanced the national coordination of investigation and enforcement efforts. As well, according to NJMT representatives, ICCUF contains essential elements of a successful national firearms enforcement strategy.

The ICCUF Initiative has led to improved investigations and enforcement relating to gun crime, through: the sharing of actionable intelligence that meets the needs of the law enforcement community; increased knowledge of investigative procedures; and, the national coordination of investigations and enforcement. There has been an increase in the number of firearms seized and in the number of RCMP CIBIN hits suggesting that investigations and enforcement have improved. Finally, stakeholders attested to the fact that ICCUF has improved investigations and enforcement relating to gun crime.

Ultimate Outcome

According to stakeholder perception, the ICCUF Initiative contributed to the prevention of the criminal use of firearms. Interviewees noted that the absence of ICCUF products and services would have a significant impact on their activities, including successful seizures, efficiency and effectiveness of investigations, and informed policy-making.

In 2008-2009, with the collaborative efforts of a multitude of ICCUF partners, a significant number of firearms (1,200 firearms or about 18% of all firearms seized by RCMP in a year) had been prevented from entering Canada. In addition, the re-classification of these firearms has prevented future shipments of these firearms from entering Canada.

5.3 Performance - Demonstration of Efficiency and Economy

NJMT representatives indicated that they have implemented changes over the course of the Initiative for the purpose of improving efficiency, including establishing processes and procedures to facilitate access to resources. Stakeholders perceived that ICCUF is being delivered efficiently. In addition, stakeholders reported examples of cost savings to their organizations that have resulted from their use of ICCUF products and services. The current ICCUF approach was viewed as a good model providing value for money.

Overall, partners are producing the expected level of outputs and users are mostly satisfied with the products and services. However within the RCMP, there is a strong correlation between spending over the ICCUF budgeted amount and the ability to produce desired outputs and generate a high level of user satisfaction. Units that have spent more than the budgeted amount at RCMP include NWEST, FRT, and CIBIN. There are two RCMP units that have under-expended (CID and CISC). If CID and CISC were to eventually expend the budgeted amount for their activities, RCMP may not be able to balance its overall ICCUF budget. Between 2004-2005 and 2008-2009, RCMP spent 2% less than the budgeted ICCUF amount.

The ICCUF Initiative was demonstrating increased economy. The cost per seized firearm showed a decreasing trend, starting in 2007.

6. Recommendations

Two recommendations emerge from the conduct of this evaluation. It is recommended that:

1. Royal Canadian Mounted Police examines what improvements could be made to ensure that the tactical reports and tactical advice provided by the Regional Firearms Intelligence Coordinators of the Criminal Intelligence Directorate are as successful as the other products and services provided by the Initiative; and
2. Royal Canadian Mounted Police assesses the funding allocation of its various activities within the Initiative. Overall, spending by the Royal Canadian Mounted Police is slightly less than the budgeted amount allotted under the Initiative. Royal Canadian Mounted Police needs to address the reasons why some units spend more than the budgeted amount (National Weapons Enforcement Support Teams, Firearms Reference Table, Canadian Integrated Ballistics Identification Network) and why some units spend less than the budgeted amount (Criminal Intelligence Directorate, Criminal Intelligence Service Canada).

7. Management Response and Action Plan

This evaluation report has been reviewed and approved by deputy heads of all three participating department/agencies of the Initiative. In addition to Royal Canadian Mounted Police providing management action plan in response to the evaluation's recommendations, Public Safety Canada and Canada Border Services Agency were provided the opportunity for commenting on this report and the evaluation.

Royal Canadian Mounted Police

Response: Royal Canadian Mounted Police accepts and fully supports recommendations 1 and 2.

Recommendation 1: Royal Canadian Mounted Police acknowledges that tactical reports and tactical advice provided by the Regional Firearms Intelligence Coordinators of the Criminal Intelligence Directorate are not as successful as the other products and services of the Initiative because there are limited amounts of tactical information being produced in certain areas due to: the high rate of turnover and vacancy; slow human resource processes; and, co-location issues and geo-spatial coverage.

In order to provide tactical reports and tactical advice that are more useful to end-users, Criminal Intelligence Directorate will develop and implement innovative service delivery models to ensure the adequacy, timeliness, and quality of the products and services. This will be done with the assistance of other organizational units of the Royal Canadian Mounted Police. Moreover, to make tactical information more accessible for general law enforcement officers, the Directorate will improve and streamline the process of uploading tactical information onto various databanks. In the meantime, it will educate clients on how to extract the tactical information from various databanks.

Recommendation 2: Royal Canadian Mounted Police continues to support initiatives that enhance the national collection, analysis and sharing of firearms-related intelligence, in order to improve investigations and increase knowledge of patterns of smuggling and trafficking of illegal firearms.

Royal Canadian Mounted Police will conduct a review of the funding allotted under the Initiative among the various units of the organization, to determine whether re-allocation between program areas will better achieve program objectives and whether efficiencies can be realized.

The target date for implementing recommendations 1 and 2 will be April 2011. Responsible units are the Criminal Intelligence Directorate, Criminal Intelligence Service Canada, Canadian Integrated Ballistics Identification Network, Firearms Investigative and Enforcement Services Directorate, and the Firearms Reference Table. Royal Canadian Mounted Police remains committed to the Initiative in conjunction with other participating department/agencies.

Canada Border Services Agency

Canada Border Services Agency agrees with the main findings of the report, including the conclusion that there is a continuing need for the Initiative and that it has successfully achieved its expected outcomes. It is evident that, through the collaborative efforts of partners, the Initiative has contributed to its ultimate outcome of preventing the criminal use of firearms.

The ongoing funding for this Initiative and the well-established and successful partnerships provides a firm foundation for core partners to develop a national strategy to combat the smuggling, trafficking and further criminal use of firearms.

Public Safety Canada

Public Safety Canada has reviewed and accepts the key findings of the evaluation, in particular that the Investments to Combat the Criminal Use of Firearms Initiative continues to be significant and has been successful in achieving the outcomes that were initially set. As the only national initiative to combat gun crime and the illegal movement of firearms, the Initiative has demonstrated considerable progress in the collection and sharing of firearms-related intelligence and knowledge. There now exists a substantial level of co-operation and cohesiveness among participating department/agencies in both operational and policy pursuits. A solid base of tangible achievements has been realized.

There are no recommendations arising from this evaluation that are addressed to the Department. Public Safety Canada is encouraged by the finding that there is a high level of satisfaction among users of its products, such as its policy advice and research. The presence of the Initiative, with its access to the diverse expertise and intelligence of all the participating department/agencies, has ensured the formulation of relevant and timely policy advice.

Annex A: Description of activities of ICCUF Partners

Annex B: Summary of Recommendations from the 2007 ICCUF Formative Evaluation and responses by partner department/agencies

Annex C: List of Documents Reviewed

Document List

• CBSA Firearms Quarterly Reports, Strategic Intelligence Analysis Division, Intelligence Directorate
• CBSA Firearms Smuggling Threat Assessment (September 2009), Intelligence Risk Assessment and Analysis Division Intelligence Directorate
• CBSA Report on Plans and Priorities, 2010-2011
• CBSA, Canadian Illegal Firearms Enforcement Workshops (January, 2009), Workshop evaluation
• CISC, 2007 Annual Report, Organized Crime in Canada
• CISC, Canadian Illegal Firearms Enforcement Workshop (CIFEW) (January 20^th & 21^st, 2009, Ottawa, Ontario)
• CISC, CISC May 26 ICCUF-Mid-term Evaluation - CISC Updated Responses
• CISC, Memo to CISC Community, 2007-11-05, Release of the 2007 National Strategic Firearms Threat Assessment
• CISC, Memo to CISC Community, 2009-02-01, Release of the 2008 National Strategic Firearms Threat Assessment
• CISC, National Criminal Intelligence Estimate, 2009
• [ * ]
• CISC, National Strategic Firearms Threat Assessment, 2006
• CISC, National Strategic Firearms Threat Assessment, 2007
• CISC, National Strategic Firearms Threat Assessment, 2008
• CISC, Report on Organized Crime, 2009
• Department of Justice Canada, The illegal movement of firearms in Canada: report of the Firearm Smuggling Working Group, 1995
• (DRAFT) Response to the Recommendations of the Formative Evaluation of the ICCUF Initiative, December 2008
• GCS, Formative evaluation final report, May 30, 2009
• Government of Canada Caps Costs of Registry and Improves Gun Crime Measures (http://www.psepc.gc.ca/publications/news/20040520 e.asp)
• ICCUF Initiative - Research Report #2 - Directory of Organizations with Firearms Related Responsibilities in the Federal Government, March 2009
• ICCUF Initiative - Research Report #1 - Literature Search and Bibliography, March 2008
• ICCUF Initiative - Research Report #3: Guns and Gangs, March 13, 2009
• ICCUF mid-term evaluation response, December 2008
• ICCUF NJMT - June 4, 2009 Record of Decision
• NWEST, Monthly Activity Summaries
• OAG document, "What funding has CBSA received and what has it done under the Investments to Combat the Criminal Use of Firearms initiative?"
• PS Report on Plans and Priorities, 2010-2011
• [ * ]
• PSEPC, Final Report: Youth, Weapons and Violence in Toronto and Montreal, March 2006
• PSEPC, Government of Canada Caps Costs of Registry and Improves Gun Crime Measures, May 2004
• RCMP Report on Plans and Priorities, 2010-2011
• RCMP, CID, Current Trends in Firearms Trafficking and Smuggling in Canada, November, 2007
• RCMP, NWEST, Investments to Combat the Criminal Use of Firearms; Framework for Delivery
  Prepared for: ICCUF Joint Management Team, February 2006
• [ * ]
• Speech from the Throne - 2007, 2008, 2010
• Statistic Canada, Juristat, Homicide in Canada, 2008, by Sara Beatie, October 2009, component of Statistics Canada catalogue no. 85-002-X, Vol. 29, no.4
• Tony Heemskerk and Eric Davies, A report on the Illegal Movement of Firearms in British Columbia, November 2008

Annex D: Law Enforcement Interviewees by Product/Service and Province - heavy multiple-product users

Annex E: Key Informant Interview Guide

Interview Guide for Law Enforcement Personnel- heavy multiple-product users

Evaluation of the Investments to Combat the Criminal Use of Firearms (ICCUF)

1. Which of the following products or services have you or your organization accessed over the past five years? (select all that apply)

• Intelligence:
  • Ballistic data queries (IBIS checks) provided by the Canadian Integrated Ballistic Identification Network (CIBIN)
  • The Firearms Reference Table (FRT)
  • RCMP Criminal Intelligence monthly reports relating to firearms (available on various databases such as NCDB, PROS/BC Prime, ACIIS)
  • CISC's National Strategic Firearms Threat Assessments and / or Strategic Intelligence Briefs
  • Intelligence provided by CBSA Regional Intelligence Firearms Liaison Officers (RIFLOs) (e.g., intelligence reports and bulletins)
• Investigative Support:
  • NWEST training and/or information sessions
  • Investigative advice or support provided by the Firearms Call Centre or NWEST representatives
  • Firearms tracing provided by Canadian Firearms Tracing Centre
  • NWEST firearms related expert advice and witnesses
  • Tactical advice supporting operations, from the RCMP Criminal Intelligence (Regional Firearms Intelligence Coordinators - RFIC)
  • Intelligence support to police investigations by CBSA Regional Intelligence Firearms Liaison Officers (RIFLOs) (e.g., functional advice and guidance, training)
  • Other; please specify:
  • Do not recall

2. Over the course of one year, on average, how often would you use these products/services?

3. To what extent do you agree that the following products/services provided you / your organization with actionable intelligence?

Note: Actionable intelligence refers to the direct link between the delivery of timely, adequate and useful intelligence and police action begin taken.

4. How were these products/services used?

5. To what extent has the intelligence generated by these products/services improved the coordination of investigations and enforcement activities...

6. a) (if applicable) To what extent has the strategic intelligence generated by these products/services assisted with informed policy making and strategic planning?

• Not at all
• To some extent
• To a large extent
• Don't know

In what way? Please elaborate and please provide examples to support your opinion.

6. b) (if applicable) To what extent has the strategic intelligence generated by these products/services increased knowledge of gun crime issues, trends and threats?

• Not at all
• To some extent
• To a large extent
• Don't know

In what way? Please elaborate and please provide examples to support your opinion.

7. a) To what extent do you agree that the following products and services Increased your / your organization's knowledge of investigative procedures?

7. b) In what ways have these products or services impacted your, or your organization's, knowledge of investigative procedures?

7. c) Or if you indicated the products/services have not increased your / your organization's knowledge of investigative procedures, what have been the barriers?

8. In the last five years, have the products and services you used (as identified in question #1) led to more successful investigations and enforcement relating to gun crime?

• Not at all
• To some extent
• To a large extent
• Don't know

What helped most? Least?

In what way?

Please provide some examples to support your opinion. (e.g., linked crimes, concluded investigations, firearms seizures, timely investigations, successful court outcomes, better evidence to court)

9. Based on everything we've talked about so far, to what extent do you believe the products and services you used (as identified in question #1) contributed to the prevention of the criminal use of firearms?

• Not at all
• To some extent
• To a large extent
• Don't know

10. What would the impact have been in the absence of these products and services?

11. Do the products and services you used (as identified in question #1) result in any cost savings to you or your organization? (e.g., in the production of search warrants, avoidance of cost for finding firearms matches, administrative file closures, helping to make decisions about more efficient use of resources, etc.)

• Yes - please provide specific examples of savings of time or money:
• No
• Don't know

12. Overall, how useful, to you or your organization, were the products and services you used (as identified in question #1)? Would you say they were ...

• Very useful
• Somewhat useful
• Neutral
• Not very useful
• Not at all useful
• Do not know

Which products and services helped most? Least?

In what way? Please provide some examples to support your opinion.

13. How likely are you to continue to use the products and services you identified (as identified in question #1)?

• Very likely
• Somewhat likely
• Neutral
• Not very likely
• Not at all likely
• Do not know

14. If these products and services are not available, to what extent will the overall effectiveness of your job be affected?

• Not at all
• To some extent
• To a large extent
• Don't know

Please elaborate.

15. Have there been any unintended impacts (positive or negative) as a result of your use of the products and services (as identified in question #1)?

• Yes - please describe:
• No
• Don't know

16. Are you aware of any other sources of similar products and services?

• Yes - please describe:
• No
• Don't know

17. (If yes,) Have you used these other products and services? Why or why not?

18. (If used) In what ways do these other products and services complement or duplicate the products or services you indicated you use (as identified in question #1)?

• Don't know

19. Is there anything else you would like to add?

Interview Guide for Public Safety Policy Users for the Evaluation of the Investments to Combat the Criminal Use of Firearms (ICCUF)

1. Can you please describe the products or services you received from PS Canada's Firearms and Operational Policing Policy Division?

• Policy papers and advice - please identify
• Research papers - please identify
• Other - please describe

2. Over the course of one year, on average, how often would you use these products/services?

3. To what extent do you agree that the products/services provided you / your organization with increased knowledge of gun crime issues, trends and threats?

4. How were these products/services used?

5. To what extent has the strategic intelligence generated by these products/services assisted with informed policy making and strategic planning?

• Not at all
• To some extent
• To a large extent
• Don't know

In what way? Please elaborate and please provide examples to support your opinion.

6. a) To what extent do you believe the products and services you used (as identified in question #1) contributed to the prevention of the criminal use of firearms?

• Not at all
• To some extent
• To a large extent
• Don't know

6. b) What would the impact have been in the absence of these products and services?

7. Do the products and services you used (as identified in question #1) result in any cost savings to you or your organization?

• Yes - please provide specific examples of savings of time or money:
• No
• Don't know

8. Overall, how useful, to you or your organization, were the products and services you used (as identified in question #1)? Would you say they were ...

• Very useful
• Somewhat useful
• Neutral
• Not very useful
• Not at all useful
• Do not know

Which products and services helped most? Least?

In what way? Please provide some examples to support your opinion.

9. How likely are you to continue to use the products and services you identified (as identified in question #1)?

• Very likely
• Somewhat likely
• Neutral
• Not very likely
• Not at all likely
• Do not know

10. If these products and services are not available, to what extent will the overall effectiveness of your job be affected?

• Not at all
• To some extent
• To a large extent
• Don't know

Please elaborate.

11. Have there been any unintended impacts (positive or negative) as a result of your use of the products and services (as identified in question #1)?

• Yes - please describe:
• No
• Don't know

12. Are you aware of any other sources of similar products and services?

• Yes - please describe:
• No
• Don't know

13. (If yes,) Have you used these other products and services? Why or why not?

14. (If used) In what ways do these other products and services complement or duplicate the products or services provided by PS Firearms and Operational Policing Policy Division (as identified in question #1)?

• Don't know

15. Is there anything else you would like to add?

Interview Guide - Relevance - Evaluation of the Investments to Combat the Criminal Use of Firearms

1. In your view, is there a continued need for the ICCUF Initiative? Why or why not?

2. According to what Budget or government priorities were ICCUF created? How does ICCUF relate to the current government priorities (i.e., how the program's expected results aligned to the current Government priorities)?

3. What is the nature of the federal government's role and mandate to deliver ICCUF? Are these roles and mandate appropriate? (vis-à-vis the appropriateness of roles of other jurisdictions)

4. Are you aware of any other programs that might complement or duplicate the ICCUF Initiative (e.g., other initiatives or taskforces of provincial/ municipal law enforcement organizations, other firearms intelligence programs)?

1. (If aware) In what ways do these other programs complement or duplicate the ICCUF Initiative?
2. What is the level of coordination and cooperation between ICCUF and these other initiatives/taskforces addressing similar needs?

5. a) Are there alternative approaches that could be used to achieve the objectives of the ICCUF Initiative (see note below)?

Note: the ICCUF initiative aims to achieve the following outcomes:

• the sharing of actionable intelligence,
• increased knowledge of investigative procedures,
• increased knowledge of gun crime issues trends and threats,
• enhanced national coordination of investigations and enforcement,
• improved investigations and enforcement, ultimately leading to
• the prevention of the criminal use of firearms

5. b) Would these approaches be more efficient or effective than the ICCUF Initiative? If yes, please describe how.

Annex F: Focus Group Guide - for National Joint Management Team

Focus Group Guide for the Evaluation of the Investments to Combat the Criminal Use of Firearms (ICCUF)

Relevance

• 1. Is there a continuing need for ICCUF? Why do you say so?
• 2. According to what Budget or government priorities were ICCUF created? Can you provide documentation to support it?
• 3. How does ICCUF relate to the current government priorities (i.e., how the Initiative's expected results are aligned to the current Government priorities?)
• 4. What is the nature of the federal government's role and mandate to deliver ICCUF? Are these roles and mandate appropriate (vis-à-vis the appropriateness of roles of other jurisdictions)?

Effectiveness - Assessment of Success

• 5. To what extent did ICCUF resources participate in integrated teams (e.g., JFOs)?
• 6. To what extent has ICCUF increased knowledge of gun crime issues, trends, and threats?
• 7. To what extent does the ICCUF Initiative inform the development of an intelligence-led national enforcement strategy?
• 8. Have there been barriers? If so, what are they?
• 9. To what extent has ICCUF contributed to the improvement of gun crime investigation and enforcement? (including the national coordination of investigative efforts)

Efficiency and Economy

• 10. Have any changes been made to the ICCUF Initiative that has increased the efficiency of the Initiative (e.g., changing technology, environment)? If yes, please describe.
• 11. Does the use of ICCUF products and services result in any cost savings to you/your organization
  1. If yes, please provide specific examples of savings of time or money.
• 12. In your opinion is the ICCUF Initiative the most efficient model to achieve the ICCUF objectives?
  1. Are there alternative models that could be used?
  2. Would these be more or less efficient and effective than ICCUF? Please explain why.
• 13. Are there any improvements that could be made to the ICCUF Initiative (i.e., to make the program more efficient)? Please explain.
• 14. Have there been any unintended impacts (positive or negative) as a result of the ICCUF Initiative?
• 15. Are you aware of any other programs that might complement or duplicate the ICCUF Initiative (e.g., other initiatives or taskforces of provincial/ municipal law enforcement organizations)?
  1. (If aware) In what ways do these other programs complement or duplicate the ICCUF Initiative?
• 16. Is there any coordination / cooperation between ICCUF and these other programs? If yes, please describe.
• 17. Is there anything else you would like to add?

Annex G: Quantitative Data Template

Annex H: Financial Data Template

Note: ^* Budgeted amounts were pre-filled for the partner organizations based on program foundational documents.

Annex I: Tactical and Strategic Intelligence Products and Services

Notes

1 In this report, Firearms Reference Table (FRT) refers to both the organizational unit within RCMP and the tactical intelligence product produced by the FRT unit.

2 Criminal Intelligence Service Canada, under the stewardship of the Royal Canadian Mounted Police, is comprised of nearly 400 member agencies representing the federal, provincial, and municipal levels of law enforcement. It is governed by the National Executive Committee that consists of 25 leaders from Canada's law enforcement community.

3 Program representatives from these three units within the Royal Canadian Mounted Police indicated they spent more than their allotted budget under the Initiative was because the allotted amount was not sufficient to support their operations. Thus, a deliberate decision was made within the Royal Canadian Mounted Police to re-allocate its budget in-house. Between 2004-2005 and 2008-2009, these three units had funding from sources within the Royal Canadian Mounted Police on top of the budgeted amount allotted under the Initiative.

4 For the purpose of this evaluation, the words "firearms" and "guns" are used interchangeably.

5 Firearm Smuggling Working Group (1995). "The illegal movement of firearms in Canada: report of the Firearm Smuggling Working Group", Department of Justice Canada.

6 Consulting and Audit Canada, (August 31, 2005). "Investments to Combat the Criminal Use of Firearms - Evaluation Framework" prepared for Public Safety and Emergency Preparedness Canada.

7 Government Consulting Services (August 31, 2009). "Methodology Report - Evaluation of the Investments to Combat the Criminal Use of Firearms" prepared for Public Safety Canada.

8 Treasury Board of Canada (2009). Policy on Evaluation, April 1, 2009. Accessed online at http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15024&section=text#cha4

9 Government Consulting Services, Public Works and Government Services Canada (May 30, 2007). "Formative Evaluation of the Investments to Combat the Criminal Use of Firearms Initiative - Final Report" prepared for Public Safety Canada, Royal Canadian Mounted Police, and Canada Border Services Agency.

10 For the most part, at the data collection stage of the evaluation (February to June 2010), quantitative and financial data were only available for the full fiscal year between 2004-2005 and 2008-2009.

11 There are five cores issues described in the Directive on the Evaluation Function, as follows:

• Core Issue 1: Continued Need for Program: an assessment of the extent to which the program continues to address a demonstrable need and is responsive to the needs of Canadians;
• Core Issue 2: Alignment with Government Priorities: Assessment of the linkages between program objectives and (i) federal government priorities and (ii) departmental strategic outcomes;
• Core Issue 3: Consistency with Federal Roles and Responsibilities: Assessment of the role and responsibilities for the federal government in delivering the program;
• Core Issue 4: Achievement of Expected Outcomes (inc. progress towards immediate, intermediate and ultimate outcomes) with reference to performance targets and program reach, program design, including the linkage and contribution of outputs to outcomes; and,
• Core Issue 5: Demonstrations of Efficiency and Economy: Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes.

12 Two other questions are identified in the Evaluation Matrix of the 2009 Methodology Report and they are:

• Have the recommendations of the formative evaluation been implemented? Please refer to Annex B for a summary of the responses of the recommendation provided by partner department/agencies.
• Have there been any unintended impacts (positive or negative)? During the course of the evaluation, the evaluation team did not find any significant unintended impacts.

13 Treasury Board of Canada Secretariat (2009). Standard on Evaluation for the Government of Canada, April 1, 2009. Accessed online at http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15688

14 Interview guides are provided in Annex E. Interview guides for groups B and C are variants of the law enforcement guide (group A) and therefore, are not included in the Annex.

15 On average, these users utilize over 3 ICCUF products and services, more than 10 times per year.

16 A shortened version of the Group A law enforcement interview guide was used for this group. The emphasis of these interviews was to examine if these users might express a different viewpoint (than group A users) regarding the impact it would have had on them if ICCUF did not exist.

17 Focus group guide was provided in Annex F.

18 NJMT representatives filled out the same interview guide as the law enforcement interviewees.

19 PS research report #3: "Guns and Gangs" (March 13, 2009) and "A Report on the Illegal Movement of Firearms in British Columbia" (November 2008), by T. Heemskerk and E. Davies.

20 Royal Canadian Mounted Police (2006). "Feature Focus: Youth Gangs and Guns, RCMP Environmental Scan", p. 5. See also Statistics Canada (2009), "Juristat, Homicide in Canada, 2008".

21 Government Consulting Services, Public Works and Government Services Canada (March 2009). "Investments to Combat the Criminal Use of Firearms Initiative, Research Report #3: Guns and Gangs".

22 Public Safety and Emergency Preparedness Canada (2006). "Final Report: Youth, Weapons and Violence in Toronto and Montreal".

23 The linkages between these intended outcomes are depicted in the Logic Model.

24 According to NWEST representatives, prior to 2007, the measurement method to account for NWEST trace requests had not been accurate. Going forward, NWEST representatives expect that data capture for trace requests will be significantly improved.

25 A different data collection method was employed prior to 2006-2007 and therefore, figures for 2004-2005 and 2005-2006 are not reported.

26 The sharp increase in the requests for assistance in the production of search warrants in 2009 can be partially explained by the implementation of the Centralized Firearms Occurrence and Reporting System by the RCMP's Firearms Investigative and Enforcement Services Directorate (which includes NWEST and FRT) in 2009. As a result of the launch of this system, all firearms occurrences within the directorate were centrally reported, which allowed NWEST to increase its collection, collation and analysis capabilities. Meantime, NWEST also implemented strict protocols regarding collection of data such as properly filling out the circumstance of seizure field in the forms which might also have attributed to the increase in the reporting of the requests for assistance in the production of search warrants in 2009. This suggests that prior to 2009, this statistic might have been under-reported..

27 1) Literature Search and Bibliography (March, 2008), 2) Directory of Organizations with Firearms Related Responsibilities within the Federal Government (March, 2009), 3) Guns and Gangs (March, 2009), 4) Youth, Weapons and Violence in Toronto and Montreal (March, 2006), 5) Data on Canadian Importers of Firearms, 6) Study of Firearms Markings (2009) (counted as four separate reports).

28 1) Analysis of the sources of crime guns in police custody in Canada, 2) Cost of firearms crimes in Canada, 3) Identifying illicit firearms market acquisition patterns.

29 [ * ]

30 [ * ]

31 ICCUF was initiated with an aim to enhance the national collection, analysis and sharing of firearms-related intelligence in Canada. This was seen as a first step in the formulation of an informed national enforcement strategy to combat gun crime and firearm trafficking.

32 NWEST seized firearms are all directly connected to the potential for criminal use in Canada, and these firearms are all seized in Canada.

33 A vast majority (about 95%) of firearms seized by CBSA is related to personal firearms of U.S. residents who are carrying them into Canada for their own protection or for hunting. Though illegal in Canada, it is unlikely that these seized firearms were destined for the Canadian illegal firearms market. Between 2006 and 2009, the total number of firearms seized (with and without criminal intent) by CBSA at the border amounted to 2,117.

34 Besides the NWEST (RCMP) and CBSA, many other Canadian law enforcement agencies conduct firearms seizures (e.g., Toronto Guns and Gangs Unit, Winnipeg Police Agency).

35 Restricted firearms are, broadly speaking, handguns suitable for target practice, and certain military style tactical rifles and shotguns.

36 Prohibited firearms are, broadly speaking, small-frame handguns intended for self-defence, and certain rifles and shotguns derived from fully automatic military firearms.

37  [ * ]

38 High = 85% to 100%; Moderate is 50% to 84%; Low is less than 49% agreed that the products are satisfactory.

39 During the same period that FRT received funding from the RCMP's Canada Firearms Centre, FRT also received funding from the U.S. National Institute of Justice.

40 Although the logic model might have indicated that certain activities (e.g., policy advice) are only provided by certain ICCUF partners (e.g., PS), some additional ICCUF partners indicated that they also participate in these activities (e.g., for policy advice, in addition to PS, NWEST, FRT, CBSA also participate).

41 The cost of CBSA seized firearms is not included, for two reasons: 1) RCMP and CBSA firearm seizures are not of the same nature and are not conducted in the same environment, and thus the two organizations have different cost structure as it relates to firearm seizure; 2) As compared to CBSA, the volume of seized firearms by RCMP is huge making it a good candidate to conduct the cost analysis. Between 2006 and 2009, CBSA seized a total of 109 firearms intended for criminal use and RCMP seized 26,911 firearms.

Date modified:

2022-08-12