2015-2016 Evaluation of the Kanishka Project Research Initiative

2015-2016 Evaluation of the Kanishka Project Research Initiative PDF Version (601 KB)

Table of contents

• Executive Summary
• 1. Introduction
• 2. Profile
  • 2.1 Background
  • 2.2 Governance
  • 2.3 Resources
  • 2.4 Program Logic Model
• 3. About the Evaluation
  • 3.1 Objective and scope
  • 3.2 Methodology
  • 3.3 Limitations
  • 3.4 Protocols
• 4. Findings
  • 4.1 Relevance
  • 4.2 Performance - Effectiveness
  • 4.3 Performance - Efficiency and Economy
• 5. Conclusions
• 6. Recommendations
• 7. Management Response and Action Plan
• Annex A: Logic Model
• Annex B: Documents Reviewed
• Annex C: Intiative Sponsored Events
• Annex D; Financial Information

Final Report

2016-06-28

Executive Summary

Program evaluations support accountability to Parliament and Canadians by helping the Government of Canada credibly report on the results achieved with resources invested in programs. They also support deputy heads in managing for results by informing them about whether their programs are producing the expected outcomes efficiently and cost-effectively. Program evaluations support policy and program improvements by helping identify lessons learned and best practices.

What we examined

This evaluation examined the relevance and performance of the Kanishka Project Research Initiative, a $10-million, five-year initiative established in 2011, to address gaps in understanding of terrorism in Canada and the way it manifests itself in Canadian communities. The Initiative has several components, including a grants and contributions component that is designed to fund research studies and support direct engagement with researchers. Given that the Initiative includes a grants and contributions component, the evaluation assessed the extent to which the design, delivery and administration of this component of the Initiative conformed to the requirements of the Government of Canada Policy on Transfer Payments.

Why it is important

Terrorism is considered a threat to Canada’s national interest and security. In recent years, the number of terrorist incidents has been increasing steadily, both in Canada and around the world. Many countries, including Canada, are facing radicalization to violence, particularly of youth. More than 180 Canadians are known to have gone abroad to take part in foreign armed conflicts. Preventing, detecting, denying terrorists the means and opportunity to carry out their activities and responding to these developments are among the Government of Canada’s, and by extension, the Department of Public Safety’s highest priorities. The Initiative plays an important role in creating networks across sectors, generating knowledge for decision-makers, and increasing Canadians’ understanding of terrorism and counter-radicalization to violence, which is increasingly needed to contribute to building a safe and resilient Canada.   

What we found

Relevance

The raison d'être of the Initiative was to invest in research on pressing questions to enable Canada to better understand what terrorism meant in the Canadian context and what could be done to support effective policies and programs to counter-terrorism and violent extremism in Canada. Despite its contribution to date, there is still a continued need for the Initiative to shed more light on these issues.

The Initiative is well aligned with the federal government and PS’s priorities, as ensuring the safety and security of Canadians at home and abroad continues to be among the top priorities of the government. The emphasis on the need for further research communicated as part of the Government’s commitment to create the Office of the Community Outreach and Counter-radicalization attests to the relevance of the Initiative and ongoing need for similar activities.  

Performance

To a large extent, the Initiative has contributed to the achievement of its expected outcomes: It has supported the creation of various networks and other mechanisms for ongoing dialogue across different sectors on terrorism and counter radicalization; through funding research studies and other mechanisms, the Initiative has facilitated the generation of knowledge and tools to ensure that Canadian policy and decision-makers, as well as frontline officers and other practitioners have access to more relevant and timely information to do their work; and researchers affiliated with the Initiative have more resources and support at their disposal to conduct research and to study the identified priority areas.

The Department has put in place a robust governance framework to oversee the delivery of the Initiative. For the most part, the Initiative was delivered efficiently and economically. The design, delivery and administration of the grants and contribution components of the Initiative were found to generally conform to the requirements of Government of Canada Policy on Transfer Payments.

Notwithstanding the above achievements, the evaluation identified a few opportunities for improvement. The following recommendations are provided in the spirit of continuous improvement.

Recommendation

The ADM of the Portfolio Affairs and Communications Branch and/or the future Senior Departmental Officer responsible for the Office for Community Outreach and Countering Radicalization to Violence should ensure that:

• Kanishka-related research findings are better communicated to policy and decision makers and the general public.

In collaboration with the Assistant Deputy Minister Corporate Management Branch ensure that:

• funding recipients’ reporting requirements are proportionate to their current risk profile.

Management Response and Action Plan

Management accepts all recommendations and will implement an action plan.

1. Introduction

This report presents the results of the Public Safety Canada (PS) 2015-2016 Evaluation of the Kanishka Project Research Initiative (here on referred to as the Initiative).

This evaluation was conducted in compliance with Section 42.1 of the Financial Administration Act, which requires all programs of grants and contributions be evaluated at least once every five years, the 2009 TB Policy on Evaluation, the TB Policy on Transfer Payments, the Directive on the Evaluation Function and the Evaluation Standard, as well as the PS Internal Audit and Evaluation Directorate processes and standards. The evaluation is intended to provide Canadians, parliamentarians, Ministers, central agencies, and the Deputy Minister of Public Safety with a neutral, evidence-based assessment of the relevance and performance (effectiveness, efficiency and economy) of this program. 

2. Profile

2.1 Background

Established in 2011, the Initiative responded to a proposal made in the final report of the Commission of Inquiry into the Investigation of the Bombing of Air India Flight 182. In his report, the Head of the Inquiry, Justice Major, proposed the establishment of an academic program to study terrorism and counter-terrorism in order to address what was described to be “a serious gap in Canadians’ understanding about terrorism, and how it manifests itself in Canadian communities.”

The Initiative focuses on direct engagement with scholars and students to support research on pressing questions for the Canadian Government and society about terrorism and counter-terrorism; to expand the community of researchers, disciplines and universities addressing such questions; and to better connect that research community with officials responsible for national security, as well as bring the new knowledge to the general public.

By engaging with permanent research institutions, including the Social Sciences and Humanities Research Council (SSHRC) and the Canadian Safety and Security Program (CSSP), the Initiative is designed to leverage further funds and ensure that these institutions are more responsive to government needs, even after the termination of funding through the Initiative.

2.2 Governance

The Initiative’s governance structure includes a Lead Team at PS, comprising the Manager of Research and Academic Relations, a Project Manager, and a Project Support Officer. This Lead Team is accountable for the overall performance of the Initiative and works closely with an Assistant Deputy Minister (ADM) level Steering Committee that has decision making authority to select projects for funding. This committee is co-chaired by the ADM of the Portfolio Affairs and Communications Branch at PS and a representative of the Air India Flight 182 Victims’ Families Association, providing this organization direct involvement in project selection. Membership on the Steering committee includes representatives from the Royal Canadian Mounted Police, Canadian Security Intelligence Service, Financial Transactions and Reports Analysis Centre of Canada, Correctional Service of Canada, Transport Canada, Immigration, Refugees and Citizenship Canada, Defence Research and Development Canada and Justice Canada. A separate Working Group of senior analysts and Directors support the Steering Committee by reviewing and making recommendations on project proposals and by briefing their ADM Steering Committee member on these recommendations.

A separate Working Group of officials was created to support the Steering Committee, which is comprised of senior analysts and Directors that review project proposals and make recommendations to the Steering Committee. Representatives on the Working Group generally correspond to the representation on the Steering Committee to ensure officials can brief up appropriately on their recommendations.

The Lead Team consults with a small group of external advisors who represent a broader set of interests. Members of this Advisory Group come from industry, those affected by terrorism, such as the Air India Flight 182 Victims’ Families Association, as well as government and academia. Its mandate is to assist PS by participating in the review of research priorities for the Project, reviewing research produced by Kanishka-affiliated researchers, and supporting the expansion of public awareness about the Initiative by engaging the broader public.

Image Description

The figure describes the Lead Team (PS Strategic Policy Branch) as the primary decision making body for the project, performs project management and secretariat functions, provides direction, establishes themes, responsible for overall project performance.

The figure shows that there are links between the Lead Team and the three following groups: (1) the Advisory Group : comprising of external representatives (community , industry , academics, etc.); provides advice, guidance and connection to networks; (2) the Working Group: Director/senior analyst working group pre-assesses proposals, provide expertise and prepare recommendations to the Steering Committee; and (3) the Steering Committee: ADM level committee that provides advice, guidance and overall approval of work plans and proposals.  

The figure shows the input of the research community (Institutions, researchers, students, networks) to the PS team.

2.3 Resources

The Initiative had $10 million in funding over five years and was expected to attract additional contributions as well as in-kind support for research, facilities, training programs, regional events, and data collection from universities, other levels of government, and/or other initiatives such as SSHRC. In addition, following ex gratia payments to families of the Air India tragedy, the $1.3 million remaining was reallocated to the research initiative, thereby increasing the total commitment to $11.3 million.

Note: PS funded an additional $1,054,905 over five years from existing reference levels (Vote 1). Total program value over five years is $11,050,906.

There are four main areas of investment in this Initiative:

1. A contribution program, managed by PS, to directly support scholars and students working on promising projects (Vote 5: $2.5M over five years).
2. Funds to partner with the SSHRC Insight Grant research funding program to ensure quality through peer review of proposals (Vote 5: $2.1M over five years).
3. Funds to work with the SSHRC Partnership Grant Program to create and support a national network of scholars and students (Vote 5: $1.6M over five years).
4. Direct activities led by PS to support the development of a stronger research community and knowledge base, informing government and public understanding, and to support management of the program (Vote 1: $3.8M over five years).

2.4 Program Logic Model

The Program’s logic model, a visual representation that links what the program was funded to do (its activities) with what it produced (its outputs) and what it intended to achieve (its outcomes), is presented in Annex A. The logic model provided the basis for developing the evaluation matrix, which gave the evaluation team a roadmap for conducting this evaluation. As depicted in the logic model the Initiative’s expected outcomes were as follows:

Immediate:

• Researchers engaged in terrorism-related issues have resources to study priority areas and relevant policy gaps;
• Policymakers, decision-makers and practitioners have access to timely and relevant research;
• Mechanisms are in place to collaboratively consider existing and emerging terrorism-related issues.

Intermediate:

• Research organizations have the capacity to undertake terrorism-related research relevant to Canada;
• Those engaged in addressing terrorism-related issues are equipped with relevant knowledge and tools.

Ultimate:

• Policies and programs related to Canada’s national security are informed by relevant research.

3. About The Evaluation

3.1 Objective and Scope

The objective of the evaluation was to assess the relevance and performance of the Initiative in order to support:

• accountability to Parliament and Canadians by helping the Government to credibly report on the results achieved with resources invested in these activities;
• the Deputy Minister of Public Safety in managing for results by informing him about whether these activities are producing the outcomes that they were designed to produce, at an affordable cost; and,
• policy and program improvements.

The evaluation covered the Initiative activities and performance since its inception in 2011.

3.2 Methodology

The evaluation approach and methodology was established during a planning phase that included consultation with program management, particularly with members of the Initiative Lead Team. It was designed both to assess the relevance and performance of the Initiative and the appropriateness of the Program design and theory.

3.2.1 Rapid Impact Evaluation

The evaluation planning phase for this evaluation coincided with the Treasury Board Secretariat Center of Excellence for Evaluation’s launch of a pilot project intended to test a new approach to program evaluation called Rapid Impact Evaluation (RIE).^Footnote1 RIE has been described as “an intensive, team-based and program-focused investigation that has an iterative process for data collection and analysis and relies on community [stakeholder] participation in order to quickly develop a holistic understanding of a program from both an insider’s perspective, particularly those who were involved in the project design and inception, and an outsider’s perspective.”^Footnote2

This approach to evaluation is described as less formal, less resource- and time-intensive, and more useful for decision-making purposes, while maintaining the rigour of a conventional evaluation.

An important aspect of a Rapid Impact Evaluation approach is the use of counterfactual analysis, which is a “comparison between what actually happened and what would have happened in the absence of the intervention.”^Footnote3 In the case of the Initiative, the evaluation explored this issue through a closer examination of two of the three potential alternatives to the Project’s design. In particular, the evaluation attempted to find out if the Project would have been better able to contribute to the achievement of its expected outcomes had it been designed differently.

The Rapid Impact Evaluation aspect of the methodology was implemented in three phases:

• Phase I: development of a short program summary based on existing inception documents that was validated by program management and stakeholders to ensure all had the same understanding of what the Initiative was intended to achieve.
• Phase II: development of relevant data collection tools, including a survey questionnaire that was launched to elicit the perspectives of the key stakeholders and subject matter experts on the relevance, design and performance of the Initiative.
• Phase III: conduct analyses, verification and reporting of the evaluation findings.  

3.2.2 Evaluation Core Issues and Questions

The following evaluation issues and questions were addressed in the evaluation:

Relevance

1. To what extent is there a continued need for the Initiative? Has the Initiative evolved to meet changing needs?
2. To what extent are the objectives and activities funded by the Initiative consistent with government priorities, roles and responsibilities?
3. To what extent are the Initiative mandate, activities and outputs consistent with the federal government’s roles and responsibilities?

Performance—Effectiveness  

4. To what extent has the Initiative contributed to the achievement of its outcomes?
5. To what extent has the Initiative been able to develop partnership with other research organizations, particularly with SSHRC, to leverage funds?
   1. What have been the main facilitators and/or impediments towards forging partnerships and leveraging funds? 
6. To what extent have the Initiative-funded research findings/products contributed, directly or indirectly, to informed and evidence-based decision-making? 

Design and Delivery

7. To what extent has the Initiative been implemented according to its original design?

Assessment of Alternatives (counterfactual)

8. What would have happened if the Initiative had been designed differently? For example, if instead of its current design, four research chairs had been created in various universities to lead on the study of terrorism and counter-terrorism in Canada. To what extent would that model have yielded different results?

Governance

9. To what extent is the governance structure appropriate and effective?
10. To what extent does the Project governance meet the requirements as outlined in sections 3.6 and 3.7 of the Policy on Transfer Payments (i.e., that transfer payments be managed in a manner that is sensitive to risks; strikes an appropriate balance between control and flexibility; and establishes the right combination of good management practices, streamlined administration and clear requirements for performance)?

Performance—Program Administration/Efficiency and Economy

11. To what extent has the Project been managed / delivered efficiently and economically?
12. Does the program duplicate any other PS and/or federal government initiatives?

3.2.3 Lines of Evidence

The data collection for this evaluation was mainly carried out in the fall of 2015. The stakeholder survey component was launched on December 22, 2015, and closed three weeks later on January 15, 2016. The methodology for the evaluation included the following lines of evidence:

3.3 Limitations

• We used one comprehensive survey questionnaire to elicit information from various stakeholder groups. This limited the ability of a number of stakeholders to respond to some of the questions due to their level of involvement in and lack of familiarity with the inner workings of the Initiative. Although we conducted a limited number of follow-up phone interviews to seek clarification or further elaboration on certain issues, the evaluation would have been more enriched by developing stakeholder-specific questionnaires.  

• The Initiative had in place a performance measurement strategy. However, it has not been implemented. Although program management was able to provide the evaluation team with an appropriate amount of performance information to conduct the evaluation, the evaluation findings and conclusions could have been stronger and better supported had there been a systematic data collection system in place.

3.4 Protocols

This report was submitted to program managers and to the responsible Assistant Deputy Minister for review and acceptance. A Management Response and Action Plan was prepared in response to the evaluation recommendations. These documents were presented to the PS Departmental Evaluation Committee for consideration and for final approval by the Deputy Minister of Public Safety.

4. Findings

4.1 Relevance

4.1.1 Continued Need

As indicated above, the Initiative was established to address a gap in Canada of understanding about terrorism and how it manifests itself in Canadian communities. Document and literature review suggest terrorism is widely viewed as a serious threat to Canada’s national interest and security. Since the inception of the Initiative, the context for terrorism and the threat it imposes have evolved, the number of attacks has grown and their nature has changed. Terrorist threats have become “more diverse…, dispersed across a wider geographical area, and often emanate from countries without effective government.”^Footnote4 There is a growing phenomenon of individuals leaving their countries of residence to engage in foreign conflicts,^Footnote5 as well as the “lone actor” phenomenon, in which individuals, who may not be connected or are loosely connected (i.e., via Internet and Internet-based social networks) to an external terrorist entity but for various reasons, including ideological sympathy with a particular group, choose to take up their cause. Canada has experienced both of these phenomena. At the time of writing this report, the Government was aware of 180 individuals with Canadian connections who had travelled abroad potentially to take part in various terrorism-related activities.^Footnote6 Two fatal terrorist acts in October 2014, in Ottawa and St. Jean-sur-Richelieu, Quebec, were suspected of being “lone wolves” and self-radicalized. Between 2011 and 2014, the number of reported terrorist attacks worldwide has increased by 31% (from 10,283 to 13,463),^Footnote7 while in Canada the number of police reported terrorist incidences^Footnote8 was 59 in 2011 and 100 in 2014, an increase of 69%.^Footnote9 All of these indicate the growing problem of terrorism and radicalization to violent extremism in Canada and the need to understand these phenomena in the Canadian context. 

There is a lack of Canada-specific empirical data on issues of terrorism and counter-terrorism in the Canadian context that the Initiative is designed to address. Through Kanishka, new work has been produced about the online dynamics of violent extremism. The Initiative also funded the first study ever conducted in Canada^Footnote10 to look at why Canadians convert to Islam. Despite funding approximately 50 major academic projects to improve the understanding of terrorism in the Canadian context and contributing to the creation of a domestic academic and research community to work with government on national security issues, including international partners, there are still gaps in the availability of empirical data and evidence-based analysis both in Canada and abroad on these issues, acknowledged by many researchers and practitioners/ policy makers alike. A recent literature review to understand the experiences of exclusion by marginalized youth and their relationship to radicalization to violent extremism and community-level security interventions, the researchers found the bulk of the available literature was from the UK, which has a quite different context from Canada.^Footnote11 This underlines the need for projects such as Kanishka both to contribute to the theoretical aspects of these discussions, as well as to conduct empirical research to support policy makers’ and practitioners’ efforts in developing appropriate policies and programs to counter violent extremism.

All of the survey respondents indicated a continued need for the Initiative and most (89%) also agreed that the Initiative has evolved to meet changing needs.

Radicalization to violence at its core is a process in which certain individuals take up an ideological position that moves them towards extremism and, ultimately, terrorism. To be contained, it requires multifaceted strategies, long-term planning and intervention. As it is stated in the Global Counter Terrorism Forum’s Ankara Memorandum on Good Practice in Counter Violence Extremism, “States are encouraged to consider comprehensive action in preventing and countering violent extremism, in co-operation with governmental and non-governmental actors.” The majority of the survey respondents viewed the Initiative in this light and regarded it as a good starting point in Canada’s comprehensive approach to counter violent extremism.^Footnote12 However, they found a five-year mandate to be too short to allow the Initiative to achieve its full potential. As one of the respondents puts it, the Initiative “provided the platform for some much needed Canadian research on countering violent extremism-related issues, but this kind of research and policy dialogue, in order to be effective, must continue in earnest.”  

4.1.2 Alignment with Federal and Departmental Priorities

The Initiative is well aligned with federal government as well as PS priorities. Key documents have identified ensuring the safety and security of Canadians at home and abroad as one of the first priorities of the Government.^Footnote13 Canada’s Counter-Terrorism Strategy has a prevention component that is intended "to prevent individuals from engaging in terrorism" and tries to prevent radicalization to violence by addressing the factors that motivate individuals to engage in terrorism-related activities.”^Footnote14 The Initiative is considered an integral part of the “Prevent” element of this strategy and one of the contributors to building resilience in communities to counter radicalization and violent extremism and in contributing to research in the Canadian context.The Government has acknowledged that research is needed to better understand the factors that lead to terrorism and radicalization. In November 2015, at the G20 Summit, the Prime Minister stated that “the fight against terrorism is a major priority for all of our countries… counter terrorism actions must continue to be part of a comprehensive approach based on addressing the conditions conducive to terrorism as stipulated in UN Security Council Resolution 2178, countering violent extremism, combatting radicalization and recruitment, hampering terrorist movements, countering terrorist propaganda.”^Footnote15   

In the 2016 Budget, the Government made a commitment to protecting Canadians by ensuring that public safety officers have the necessary equipment and facilities to investigate crime and enhance their understanding of radicalization at home.^Footnote16 As such, the Government proposed to establish an Office of the Community Outreach and Counter-radicalization Coordinator. The Office will be provided with a budget of $35 million over five years, starting in 2016-17, with $10 million per year ongoing. This initiative is intended to provide “leadership on Canada’s response to radicalization to violence, coordinate federal/provincial/territorial and international initiatives, and support community outreach and research.”^Footnote17

Advancing counter-radicalization and counter-terrorism efforts with all levels of government, international partners, and other stakeholders, with an emphasis on outreach, prevention and accountability, is one of PS’s priorities.^Footnote18 The Initiative contributes to the achievement of this priority and departmental legislative objectives by supporting, among other issues, projects and activities that have strategic importance to the Government of Canada. It also contributes to the enhancement of national security by supporting approaches and the acquisition of knowledge and its translation into action for communities and community-based organizations. The majority of stakeholders (96% of the survey respondents) consulted in this evaluation fully agreed that the mandate and activities of the Initiative were consistent with federal government priorities.

4.1.3 Alignment with Federal Roles and Responsibilities

Although the responsibility for managing the terrorist threat overlaps between the federal and provincial governments, legal and policy authorities to address counter-terrorism rest with the federal government.^Footnote19

Under the Department of Public Safety and Emergency Preparedness Act, the Minister of Public Safety and Emergency Preparedness is responsible for exercising leadership at the national level relating to public safety.^Footnote20 To that end, the Minister is the lead for overall counter-terrorism planning, preparedness and response within the Government of Canada, and for national leadership and coordination on matters relevant to national security.^Footnote21

The Minister’s mandate letter reiterated this responsibility and identified the overarching goal of the Minister of Public Safety as leading the government’s work in ensuring that Canadians are safe while their rights are protected. The Minister was also mandated to create an Office of the Community Outreach and Counter-radicalization Coordinator.^Footnote22

In spite of the shared responsibilities and the efforts by other levels of government in developing and implementing counter-terrorism and counter-radicalization programs, the federal government’s roles and responsibilities to coordinate the overall activities, particularly its participation in funding research was deemed to be critical by the majority of survey respondents. In fact, 98% of the survey respondents agreed that the mandate and activities of the Initiative are consistent with federal roles and responsibilities.

4.1.4 Duplication with Other Initiatives

Survey results, review of electronic databases, and document review were used to assess whether there is duplication between the Initiative and other PS or federal government initiatives. Although 58% of the survey respondents indicated that the Initiative did not duplicate other Government initiatives, many were unsure (33%) and a few respondents (9%) noted that there may be some duplication between research projects funded by Kanishka and other federal initiatives.

The evaluation examined four federally-funded initiatives to identify any duplication or potential opportunities for further synergy: The Canadian Safety and Security Program (CSSP), Defence Research and Development Canada (DRDC), SSHRC, and the Policy Development Contribution Program (PDCP). These initiatives were selected because they all fund and/or conduct research studies related to national security, and some of them were also identified by some of the survey respondents as potential duplication.   

1. CSSP is a federally-funded program led by DRDC’s Centre for Security Science, in partnership with PS. CSSP seeks to “strengthen Canada's ability to anticipate, prevent, mitigate, prepare for, respond to, and recover from natural disasters, serious accidents, crime and terrorism through the convergence of science and technology (S&T) with policy, operations, and intelligence.”^Footnote23 From 2013-15, only three CSSP-funded projects focused on issues related to terrorism and radicalization.^Footnote24 Two of these projects were led by PS to improve the understanding of radicalization and countering extreme violence. The other project was led by the Royal Canadian Mounted Police (RCMP) and Ryerson University, which evaluated and developed community-informed strategies to strengthen resiliency from violence and radicalization. All three cases built on research initially conducted through Kanishka. The emphasis on developing innovative S&T solutions for application in operational settings as part of CSSP’s Call for Proposals suggests that there is no duplication between CSSP and the Initiative. Rather, the knowledge produced by CSSP-funded research studies can complement the Initiative’s emphasis on building new academic research capabilities on terrorism and counter-terrorism, especially in the social and behavioural sciences, by applying an S&T approach towards developing operational solutions.
2. As an agency of Canada’s Department of National Defence (DND), DRDC comprises eight research centres across Canada and is the national leader in defence and security science and technology. DRDC “provides DND, the Canadian Armed Forces (CAF), other government departments as well as public safety and national security communities with the knowledge and technological advantage needed to defend and protect Canada’s interests at home and abroad.”^Footnote25 After review of DRDC funded projects since its inception, only four projects were found to be related to terrorism and radicalization. These studies mainly focused on the process of radicalization through historical data and empirical literature review. Upon examination, there is no evidence of duplication between DRDC and the Initiative. Similar to CSSP, DRDC’s focus on security science and technology can complement the social behavior research funded through the Initiative and create potential opportunities for synergy.
3. SSHRC is a “federal research funding agency that promotes and supports postsecondary-based research and research training in the humanities and social sciences.”^Footnote26 After review of successful proposals funded through SSHRC in 2014-15, twelve projects were found to be related to terrorism and radicalization^Footnote27 Of these 12, one project was also funded through the Initiative.^Footnote28 While there were many SSHRC-funded research projects related to terrorism and radicalization, further examination found subject matter differences between research funded through SSHRC and those funded through the Initiative. In addition, by design, the Initiative is meant to engage with SSHRC to leverage further funding and to create a network of researchers. This may explain the number of terrorism-related projects funded through SSHRC. Since there is already collaboration between SSHRC and the Initiative and the research topics appear to differ, there is no clear indication of duplication.
4. PDCP is a grant and contribution program at PS that supports strategic projects undertaken by the Department’s stakeholders that contribute to policy making and improved service delivery. PDCP funds an average of 10-15 projects each fiscal year by providing contributions to Canadian provinces, territories, public and private non-profit organizations, aboriginal governments, local non-government organizations and national voluntary organizations. These projects can be related to any of the policy areas within the Department such as Emergency Management, Countering Crime, and National Security. Examination of projects funded through the PDCP within the last six years found only one project related to radicalization, which predates Kanishka.^Footnote29

The evaluation concluded that there is no evidence to suggest that there is duplication between what is funded by CSSP, DRDC, SSHRC, or PDCP and the Initiative. CSSP and DRDC’s focus on security science and technology complements the social science research and capacity building funded through the Initiative. The current collaboration between SSHRC and the Initiative as well as the difference in research topics suggests that there is no duplication of research. Finally, examination of PDCP-funded research shows no indication of duplication within the Department.

4.2 Performance - Effectiveness

This section assesses the performance of the Initiative in contributing to the achievement of its following three immediate expected outcomes:

• Mechanisms are in place to collaboratively consider existing and emerging terrorism-related issues.
• Policymakers, decision-makers and practitioners have access to timely and relevant research.
• Researchers engaged in terrorism-related issues have resources to study priority areas and relevant policy gaps.

4.2.1 Mechanisms and networks are in place for ongoing dialogues on terrorism-related issues

One of the expected immediate outcomes of the Initiative was to establish a core network of multidisciplinary scholars and students from across Canada and international experts support collaboration and knowledge transfer and ongoing dialogue across different sectors on terrorism and counter-radicalization related issues. It was envisioned that through these dialogues, the stakeholders would collaboratively assess issues such as threat environment, public perceptions, etc., and identify challenges (i.e., lack of resources, knowledge gaps, problems with available tools), and ways of addressing those challenges. The activities for the achievement of this expected outcome were to be funded both through the $1.6 million over five years that were earmarked for the SSHRC Partnership Grant Program, as well as a $3.8 million over five years funding that was allocated to a set of activities directly led by PS.

A review of the baseline information indicates that at the time of the Initiative’s inception, there was no known community of scholarship in Canada that produced research on terrorism and counter-terrorism relevant to the Canadian context. Researchers appeared to have been largely isolated from one another and their research products had limited practical applicability for policy-makers and practitioners. Document and literature reviews and the stakeholder survey suggest the Initiative has been, to a large extent, successful in establishing a network of multidisciplinary scholars and students across Canada, and among some international experts, to support collaboration, knowledge transfer and ongoing dialogue across different sectors on terrorism- and counter-radicalization related issues.

In an effort to support the development of a stronger research community and knowledge base, and to inform government and public understanding of countering violent extremism related issues, the Initiative’s Lead Team at PS has organized or attended numerous events both in and outside Canada over the past five years, including workshops, research symposiums, conferences and other sessions. PS also directed 9 research studies, funded at least 24 student research projects and hosted small and large-scale events to bring together officials, researchers and stakeholders, including family members of victims of the Air India bombing.^Footnote30 In 2013, for example, a workshop in Ottawa brought together leading experts on terrorism and extremist crime data from across North America and the United Kingdom. The objective of the event was to share lessons learned on open source terrorism-related databases and to outline the key components of a Canadian database that could be accessed by researchers across Canada to support high-quality, relevant research on understanding terrorist and other extremist crime trends.^Footnote31

The Initiative also contributed to the establishment of the Canadian Network for Research on Terrorism, Security and Society (TSAS) in 2012 by providing the core funding for its creation. Led by a consortium of universities including the University of British Columbia and the University of Waterloo, TSAS engages in policy-relevant research and dissemination on terrorism, security and society and cultivation of a new generation of young scholars interested in studying terrorism and other national security-related issues. It fosters communication and collaboration on national security and terrorism issues among academic researchers in Canada and between researchers and policy officials. By having approximately 90 affiliated researchers and over 100 affiliated students from a variety of disciplines and universities, and establishing partnerships across government, civil society groups and internationally, TSAS has been described as Canada’s focal point for learning about and contributing to the growing knowledge base on terrorism and counter terrorism in Canada.

The majority of survey respondents (62%) support the findings of the literature and document review, indicating the Initiative has been fully or moderately successful in putting in place mechanisms for the Canadian policy-makers, practitioners and researchers to collaboratively consider existing and emerging terrorism-related issues. A number of the stakeholders described the Initiative as contributing to the creation of a set of formal and informal communities of science and practice in Canada that work collaboratively with government officials to address national security issues, such as terrorism and radicalization to violence. This was demonstrated in practice after the October 22 Ottawa shooting where TSAS and Kanishka affiliated researchers participated in over 100 media appearances across Canada.  

4.2.2 Policy-makers, decision-makers and practitioners have access to timely and relevant research

Since its inception in 2011, the Initiative has administered five rounds of open calls for proposals through its contribution program, which resulted in funding 37 research projects, with a total cost of approximately $6.2M. PS has also funded several targeted research studies to address specific information needs of the Department and other government partners in response to changing priorities or certain events with national security implications. These studies cover topics such as the online activities of individuals involved with violent extremist groups to understanding and supporting the needs of victims of terrorism, to creating and testing approaches for police, community groups, teachers and other practitioners to counter violent extremism, etc. To date, well over 30 of these research studies have been completed, the majority of which have been published on line.

An important factor in optimizing the use of research results is the extent to which the government’s research needs and priorities are articulated and communicated with the researchers and vice-versa. To ensure research is relevant to the needs of the Government of Canada and to address gaps in knowledge, the Lead Team at PS has the primary responsibility to obtain input from stakeholders and communicate needs to researchers. At the outset of the Initiative, the Lead Team, in collaboration with departments and agencies involved in national security, including the Privy Council Office, developed a set of research themes to provide guidance in addressing relevant gaps in research or tools for front-line officers. Using its existing domestic^Footnote32 and international networks,^Footnote33 the Lead Team shared information, determined research themes, questions, and priorities, identified collaboration mechanisms and asked for feedback on projects. The same formal and informal networks were used to review the research themes, mid-course and at the last call for proposals, as well as for five major calls for contract research, to ensure that they remained relevant to the research needs of government partners, reflected new understanding and developments, and remained with the scope of the Initiative’s Terms and Conditions. Subsequently, research themes were communicated to researchers in the calls for proposals, posted online, and conveyed through communication by the Chair of the ADM Steering Committee to the academic community. In addition, the themes were part of the Memorandum of Understanding between PS and SSHRC concerning collaboration on the management of the Initiative. The final assurance that the proposals supported national security needs and government priorities came from the ADM Steering Committee, which only considered research proposals that addressed one or more priority areas. These processes have supported the Initiative’s evolution and allowed the Initiative to fund only those research studies that were within the scope of the Initiative’s Terms and Conditions and addressed the most current and pressing research areas as identified by government partners.   

As the intermediary between the researchers and the information users, the Lead Team is responsible for transferring research findings to policy and decision-makers. Although there is no formal process in place, the Lead Team uses its interpersonal relationships and networks to maintain regular conversations with user communities, and as the main means of conveying the information to them. It also sends out broad mailings of relevant reports to members of various networks and facilitates some degree of direct working relationships between the relevant experts and the users.

The evaluation team was also provided with some evidence, including a number of memos, briefing notes, deck presentations and others that were developed by the Lead Team using, in part or fully, the research findings. The majority of these documents had been prepared for departmental senior management, the Minister and other parliamentarians for the purpose of informing policy and decision-making. There is also evidence to indicate that research findings were used to inform front line officers, community workers and other practitioners. For example, findings from a study entitled The Syria Conflict: the Evolution of al Qaeda and other Militant Movements after the Arab Spring, by the International Centre for the Study of Radicalization and Political Violence, were used to brief frontline police officers on the phenomenon of young females traveling to Syria, their method of recruitment, and signs to consider in assessing risk and supporting appropriate early intervention. Similarly, researchers from Ryerson University and the University of Alberta are working with communities and police in Toronto and Edmonton to study factors that heighten or reduce the risk of violent extremism, how they differ across communities, and how police services can strengthen partnerships tailored to local needs. 

Although the Initiative has been successful in identifying information gaps and funding appropriate research studies to address those gaps, it was less successful in transferring knowledge to the appropriate players, particularly the general public.^Footnote34 The Initiative has endeavoured, either directly or through its affiliated research networks, to have a web presence; however, the evaluation was unable to determine the existence of a systematic approach to produce appropriate products for disseminating the research findings and other information for policy and decision-makers and/or public consumption.^Footnote35

As this is often found to generally be the case, the findings of academic research papers may not be immediately relevant to policy and decision-making and/or their significance understood by the general public. The absence of a systematic approach for communicating research findings with the public has been identified by the evaluation as a missing link in the transfer of knowledge. As it was indicated by one of the survey respondents, “the Initiative has generated a lot of very interesting findings, but outside of a selected number of recipients who have written public articles (non-technical), blogs and carried out conversations on social media, little has been done to drive the awareness of this work with the public.” This presents the Department with an opportunity to put in place mechanisms to better ensure the translation of academic research findings to inform policy/decision-making, as well as the public discourse in Canada on issues surrounding terrorism and countering violent extremism.^Footnote36 According to another respondent, “the work done by the various parties over the duration of the Initiative needs to expand to include knowledge exchange – both from a scientific and public pedagogical standpoint. Sensitizing communities, empowering the general public with cognitive tools such as critical thinking, digital literacy and information literacy through sustained collaborations with media representatives, educators and political leaders will ensure that we are able to take steps to prevent terrorism and radicalisation that could lead to violent extremism.”      

Recommendation 1: The ADM of the Portfolio Affairs and Communications Branch and/or the future Senior Departmental Officer responsible for the Office for Community Outreach and Countering Radicalization to Violence should ensure that Kanishka-related research findings are better communicated to policy and decision makers and the general public.

4.2.3 Researchers engaged in terrorism-related issues have resources to study priority areas and relevant policy gaps

Kanishka was designed to address the lack of empirical data on issues of terrorism and counter-terrorism in the Canadian context by providing funding to researchers to undertake research projects and build networks to exchange ideas. The Initiative did this by providing over $10M in funding over 5 years. By using a number of mechanisms, discussed in Section 4.2.2, the Initiative’s funded research projects were relevant to government priorities, addressed policy gaps and remained current with emerging needs. Given the scarcity of social science research funding opportunities in Canada for terrorism and counter-terrorism, many stakeholders perceived the injection of Kanishka funding as very positive, and lamented the prospect of discontinuation of this funding.

The level of interest in terrorism and counter-terrorism and the need for funds outstripped what was available under Kanishka. Over the past five rounds of calls for proposals, 160 proposals were received, representing more than $36M in requests for funding. With each call, the number of proposals received increased by twofold. Although the Initiative could only fund a small fraction of these proposals, the stakeholders nevertheless described Kanishka funding as “instrumental in providing a boost to research in this area.”

The majority of stakeholders surveyed (65%) indicated a high or moderate level of confidence in the adequacy of the resources provided to the researchers and 29% expressed a modest level of confidence. This percentage is slightly lower when looking at just the funding recipients (63% expressed high or moderate levels of confidence and 31% indicated a modest level of confidence). Only 4% of all stakeholders addressing this question indicated that researchers engaged in terrorism-related issues did not have adequate level of resources to conduct their studies.   

4.2.4  Governance

The evaluation team reviewed various internal documents to examine the extent to which the Initiative has been able to put in place a rigorous governance framework. The existing governance structure is robust; the roles and responsibilities of different committees and advisory groups are clearly defined in their respective terms of reference and followed.

Over the course of the Initiative, the oversight committees met on a regular basis as specified in the inception documents. The Working Group met once per round of funding, including 5 face-to-face meetings, 1 teleconference and 3 virtual meetings. The Steering Committee also met at least once per round of funding, which included 5 face-to-face, and 3 times virtually. The Advisory Group met 3 times on an ad hoc basis. The Working Group and Steering Committee have records of decisions on file.

Generally, the majority of the stakeholders indicated that the appropriate people are involved in the Initiative.

4.2.5 Program Design and Delivery

Various options were considered in designing the Initiative. However, for reasons that will be described below, the existing option, of a national initiative consisting of four complementary elements, including a grants and contribution program, two partnership arrangements with SSHRC and a program component designed to support direct activities by the Department, was chosen. The evaluation assessed the extent to which the Initiative was implemented according to its original design, and to what extent this particular design has impacted the achievement of the expected outcomes.

The assessment was done using the Rapid Impact Evaluation approach, which was being piloted by TBS to test its applicability in the Government of Canada context. This approach required the evaluation to engage distinct groups of experts and program stakeholders to weigh the impact of the actual and alternative designs of Initiative in achieving the stated objectives. By assessing what might have been the impacts under an alternative design (i.e., counterfactual), the evaluation tried to identify impacts that could be attributed to the current design of the Initiative. This section presents, first, the rationale and main assumptions behind the current design of the Initiative and, second, its impact on achieving the desired outcomes.     

A review of the Initiative’s inception documents indicates that the option of a national program, as described above, was chosen primarily for two main reasons: 1) as the most direct means to ensure that research was focused on pressing questions about terrorism and violent extremism facing the Canadian government and society; 2) as the shortest timeframe to build capacity nationally and across multiple disciplines to conduct research and to produce results. The following table illustrates some of the main assumptions behind the inclusion of each element in the Initiative’s original design.

Our assessment indicated that the Initiative was implemented, to a large extent, according to its original design; however, not all the assumptions have materialized in the way they were envisioned. This is particularly the case with respect to the two SSHRC-related components, where it appears that the Initiative has had limited success in establishing an active partnership with the organization.

Evidence from documents and follow-up interviews indicated that, although SSHRC was able to connect the Initiative’s Lead Team to its established networks of researchers, particularly at the outset of establishing the Initiative, there were only a limited number of research proposals received through the Insight Grant Program, resulting in funding only one research study over the past five years. Similarly, there was only one proposal that received Kanishka funding under SSHRC’s Partnership Grant Program. This could be partly attributed to SSHRC’s decision, shortly before the Initiative launch, to eliminate a partnership program that was in place to allow SSHRC to formally partner with government departments to conduct joint “fenced calls for proposals.” Under that program, government partners were granted a lead role in shaping the calls for proposals, and there was a formal commitment by SSHRC for the upfront matching of funds. The elimination of the program meant there were no Kanishka-specific calls for proposals by SSHRC. Instead, anyone interested in doing Kanishka-related research competed in the general calls for proposals against proposals in all other subject areas. Only proposals successful in that general competition were provided to PS for funding consideration. Some of the survey respondents have attributed this apparent lack of interest in seeking funding through SSHRC mechanisms to the complexity and highly labour-intensive nature of SSHRC’s funding processes.

As a result of these developments, the Initiative did not utilize all the funding that was earmarked for the two SSHRC-related components as it was originally planned. Rather a decision was made to reallocate a large portion of the funding to the Kanishka Project Contribution Program. Follow-up interviews indicated that the management team consulted PS Finance and relevant TBS officials to ensure the appropriateness of the reallocation. However, from an evaluation perspective, this, nonetheless, represents a slight deviation from the Initiative’s original design and signifies a predicament that was not foreseen in the Initiative’s original design. Ideally, it would have been more prudent to have had included a mechanism in the inception document to mitigate a situation such as this.^Footnote37  

4.2.6 Assessment of Alternative Designs

To assess the viability of the Initiative’s current design and the extent to which the Initiative in its current form has contributed to the achievement of the Initiative’s expected outcomes, the evaluation presented the following two scenarios as alternatives in the stakeholders’ survey.

whether or not the Initiative’s expected outcomes were negatively affected by the Initiative’s relationship with SSHRC, which did not develop as envisioned.

The second alternative was a scenario allocated Kanishka’s $10M funding to create four research chairs at four major Canadian universities to lead studies of terrorism and counter-terrorism in Canada. This alternative was chosen to examine if housing the Initiative outside the Department would have produced better results.

Image Description

The graph shows the percentage likelihood of achieving outcomes between the original design and 2 alternative counterfactuals in the short-term (2015) and long-term (2020).

For 2015, program stakeholders believe that the current program is 3% more likely to achieve outcomes than Alternative 1; funding recipients believe that the current program is 3% less likely to achieve outcomes than Alternative 1; subject matter experts believe that the current program is 17% less likely to achieve outcomes than Alternative 1. Together, all program stakeholders believe that the current program is 1% less likely to achieve outcomes than Alternative 1.

For 2020, programs stakeholders believe that the current program is 6% more likely to achieve outcomes than Alternative 1; funding recipients believe that the current program is 4% more likely to achieve outcomes than Alternative1; subject matter experts believe that the current program is 6% less likely to achieve outcomes than Alternative 1; and together, all program stakeholders believe that the current program is 4% more likely to achieve outcomes than Alternative 1.

For 2015, program stakeholders believe that the current program is 24% more likely to achieve outcomes than Alternative 2; funding recipients believe that the current program is 7% more likely to achieve outcomes than Alternative 2; subject matter experts believe that the current program is 21% more likely to achieve outcomes than Alternative 2; and together, all program stakeholders believe that the current program is 19% more likely to achieve outcomes.

For 2020, program stakeholders believe that the current program is 42% more likely to achieve outcomes than Alternative 2; funding recipients believe that the current program is 25% more likely to achieve outcomes than Alternative 2; subject matter experts believe that the current program is 47% more likely to achieve outcomes than Alternative 2; and together, all program stakeholders believe that the current program is 37% more likely to achieve outcomes than Alternative 2.

The majority of respondents agreed that the Initiative’s current design was the preferred option. The omission of SSHRC components from the first alternative design was seen to have a marginal negative impact on the program in the short-run and no difference in the Initiative’s outcomes in the long run. In other words, inclusion of SSHRC was not expected to make a difference in Canadian policy makers, decision makers, and practitioners’ access to timely and relevant research.

Similarly, the majority of the respondents found Kanishka’s existing program design to be much better than the second proposed alternative, i.e. Research Chairs. In fact, the existing program design was assessed to provide around 19% more contributions to informing policy-making in the short term and around 37% more in the long term than the Research Chairs option.

To house the Initiative at PS was also perceived by the majority of the stakeholders (82%) as positive, mainly because it would better ensure stable funding for the Initiative, as well as connecting researchers with government officials and providing pathways for transferring knowledge to policy and decision-makers.     

4.2.7 Consistency with Government of Canada Policy on Transfer Payments

Given that the majority of the Initiative’s allocated funding is through grants and contributions (Vote 5), the evaluation assessed the extent to which the Initiative’s design, delivery and administration conformed to the best practices requirements as outlined in the Government’s Policy on Transfer Payments, in particular sections 3.6 and 3.7. These sections require that transfer payments be managed in a manner that is sensitive to risks, strikes an appropriate balance between control and flexibility, and establishes the right combination of good management practices, streamlined administration and clear requirements for performance.  

The majority of the stakeholders agreed that the design, delivery and administration of the Initiative conformed to the requirements of the Policy on Transfer Payments. The following graphs present the stakeholders’ assessment of each particular question:

To what extent do you think the Initiative is being managed in a manner that:

Image Description

Respects sound stewardship and the highest level of integrity: Fully (76%), Moderately (13%), Somewhat (2%), Not at all (0%), Don’t know (9%)

Image Description

Respects transparency and accountability: Fully (73%), Moderately (11%), Somewhat (4%), Not at all (0%), Don’t know (11%)

Image Description

Is fair, accessible and effective for all involved (departments, applicants and funding recipients): Fully (69%), Moderately (13%), Somewhat (2%), Not at all (4%), Don’t know (11%)

Image Description

Strikes an appropriate balance between control and flexibility: Fully (67%), Moderately (16%), Somewhat (4%), Not at all (0%), Don’t know (13%)

Image Description

Establishes the right combination of good management practices, streamlined administration and clear requirement for performance: Fully (51%), Moderately (29%), Somewhat (4%), Not at all (2%), Don’t know (13%)

The issue of financial reporting was described by some of the respondents, particularly funding recipients, as problematic. From the funding recipients’ perspective, the financial reporting processes are complicated and lack clarity in terms of expectations. Furthermore, the requirement to provide quarterly financial reports was described as cumbersome, given that most researchers do not have staff or expertise in financial matters, nor do their institutions collect financial information on a basis that allows them to feed into these quarterly reports. From the departmental staff perspective, changing requirements and processes for financial management have been burdensome. They have also identified cash-flow reporting as a significant problem in working with funding recipients. This may be the reason for the Initiative receiving a lower rating in meeting the requirement to establish good management practices, streamlined administration and clear requirement for performance.

According to Section 6.5.7 of the Policy on Transfer Payments, the administrative requirements on funding recipients should be proportionate to the level of risk, which means that monitoring, reporting, and auditing practices should be reflective of the risks specific to the program, the value of funding in relation to administrative costs, and the risk profile of the recipient. Our review of the financial reporting practices and cash-flow statement template indicate that there is only one set of requirements in place for all, which requires every recipient, regardless of the value of funding or the risk profile of the recipient, to provide the same information on a quarterly basis.

Documents reviewed also indicate that following the 2007 Independent Blue Ribbon Panel on Grants and Contributions, PS introduced a set of changes, including the development of a tailored risk-based tolerance levels and corresponding monitoring plans, as well as several administrative tools to improve the administration of grants and contributions in the Department. Although the Initiative benefitted from these improvements, given the above discussion on the current reporting requirement, there may be still opportunities for further improvement in this area. To meet the reporting objectives of the Policy on Transfer Payments, particularly with respect to simplifying administration and reducing the burden of reporting on the funding recipients, it might be prudent for the Department to reassess the necessity of its quarterly reporting requirement for funding recipients to ensure that it is proportionate to the level of risk. The Department should also review the template for obtaining financial information (i.e., cash-flow statement) to ensure that it is easy to understand and requests only the type of information that is absolutely necessary for accountability and project management purposes. 

Recommendation: The ADM of the Portfolio Affairs and Communications Branch and/or the future Senior Departmental Officer responsible for the Office for Community Outreach and Countering Radicalization to Violence should in collaboration with the Assistant Deputy Minister Corporate Management Branch ensure that funding recipients reporting requirements are proportionate to their current risk profile.

4.2.8 State of the Initiative’s Performance Measurement

The Initiative developed a Performance Measurement Strategy (PMS) at its onset. For the first two years after its launch, three performance reports were produced and tabled at Treasury Board. However, as the Initiative matured, Treasury Board cancelled the requirement for producing these semi-annual performance reports and asked that any future reporting be done through the Departmental Performance Reports.

In spite of producing these performance reports, the Initiative’s original PMS was never implemented. In its 2013 assessment of the Initiative’s PMS, Internal Audit and Evaluation identified several areas of improvement that were communicated to the Initiative Lead Team for consideration. Although the Initiative’s logic model has been modified since then, the PMS as a whole has remained unchanged. Any renewal efforts should include the development of a comprehensive Performance Measurement Strategy for the Initiative in accordance with existing TBS and PS guidelines. The Strategy should include a scheme for gathering and reporting performance information on an ongoing basis.

4.3 Performance - Efficiency and Economy

Overall, the Initiative’s operations and activities were delivered efficiently and economically. See Annex D for the Initiative’s budget and expenditures from its launch in 2011-12 to 2015-16. As illustrated, both salary and operations and maintenance costs (including grants and contributions paid, cost of research contracts and events, etc.) stayed within and, in some cases, below the allocated budgeted amount. However, some survey respondents have commented that this efficiency might have been attained at the expense of sacrificing part of the Initiative’s effectiveness.

Survey respondents commented that the Initiative was understaffed from the outset. As indicated under section 2.2 above, the Project Lead Team at PS that was responsible for day-to-day administration of the Initiative comprised of mainly three staff (i.e., the Manager of Research and Academic Relations, a Project Manager, and a Project Support Officer). Although this might have contributed to lowering overhead costs, the drawback has been the limitation it has imposed on achieving the Initiative’s full potential had it been staffed adequately.

The issue of the Initiative being understaffed and the impact that it has had on its operations, including the achievement of its expected outcomes, was raised by many of the survey respondents, including program staff and funding recipients alike. As indicated by one of the survey respondents, “Much was accomplished under the current model, but things would be much further along in terms of policy development and knowledge dissemination if the Initiative was fully staffed.” Another survey respondent identified understaffing as “the largest barrier to advancing [the Initiative’s] goals.” Accordingly, “Given how much effort it takes to assess research, build and manage ongoing relationships, provide feedback, determine how to best package and communicate findings to key audiences, the fact that no analyst was assigned to the team was a huge barrier.” Others indicated that understaffing made it difficult for the Initiative “to achieve service standards of the Department”, as it takes too long “to respond to the needs of recipients i.e., payment requests, amendment requests, questions about their respective contribution agreements.”   

The program administration ratios were high for the first two years.^Footnote38 This was mainly because during this time the main focus was on establishing the Initiative and, hence, there was a limited amount of contributions paid out.^Footnote39 As the Initiative became more established and began funding more projects, the administration ratio fell to 12% in 2013-14 and 15% in 2014-15. This suggests that when the Initiative started to pay out its contributions, it was able to do so in a relatively efficient manner. In 2015-16, the administration ratio increased to 24% due to a combination of higher administration costs and lower amounts of contributions paid by the Initiative. However, this is normal for a contribution program of this size. Our review of similar programs indicated that this is usually the case for grants and contributions programs to have a high program administration ratio in the first year.

5. Conclusions

Relevance

The original impetus for the establishment of the Initiative in 2011 was to invest in research on pressing questions for Canada to better understand what terrorism meant in the Canadian context, how it was changing over time, and what could be done to support effective policies and programs to counter terrorism and violent extremism in Canada. In 2016, these needs persist.

Terrorism is still being seen as a threat to Canada’s national interest and security. The number of terrorist incidents has increased both in Canada and around the world. The nature of terrorism has changed as well. Many countries, including Canada, are facing radicalization to violence, particularly of youth. This has manifested itself in many forms, including some of these individual leaving their country of residence to take part in foreign conflicts. There are 180 Canadians who are known to have done so.

The Government of Canada has put in place a comprehensive Counter-Terrorism Strategy to prevent, detect, deny and respond to domestic and international terrorism. Given the complexity of the issues involved, there is a continued need to learn more about what factors motivate these individuals and how to prevent them. The Initiative has contributed to the prevention of terrorism and radicalization through funding research studies, generating knowledge and awareness and building networks and communities of practice. However, in spite of what has been achieved, there is a continued need for the Initiative.

Recent budget announcements and the articulation of the government’s national security priorities, including the creation of the Office of the Community Outreach and Counter-radicalization, demonstrated that the Initiative’s functions are well aligned with the Government of Canada’s public safety priorities, roles and responsibilities.  

Performance

The Initiative has contributed, to a large extent, to the achievement of its expected outcomes. In particular, the Initiative has put in place various mechanisms for having ongoing dialogue across different sectors on terrorism and counter radicalization. Canadian policy and decision-makers, as well as frontline officers and other practitioners, have access to more relevant and timely information, as well as to experts and community leaders to inform policy and decision-making. Researchers also have more resources and support at their disposal to conduct research and to study priority areas.

The Department has put in place a robust governance framework to oversee the delivery of the Initiative. Overall, the Initiative was delivered efficiently and economically. The design, delivery and administration of the grants and contributions components of the Initiative were found to generally conform to the requirements of the Government of Canada’s Policy on Transfer Payments.

The Initiative was found to have conducted its operations efficiently and economically. Although the Initiative was able to keep its overhead cost low and within budget, suggestions were made that this might have negatively affected fuller achievement of its expected outcomes.     

Notwithstanding the above-described achievements, the evaluation has identified a few opportunities for improvement. The following recommendations have been made in the spirit of continuous improvement.                                        

6. Recomendations

The ADM of the Portfolio Affairs and Communications Branch and/or the future Senior Departmental Officer responsible for the Office for Community Outreach and Countering Radicalization to Violence should ensure that:

• Kanishka-related research findings are better communicated to policy and decision makers and the general public.

In collaboration with the Assistant Deputy Minister Corporate Management Branch ensure that:

• funding recipients’ reporting requirements are proportionate to their current risk profile.

7. Management Response and Action Plan

The Kanishka Project has built significant policy-relevant research capacity in support of Canadian policy and program needs, as well as extensive multi-disciplinary networks bringing together researchers, officials, practitioners and community members, domestically and abroad. As noted by the Evaluation, more work is needed to address ongoing and new knowledge needs, and to support such work with a stronger emphasis on knowledge mobilization, as well as continuing commitment to addressing process burden. Importantly, the new Office for Community Outreach and Countering Radicalization to Violence will have more staff support available to help drive the knowledge mobilization function, and will draw more on the program capabilities of Emergency Management and Programs Branch to support grants and contributions functions.

Annex A: Logic Model

Image Description

The Kanishka Project activities are: Transfer payment program and contract management; terrorism and counter-terrorism research assessment; liaising with key partners (government, civil society and international partners); leadership, direction-setting and coordination. The outputs are: signed contracts and contribution agreements; information and advice; collaboration and networking mechanisms. The outputs lead to the following immediate outcomes: Researchers engaged in terrorism-related issues have resources to study priority areas and relevant policy gaps; policymakers, decision-makers and practitioners have access to timely and relevant research; mechanisms are in place to collaboratively consider existing and emerging terrorism-related issues. The immediate outcomes lead to the following intermediate outcomes: Funded research organizations have the capacity to undertake terrorism-related research relevant to Canada; those engaged in addressing terrorism-related issues are equipped with relevant knowledge and tools. The intermediate outcomes lead to the ultimate outcome: Policies and program related to Canada’s national security are informed by relevant research.

Annex B: Documents Reviewed

• Amarasingam, Amarnath (2014). Barriers to violent radicalization: Understanding pathways to resilience among Canadian youth.
  
• Budget 2016.
  
• Centre for Studies in Religion and Society, UVIC. Book on Religion, Radicalization & Securitization: A Canadian Perspective. 2010-2011.
  
• Dawson, L., Edwards, C. & Jeffray, C. (2014). Learning and Adapting: the use of monitoring and evaluation in countering violent extremism. A Handbook for Practitioners.
  
• Global Counterterrorism Forum. Ankara Memorandum on Good Practices for a Multi-Sectoral Approach to Countering Violent Extremism
  
• Government of Canada (2013). Building Resilience Against Terrorism, Canada’s Counter-Terrorism Strategy.  
  
• Minister of Public Safety and Emergency Preparedness Mandate Letter (2016).
  
• Parliament of Canada. Standing Senate Committee on National Security and Defence, March 7, 2016
  
• Nolan, E. & Hiebert, D. (2014). TSAS Working Paper Series: Social Perspectives on National Security: A Review of Recent Literature.
  
• Statistics Canada (2015). Police-reported crime statistics in Canada, 2014.
  
• Public Safety Canada (2014). Public Report on the Terrorist Threat to Canada.
  
• Public Safety Canada. Kanishka Project Third Report to Treasury Board, January-June 2013. 
  
• Public Safety Canada. Report on Plans and Priorities 2016-17.
  
• Rapid Evaluation. I-Tech technical Implementation Guide # 6 www.go2itech.org/resources/technical-implementation...RapidEval.../file.
  
• Speeches from the throne (2013, 2015).
  
• United States Department of State (2015). Bureau of Counterterrorism, Country Reports on Terrorism 2014 Report, Chapter 1.
• World Bank. Independent Evaluation Group, Impact Evaluation: The Experience of the Independent Evaluation Group of the World Bank.

Media:

• CBC News. Canadian Converts to Islam Focus of Study by Australian Sociologist: http://www.cbc.ca/news/politics/canadian-converts-to-islam-focus-of-study-by-australian-sociologist-1.3165573
• Office of the Prime Minister. G20 Statement on the Fight Against Terrorism http://pm.gc.ca/eng/news/2015/11/16/g20-statement-fight-against-terrorism#sthash.nTh4GCSs.dpuf

Websites:

• Canadian Safety and Security Program http://www.science.gc.ca/default.asp?lang=En&n=D6358D2D-1
• Canadian Safety and Security Program, Call for Proposals
  www.science.gc.ca/default.asp?lang=en&n=A4F3D66C-1
• Defence Research and Development Canada
  http://www.drdc-rddc.gc.ca/en/index.page
• Public Safety Canada
  http:publicsafet.gc.ca/cnt/ntnl-scrt/cntr-trrrsm/index-en.aspx
• Social Sciences and Humanities Research Council
  http://www.sshrc-crsh.gc.ca/about-au_sujet/index-eng.aspx

Annex C: Intiative Sponsored Events

1. Kanishka London Symposium – December 14-15, 2015 (London, UK)
2. Workshop on Behaviour patterns of lone actor terrorists, solo actors, and mass casualty shooters  (Gill/Corner/Amarasingam ) – December 2015
3. Open Source Intelligence Workshop – November 2015
4. Radicalization and Violent Extremism: Lessons Learned from Canada, the UK and the US - July 2015 (Washington, DC)
5. TSAS Chatham House Event with Dr. Nico Prucha on Counter-Narratives – June 2015
6. Kanishka Project Research Affiliate Program Workshop – March 2015
7. Kanishka Project Symposium on Social Media and Online Analytics – March 2015
8. Children & Youth in Challenging Contexts (CYCC) Network and Wisdom2Action Calgary -
   “Best Practices to Promote the Resilience of Youth Against Violent Extremism” (funding through contribution program) – March 2015
9. Metropolis Pre-forum: Kanishka Project Symposium (17th National Metropolis Conference) – March 2015
10. Kanishka Project Symposium on Community-Based Research – Feb 2015
11. Symposium on Early Intervention models for preventing violent extremism (Khan/Kohler/Vidino) – December 2014
12. Kanishka Project Research Affiliate Program Event – March 2014
13. CIDB Database Event – March 2014
14. Kanishka Project Sessions at the 16th National Metropolis Conference – March 2014 (RDIMS#1057916)
15. Air India Families’ Event – Feb 2014
16. Interdepartmental workshop to examine connections between research on hate speech and preventing/responding to violent extremism – June 2013
17. Kanishka retreat of officials and key subject matter experts to conduct a mid-term progress review of Kanishka, as well as the state of terrorism research and the threat picture more generally, to help focus priorities and operation of the program – June 2013
18. Ottawa – breakfast session for Minister of Public Safety with Kanishka graduate student Research Affiliate Program award winners, along with select Kanishka-funded scholars – April 2013
19. Ottawa series of events on risk and resilience at Metropolis conference including plenary ‘Understanding risk and fostering resilience in a diverse society’, as well as workshops – March 2013
20. Organizational and funding support for NS Policy led international Symposium on Measuring Effectiveness of Countering Violent Extremism Programming which featured several Kanishka-supported researchers – March 2013
21. Kanishka Opening Conference – November 2012
22. Montreal/Quebec Metropolis Centre ‘justice, policing and security’ capstone event – November 2012
23. Ottawa universities collaborative research design workshop for researchers and officials – September 2012
24. Ottawa Kanishka collaborative research design symposium for researchers and officials – May 2012
25. UBC workshop on migration and security, and research network development workshop – April 2012
26. Metropolis Toronto pre-conference: ‘Fear and Polarization: What can we learn from Europe?’, a plenary on resilience, multiple workshops, and a keynote by the Minister of Public Safety – Feb 2012
27. Information session for families of Air India victims on Kanishka and the ex gratia payment program – January 2012
28. Deputy Minister-level policy conversation, and working level roundtable with scholars and officials on countering violent extremism (co-host with NS Policy) – January 2012
29. Multiple engagement sessions with individual researchers to discuss their Kanishka research – including for multiple meetings of the Cross Cultural Roundtable on Security – or research related to Kanishka to gauge if of relevance to Canadian policy and program needs on counter-terrorism (estimate well over 20)

Annex D: Financial Information

Note: In 2014-2015, $102,156 was transferred to Social Sciences and Humanities Research Council for the research on right wing extremist networks in Canada; $89,635 was also transferred in 2015-2016.

Date modified:

2022-08-16