2017-2018 Evaluation of the Major International Event Security Cost Framework

PDF Version (922 KB)

Table of contents

• Executive Summary
• 1. Introduction
• 2. Profile
  
  • 2.1 Background
  • 2.2 Program Objectives
  • 2.3 Roles and Responsibilities
  • 2.4 Resources
  • 2.5 Logic Model
• 3. About The Evaluation
  • 3.1 Objective
  • 3.2 Scope
  • 3.3 Methodology
  • 3.4 Limitations
• 4. Findings
  • 4.1 Relevance
  • 4.2 Performance - Effectiveness of the Framework's Design and Implementation
  • 4.3 Performance - Efficiency and Economy of Major International Event Security Cost Management
• 5. Conclusions
  • 5.1 Relevance
  • 5.2 Performance - Effectiveness
  • 5.3 Performance - Efficiency and Economy
• 6. Recommendations
• 7. Management Response and Action Plan
• Annex A: 2016 North American Leaders Summit Financial Information
• Annex B: Logic Model
• Annex C: Documents Reviewed

Executive Summary

This report presents the results of the 2017-18 Evaluation of the Major International Event (MIE) Security Cost Framework Policy (hereafter referred to as the Framework). The Framework is a contribution program created in June 2001 to provide financial relief to host jurisdictions for incremental, extraordinary, reasonable and justifiable policy and security-related costs incurred in support of Royal Canadian Mounted Police (RCMP)-led security operations for MIEs. Two types of events are eligible under the Framework: events that are pre-designated and those that meet three specific eligibility criteria. The Framework was last evaluated in 2012-13.

What we examined

The evaluation assessed the relevance of the Framework; its conformance with the Treasury Board Policy on Transfer Payments; the clarity of partners' roles and responsibilities; the extent to which 2014 revisions to the program's policy and Terms and Conditions have met their intended objectives; and the extent to which the current performance measures are relevant for decision making. The evaluation covered the time period from 2013-14 to 2016-17.

What we found

Relevance

The evaluation found that there is a continuing need for the Framework as host jurisdictions continue to require federal financial assistance to support RCMP-led security plans. The Framework is aligned with the priorities, roles and responsibilities of both Public Safety Canada (PS) and the federal government.

Performance – Effectiveness of the Framework's Design and Implementation

The evaluation found that once activated, the Framework provides a clear and structured mechanism for the reimbursement of policing and security costs to host jurisdictions. However, applicability of the Framework is limited due to the criteria used to designate an event and assess cost eligibility. These gaps affect applicability of the Framework and its ability to address security cost requirements for current and future events.

The roles and responsibilities of key partners involved in the Framework's application were generally well-defined, understood and adhered to. However, there is a need to better communicate PS and RCMP roles with regards to activating the Framework.

Although collaboration among federal organizations improved during the 2016 North American Leaders Summit (NALS), tight planning timelines continued to adversely impact collaborative activities, which speaks to a need to ensure a shared understanding of all partners' roles and responsibilities, beyond PS and the host jurisdiction.

The evaluation found that the Peer Advisory Expert (PAE) and Independent Auditor added value and assisted PS in fulfilling its oversight and challenge function responsibilities. They were found to assist the administration of the Framework, and were deemed helpful for both the Department and the host jurisdiction. Both functions contributed to increased accountability to Canadians, transparency for the Framework and host jurisdiction, and a level of assurance for PS in terms of management responsibility.

Furthermore, the evaluation determined that the Framework had a performance measurement strategy that included appropriate indicators and targets; however, data collection, analysis and reporting were not conducted on a consistent basis. This may be due to a lack of dedicated resources managing the Framework between MIEs.

Performance – Efficiency and Economy of MIE Security Cost Management

The evaluation identified two other federal contribution programs which support extraordinary policing costs incurred by host jurisdictions for major international events: the Nation's Capital Extraordinary Policing Costs Program (NCEPCP) (managed by PS);^{Note 1} and the 1996 Framework for the Management and Funding of Prime Minister-led Summits of an International Nature (managed by Global Affairs Canada). While these programs were found to be complementary and fill gaps in the Framework's applicability, a centralized Government of Canada approach to hosting international events could further reduce inefficiencies.

Multiple factors affect MIE costs, including the lead time between the official announcement and the date of the event, as well as its location. Personnel, operations and maintenance costs were the main cost drivers for the 2016 NALS. PS's ability to reduce cost lies in the planning phase and through the application of its challenge function. Inefficiencies in procurement and asset management were found, particularly as it relates to processes for assets acquired from and returned to the RCMP asset inventory.

Recommendations

Given Canada's continued commitment to hosting international events and their associated security risks, the ADM of CSCCB should:

1. Review existing event designation criteria to ensure flexibility to meet and validate the overall objectives of the Framework.
2. Reassess the eligible cost criteria to ensure that they are sensitive to risk and strike an appropriate balance between control and flexibility in order to support jurisdictions in hosting Major International Events.
3. Ensure that performance information is collected and reported on to support timely and evidence-based decision making.

Management Response and Action Plan

Management accepts all recommendations and will implement an action plan.

1. Introduction

The report presents the results of the 2017-18 evaluation of the Major International Event (MIE) Security Cost Framework Policy (hereafter referred to as the Framework). The evaluation was conducted in accordance with the Treasury Board of Canada (TB) Policy on Results and Policy on Transfer Payments, and in compliance with section 42.1(1) of the Financial Administration Act.^{Note 2} The Framework was last evaluated in 2012-13.

2. Profile

2.1 Background

The Framework is a contribution program^{Note 3} created in June 2001 to provide financial relief to host jurisdictions for the implementation of incremental, extraordinary, reasonable and justifiable policing and/or security-related services^{Note 4} in support of Royal Canadian Mounted Police (RCMP)-led security operations for an MIE. MIEs require significant security measures and the assistance of multiple jurisdictions and agencies from all levels of government to safeguard foreign officials, citizens and Canadian communities hosting such events. Prior to 2000, provincial, territorial and municipal jurisdictions^{Note 5} (host jurisdictions) involved in federally-led MIEs covered incremental security costs from their own budgets. In 2000, the federal government recognized that these costs were no longer sustainable for host jurisdictions given the increased level of security required.

There are two types of MIE eligible under the Framework: events that are pre-designated and those designated after meeting three specific eligibility criteria. Pre-designated events are hosted by the Prime Minister (PM) or a federal Minister and involve several heads of state or heads of government. These events have a high level of threat and require security support from other jurisdictions. Examples of pre-designated events include Asia-Pacific Economic Cooperation (APEC) Summits, Commonwealth Heads of Government Meetings, G7/20 type Summits, North American Leaders Summit (NALS), La Francophonie Summits, Organization of American States (OAS), Summit of the Americas and meetings between the PM and the President of the United States.

Other events can be designated as an MIE under the Framework if they satisfy all three of the following conditions:

1. the event is held in Canada and hosted by the PM or a Minister;
2. the RCMP is the lead security agency responsible for the event; and
3. the threat assessment conducted by federal partners identifies a significant security risk.

Based on the recommendations stemming from the 2012-13 Evaluation, a new Framework Policy and revised Terms and Conditions (Ts&Cs) were approved and implemented in April 2014. A key change was the establishment of a Peer Advisory Expert (PAE) to strengthen the program's challenge function by advising PS on operational requirements for the resources requested by the host jurisdiction during event planning and procurement to ensure they are justifiable and reasonable.

2.2 Program Objectives

The objective of the Framework is to facilitate cooperation with host jurisdictions by reimbursing them for incremental, extraordinary, justifiable and reasonable policing and security-related costs incurred in support of RCMP-led security operations for an MIE.

2.3 Roles and Responsibilities

The table below presents an overview of the roles and responsibilities of the partners generally involved in the application of the Framework.

2.4 Resources

Since its inception in 2001, the Framework has been applied to 12 MIEs and has provided approximately $260M to assist host jurisdictions with eligible security costs incurred for designated MIEs held in Canada. For the 2016 NALS, $4.8 M in eligible costs was reimbursed to the host jurisdiction (Annex A). For the 2016 NALS program administrative costs were $215,857. This represents 4.5% of the contribution paid to the 2016 NALS and includes the salaries for PS staff and the contracts for the PAE and the Independent Auditor.    

2.5 Logic Model

The logic model found in Annex B illustrates the activities to be undertaken and the outputs needed to reach the Framework's ultimate objective of Canada's safe hosting of MIEs. The achievement of this objective is facilitated through the realization of the following intermediate outcomes:

• sound management of eligible incremental security costs incurred by designated MIEs;
• early federal engagement with jurisdictions to advance cost effective planning and procurement activities; and
• jurisdictions have the incremental security resources (i.e. assets, equipment, personnel) required to support RCMP-led security operations.

3. About The Evaluation

3.1 Objective

The main objective of this evaluation is to provide a neutral, evidence-based assessment of the design and implementation of the Framework. To this end, the evaluation examined:

• the relevance of the Framework;
• the effectiveness of the government-wide approach to the application of the Framework;
• conformance of the Framework with the TB Policy on Transfer Payments;
• the extent to which the 2014 Ts&Cs revisions have met their intended objectives; and
• the implementation and relevance of the current performance measures in support of evidence-based decision making.

3.2 Scope

The evaluation covered the period of 2013-14 to 2016-17. Although the North American Leader Summit (NALS) held in Ottawa on June 29, 2016, was the only MIE that qualified under the Framework during the time period under review, a number of other events have been held in Canada which did not meet the Framework's eligibility criteria for funding. Therefore, analysis of Framework administration and implementation focused primarily on the 2016 NALS. Four other events which were ineligible for Framework coverage^{Note 6} were also reviewed to analyze Framework applicability. For the purpose of this evaluation, the term 'MIE' is used to distinguish between events designated under the Framework and those not covered by this program.

3.3 Methodology

This evaluation was conducted in accordance with the TB Policy on Results and Directive on Results.

3.3.1 Evaluation Topics and Core Issues

3.3.2 Lines of Evidence

Data collection for the evaluation occurred from July to November 2017 and included the following lines of evidence:

• Document Review: included relevant reports, studies and findings, as well as proceedings from internal review workshops recently conducted by program management to assess Framework performance during the 2016 NALS. The full list of documents consulted can be found in Annex C.
• Interviews: 20 interviews were conducted with PS program staff and senior management, other federal partners,^{Note 7} and the host jurisdiction, Independent Auditor and PAE for the 2016 NALS. The interviews were used to assess the relevance and effectiveness of the design and implementation of the Framework, and to identify any potential best practices.
• Analysis of Performance and Financial Information: The evaluation reviewed the Framework's performance data and analyzed available financial information to assess the efficiency and economy of MIE cost management.

3.4 Limitations

The sample size of events available for analysis was small, as only one MIE occurred during the time period under review. Therefore, the evaluation was limited in its ability to fully assess the impact of the 2014 Ts&Cs revisions.

4. Findings

4.1 Relevance

There is a continued need for the Framework. With the increasing security costs of MIEs, host jurisdictions continue to require federal assistance to cover the costs of policing and other security measures to support the RCMP-led security plans. The Framework is aligned with the priorities and roles and responsibilities of PS and the federal government.

Following the large-scale mobilized protests of the 1999 World Trade Organization Summit in Seattle and the 2000 annual meetings of the International Monetary Fund and World Bank in Washington D.C., not to mention the rise in global terrorism in the wake of 9/11, security demands associated with hosting major international events have increased substantially. To address this issue, the Government of Canada developed the Framework policy to provide security–related financial assistance to jurisdictions hosting PM or Minister-led international meetings; it was approved in 2001. The threats necessitating this policy persist in today's global context as does Canada's commitment to host MIEs, such as the G7 Summit.

The Department of Public Safety and Emergency Preparedness Act,^{Note 8} 2005, stipulates that the Minister is responsible for exercising leadership at a national level relating to public safety and emergency preparedness. The December 2015 Speech from the Throne^{Note 9} reiterated the government's commitment to providing greater security for Canadians and to strengthen its relationship with allies, such as G7 partners. The 2018 G7 Summit will be hosted in Canada and is a pre-designated event under the Framework. By providing financial assistance to ensure sufficient security coverage for this, and other MIEs, the Framework is aligned with the department's mandate and supports both the federal government and PS priorities for a safe and resilient Canada.

4.2 Performance – Effectiveness of the Framework's Design and Implementation

4.2.1 Conformance with the TB Policy on Transfer Payments

The Framework provides a clear and structured mechanism for the reimbursement of policing and security costs to local jurisdictions and has sufficient challenge functions to provide oversight. However, applicability of the Framework is limited due to the criteria used to designate those events not already pre-established as eligible under the Framework. A lack of flexibility was also found in the determination of cost eligibility as outlined in the Ts&Cs. These limitations affect the Framework's ability to address security cost requirements for current and future events.

As a Grants and Contributions Program, the Framework is administered in accordance with the TB Policy and Directive on Transfer Payments. Section 3.6 of the policy states that the government is resolved to ensuring that transfer payment programs are designed, delivered and managed in a manner that is fair, accessible and effective for all involved. Section 3.7 requires that programs be managed in a manner that is sensitive to risks, strikes an appropriate balance between control and flexibility, and establishes the right combination of good management practices, streamlined administration and clear requirements for performance. To this end, the evaluation assessed the extent to which the Framework sufficiently addresses the security cost requirements of current and future MIEs. It also examined factors affecting the Framework's applicability with regards to event and cost eligibility, and assessed the effectiveness of the challenge functions in providing appropriate oversight.

Event Eligibility

Multiple respondents raised the following concerns regarding the criteria used to designate an event under the Framework:

1. PM or Minister-led: Interviewees indicated that in today's reality, all events require some form of security and a threat risk assessment should be conducted whether or not the events are PM or Minister-led. It was also suggested that the Framework should take into account the definition of internationally protected persons (IPPs)^{Note 10} as per the Canada Criminal Code, rather than heads of state. The RCMP is responsible for the security of IPPs, as well as the planning of security for MIEs when the Framework is activated. 
2. Significant security risk: Interviewees felt that there is a need to clarify the definition of "significant risk" and questioned its relevance as a criterion for event designation under the Framework. They stated that the risk assessment is an ongoing process and the outcome can change at any time before or during an event. This means that an event may be assessed as having no security threat at the planning phase, yet become subject to significant security measures due to changes to event location, agenda, or geo-political context. Further, interviewees indicated that all events – designated or otherwise – should be subject to a threat risk assessment whether or not the RCMP is responsible for event security.

During the evaluation period, four (4) events did not meet the eligibility criteria for Framework coverage. The table below presents these four events, the reasons why they failed to satisfy the criteria, and actions (if any) taken to address the financial burden related to increased security costs placed on the local jurisdictions.

Although these events were not eligible under the Framework, the 2017 UN Disaster Reduction and Recovery Regional Meeting and the 2017 Canada Day Royal Visit were funded under other federal programs. According to this data and interviewees, the Framework does not sufficiently address security costs due to its narrow criteria for eligibility. A more flexible framework may be needed to take into account various factors that can impact event security and to allow both small and large events to be eligible, even if they are not PM or Minister-led.

Cost Eligibility

Eligible costs are detailed in the Framework's Ts&Cs^{Note 11} and outlined in Contribution Agreements established with host jurisdictions.

In terms of personnel costs, clause 15.1 of the Ts&Cs relates to the regular salaries and benefits of full-time dedicated planning teams, and outlines the conditions required to reimburse these eligible costs. According to clause 15.1.3, the recipient must backfill the positions of employees assigned to the full time planning team for those costs to be reimbursed. Interviewees suggested this clause was a barrier to receiving compensation for personnel costs dedicated to the planning team for the 2016 NALS. An activity report prepared by the recipient following NALS 2016 indicated that the short timeframe for event planning, difficulty finding the appropriate level of expertise required, and pre-existing staffing demands within the recipient organization prevented the backfilling of positions for employees who worked on the planning phase of the 2016 NALS.

The report also stated that additional costs were incurred as a direct result of the 2016 NALS for non-claimable hours^{Note 12} of police officers assigned to the MIE. These hours were not covered under the Contribution Agreement and were therefore not reported in their reimbursable expenses.

An activity report prepared by the recipient following NALS 2016 noted that local jurisdictions and other service providers should be compensated for costs associated with providing support for an MIE, and that this should extend beyond security-related costs to include costs related to fire services, paramedics, and the dedicated planning team (even if the positions are not backfilled). Interviewees further supported this consideration and identified that not reimbursing these costs penalized the recipient and the negative impact would be greater for smaller cities than major cities. It should be noted that the activity report recommended that an administration burden rate of 10% of the total reimbursable claim amount be permitted in recognition of these types of costs incurred by the host jurisdiction.

Among other cost eligibility issues, the wording contained in the Framework's Ts&Cs focuses mainly on police services. Interviewees raised concerns about the necessity of fire and paramedic services to hire additional staff in order to meet their requirements for the 2016 NALS, and that these costs were not eligible for reimbursement. Expanding the language of clause 15.37 of the Contribution Agreement would allow these emergency services to contract additional resources to support MIEs as needed.

Challenge Functions

PS performs a challenge function by conducting a review of the reasonableness of event cost estimates and eligibility of expenses through the PAE and the Independent Auditor, respectively.

The PAE focuses on the reasonableness of resources required by the host jurisdiction to support RCMP-led security operations. This review, conducted prior to the actual MIE, helps PS validate requested resources and estimates for the security-related costs. The 2016 NALS was the first MIE for which the PAE function was utilized. The role was well-received by the host jurisdiction and regarded as a collaborative process that allowed for effective intervention and problem resolution. This function provided PS and the Independent Auditor with a common understanding of the security plan and any issues to be considered before validating the resources requested and performing the audit.

Following an MIE but prior to reimbursement of host jurisdiction costs, the Independent Auditor verifies actual costs of the security operations. The Auditor validates whether or not the host jurisdiction complied with the Contribution Agreement and ensures that the amounts claimed are eligible for reimbursement. The Framework's Guide for Recipient Audit was developed to assist host jurisdictions by outlining necessary requirements for the audit. The Guide provides information on audit objectives, audit scope, audit methodology, and management letters. For the 2016 NALS, the Guide was deemed helpful by the host jurisdiction.

The role of the Independent Auditor was found to be critical to ensure accountability and transparency in the process of reimbursing eligible security-related costs claimed by the host jurisdiction. It was reported that the role went smoothly during the 2016 NALS, due in large part to the early involvement of the Independent Auditor in the planning phase of the MIE.

Both the PAE and the Independent Auditor functions contribute to increased accountability to Canadians, transparency for the Framework and host jurisdiction, and a level of assurance for PS in terms of management responsibility for oversight.

4.2.2 Clarity of Partners' Roles and Responsibilities

The roles and responsibilities of the key partners involved in the Framework's application were generally well defined, understood and adhered to. However, there is a need to better communicate PS and RCMP roles with regard to event eligibility.

Overall, the roles and responsibilities of the key partners involved in the administration and application of the Framework (i.e., PS, the RCMP and the host jurisdiction) were generally well defined in official documents supporting the administration of the policy such as the Ts&Cs and the Contribution Agreement for the 2016 NALS. However, interviewees identified that clearer language needs to be employed in certain sections of the Ts&Cs to improve understanding of the Framework's application and administration. For example, paragraph 7.1 describes supporting emergency management services without clearly identifying the specific costs that are eligible for reimbursement.

The majority of interviewees stated that the relationship between PS and the RCMP is positive, and that the roles and responsibilities of these two key partners have become clearer and better adhered to since the 2014 revisions to the Framework. However, they also shared that further clarification of roles could allow for greater alignment of activities and help avoid applicability issues such as the ineligibility of the 2016 Global Fund Replenishment Conference. Interviewees stressed the need for PS to be involved in all discussions regarding the Framework's applicability and to be solely responsible for activating the Framework, according to the Ts&Cs. It was also suggested that while the Ts&Cs do describe the roles and responsibilities of PS and the host jurisdiction, a consolidated document clarifying all partner roles would be beneficial.

The PAE and the Independent Auditor's roles and responsibilities were clearly defined in the Ts&Cs and Contribution Agreement for the 2016 NALS. The host jurisdiction indicated that having these roles explained at the outset contributed to building trust and improving relationships. This collaborative approach between the PAE and the Independent Auditor was identified as a key factor to their successful interactions with the host jurisdiction during the 2016 NALS.

There were mixed views among interviewees with regards to the overall level of collaboration among federal partners when planning for and managing MIE. Some of the interviewees felt that collaboration between federal partners was excellent, while others indicated that there are still areas for improvement. Interviewees noted that although the Framework has improved and formalized collaboration among federal partners, tight timelines and fast turnarounds during planning increases pressures and adversely impacts collaborative activities. This speaks to the need for roles and responsibilities to be clearly communicated, understood and adhered to by all partners.

4.2.3 Relevance of the Framework's Performance Measures

The Framework is included in the Performance Information Profile for the Law Enforcement and Policing program, in accordance with the requirements of the TB Policy on Results and Directive on Results. However, there is a need for PS to collect and report on the relevant data on an ongoing basis.

The program is reflected in the Performance Information Profile (PIP) for the Law Enforcement and Policing program. This replaces the performance measurement strategy that was created following the 2014 revisions of the Framework policy. To assess its relevance, the evaluation looked at conformance with the Guidelines on Performance Measurement Strategy^{Note 13} under the TB Policy on Transfer Payments^{Note 14}(2008) and the updated requirements under the TB Policy on Results and Directive on Results (2016), and the implementation of the data collection.

The evaluation found that the program's 2014 performance measurement strategy contained useful indicators and targets related to the program outcomes. However, information on data collection methods and timelines were not included. The four indicators included in the PIP address both program effectiveness and efficiency, and respond to the policy requirements to include data collection information and methodology. The timing for all of the data collection is "after a major international event". No data collection tools or forms have been developed.

Over the 18 years of Framework implementation, performance data has not be collected or reported on consistently. This may be a result of the program design as there is no permanent ongoing staff allocated to the program. Resources are assigned once an MIE has been identified.

4.3 Performance – Efficiency and Economy of Major International Event Security Cost Management

4.3.1 Duplication and Synergy with Other Federal Programs or Initiatives

There are other federal programs that address policing and security costs for events, which are complementary and fill gaps in the Framework's applicability. There is a desire for a centralized Government of Canada approach for events.

The evaluation looked at potential duplication and/or complementarity between the Framework and other federal contribution programs supporting extraordinary policing costs incurred by host jurisdictions. In addition to the Framework, the evaluation identified the NCEPCP, which is a contribution program under the authority of the Minister of PS to reimburse the City of Ottawa for eligible, reasonable and justifiable policing costs incurred in relation to policing duties specific to the Nation's Capital. The program was created following the events at the War Memorial and Parliament Hill on October 22, 2014, and is not necessarily related to international events. A maximum contribution under the Ts&Cs was set at $2 million per fiscal year starting in 2015-16, with an exception in fiscal year 2017-18 when the maximum contribution was set at $3.2 million to take into account costs associated with the 150th Canada Day celebrations. The incremental policing and security-related costs associated with the 2017 Canada Day Royal Visit to the City of Ottawa are an example of eligible costs that can be covered by the NCEPCP. This program fills gaps for events in the Nation's Capital deemed ineligible for Framework coverage.

The evaluation also identified the 1996 Framework for the Management and Funding of Prime Minister-led Summits of an International Nature as another specific policy authority regarding costs associated with PM-led summits or conferences. This fund is under the authority of the Minister of Global Affairs Canada. Interviewees indicated that because event designation criteria is unclear, GAC's framework is often used as a tool to cover security costs for major events. This may indicate a need for the federal government to implement a more flexible program with clearer eligibility criteria to address the security costs for major events held in Canada.

While these two programs are complementary to the Framework, the majority of interviewees indicated that a centralized Government of Canada approach to events would be ideal.^{Note 15} It was suggested that given the current climate, including an anticipated increase in the frequency of events to be held in Canada, a centralized federal approach could be applied to all types of events including those that are not PM or Minister-led. In that context, the Framework could be one component of the overall approach to large-scale events and only activated when needed. Possible benefits to a centralized approach could include more effective planning; knowledge management through stabilization of staff responsible for MIEs; quicker activation of the Framework; and a permanent forum for issues-management and lessons-learned.

4.3.2 Main Cost Drivers Associated with Major International Events

Multiple factors affect MIE costs, including the lead time between the official announcement and the event date, location and required security measures. For the 2016 NALS, personnel and operating and maintenance costs were the main costs drivers.

A review of the 2016 NALS financial data revealed there were five main cost drivers which accounted for 64.5% of the total amount paid to the host jurisdiction. A detailed overview of the 2016 NALS financial information can be found in Annex A.

Interviews and document review revealed that there are different factors affecting the main cost drivers and, consequently, the total costs of an MIE. These factors are largely related to the timing of event designation, the selection of a location, and the relative ease of securing the chosen location. Alterations to MIE programming impacts event costs, as additional resources are required to accommodate changes in location or scheduling.

While most of these factors are out of PS control, interviewees suggested areas for improvement and some best practices that could help mitigate the impact of these factors. Suggestions include establishing a list of pre-selected jurisdictions that have the security and logistical capacity to host MIE, and sharing the incremental security costs with the host jurisdiction based on the potential economic impacts of the MIE on the local community and businesses.

4.3.3 Measures Taken to Improve Efficiency and Economy

The PAE and Independent Auditor's functions contribute to the efficient administration of the Framework by PS. However, the procurement process for filling these roles was found to be inflexible and created challenges for aligning them within the Framework's timelines. Inefficiencies in procurement and asset management were also found.

Public Safety Challenge Functions

Both the PAE and Independent Audit roles complemented each other and assisted the program in improving the economy of the Framework. While the PAE's focus was not on the financial aspect, as they only assessed the reasonableness of resources required, this function does have an impact on the overall cost of the event as it complements the auditor's specific focus on resource costs. For the 2016 NALS, the Independent Auditor contributed to a reduction of 2.55% of the total claims from the host jurisdiction as presented in Table 5. The cost reductions were mainly related to personnel costs during the planning phase.

The recipient noted that the PAE's security expertise was instrumental for them during the planning stage, especially when questions concerning resource eligibility arose. They also viewed the earlier involvement of the Independent Auditor as a positive improvement that facilitated a timelier reimbursement. A review of historical data showed a reduction over the past three MIEs in the time required between the end of an event and the reimbursement of costs. This reduction was particularly notable for the 2016 NALS. However, it is too early to confirm the extent to which the introduction of the PAE function indirectly contributed to this trend. Other factors, such as timeliness of documentation submitted by the host jurisdiction, and the location and size of event, also impact the timeliness of reimbursement. Therefore, while the PAE and Independent Auditor may help facilitate the reimbursement process, recent increased efficiencies cannot be attributed solely to these functions.^{Note 16}

A contracting model was used to acquire the services of both the PAE and the Independent Auditor, as MIEs are only held from time to time and such expertise is not required by PS on a continuous basis. There was sufficient evidence to confirm that both functions collaborate and support each other well, as each brings a different skill set to the due diligence process.

However, given the need to staff these roles quickly in response to short planning timelines, there were challenges with this model. Contracting processes can be lengthy and inflexible. Furthermore, although these functions have different milestones, they were perceived by respondents to be a continuous function because of their involvement from the beginning of the 2016 NALS. It was mentioned that a more flexible contract incorporating better alignment of the functions throughout the full duration of the event could be of benefit to PS and the host jurisdiction. Document review and interviewees suggested this realignment could allow the PAE to advise the Independent Auditor during the audit process and further validate the reasonability of actual amounts claimed by the host jurisdictions.

Event Procurement and Asset Management

Procurement refers to all contracting activities conducted in support of the MIE and includes the acquisition of security assets such as radios and fencing. As outlined in the Ts&Cs, there are three possibilities for procuring reimbursable security assets: leasing or renting assets (host jurisdictions are reimbursed at 100% of the cost); purchasing and retaining assets (reimbursed at 50% of the cost); or purchasing and returning assets to the Government of Canada (reimbursed at 100% of the cost). Asset management refers to the process in place following an MIE to manage any security assets purchased for the event. Some interviewees felt that the current approach to procurement and asset management could be more efficient and effective.

According to the PAE's review of the 2016 NALS incremental security costs, the RCMP and host jurisdiction decided that effort would be made to procure required equipment through a cooperative approach. Joint options analysis was conducted for major resource acquisitions over $10,000. The joint procurement processes and options analysis consisted of the following risk-based criteria: planning considerations, risk factors, site requirements, strategic intervention requirements, and threat assessment.^{Note 17} Interviewees mentioned that one of the challenges the team faced was the limited availability of security fencing in the North American market due to the fact that the 2016 NALS was not the only major event occurring in North America at the time. Overall, the most significant investment in assets was the purchase of radios to be retained by the recipient after the event. Furthermore, the requirements for procurement of specific assets were done on a short turnaround with no possibility of last minute changes. This may have led to potential inefficiencies.

There is a lack of processes in place to account for host jurisdictions procuring assets from the RCMP asset inventory that may have been already purchased for previous MIEs. Further, the Ts&Cs do not clarify the RCMP's role in asset management. Hence, there is a need to address these gaps and reduce possible inefficiencies in the procurement and asset management processes.

Public Safety Ability to Reduce Event Costs

With regards to PS's ability to reduce security costs, one of the most pressing issues to consider is timing. With short timeframes for event planning, PS estimates are often high compared to actual expenses^{Note 18} due to the need for early costing as part of the funding requests. As the planning progresses, changes to the event security requirements impact the overall cost. Interviewees stated that PS's main ability to reduce costs lies in the planning phase through the application of the challenge function, particularly the PAE review to ensure reasonableness of resources requested from the host jurisdiction. Other opportunities for possible cost reduction mentioned by interviewees include:

• asset management through the use of existing government resources and the creation of a national inventory;
• streamlined interoperability of communications equipment; and
• pre-identified and assessed event locations to create efficiencies as bigger cities have more resources than smaller ones.

5. CONCLUSIONS

5.1 Relevance

Over the last twenty years the global security environment has undergone dramatic changes. This has resulted in a substantial increase in the security costs associated with hosting MIEs. The threats necessitating the Framework persist in today's global context as does Canada's commitment to host MIEs, such as the 2018 G7 Summit. Therefore, the policy remains relevant and necessary.

The Framework is aligned with federal and PS priorities related to building a safe and resilient Canada, and Canada's ability to safely host MIEs as it seeks to strengthen its global relationships. The Framework is also aligned with federal and PS roles and responsibilities with regards to providing financial assistance to ensure sufficient security coverage for MIEs, as outlined in the Department of Public Safety and Emergency Management Act (2005).

5.2 Performance-Effectiveness

Key successes of the program include the level of security provided at past events, the support provided to local jurisdictions, PS's challenge functions and having predesignated events which expedites application of the Framework.

The evaluation found that once activated, the Framework provides a clear and structured mechanism for the reimbursement of policy and security costs to host jurisdictions, and that it is aligned with the TB Policy on Transfer Payments. PS plays its administrative and challenge function roles by providing relevant information to the host jurisdiction, and by conducting a review of the reasonableness of event cost estimates and eligibility of expenses through the PAE and the Independent Auditor, respectively. Both roles were clearly defined and adhered to during the 2016 NALS, and brought a level of expertise and confidence to PS that is not available in-house. For future MIEs that PAE should be engaged for the entirety of the event, from planning to reimbursement.

Overall, the roles and responsibilities of key partners involved in the administration and application of the Framework were well defined in the Framework's official documentation. Increased collaboration was noted among federal partners during the 2016 NALS; however, tight timelines during planning negatively impacts collaborative activities. This speaks to the necessity of ensuring a common understanding for all partners' roles and responsibilities.

Interviewees noted confusion around when the Framework is triggered and the criteria used to designate event eligibility. These gaps affect applicability of the Framework and its ability to address security cost requirements for current and future events. Furthermore, there is a need to better communicate PS and RCMP roles with regard to Framework application. A more flexible Framework may be required to account for rising security costs and the various factors which impact the security of an event. As well, eligibility criteria including who has the authorization to seek event designation, and eligible security-related costs, should be clearly defined and sufficiently flexible to ensure reasonable costs incurred by the host jurisdiction are reimbursed.

Although the Framework has performance measures in place, data collection, analysis and reporting was not conducted on a consistent basis. Additional oversight between MIEs may be required.

5.3 Performance-Efficiency and Economy

In addition to the Framework, the evaluation identified two federal contribution programs supporting extraordinary policing costs incurred by host jurisdictions: the NCEPCP (managed by PS), and the 1996 Framework for the Management and Funding of Prime Minister-led Summits of an International Nature (managed by GAC). While these programs were found to be complementary and fill gaps in the Framework's applicability, a centralized Government of Canada approach with more flexible eligibility criteria could further reduce inefficiencies.

Factors affecting the cost of MIEs are largely related to the event's designation process, timing, and location. While most of these factors are out of PS control, PS's main ability to reduce costs lies in the planning phase and through the application of its challenge functions. PS has responded to the 2012-13 evaluation recommendations and strengthened its challenge function through the implementation of the PAE role. The 2016 NALS was the first event for which this function was utilized, and both the PAE and the Independent Audit roles contributed to reducing unnecessary costs, increased accountability to Canadians, transparency for the Framework and host jurisdiction, and a level of assurance for PS in terms of management responsibility for oversight.

6. Recommendations

Given Canada's continued commitment to hosting international events and their associated security risks, the Assistant Deputy Minister of the Community Safety and Countering Crime Branch should:

1. Review existing event designation criteria to ensure flexibility to meet and validate the overall objectives of the Framework.
2. Reassess the eligible cost criteria to ensure that they are sensitive to risk and strike an appropriate balance between control and flexibility in order to support jurisdictions in hosting Major International Events.
3. Ensure that performance information is collected and reported on to support timely and evidence-based decision making.

7. Management Response and Action Plan

Management accepts all recommendations and will implement an action plan.

Annex A: 2016 North American Leaders Summit Financial Information

Annex B: Logic Model

Major International Events Security Cost Framework Program Logic Model

Annex C: Documents Reviewed

B. Walton & D. Philpott (2011). Special Event Security Planning & Management: Security Best Practices for All Levels of Government, Education & Corporate Events. Government Training Inc.: Longboat Key, Florida.

Bureau of Justice Assistance, U.S. Department of Justice (2013). Managing Large-Scale Security Events: A Planning Primer for Local Law Enforcement Agencies.

City of Ottawa (2017). North American Leaders' Summit Activity Report.

City of Ottawa (2017). North American Leaders' Summit Summary of Costs.

J.D. Neily Consulting Ltd. (2016). Independent Peer Advisory Review of Incremental Security Costs for the North American Leaders Summit (NALS) 2016.

Office of the Auditor General of Canada. (2011). 2011 Spring Report of the Auditor General of Canada. Chapter 1: Expenditures for the 2010 G8 and G20 Summits.

Ottawa Police Service (2017). North American Leaders' Summit Activity Report to the Federal Government.

Public Safety Canada (2012). Security Cost Framework Policy Application for the 2010 G8 and G20 Summits. Lessons Learned Report.

Public Safety Canada (2012). 2012-2013 Evaluation of the Security Cost Framework Policy.

Public Safety Canada (2014). Performance Measurement Strategy for the 2014 Major International Event Security Cost Framework.

Public Safety Canada (2016). Contribution Agreement. North American Leaders' Summit 2016.

Public Safety Canada (2016). Major International Event Security Cost Framework Review – Workshop 1: Whole of Government Approach to Major International Events. Background Document.

Public Safety Canada (2016). Major International Event Security Cost Framework Review – Workshop 1: Whole of Government Approach to Major International Events. Meeting #2 Summary, December 13, 2016.

Public Safety Canada (2016). Terms and Conditions of the Major International Event Security Cost Framework.

Public Safety Canada (2017). Application of the Security Cost Framework Policy Summary.

Public Safety Canada (2017). Guide for Recipient Audit - Major International Event Security Cost Framework.

Public Safety Canada (2017). Major International Event Security Cost Framework – Background.

Public Safety Canada (2017). Major International Event Security Cost Framework Review – Documentation Issues.

Public Safety Canada (2017). Major International Event Security Cost Framework Review – Workshop 1: Whole of Government Approach to Major International Events. Meeting #3 Summary, May 24, 2017.

Public Safety Canada (2017). Major International Event Security Cost Framework Review – Workshop 2: RCMP Synchronization. Meeting Summary, June 13, 2017.

Public Safety Canada (2017). Major International Event Security Cost Framework Review – Workshop 3&4: Terms and Conditions/Contributions Agreement. Background Document.

Public Safety Canada (2017). Major International Event Security Cost Framework Review – Workshops 3&4: Terms and Conditions/Contribution Agreement. Meeting Summary, May 26, 2017.

Public Safety Canada (2017). Major International Event Security Cost Framework Review – Workshops 3&4: Terms and Conditions/Contribution Agreement. Meeting Summary, June 19, 2017.

Public Safety Canada (2017). Major International Event Security Cost Framework Review – Workshop 6: Peer Advisory Review and Independent Audit Functions. Background Document.

Public Safety Canada (2017). Major International Event Security Cost Framework Review – Workshop 6: Peer Advisory Review and Independent Audit Functions. Meeting Summary, May 2, 2017.

Public Safety Canada (2017). Major International Event Security Cost Framework Review – Workshop 6: Peer Advisory Review and Independent Audit Functions. Meeting Summary, May 4, 2017.

Public Safety Canada (2017). Performance Data – Major International Event Security Cost Framework.

Public Safety Canada (2017). Terms and Conditions of the Nation's Capital Extraordinary Policing Costs Program.

Spearhead Management Canada Ltd. (2017). Contribution Audit Report. The North American Leaders' Summit 2016.

Spearhead Management Canada Ltd. (2017). Report on Lessons Learned. The North American Leaders' Summit 2016.

The Treasury Board (1996). Framework for the Management and Funding of Prime Minister-Led Summits of an International Nature.

Date modified:

2018-10-01