Evaluation of the Security Cost Framework Policy (2012-2013)

Evaluation of the Security Cost Framework Policy (2012-2013) PDF Version (550 KB)

Table of contents

• Executive Summary
• 1. Introduction
• 2. Profile
  • 2.1 Background
  • 2.2 Roles and Responsibilities
  • 2.3 Resources
  • 2.4 Logic Model
• 3. About The Evaluation
  • 3.1 Objective
  • 3.2 Scope
  • 3.3 Methodology
  • 3.4 Limitations
  • 3.5 Protocols
• 4. Findings
  • 4.1 Relevance
  • 4.2 Performance-Effectiveness
  • 4.3 Performance-Efficiency and Economy
• 5. Conclusions
  • 5.1 Relevance
  • 5.2 Performance
• 6. Recommendations
• 7. Management Response and Action Plan
• Annex A: Documents Reviewed
• Annex B: Financial Analysis

Executive Summary

Evaluation supports accountability to Parliament and Canadians by helping the Government of Canada to credibly report on the results achieved with resources invested in programs. Evaluation supports deputy heads in managing for results by informing them about whether their programs are producing the outcomes that they were designed to achieve, at an affordable cost; and supports policy and program improvements by helping to identify lessons learned and best practices.

What we examined

The Evaluation of the Security Cost Framework Policy (referred to herein as the Policy) was conducted in accordance with the funding approval requirement that an evaluation be completed prior to the renewal of terms and conditions by September 2013.

The Policy was created in 2001 to provide financial relief to host jurisdictions for the implementation of incremental, extraordinary, reasonable, and justifiable security-related measures in support of Prime Minister or Minister-led international meetings held in Canada. Since its inception, the Policy has been applied to 11 major international events, and the Government of Canada has paid out approximately $250 million to provinces and municipalities involved in providing security for designated events^{Note 1}.

The scope of this evaluation covers the time period beginning on September 4, 2008, when a revised version of the Policy was implemented. The 2008 Policy allowed for the establishment of contribution agreements between the federal government and a province/territory or municipality in relation to the provision of security at a designated international meeting, held in Canada. Four designated events were included in the scope of the evaluation as follows: the 2008 Sommet de la Francophonie in Québec City; the 2009 visit of U.S. President Obama in Ottawa; the 2010 G8 Summit in Huntsville; and, the 2010 G20 Summit in Toronto.

Why it is important

Security costs related to hosting major international events are rising due to the changing global security environment over the past decade. Host Canadian jurisdictions struggle to absorb the incremental and extraordinary costs related to these events. The Government of Canada needs to have an efficient process in place to determine what costs are eligible; to negotiate cost estimates; and, to provide timely reimbursements to provinces/territories and municipalities. The Policy contributes to the optimization of policing resources and cost control of designated events by engaging provincial/territorial and municipal security partners and obtaining their cooperation.  

What we found

The need for the Policy remains. Incidents over the last decade have caused a rise in security costs and have increased the complexity of security for Prime Minister or Minister-led events. Since Canada continues to host international events, host jurisdictions cannot sustain the associated incremental and extraordinary policing and security-related costs from their existing budgets.

The Policy is aligned with federal priorities as the government continues to place a strong focus on safety and security and on the importance of hosting major international events. The Policy is also aligned with departmental objectives related to its leadership role in the Canadian law enforcement community and its role of addressing evolving threats to safety and security.

The mandate for the federal government to provide security for major international events is clearly established in legislation as is the role of Public Safety Canada related to administering the Policy through contribution agreements to provide financial assistance to provincial/territorial and municipal security partners in host jurisdictions. The program ensures that linkages are maintained with other federal initiatives related to the organization of major international events.

In terms of performance, although it requires some improvements, the 2008 Policy represents an evolution of this instrument for the federal government. The Public Safety Canada team has provided support and leadership in implementing the Policy in general. The team is described as professional and knowledgeable, maintaining excellent working relationships with federal, provincial and municipal security partners. Public Safety Canada staff have revised the Policyover time to reflect the changing security environment. Notable are changes that came into effect in 2008, when the program was converted into a contribution program and monetary caps were put in place. This has enhanced the comprehensive management of the Policy. In fact, between 2000 and 2008, prior to the Policy becoming a contribution program, the total amount claimed was about 20% over the total amount reimbursed; since 2008, this percentage has dropped to 3%. Gaps in support from Public Safety Canada include the need for clarity surrounding the eligibility of costs for reimbursement; for better communication on the audit process; and for publication of general documentation related to the Policy. In terms of monetary support, under the 2008 Policy, the federal government provided $160.7 million in reimbursements to provincial and municipal security partners for the four events included in the evaluation.

The 2008 Policy has, to some extent, enabled early engagement and cooperative participation of provincial and municipal security partners. In general, partners are more eager to engage and communication is enhanced, so expectations are better managed. However, timing of event designation and funding approval remain important contextual factors that affect the achievement of full engagement. In general, if designation is obtained within an appropriate timeframe and funding approval is in place, Public Safety Canada can fully engage and negotiate early in the process. If designation and subsequent funding approval come later in the process, Public Safety Canada cannot enter into cost negotiations until a later stage, when security partners are already in the planning phase.

The 2008 Policy has contributed to sound management of security resources, but overall event costs are highly dependent on the event site. Cost control was evident in negotiations for contribution agreements, maintenance of negotiated caps, and recipient audits. The Public Safety Canada team that administers the Policy has provided diligence and attention to detail when challenging cost estimates, but access to specialized expertise in specific areas may enhance this function. Enhanced partner engagement, procurement processes and increased expectations for provincial and municipal security partners to reuse assets have optimized resources to some extent. Outstanding issues with respect to cost management include the fact that costs are, at the outset, over-estimated by partners in an effort to manage their risks, as partners are working from planning assumptions that change and evolve. Cost estimates for some of the events could be improved since money was set aside in the fiscal framework that was later not fully expended on the event. This was evident for the G8/G20 Summits in particular, where the total reimbursement was 57% of the cap amount because the initial estimates were high. In terms of the Sommet de la Francophonie and the Obama visit, when it came time to set aside money through internal reallocation, at the end of March 2009, Public Safety Canada had more information on the actual costs as both events were completed; thus, performance against the cap was reasonable. Issues also remain relating to procurement processes and management of surplus assets.

The 2008 Policy is being administered efficiently with an average program administration ratio of 2.7% that compares favourably to other Public Safety Canada contribution programs. Timeliness of the process related to audits and reimbursements was a noted issue.

Recommendations

The Evaluation Directorate recommends that the Assistant Deputy Minister, Law Enforcement and Policing Branch, implement the following actions:

1. Further improve partner engagement through the following communication activities:
   1. Prepare and publish information, on the departmental website, regarding the Security Cost Framework Policy in general and eligibility of expenses in particular.
   2. Provide early and clear communication regarding requirements of the audit process so that provincial/territorial and municipal partners are better prepared for audits, supporting improved efficiency of the audit process.
2. Further improve event security cost management by:
   1. Developing options and seeking approval for changes to the designation and funding processes that will allow Public Safety Canada to enter into negotiations with provincial/territorial and municipal security partners at the earliest possible stage.
   2. Developing options for mechanisms that will allow for closer alignment between funding approval amounts and final reimbursements. Options should aim to strengthen the challenge function for provincial/territorial and municipal partner cost estimates, and should include, where appropriate, the assistance of specialized expertise to support the challenge function.

Management Response and Action Plan

1. Introduction

This is the Public Safety Canada (PS) 2012-2013 Evaluation of the Security Cost Framework Policy (hereafter referred to as the Policy) and administration thereof. This evaluation provides Canadians, parliamentarians, Ministers, central agencies, and the Deputy Minister of Public Safety with an evidence-based, neutral assessment of the relevance and performance (effectiveness, efficiency and economy) of this federal government policy and its administration.

The evaluation of the Policy and its administration was conducted in accordance with the funding approval requirement that an evaluation be completed prior to the renewal of terms and conditions by September 2013.

2. Profile

2.1 Background

In the past, consistent with their constitutional responsibility for the administration of justice, provinces and municipalities involved in hosting major international events led by the federal government provided security support to the Royal Canadian Mounted Police (RCMP) from within their budgets. By the early 2000s, there was recognition by the federal government that this was no longer sustainable by provinces and municipalities given the increased level of security required for such events.^{Note 2}

The Policy was created in 2001 to provide financial relief to host jurisdictions for the implementation of incremental, extraordinary, reasonable and justifiable security-related measures in support of Prime Minister or Minister-led international events designated^{Note 3} as eligible (hereafter referred to as designated events).

Since its inception, the Policy has been applied to 11 major international events, and the Government of Canada has paid out approximately $250 million to help provinces and municipalities with policing and security-related costs incurred as a result of hosting major events in Canada.

Over time, the Policy has undergone a number of revisions to reflect the changing security environment and improve its administration. In 2003, an accelerated process was put in place for designation by the Prime Minister and financial assistance coverage for other jurisdictions affected by protest activities related to the designated event.^{Note 4} Since 2008, the Policy has been administered as a transfer payment program that allows for the establishment of contribution agreements between the federal government and the affected province/territory and/or municipality in relation to the provision of security at a designated^{Note 5} international meeting. The overall objective of the Policy is to obtain the active cooperation of provincial/territorial and municipal security partners through the reimbursement of incremental, extraordinary, reasonable, and justifiable security-related costs related to hosting designated events.^{Note 6}

The 2008 Policy revisions were intended to increase federal control over costs, clarify rules for security partners, and adopt a comprehensive management regime. This was to be accomplished through such actions as engaging earlier with security partners; using contribution agreements; and establishing monetary caps on the contribution agreements.

2.2 Roles and Responsibilities

Organizing security for Prime Minister or Minister-led international meetings requires the involvement of multiple security partners. The roles of the key partners are described below.

Royal Canadian Mounted Police: The RCMP is the lead agency responsible for policing and security at major international events held in Canada. It oversees event security planning and operations and the coordination of operational security requirements with provincial/territorial and municipal security partners through its Integrated Security Unit.

Provincial/Territorial and Municipal Security Partners: Provincial/territorial and municipal police forces and security-related agencies are accountable for the administration of justice within their respective jurisdictions which includes policing.

Public Safety Canada: PS is responsible for negotiating incremental and extraordinary policing and security-related costs with security partners in the jurisdictions where designated events take place. The PS-Strategic Policing Policy and Events Division within the Law Enforcement and Policing Branch is responsible for negotiating and managing the contribution agreements, and for the financial performance of the contribution program and procedures.

Department of Foreign Affairs and International Trade: Foreign Affairs and International Trade Canada is responsible for the organization, infrastructure, and logistics of all designated events as well as providing policy advice through the creation of a Summit Management Office for each event.

Privy Council Office: For the 2010 G8/G20 Summits, the Prime Minister created the Office of the Coordinator which was responsible for coordinating security planning, budgets, funding requests, and exercises for the summits.

2.3 Resources

Table 1 illustrates estimated Vote 1 expenditures for PS staff time required to administer the 2008 Policy. These amounts are estimated since, with the exception of the G8/G20, budgets/expenditures were not separately allocated under an individual cost centre. Table 1 shows an average of yearly expenditures estimated by PS staff using percentages of staff time.

* There is no explicit budget for Vote 1 activities. The budget is derived from approved funding and expenditures incurred in the program area.                                                                                  

Table 2 presents Vote 5 approved funding allocations for events. Funding approval is sought for each individual event designated under the Policy.

* Note: The approved funding is specifically related to G8/G20 Summits. The internal reallocation of $7.7M was not reallocated from a specific program, but from the overall Grants and Contribution lapse in transfer payments that was experienced in FY 2008-2009.

2.4 Logic Model

The logic model is a visual representation that links what the Policy is funded to do (activities) with what it produces (outputs) and what it intends to achieve (outcomes). It also provides the basis for developing the evaluation matrix, which gave the evaluation team a roadmap for conducting this evaluation.

Figure 1 – Logic Model of the Security Cost Framework Policy

The following are Policy activities: develop program guidelines (e.g. eligibility, reimbursement criteria); negotiate agreements (e.g. engage security partners, review and challenge cost estimates, attend pre-planning meetings); manage agreements (e.g. maintain ongoing engagement with security partners, conduct ongoing planning, provide interpretations, address issues and make amendments); support the audit process; and coordinate the issuance of payments.

The Policy activities lead to the following Policy outputs: published terms and conditions of the Policy for security partners that govern reimbursement claims; contribution agreements; audit reports; and reimbursement of eligible expenses (financial relief).

The Policy outputs lead to the following immediate outcome: security-related support and leadership for Prime Minister or Minister-led events designated as eligible under the Policy

The immediate outcome leads to the following intermediate outcome: early engagement and cooperative participation of provincial/territorial/municipal security partners in planning for Prime Minister or Minister-led events designated as eligible under the Policy.

The intermediate outcome leads to the following ultimate outcome: early engagement and cooperative participation of provincial/territorial/municipal security partners in planning for Prime Minister or Minister-led events designated as eligible under the Policy.

The ultimate outcome leads to the PS strategic outcome: a safe and Resilient Canada

3. About The Evaluation

3.1 Objective

This evaluation supports:

• Accountability to Parliament and Canadians by helping the Government to credibly report on the results achieved with resources invested in the Policy and its administration;
• The Deputy Minister of Public Safety in managing for results by providing information about whether the Policy is producing the outcomes that it was designed to produce, at an affordable cost; and
• Policy and program improvements.

3.2 Scope

The evaluation assessed the relevance and performance of the 2008 Policy and its administration by PS for the time period beginning on September 4, 2008. The scope of this evaluation included designated events that took place after the 2008 revisions to the Policy.

These events are as follows:

• 2008 Sommet de la Francophonie in Québec City;
• 2009 Visit of U.S. President Obama in Ottawa;
• 2010 G8 Summit in Huntsville; and,
• 2010 G20 Summit in Toronto.

3.3 Methodology

This evaluation was conducted in accordance with the Treasury Board of Canada Policy on Evaluation, the Standard on Evaluation for the Government of Canada and the PS Evaluation Policy. Evaluators took into account the following factors in order to determine the evaluation effort, including the approach, scope, design, and methods, required for this evaluation:

• risks;
• quality of past evaluations;
• soundness of program theory;
• longevity of the program; and,
• contextual stability.

Specifically, the evaluation methodology and associated level of effort were calibrated taking into consideration the following issues:

• Concurrent with evaluation activities, the Strategic Policing Policy and Events Division was conducting consultations with provincial and municipal security partners as part of the renewal of the Policy. Staff of the Evaluation Directorate participated in the consultations. Thus, the evaluation used consultation data and limited the number of interviews to include only federal partners.
• The evaluation also benefited from information contained in recent reports prepared by the program following the 2010 G8/G20 Summits and studies by federal organizations such as the Office of the Auditor General and the Office of the Parliamentary Budget Officer.

3.3.1 Evaluation Core Issues and Questions

As required by the Directive on the Evaluation Function, the following issue areas and evaluation questions were addressed in the evaluation:

Relevance

• 1. What was the original need for the Policy and how has the Policy evolved? Does the need persist?
• 2. To what extent is the Policy aligned with government-wide priorities and supportive of departmental strategic outcomes?
• 3. What is the nature of the federal government's mandate to deliver the Policy?

Performance—Effectiveness  

• 4a) To what extent are key policy instruments in place to support the achievement of outcomes?
• 4b) To what extent has security-related support and leadership been provided by PS to provincial/territorial and municipal security partners for Prime Minister or Minister-led events designated as eligible under the 2008 Policy?
• 4c) To what extent has the 2008 Policy enabled early engagement and cooperative participation of security partners in planning for Prime Minister or Minister-led events designated as eligible under the Policy?
• 4d) To what extent has the 2008 Policy contributed to sound management of security and safety resources for Prime Minister or Minister-led events designated as eligible under the Policy?

Performance—Efficiency and Economy

• 5. Has the efficiency of the administration of the Policy improved over time?

3.3.2 Lines of Evidence

The methodology for the evaluation included several lines of inquiry as follows: document review, interviews, notes from consultations and analysis of performance information and financial information.

Document Review

The document review included the following types of documents: corporate documents, accountability and policy documents, inception and renewal documents, reports on plans and priorities, performance reports, speeches from the Throne, and legislative documents. A list of documents reviewed is presented at Annex A.

Interviews

Nine interviews were conducted for the evaluation with distribution among interview groups as shown in Table 3. Interview guides were developed for each group.

Consultations Conducted by the Program

Program consultations with key stakeholders included federal organizations as well as provincial and municipal security partners to collect feedback on applications of the Policy. PS Evaluation Directorate staff dovetailed data-gathering activities with the consultations, participating in the development of data-gathering tools and attending consultation meetings. The consultation notes were used as a line of evidence by the evaluators to identify key issues brought up by security partners. The security partners consulted are listed below:

• RCMP
• Toronto Police Service
• City of Toronto
• Ottawa Police Service
• Peel Regional Police Service
• Ontario Ministry of Community Safety and Correctional Services
• Ontario Provincial Police
• Sûreté du Québec
• Service de la police de la Ville de Montréal
• Service de police de Gatineau
• Service de police de Québec
• Direction des poursuites criminelles et pénales (Québec)
• · Ministère de la sécurité publique du Québec

Review of Financial and Performance Information

An analysis of financial and performance information included contribution agreements, designation letters, financial tables, and program budget and expenditure data.

3.4 Limitations

The following section describes data limitations and how the evaluation team addressed these limitations.

• The evaluation team conducted a limited number of interviews with federal organizations and relied on data collected by the program representatives from security partners during consultations. To address this limitation, consultations notes were reviewed and members of the PS Evaluation Directorate participated in the consultations.
• Of the four events examined in the evaluation, two were the 2010 G8 and G20 Summits. Security costs for these events where unprecedented and significantly higher than the other events where the Policy was applied. Much of the consultation discussion and interviews focused on the G8/G20 Summits as they were the larger and the most recent.

3.5 Protocols

During the evaluation, PS program representatives assisted in the identification of key stakeholders and provided documentation and data to support the evaluation. Collaborative participation greatly enriched the evaluation process.

This report was submitted to program managers and to the responsible Assistant Deputy Minister for review and acceptance. A management response and an action plan were prepared in response to the evaluation recommendations. These documents were presented to the PS Departmental Evaluation Committee for consideration and for final approval by the Deputy Minister of Public Safety.

4. Findings

4.1 Relevance

4.1.1 Continuing Need for the Policy

In order to establish if there is a continuing need for the Policy, the evaluation examined the larger context of trends in event costs and complexity for hosting major international events, evidence of original program rationale, and perceptions of the appropriateness of the Policy given the current need.

Document review indicates that the 1999 World Trade Organization meeting in Seattle and the 2001 Summit of the Americas in Quebec City were the drivers for the original Policy in 2001. These events began a trend of increased cost and complexity of security related to international events. It became clear that the past practice of provinces/territories and municipalities providing security support to the RCMP from within their budgets was no longer sustainable given the increased level of security required for such events. Security costs related to hosting major international events have continued to rise over the last decade. Events can now cost hundreds of millions of dollars, with 90% of expenditures related to security costs.^{Note 7} Broader threats have translated into higher costs and increased pressures on the local host jurisdictions for policing and security cost coverage.^{Note 8}

In terms of complexity of security for these events, there are a number of factors that have always been present – the need to cover multiple jurisdictions and geographic complexities, as well as the nature/size and clarity of the agenda related to designated events.^{Note 9} Federal interviewees noted that there have nevertheless been concrete changes perceivable in the level of complexity. Some of the factors that have worked to increase the complexity of hosting such events are the presence of terrorist threats; the ability to initiate large-scale protests and movements through social media; the increased legislative, procedural, and policy requirements for planning (e.g. environmental assessments and insurance reviews); and, the sheer number and type of players involved.

Following the 2007 North American Leaders Summit, the Prime Minister directed PS to move forward with changes to the Policy to achieve the following:

1. increase federal control over extraordinary security costs incurred by provinces/ territories and municipalities;
2. clarify for security partners the rules that govern reimbursement claims; and,
3. ensure that a comprehensive management regime is adopted for effective implementation of the Policy.

In order to achieve these objectives, the Policy was revised in 2008 and resulted in the creation of a contribution program. Interviewees noted that the 2008 Policy is a step forward and the contribution format is appropriate in that it allows for verification and accountability. The Policy provides a good tool for cooperation among jurisdictions as it recognizes that provincial/territorial and municipal security partners need compensation. Interviewees also noted that the Policy is only "one piece of the puzzle", indicating that these types of events would benefit from an overarching governance structure or a broader federal policy framework.

The program also provided a list of forecasted upcoming international events which may receive designation under the Policy, based on the past cycle of hosting such events. This adds evidence to the continuing need for a mechanism such as the Policy that will enable Canada to continue to host such events.

4.1.2 Alignment with Federal and Departmental Priorities

The evaluation sought to assess the degree of alignment of the Policy with federal government and departmental priorities.

Document review indicates that from a federal perspective, providing financial assistance to provinces/territories and municipalities that host designated events supports the Government of Canada's commitment to maintain a strong and resilient Canada by assisting security partners' planning efforts and ensuring that provincial/territorial and municipal security partners are able to access federal support through reimbursement.

Speeches from the Throne consistently feature safety and security as a central theme. In 2010, the Speech from the Throne highlighted the G8/G20 Summits as important events showcasing Canada. Moreover, in a 2010 television interview about costs of the G8/G20 Summits, the Prime Minister stated that, "international summits are an irreplaceable tool that world leaders use to… accomplish major tasks of diplomacy. They are necessary traditions…".^{Note 10}

Evidence indicates that the Policy is aligned with departmental priorities. It contributes to the achievement of departmental objectives related to policies addressing evolving threats to public safety, security and ongoing leadership to the Canadian law enforcement community.^{Note 11} In the context of the G8/G20 Summits, the Minister of Public Safety stated that "global security begins at home... we will take all measures necessary to ensure Canadians, delegates and international visitors remain safe."^{Note 12}

A review of PS Reports on Plans and Priorities from 2010 to 2013 shows that security arrangements play an important role in international events and place a focus on the renewal of the Terms and Conditions of the Policy. In 2010, the Minister highlighted the support to the G8/G20 Summits as key accomplishments for the Department.

4.1.3 Alignment with Federal Roles and Responsibilities

The evaluation sought evidence of accountability and authority to deliver the Policy to understand whether there is alignment with the federal mandate.

The federal role for the stewardship and oversight of delivering security for international events hosted in Canada originally began with the 1996 Department of Foreign Affairs and International Trade policy, the Management Framework for the Organization and Funding of Prime Minister-Led Summits of an International Nature.

The Foreign Missions and International Organizations Act states that "The RCMP has the primary responsibility to ensure the security for the proper functioning of any inter-governmental conference in which two or more states participate…".^{Note 13} As stated in the original iteration of the Policy (2001), the federal government is responsible for decisions to host Prime Minister or Minister-led international meetings and, as such, the RCMP is responsible for the protection of internationally protected persons. However, provinces/ territories have constitutional responsibility for the administration of justice within their province/territory and for the provision of policing services. In applying the Policy, the Government of Canada has provided financial assistance to provincial and municipal security partners for incremental extraordinary policing and security-related costs for designated events in their respective jurisdictions.

The departmental mandate to develop and deliver a transfer payment program, such as the Policy and its terms and conditions, is derived from the Department of Public Safety and Emergency Preparedness Act, wherein the Minister of Public Safety is given this authority.^{Note 14}

There are other federal initiatives that relate to the management and administration of major international events hosted by Canada. Although these processes are distinct and separate from the Policy, document review indicates that PS is aware of these and will ensure that there are synergies and linkages in terms of the overall federal security approach to managing designated events.^{Note 15}

4.2 Performance—Effectiveness

Performance was assessed by examining relevant program documents such as contribution agreements, performance reports, consultation notes drafted by the program and financial information. Qualitative evidence was also gathered through interviews. 

4.2.1 Security-related Support and Leadership

The evaluation assessed the extent which PS support and leadership has been provided to provinces and municipalities in terms of sufficiency of policy instruments and guidance; and, in terms of monetary support to provinces and municipalities.

Federal interviews and security partners consulted^{Note 16} noted that relationships with PS staff were excellent and helpful. The PS team was very professional and knowledgeable and showed an openness to review disputed items. This greatly assisted in an increased understanding of the Policy. They also noted that the Policy is flexible and needs to remain flexible to adapt to changing realities and event-specific issues. Some security partners consulted noted that the Policy is adequate, fair, and effective and that it is a good instrument.

Documented evidence supports the finding that PS staff have been proactive in providing support and leadership as needs evolve. For each version of the Policy, coverage and eligibilities were developed and outlined. Contribution agreements were drafted for each provincial and municipal security partner describing the financial provisions, including the monetary negotiated caps. PS staff requested monitoring and activity reports from partners at various stages to inform on the status of security planning activities. PS has business processes in place to administer the Policy such as financial controls and audit requirements. PS staff notes that there is currently a mechanism established for dispute resolution in the agreements, but most disagreements are resolved quickly and positively and have not reached legal processes. A review of program documentation related to the G8/G20 Summits shows negotiations between PS and provincial and municipal security partners on cost estimates and demonstrates dispute resolution over claims.

Document review and interview evidence revealed that the development of a communication tool on the Policy would be useful for all involved. The Policy and related documentation is not publicly available on the PS internet site. Publicly posting information documents on the Policy would aid provincial/territorial and municipal security partners in clarifying what can and cannot be reimbursed and which costs should or should not be claimed from the beginning in order to avoid disagreements in the later stages. Interviewees noted that recurring problems for these types of events might be avoided if guidance was available.

Federal interviewees and security partners consulted^{Note 17} agreed that the 2008 Policy needs further clarification related to eligibility of costs. They cited items such as costs related to emergency management personnel (e.g. first responders); costs related to the planning phase; intragovernmental costs; and pay considerations (e.g. overtime for senior officers during deployment).

Despite the above-noted issues, program data suggests that provincial and municipal security partners are becoming more aware of what costs are eligible. Between 2000 and 2008, prior to the Policy becoming a contribution program, the total amount claimed was about 20% over the total amount reimbursed; since 2008, this percentage dropped to 3%.

Security partners consulted stated that the audit process generally went well, however they note that the audits required significant levels of effort on their part and that improvements could be made. They expressed that there is need for increased guidance tools, checklists and description of audit requirements at the outset. They would like auditors to be involved early so that they are aware of the context. Partners also mentioned that the process was lengthy, that there were issues with regard to materiality, and suggested that audits could be more risk-based.

Interviewees believed that more integration at the federal level could have provided guidance in planning.^{Note 18} The G8/G20 lessons-learned report also highlights this issue. Even though good relationships existed between PS staff and federal partners, it is observed that the event would have benefited if PS had greater exposure to the evolution of the security planning assumptions for the Summits; and a strengthened and deeper understanding of the linkages between the provincial/territorial and municipal security operations and the overarching RCMP security operation. While PS was responsible for contribution agreement negotiations, the official security representative for the federal government with provincial and municipal partners was the Coordinator's Office at the Privy Council Office. It might have been more effective and integrated if either the Coordinator's Office would have been attached to PS or if PS had permanent representation.^{Note 19} Security partners consulted agreed and noted that it would be useful for PS to be centrally involved with other federal partners in a coordinated event planning office. As a result of not dealing directly with PS, conflicting information was given to partners regarding coverage of certain items.^{Note 20}

In terms of monetary support, Table 4 illustrates the total amount that the federal government has reimbursed provincial and municipal security partners under the 2008 Policy for incremental and extraordinary security costs related to designated events.

4.2.2 Early Engagement and Cooperative Participation of Security Partners

The evaluation sought to determine whether the 2008 Policy enables engagement and participation by provinces and municipalities for designated events. In order to do so, the evaluation team analyzed program documentation and gathered perceptions on early engagement especially during cost negotiations.

Evidence indicates that of 12 contribution agreements signed in the timeframe being evaluated, five were signed after the event. Interviewees indicate that the main contributor to this result is that PS cannot negotiate cost estimates with provincial/territorial and municipal security partners until the event is designated under the Policy. Before designation, the Department of Foreign Affairs and International Trade, the RCMP, and provincial and municipal security partners begin to prepare months in advance. The process of designation takes several months and subsequently, the process to secure federal funding, including obtaining approval of a Memorandum to Cabinet and subsequent Treasury Board submission, can take an additional period of several months.^{Note 21} Security partners consulted state that the delay in obtaining designation for the events does not allow procurement to proceed in a timely way. Tight timelines limit the ability to negotiate in advance. This situation leads to the contribution agreements being signed just prior to or after the events.^{Note 22}   

Federal interviewees note that the 2008 Policy has enabled PS to effectively engage provincial and municipal security partners in discussions; however cost negotiations cannot always take place in a timely manner. Some noted that there may be a need for a federal mechanism to start to enable PS to move forward earlier, e.g. a federal overarching process to determine a preliminary source of funds. They also noted that PS has done well in terms of early planning given these factors and circumstances. For example, for the 2009 Obama visit, the contribution agreement was processed and signed after the event because the event was announced only one month prior to it being held and designation was not obtained until a few days after the event. Negotiations were challenging because of the lack of advance notice. The contribution agreement was signed just under one year post-event. For the 2010 G8/20 Summits, a number of federal organizations as well as multiple provincial and municipal security partners began planning security in late 2008. The contribution negotiations were all completed prior to hosting the Summits. Nine of the 10 agreements/amendments were signed by the Minister of Public Safety prior to the Summits and, of these, seven were ratified by the provincial and municipal security partners in advance of the Summits. All remaining signatures were officially obtained shortly after the events.^{Note 23} Performance reports indicated that negotiations started in the year before the events. For the G8 Summit, planning was facilitated by having sufficient notice. For the G20 Summit, there was not enough notice; this made the relationships with provincial and municipal security partners more challenging.

When asked whether the revisions in 2008 had an effect on early planning, some federal interviewees stated that it has improved, that provincial/territorial and municipal security partners are more eager to engage and that communication is enhanced, so expectations are better managed. Others believed that there are no real changes for recipients. The payments are made, but timing of the designation is an important factor for cost negotiations.

Security partners consulted indicated that, overall, developing cost estimates has been very challenging because of short timelines; the time constraints required for ratification signatures; and the requirement to plan without complete information about the event/program. They also mentioned that security plans are based on planning assumptions and are variable, stating that it is difficult to plan when the threat level changes as the event moves from planning to mobilization (i.e. hard to plan for overtime).^{Note 24}

Nonetheless, program documents illustrate that partner estimates present high levels of detail and justifications for costs demonstrating their willingness to be engaged and participate cooperatively. PS consultations with the RCMP show levels of cooperation between federal partners and mutual objectives for effective security planning.

4.2.3 Sound Management of Security Resources

The evaluation assessed the extent to which the 2008 Policy contributed to sound management of security resources. This assessment focused on the overall impact of the 2008 Policy on controlling the costs of designated events and optimizing resources (equipment and personnel).

Federal interviewees had mixed views on whether the 2008 Policy assisted in terms of controlling the overall cost of security for a designated event. On the positive side, they believe the 2008 Policy has allowed the federal government to provide reimbursements and is a good negotiation tool for PS. Some interviewees stated that costs are entirely dependent on the type and location of an event. Costs vary significantly by each event and since needs and locations always change accordingly, it is difficult to control costs except if a site is a known, secured site.

As shown in Table 5, better cost control was evident the fact that the caps were maintained. However, cost estimates for some of the events could be improved since money was set aside in the fiscal framework that was later not fully expended on the event. This was evident for the G8/G20 Summits in particular, where the total reimbursement was 57% of the cap amount because the initial estimates were high. In terms of the Sommet de la Francophonie and the Obama visit, when it came time to set aside money through internal reallocation, at the end of March 2009, PS had more information on the actual costs as both events were completed; thus, the reimbursements shown in Table 5 are closer to the cap.

Some interviewees state that provincial and municipal security partners continue to submit higher cost estimates in order to manage their risks and to help manage evolving planning assumptions. They add that costs are not being negotiated early enough and this contributes to this issue.

One of the challenges noted by federal interviewees is that the cost estimates are highly dependent on timing of decisions and lead times for planning. The fact that costs cannot be negotiated during "pre-designation" contributes to the problem. The 2011 Auditor General report states that plans and budgets for the G8 Summit and, in particular, the G20 Summit were prepared within a limited time frame and with incomplete information on which to base cost estimates. As a result, assumptions were made that resulted in requests for more funding than was ultimately required. With the exception of a lack of an overall assessment, and given the unique and challenging conditions under which departments worked, there was reasonable senior management challenge of the departmental business plans in these circumstances.^{Note 26}

Despite these challenges, interviewees note that reductions have been made from initial cost estimates submitted by provincial and municipal security partners. From the records of the negotiations with partners kept by the PS Program, there was evidence of diligence and attention to detail in cost management. This was seen in the flagging of partner cost estimates for verification. Challenges made by PS Program to partner cost estimates in negotiations was also evident in correspondence demonstrating cost-avoidance and cost-reduction efforts made by PS. Federal interviewees noted that, while PS did an excellent job throughout the negotiations, the program lacks specialized expertise in the challenge function on the overall security cost estimates. Interviewees believed that a third-party peer review would allow for neutral validation and independent perspective on the estimates, and would improve planning and accountability for the federal government. Security partners consulted expressed that they do not favor peer review, with some noting that an expert advisor, involved from the beginning of the process, would be more effective.^{Note 27}

The Auditor General reported that PS relied on the RCMP for assurance that the police partners' security plans, for which they were seeking reimbursement, were consistent with RCMP plans. However, PS did not have a formal arrangement with the RCMP to do this. RCMP officials did not review partner business plans but assisted PS to a degree. PS ultimately relied on cost estimates provided by partners to identify the majority of their funding requests and therefore obtained limited assurance that all business cases were costed appropriately.^{Note 28}

During interviews conducted specifically on the audit process, interviewee's stated that the audit component of the 2008 Policy helps with cost control. In the previous process, provincial and municipal security partners were familiar with grants and contributions which did not always have an audit component whereas now they know there will be an audit; this adds rigor to the process.

In terms of overall resource optimization, federal interviewees agree that the 2008 Policy has optimized resources by enhancing partner engagement, by focusing on joint procurement and increasing expectations for provincial and municipal security partners to reuse assets. A key challenge noted in this regard is that there are no national standards on assets used by the various jurisdictions. It was mentioned that resource optimization is also attributable to other factors such as the evolution of the security environment related to these events that includes the involvement of more experienced groups in the planning and the establishment of Integrated Security Units.

Security partners consulted expressed that the federal government needs to better manage surplus asset issues, with about half stating that joint procurement was not efficient. Some of the reasons expressed were as follows: each partner has different timing and needs; standards for equipment vary from one jurisdiction to another; the RCMP has different standards; delays in agreements are not favorable to joint procurement; and, each jurisdiction has to follow its own procurement process. It was noted that joint procurement may work if there were more time for planning, if work could be done on the standardization of equipment, or if it were used only for unforeseen items.^{Note 29} The program has identified opportunities for improvement related to joint procurement and asset management.^{Note 30}

4.3 Performance—Efficiency and Economy

The efficiency of the administration of the 2008 Policy by PS was assessed by determining how the average program administration ratio compares to other PS programs. The evaluators also gathered perceptions related to program administration and with regard to the audit process for the G8/G20 Summits.

A review of financial information combined with program estimates approximates the annual administrative cost of the 2008 Policy to have varied between $0.3 million to $1.7 million over the last five years. The year of the G8/G20 Summits and the year after represent the highest costs. The average program administration ratio^{Note 31} is 2.7% which compares favourably to other PS contribution programs. Details of the calculations are contained in Annex B.

In the G8/G20 lessons learned report, the program notes that the audit process went very well considering the unprecedented number of contribution agreements ratified with provincial and municipal security partners. However, the program and the external auditors noted some challenges and opportunities for improvement. The biggest issue identified was the delays of processing and reimbursing partner claims. This was mostly due to the availability of partners and the large number and complexity of documents to be reviewed. The average processing time for the claims related to the G8/G20 claims was 43 weeks.^{Note 32}

Through performance reports following the G8/G20 Summits, provincial and municipal security partners expressed that delays in funding approval are an impediment to proper planning and procurement of security resources for events. They indicate that earlier funding approvals would be beneficial and assurance of costs to be reimbursed should be provided to partners if funding is delayed. They noted that the federal government should provide advance/interim payments as it is difficult to plan without the money in their budget.^{Note 33}

Federal interviewees agree that the process is slow as it is tied to the fiscal framework. There is no ongoing budget for the Policy and therefore no set process. It is often not possible to have accurate estimates before the event and partners are not getting compensation quick enough. In some cases, interim payments were possible through the planning phase. These payments reflected the spirit of the Policy by reimbursing incremental and extraordinary eligible policing and security-related expenses following the audit review of the partner's claim by the audit function.^{Note 34}

The former Audit Services Canada provided a detailed report to PS outlining a number of observations. Twenty recommendations were made to improve and strengthen the overall Policy application process. In addition, the program also identified a number of areas where improvement could be made in the application of the Policy and the coordination of the audit process for future designated international events under the Policy.

It was also noted that the files related to the G8/G20 Summits were very complex and some provincial and municipal security partners did not conduct internal audits. Harmonizing decisions added to the complexity as the auditors were dealing with seven different provincial and municipal security partners. More uniformity is required with regards to the application of the Policy as it relates to audits.

Another point made by the security partners consulted was that interpretations of eligibility provided by PS throughout the process should be taken into account in the final audits of claims and be reflected in the contribution agreements.^{Note 35} The program agrees and notes that it would be useful to either have a standardized process to address interpretations for partners or to include some additional language in the contribution agreements.^{Note 36}

5. Conclusions

5.1 Relevance

The need for the Policy remains. Incidents over the last decade have caused a rise in security costs and have increased the complexity of security for Prime Minister or Minister-led events. Since Canada continues to host international events, host jurisdictions cannot sustain the associated incremental and extraordinary policing and security-related costs from their existing budgets.

The Policy is aligned with federal priorities as the government continues to place a strong focus on safety and security and on the importance of hosting major international events. The Policy is also aligned with departmental objectives related to its leadership role in the Canadian law enforcement community and its role of addressing evolving threats to safety and security.

The mandate for the federal government to provide security for major international events is clearly established in legislative documents as is the role of PS related to administering the Policy through contribution agreements to provide financial assistance to provincial/territorial and municipal security partners in host jurisdictions. The program ensures that linkages are maintained with other federal initiatives related to the organization of major international events.

5.2 Performance

In terms of performance, although it requires some improvements, the 2008 Policy represents an evolution of this instrument for the federal government. The PS team has provided support and leadership in implementing the 2008 Policy. The team is described as professional and knowledgeable, maintaining excellent working relationships with federal and provincial and municipal security partners. PS staff have revised the Policyover time to reflect the changing security environment. Notable are changes that came into effect in 2008, when the program was converted into a contribution program and monetary caps were put in place. This has enhanced the comprehensive management of the Policy. In fact, between 2000 and 2008, prior to the Policy becoming a contribution program, the total amount claimed was about 20% over the total amount reimbursed; since 2008, this percentage has dropped to 3%. Gaps in support from PS include the need for clarity surrounding the eligibility of costs for reimbursement; for better communication on the audit process; and for publication of general documentation related to the Policy. In terms of monetary support, under the 2008 Policy, the federal government provided $160.7 million in reimbursements to provincial and municipal security partners for the four events included in the evaluation.

The 2008 Policy has, to some extent, enabled early engagement and cooperative participation of provincial and municipal security partners. In general, partners are more eager to engage and communication is enhanced, so expectations are better managed. However, timing of event designation and funding approval remain important contextual factors that affect the achievement of this outcome. In general, if designation is obtained within an appropriate timeframe and funding approval is in place, PS can fully engage and negotiate early in the process. If designation and subsequent funding approval come later in the process, PS cannot enter into cost negotiations until a later stage, when security partners are already in the planning phase.

The 2008 Policy has contributed to sound management of security resources, but overall event costs are highly dependent on the event site. Cost control was evident in negotiations for contribution agreements, maintenance of negotiated caps, and recipient audits. The PS team that administers the Policy has provided diligence and attention to detail when challenging cost estimates, but access to specialized expertise in some specific areas may enhance this function. Enhanced partner engagement, procurement processes and increased expectations for provincial and municipal security partners to reuse assets have optimized resources to some extent. Outstanding issues with respect to cost management include the fact that costs are, at the outset, over-estimated by partners in an effort to manage their risks, as partners are working from planning assumptions that change and evolve. Cost estimates for some of the events could be improved since money was set aside in the fiscal framework that was later not fully expended on the event. This was evident for the G8/G20 Summits in particular, where the total reimbursement was 57% of the cap amount because the initial estimates were high. In terms of the Sommet de la Francophonie and the Obama visit, when it came time to set aside money through internal reallocation, at the end of March 2009, PS had more information on the actual costs as both events were completed; thus, performance against the cap was reasonable. Issues also remain relating to procurement processes and management of surplus assets.

The 2008 Policy is being administered efficiently with an average program administration ratio of 2.7% that compares favourably to other PS contribution programs. Timeliness of the process related to audits and reimbursements was a noted issue.

6. Recommendations

The Evaluation Directorate recommends that the Assistant Deputy Minister, Law Enforcement and Policing Branch, implement the following actions:

1. Further improve partner engagement through the following communication activities:
   1. Prepare and publish information, on the departmental website, regarding the Security Cost Framework Policy in general and eligibility of expenses in particular.
   2. Provide early and clear communication regarding requirements of the audit process so that provincial/territorial and municipal partners are better prepared for audits, supporting improved efficiency of the audit process.
2. Further improve event security cost management by:
   1. Developing options and seeking approval for changes to the designation and funding processes that will allow PS to enter into negotiations with provincial/territorial and municipal security partners at the earliest possible stage.
   2. Developing options for mechanisms that will allow for closer alignment between funding approval amounts and final reimbursements. Options should aim to strengthen the challenge function for provincial/territorial and municipal partner cost estimates, and should include, where appropriate, the assistance of specialized expertise to support the challenge function.

7. Management Response and Action Plan

Annex A: Documents Reviewed

CBC News (May 25, 2010). G8, G20 security bill at least $833M.

CTV Canada (June 28, 2010). Canada AM Television Interview with Prime Minister Stephen Harper.

Department of Justice Canada. Department of Public Safety and Emergency Preparedness Act S.C. 2005, c.10.

Department of Justice Canada. Foreign Missions and International Organizations Act S.C. 1991, c.41.

Public Safety Canada (2011). A Discussion Paper on the Security Cost Framework Policy.

Public Safety Canada (2012). Renewal of the Security Cost Framework Policy Consultation Deck.

Public Safety Canada (2011). Security Cost Framework Policy Application for the 2010 G8 and G20 Summits, Management of Assets and Equipment Summary.

Public Safety Canada (2012). Security Cost Framework Policy Application for the 2010 G8 and G20 Summits, Lessons Learned Report.

Public Safety Canada (2012). Security Cost Framework Policy Renewal Guide for Consultations.

Public Safety Canada (2012). Security Cost Framework Consultation Summary

Public Safety Canada. 2010-2011 Report on Plans and Priorities.

Public Safety Canada. 2011-2012 Report on Plans and Priorities.

Public Safety Canada. 2012-2013 Report on Plans and Priorities.

Office of the Auditor General of Canada (2011). Report of the Auditor General of Canada to the House of Commons Spring 2011, Chapter 1: Expenditures for the G8 and G20 Summits.

Office of the Parliamentary Budget Officer (2010). Assessment of Planned Security Costs for the 2010 G8 and G20 Summits.

Commission for Public Complaints against the Royal Canadian Mounted Police (2012). Public Interest Investigation into RCMP Member Conduct Related to the 2010 G8 and G20 Summits.

Office of the Independent Police Review Director (2012). Policing the Right to Protest: G20 Systemic Review Report.

Annex B: Financial Analysis

Date modified:

2022-07-26