2011-2012 Evaluation of the Disaster Financial Assistance Arrangements Program

2011-2012 Evaluation of the Disaster Financial Assistance Arrangements Program PDF Version (406 KB)

Table of contents

• Executive Summary
• 1. Introduction
• 2. Profile
  • 2.1 Background
  • 2.2 Program Administration
  • 2.3 Resources
  • 2.4 Logic Model
• 3. About The Evaluation
  • 3.1 Objective
  • 3.2 Scope
  • 3.3 Methodology
  • 3.4 Limitations
  • 3.5 Protocols
• 4. Findings
  • 4.1 Relevance
  • 4.2 Performance—Effectiveness
  • 4.3 Performance—Efficiency and Economy
• 5. Conclusions
  • 5.1 Relevance
  • 5.2 Performance—Effectiveness
  • 5.3 Performance—Efficiency and Economy
• 6. Recommendations
• 7. Management Response and Action Plan
• Annex A: List of Documents Reviewed
• Annex B: Scan of Other Federal Programs
• Annex C: Cost of the Disaster Financial Assistance arrangements program

Executive Summary

This is the 2011-2012 Evaluation of the Disaster Financial Assistance Arrangements Program. Evaluation supports accountability to Parliament and Canadians by helping the Government of Canada to credibly report on the results achieved with resources invested in programs. Evaluation supports deputy heads in managing for results by informing them about whether their programs are producing the outcomes that they were designed to achieve, at an affordable cost; and supports policy and program improvements by helping to identify lessons learned and best practices.

What we examined

The Evaluation of the Disaster Financial Assistance Arrangements Program was conducted in accordance with funding approval requirements that an evaluation be completed prior to the renewal of terms and conditions by December 2012. The scope of the evaluation included Public Safety Canada's activities in administering the Disaster Financial Assistance Arrangements (the transfer payment program) starting April 1, 2006.^{Note 1}

The Disaster Financial Assistance Arrangements Program is a cost-sharing reimbursement program between the federal government and a province/territory. The purpose of the Program is to support an affected province/territory after a natural disaster, in order to assist with response and recovery costs that might otherwise place a significant burden on the provincial/territorial economy and would exceed what the province/territory might reasonably be expected to fully bear on its own.

Since the inception of the Disaster Financial Assistance Arrangements Program in 1970, the Government of Canada has paid out approximately $2 billion in post-disaster assistance to help provinces/territories with the costs of response and of returning infrastructure and personal property to pre-disaster condition.

The funding profile for the Disaster Financial Assistance Arrangements Program is about $102 million in ongoing funding per year, the majority of which is profiled for financial assistance to provinces/territories.

Why it is important

The Disaster Financial Assistance Arrangements Program is a quasi-statutorial program that promotes partnerships between the federal and provincial/territorial levels of government and promotes federal policy objectives of supporting provinces/territories when disasters threaten to overburden their financial resources.

The Program is materially significant; since inception, it has committed over $2 billion in post-disaster assistance, and on average, just over $83 million annually over the past five years, representing about 20% of Public Safety Canada's annual spending.

What we found

There is a continuing need for the Disaster Financial Assistance Arrangements Program. The activities of Public Safety Canada in delivering the Program continue to provide a mechanism by which the Government of Canada helps Canadians in their "time of need", as it has done since 1970. Nationally and internationally, there is an upward trend in the number of major natural catastrophes, and the severity and costs of the events are increasing. In 2011, there continues to be disasters that overburden provincial/territorial resources.

While the Disaster Financial Assistance Arrangements Program is aligned with federal and departmental priorities, there is emergent policy direction for consideration. Emergent policy direction includes the concept of prevention/mitigation which is being implemented in two emerging Public Safety Canada programs^{Note 2} and has been integrated into the existing Guidelines for the Disaster Financial Assistance Arrangements. In light of these emergent programs and policy direction, it will be important to position the Disaster Financial Assistance Arrangements Program in such a way as to balance the federal support offered by the combination of these programs; balance between short-term (response/recovery) and long-term (prevention/ mitigation) objectives will need to be considered.

Legislative and policy authorities demonstrate that it is within the role of the federal government, under certain circumstances, to assist provinces/territories in response and recovery efforts. There is no duplication with regard to provincial disaster financial assistance programs or other Public Safety Canada programs. However, according to interviewees, there exists the possibility for duplicative payments from other federal organizations within the sectors that the Disaster Financial Assistance Arrangements Program serves.

Through the efforts of Public Safety Canada staff that administer the Program and through financial reimbursements, provinces/territories have been supported in their response and recovery efforts for large-scale natural disasters. Program guidelines have been modified based on engagement of provinces/territories and these changes have been generally well received. The new guidelines bring new challenges that will require monitoring, examination and engagement for resolution in order to support provinces/territories in their response and recovery efforts. Overall, the Disaster Financial Assistance Arrangements Program has supported provinces/territories in easing the financial burden of major natural disasters as the federal government has paid out about $300 million over the past five years or about half of the total provincial/territorial cost of eligible disasters.

In terms of supporting Canadian communities, because the Program is accessed more often in less populous provinces, the Program has more of a direct impact in these provinces. Interviewees from these provinces highlighted the importance of the federal program in supporting provincial efforts benefiting individuals and business in the communities. More populous provinces noted a smaller impact on their communities due to the Disaster Financial Assistance Arrangements Program and attributed community benefits to provincial support.

The Disaster Financial Assistance Arrangements Program is administered efficiently but there are gaps in strategic management. In terms of efficiency, the average annual Program administration ratio (4.8%) compares very favourably to other Public Safety Canada transfer payment programs. Actions have been taken to improve program administration during the past five years. From a strategic management perspective, the program is at risk because it has no succession plan and there is the potential for loss of corporate memory, particularly at this juncture with the dissolution of Audit Services Canada and a Director who has been involved with the Program for 25 years.

Improvements to the federal audit process are needed in order to save time and money. The dissolution of Audit Services Canada provides an opportunity to update audit methodologies to a risk-based approach. A sample of six audits revealed an average delivery time of 14 months; however, because there are no service standards in place, it is not possible to state, in quantitative or absolute terms, whether the audit process has been timely.

Mitigation enhancements are poised to provide economic benefits. There are documented economic benefits to prevention/mitigation programs and the Disaster Financial Assistance Arrangements Program has integrated mitigation enhancements into its programming; however, as noted above, going forward the Disaster Financial Assistance Arrangements Program must consider its role in the larger policy context and in relation to emerging programs.

Recommendations

We recommend that the Assistant Deputy Minister, Emergency Management and Regional Operations Branch implement the following recommendations.

1. Going forward, ensure proper positioning of the Disaster Financial Assistance Arrangements Program with respect to related emergent prevention/mitigation programs. Proper positioning will help ensure that the support offered by the Disaster Financial Assistance Arrangements Program does not create deterrents for provinces/territories in accessing these other programs, and should promote balance between short-term (response/recovery) and long-term (prevention/mitigation) economic objectives.
2. Ensure that the Program's management:
   1. Include other federal departments and agencies that operate in sectors similar to the Disaster Financial Assistance Arrangements Program (e.g. agriculture, infrastructure, and Aboriginal affairs) in future consultations to explore possible duplication issues; and
   2. Develop and implement a staff succession plan to protect program and corporate memory.
3. Review the federal audit process with a view to a more risk-based, streamlined approach to enhance the program delivery.

Management Response and Action Plan

Emergency Management and Regional Operations Branch will:

1. Ensure that future Emergency Management and Regional Operations policy and program activities include and document how the Disaster Financial Assistance Arrangements program affects, and may be affected by, emergent prevention/mitigation programs.
2.  
   1. Include other federal departments/agencies such as Agriculture and Agri-food Canada, Aboriginal Affairs and Northern Development Canada, and Infrastructure Canada in program consultations, which are planned to occur prior to 2016, as part of Disaster Financial Assistance Arrangements program renewal in order to ensure that there is no duplication among federal program criteria.  Program management will consult with other departments and agencies as required during the application of the Disaster Financial Assistance Arrangements to specific natural disasters to ensure that there is no program overlap. 
   2. Develop and implement a staff succession plan in 2012-2013 to preserve program expertise and corporate memory. 
3. Review the federal audit process to evolve to a more risk-based approach during the federal audit transition process.   The initiative will begin in 2011-2012, in consultation with the Internal Audit Directorate, and should be completed and implemented in 2012-2013.

1. Introduction

This is the 2011-2012 Evaluation of the Disaster Financial Assistance Arrangements (DFAA) Program. This evaluation provides Canadians, parliamentarians, Ministers, central agencies, and the Deputy Minister of Public Safety an evidence-based, neutral assessment of the relevance and performance (effectiveness, efficiency and economy) of this federal government program.

The Evaluation of the DFAA Program was conducted in accordance with the funding approval requirements that an evaluation be completed prior to the renewal of terms and conditions by December 2012.

2. Profile

2.1 Background

The DFAA Program is a cost-sharing reimbursement program between the federal government and a province/territory. Provincial/territorial governments design, develop and deliver disaster response and financial assistance programs within their own jurisdictions. The purpose of the DFAA Program is to support an affected province/territory after a major natural disaster in order to assist with costs that might otherwise place a significant burden on the provincial economy and would exceed what the province/territory might reasonably be expected to bear on its own. Assistance is not provided directly to affected individuals, small businesses or communities.

Since the inception of the DFAA Program in 1970, the Government of Canada has paid out approximately $2 billion in post-disaster assistance to help provinces/territories with the costs of response and of returning infrastructure and personal property to pre-disaster condition.

A review of the DFAA Program was undertaken in 2001 and concluded in 2006 based on a federal mandate approved in 1999 to consult with the provinces/territories on specific proposed changes to the DFAA Program. The goal of the review was to modernize the DFAA Program and to address several issues raised by stakeholders. Following this review, revised program guidelines were issued in 2008. Public Safety Canada (PS) proposed subsequent reviews every 10 years given the nature of the DFAA Program; thus, the next review is planned to conclude by 2016.

2.2 Program Administration

The DFAA Program is administered based on the guidelines developed by PS. Requests for DFAA Program financial assistance must be made by the province/territory to the Prime Minister or the Minister of Public Safety. The Emergency Management Act requires that a federal Order-in-Council be issued declaring that a provincial/territorial emergency is of concern to the Government of Canada and authorizing the provision of financial assistance. The PS Regional Director provides federal liaison with provincial/territorial officials and supports headquarters with the administration of the Program.

Provinces/territories must keep accounts and records of all related financial estimates and expenditures related to the disaster. A province/territory may request advance payments to address early requirements. Interim payments may also be made, if requested, and deemed appropriate by the Minister of Public Safety.

The final provincial/territorial request for payment must be validated at the provincial/territorial level to assert that the request for payment reconciles to the audited financial statements of the province/territory. Once the provincial/territorial request for payment is submitted, PS conducts a federal audit for DFAA Program compliance. The Minister of Public Safety makes final decisions as to the compliance and eligibility of payment requests. The final payments are made based on a per-capita cost-sharing formula as illustrated in Table 1.

Annual DFAA Managers Meetings are co-chaired by PS Program representatives. Representatives from all provinces/territories and PS regions are invited to attend the meetings.

2.3 Resources

As illustrated in Table 2, the funding profile (the approved budget under the management of the Program) is about $102 million in ongoing funding per year.

It is noted that PS Regional Directors are also involved in the delivery of the DFAA Program in the regions depending on which province/territory is experiencing a disaster event. These amounts are not included in the Table 2.

Over the years, Audit Services Canada has performed the federal audit of provincial/territorial submissions made under the DFAA Program. As such, a Memorandum of Understanding with Audit Services Canada accounts for most of the operating and maintenance budget. In the spring of 2011 it was announced that Audit Services Canada will no longer operate in fiscal year 2012-2013; however, operations will continue through 2011-2012.

2.4 Logic Model

The logic model presented in Figure 1 is a visual representation that links what the program is funded to do (activities) with what the program produces (outputs) and what the program intends to achieve (outcomes). It also provides the basis for developing the evaluation matrix, which gave the evaluation team a roadmap for conducting this evaluation.

Figure 1 – Logic Model for the DFAA program

The following are the activities of the Program: develop program guidelines and provide interpretation of DFAA criteria as required; review provincial/territorial requests for federal financial assistance and prepare Order in Council documentation; respond to request for advance, interim and final payments; conduct audits for requested payments.

The activities lead to the following outputs: DFAA guidelines and interpretation bulletins; Orders in Council; cost-sharing reimbursements of eligible provincial/territorial expenditures; audits.

The outputs lead to the following program outcome: provinces and territories are supported in their response and recovery efforts for large-scale natural disasters.

The program outcome leads to the PAA Outcome: the DFAA supports Canada in its response and recovery from events affecting the national interest (PMF 1.4.2).

3. About The Evaluation

3.1 Objective

This evaluation supports:

• Accountability to Parliament and Canadians by helping the Government to credibly report on the results achieved with resources invested in this program;
• The Deputy Minister in managing for results by informing him about whether this program is producing the outcomes that it was designed to produce, at an affordable cost; and
• Policy and program improvements.

3.2 Scope

The evaluation assessed the administration of the DFAA Program by PS and the outcomes of the program starting April 2, 2006. The scope of the evaluation included PS program activities to administer the DFAA contribution program. Thus, the term DFAA Program refers to both the contribution program and the PS administrative aspects thereof.

3.3 Methodology

The evaluation was conducted in accordance with the Treasury Board Policy on Evaluation, the Directive on the Evaluation Function and the Standard on Evaluation for the Government of Canada, as well as the PS Evaluation Policy. Evaluators took into account the following factors in order to determine the evaluation effort, including the approach, scope, design, and methods, required for this evaluation:

• Risks
• Quality of past evaluations
• Soundness of program theory
• Longevity of program
• Contextual stability

Specifically, the evaluation methodology and associated level of effort were calibrated taking into consideration the characteristics of the DFAA Program.

1. The program is low risk: The risk of "non-performance" is low; the DFAA Program is a formula-based program that provides reimbursements to provincial/ territorial recipients and is paid out based on a federal audit of provincial/territorial requests for payment. An Order in Council is required to respond to each request from a province/ territory.
2. The program is considered to be quasi-statutorial. Although there is no written definition of quasi-statutorial, the term refers to a program that is considered to be a statutory or legal obligation because of a commitment by Ministers to their provincial/territorial counterparts and because of historical agreement among the Department of Finance, Ministers and the Treasury Board. Statutory characteristics of the DFAA Program include the fact that there is no limit to the amount of money to be associated with it and that the Program is not designed to create/drive change within a certain target population. Having stated this, the Program is not considered to be purely statutory because the source of funds (in this case Vote 5) is still voted annually in Parliament, and there is a requirement to renew terms and conditions periodically.
3. Policy on Evaluation includes expectations regarding evaluation of statutory spending: The Policy on Evaluation states that "Deputy heads ensure that all ongoing programs of grants and contributions are evaluated every five years, as required by section 42.1 of the Financial Administration Act; and that the administrative aspect^{Note 4} of major statutory spending is evaluated every five years.^{Note 5} The DFAA Program is not considered "major" statutory spending.
4. Policy consultations for the Program are planned in the near future: Full consultations with Program stakeholders, for the purposes of policy renewal, are planned to be completed by 2016.

Taking the above factors into consideration, the evaluation team calibrated the methodology to focus less on issue areas related to relevance and effectiveness and more on administrative aspects (efficiency and economy of the Program). Having stated this, although the evaluation focused on administrative aspects, each core issue was addressed in the evaluation.

3.3.1 Evaluation Cores Issues and Questions

As required by the Directive on the Evaluation Function, the following issue areas and evaluation questions were addressed in the evaluation:

Relevance

1. What need is the DFAA Program addressing?

2. To what extent is the DFAA Program aligned with government-wide priorities and supportive of departmental strategic outcomes?

3. What is the nature of the federal government's mandate to deliver the DFAA Program?

Performance—Effectiveness  

4. To what extent have outcomes been achieved?

a) Provinces/territories are supported in their response and recovery efforts for large-scale natural disasters.

b) The DFAA supports Canada in its response and recovery from events affecting the national interest.

Performance—Efficiency and Economy

5. a) Has the efficiency of the Program improved over time?

b) To what extent does the Program's delivery mechanism minimize the use of resources in realizing outputs and outcomes?

3.3.2 Lines of Evidence

The evaluation team used the following lines of evidence to assess the Program: document review, interviews, and a review of performance and financial data. Each of these methods is described in more detail below.

Document Review

The document review included the following types of documents: corporate documents, accountability and policy documents, Program inception and renewal documents, reports on plans and priorities, performance reports, speeches from the Throne, legislative documents and Program-specific documents. A list of documents reviewed is presented at Annex A.

Interviews

Nine interviews were conducted using interview guides developed for each of the interview groups described in Table 3. Provincial interviewees were chosen based on the number of events, the amounts reimbursed through the DFAA Program and to ensure regional representation.

Financial and Quantitative Data Review

An analysis of financial and quantitative data included Program performance documents such as the list of events per province/territory; a compilation of timeframes for producing audit reports (for a sample of six audits); a list of DFAA Program payments per year by province/territory; and expenditure data.

3.4 Limitations

The evaluation team interviewed representatives from only a sample of provinces/territories for the evaluation in order to calibrate the level of effort for the evaluation and to avoid interview fatigue given the planned policy consultations, which will be more comprehensive in nature. Where possible, the evaluation addressed this limitation by supplementing interview information with quantitative information and information from document review.

3.5 Protocols

During the conduct of the evaluation, PS Program representatives assisted in the identification of key stakeholders and provided documentation and data to support the evaluation. Collaborative participation greatly enriched the evaluation process.

This report was submitted to Program representatives and to the responsible Assistant Deputy Minister for review and acceptance. A management response and action plan was prepared in response to the evaluation recommendations. These documents were presented to the PS Departmental Evaluation Committee for consideration and for final approval by the Deputy Minister of Public Safety.

4. Findings

4.1 Relevance

4.1.1 Need for the Program

In order to establish if there is continuing need for the DFAA Program, the evaluation examined the larger context of trends in incidence, severity and costs of natural disasters in Canada and the continuing need for PS activities to administer the DFAA Program.

Document review indicates that "nationally and internationally, the frequency of natural disasters is increasing and that flooding is an important consideration in the types of disasters affecting Canadians. The cumulative effect of these disasters produces a significant personal, material and economic strain on individuals, communities and the fiscal capacity of all levels of governments."^{Note 6} An international insurance study indicates that "of the 304 catastrophic events that occurred in 2010, 167 were natural catastrophes", and that "2010 set a new record for the number of natural catastrophes."^{Note 7} This same study indicates that, in 1970, the number of natural catastrophic events was less than 50 while in 2010 the number was more than 150 with numbers generally trending upward since 1970. The report also shows insured catastrophe losses from weather-related events trending upward from about $5 billion in 1970 to more than $40 billion in 2009.^{Note 8} Another recent 2011 study that forecasts climate change trends for Canada indicates that "the coastal land area exposed to climate change–induced flooding from sea-level rise and increased storminess across Canada by the 2050s is roughly equivalent to the size of the Greater Toronto Area. The costs of flooding from climate change could be between $1 billion and $8 billion per year by the 2050s.^{Note 9}

Interview responses echoed these findings. Interviewees indicated that the number of natural disasters is trending upward basing their opinions on both amounts paid out in recovery efforts and data from the insurance industry and climate change statistics. The most prominent themes among interviewees were that the cost and magnitude of natural disasters are going up, and that DFAA-eligible events are increasing. Events that were thought to be anomalies such as the 1998 ice storm, the 2011 provincial floods (Manitoba, Saskatchewan, and Quebec) or the Slave Lake fires may now be occurring every two years.

According to the DFAA Program, overall, from 1970-1971 to 1995-1996, the Program paid out on average $10 million annually. Since 1996-1997, this average has been approximately $100 million annually. From 2000-2001 to 2009-2010, the DFAA Program had seven to nine events per year. In 2010-2011, the DFAA Program had 26 requests for reimbursement.

Some interviewees also noted that, in 2011, while the number of events is less significant, the magnitude is incredible in terms of cost. The Manitoba floods this year could be the second most expensive event in the history of the DFAA Program. Some interviewees also noted that atypical weather patterns and unusual events are increasing. In this regard, interviewees provided examples such as: very late thaws, unprecedented snowfall, rivers changing their course, and properties losses due to bank erosion. As illustrated in Figure 2 below, analysis of quantitative data confirms interviewee perceptions regarding the upward trend in the number of events.

Figure 2 – Trend in Number of Events Requesting DFAA Assistance

Figure 2 shows a trend going upwards by number of events per year since 1970.

1970: 2
1971: 2
1972: 1
1973: 4
1974: 7
1975: 2
1976: 6
1977: 0
1978: 3
1979: 4
1980: 1
1981: 1
1982: 3
1983: 4
1984: 5
1985: 0
1986: 2
1987: 3
1988: 5
1989: 3
1990: 5
1991: 2
1992: 1
1993: 3
1994: 1
1995: 7
1996: 5
1997:5
1998:5
1999:10
2000: 7
2001: 4
2002: 2
2003: 10
2004: 5
2005: 11
2006: 5
2007: 12
2008: 9
2009: 5
2010: 10

Renewal documents from 2006, 2007 and 2010 indicate that the DFAA Program has remained relevant to Canadians, thousands of whom have been affected by natural disasters. The Saguenay River Flood (1996), the Red River Flood (1997), and the Eastern Canada Ice Storm (1998) affected approximately 20% of the Canadian population.^{Note 10} In 2011, there continues to be disasters that overburden provincial resources. These activities include providing guidance, administering payments, preparing Orders-in-Council and providing federal audits of the requests for payment.

4.1.2 Alignment with Federal Government and Departmental Priorities

The evaluation sought to assess the degree of alignment with federal government and departmental priorities mainly through document review.

Program documents from 2010 indicate that, from a federal perspective, providing financial assistance to provinces/territories following natural disasters supports the Government of Canada's commitment to maintain a strong a resilient Canada by assisting community recovery efforts and ensuring that individual Canadians are able to access federal support through cost-sharing programs with provinces/territories^{Note 11}. Providing this support to provinces/territories continues to be a priority for the federal government as illustrated in the speeches from the Throne and in media releases from the Prime Minister's office. The Speech from the Throne in June 2011 stated that the "Government recognizes the resolve and courage shown by Canadian communities in the face of adversity (i.e. floods and wildfires) and it will stand by them in their times of need." While visiting the Quebec floods in the spring of 2011, the Prime Minister indicated that the federal government was "working closely with the province and affected communities to keep them safe and to help them rebuild their communities and lives" ^{Note 12}

Commitment to provide assistance to provinces/territories for prevention/mitigation is also illustrated by the commitment to implement a one-time cost-sharing program with provinces for the costs of permanent flood mitigation measures, taken specifically for the 2011 spring flooding in several provinces, for costs that would not otherwise eligible under the Disaster Financial Assistance Arrangements.^{Note 13} The Prime Minister also announced the Government is prepared to discuss a mitigation program that would apply to all provinces/territories to help enhance infrastructure to better withstand future floods.

Departmental planning and reporting documents illustrate that there is also alignment of the DFAA Program with departmental priorities. For example, one of the key priorities of the Department is to strengthen its emergency management function^{Note 14} "The Department develops and maintains the federal government's capacity to manage emergencies, monitors and coordinates the federal response and provides support to provinces/territories when federal assistance is needed".15 The DFAA is a key aspect of the recovery program and contributes to the PS strategic outcome of "a safe and resilient Canada".

In terms of the departmental Program Activity Architecture, the DFAA Program aligns with Program Activity 1.4, Emergency Management and contributes under program sub-activity 1.4.2 Emergency Response and Recovery.

4.1.3 Alignment with Federal Mandate

The evaluation sought evidence of accountability and authority to deliver the DFAA Program within existing legislation and policy in order to understand whether there is alignment with the federal mandate. The evaluation also studied the degree of duplication or synergy in the delivery of the DFAA Program.

The DFAA may be considered a "quasi-statutory" program in that the provincial and territorial governments expect financial assistance from the Government of Canada following significant natural disasters, and such assistance has been provided under the DFAA for more than 140 events since 1970.^{Note 16} More formally, authority for delivery of the DFAA Program is in accordance with the Emergency Management Act (S.C. 2007, c. 15). Under section 3, the Minister of Public Safety is responsible for coordinating, among government institutions and in cooperation with the provinces/territories and other entities, emergency management activities. Under section 4 (j), one of the Minister's responsibilities is to provide financial assistance to a province/territory if: (i) a provincial emergency in the province/territory has been declared to be of concern to the federal government under section 7, (ii) the Minister is authorized under that section to provide the assistance, and (iii) the province/territory has requested the assistance.

Authority to provide financial assistance rests with the Governor in Council who may, on the recommendation the Minister of Public Safety, make an order as required under the Emergency Management Act declaring a provincial emergency to be of concern to the Government of Canada and authorizing the provision of financial assistance to the affected province/territory.^{Note 17} Thus, each disaster event requires that an Order in Council be enacted to provide the authority for the DFAA Program to assist provinces/territories in their recovery efforts.

Duplication and Synergy

It is important to note that the DFAA Program works in synergy with provincial disaster financial assistance programs. Thus, there is no duplication between the provincial disaster financial assistance programs and the federal DFAA Program.

In terms of possible duplication at the federal level, foundation documents state that "The DFAA are the only standing disaster relief instrument for federal financial assistance to provinces/territories."^{Note 18} While it is the Government of Canada's primary instrument for responding to the financial needs of provinces/territories in the wake of major natural disasters, other forms of assistance are available from the federal government to address certain needs or aid specific economic and social sectors affected by a disaster not covered by the DFAA.^{Note 19}

Interviewees expressed that there remains the possibility of duplicative payments to end users within the sectors covered by the DFAA Program. They cited for example, programs delivered by Agriculture and Agri-food Canada that address farm and crop losses; programs delivered by Aboriginal Affairs and Northern Development Canada that cover broad emergency management areas including recovery; and programs delivered by Infrastructure Canada that provide for community infrastructure projects. The extent of this issue could not be quantified by interviewees and insufficient information was available to the evaluation to confirm the existence of this phenomenon; however, a scan of possible programs in departments/agencies mentioned by interviewees was conducted by the evaluators. A summary is listed in Annex B.

The DFAA Guidelines address the issue of duplicative payments stating that ineligible expenses include "expenditures for which provision is made for full or partial reimbursement to the province/territory under any other federal program existing at the time of the emergency, whether or not the province/territory accessed the Program".^{Note 20} The guidelines also make provision for the division of responsibility between Aboriginal Affairs and Northern Development Canada and PS in the case of off-reserve lands and First Nations reserve lands, clearly outlining where the DFAA Program will and will not apply.^{Note 21} Despite these provisions, interviewees noted that there is a possibility for duplicative payments from other federal organizations within the sectors that the DFAA Program serves.

Internal PS programs do not duplicate the DFAA Program. The Joint Emergency Preparedness Program focuses on expenditures related to preparedness and includes contribution funding for such things as training, emergency planning and exercises, the purchase of emergency response equipment or the reimbursement of costs to sustain urban search and rescue initiatives.

In addition, the planned one-time flood mitigation and disaster mitigation programs will not duplicate the work of the DFAA Program. These programs are purely preventative in nature; they are forward-looking and mitigate based on an assessment of risk and vulnerabilities. The DFAA Program is reactive to an actual event and the mitigation enhancement criteria under the Program is limited to cost-sharing for up to 15% to reimburse for the enhancement of damaged properties from a particular natural disaster.

4.2 Performance—Effectiveness

4.2.1 Support to Provinces/Territories

In order to assess whether support has been provided to provinces/ territories, the evaluation examined the support provided by PS in the administration of the Program. The evaluation also examined monetary support in terms of payments to provinces and territories and impact on the provincial financial burden.

Administrative Support

The DFAA Program Guidelines elaborate which provincial/territorial expenses are eligible for federal cost-sharing and how the funding arrangements are administered. Examples of expenses eligible for cost-sharing under the DFAA Program include: evacuation, food, shelter and clothing and provision of essential community services; security measures such as removal of hazardous materials, and removal of damaged structures for public safety; repairs to public buildings and infrastructure; restoration of small business and farmsteads, individuals' principal residences; and the cost of damage inspection, appraisal and clean-up.

Further to consultations that took place from 2001 to 2006, the DFAA Guidelines were revised in 2008 as follows:

• Clarification of the eligible disasters definition: the previous guidelines were not specific and had included non-natural disaster events. While these events may be worthy of some type of federal assistance, it was determined that the DFAA Program could only be accessed for specific types of events. The clarification was important for provinces/territories to know what to expect, and for the federal government to appropriately respond to major events.
• Expansion of the eligibility of small businesses and farms: provinces had concerns related to the required percentage of ownership (51% of income) to qualify for DFAA Program. For example, part-time farmers relying on two or three sources of income were not eligible. The new guidelines have included businesses in those types of situations.
• Inclusion of charitable, non-profit or voluntary organizations: it was determined that such organizations should receive financial assistance as they are affected by disasters in the same way as businesses and farms.
• Mitigative enhancements to damaged infrastructure (an additional 15% of eligible costs are covered): this is a new process and the Program is and will continue to adjust as it is used, with input from the provinces/territories. For example, in January 2011, it was announced that provinces could group different sites as strategic investments rather than doing it site by site, which was less efficient.

Interviewees were generally positive about the 2008 changes, although it is too soon to determine if they have actually facilitated response and recovery since no final requests for payment has been made yet (as of September 2011) under the new guidelines.

Most provincial representatives interviewed for the evaluation agreed that the DFAA Guidelines are clear, useful and accessible; that they have no problems applying them; and that they can submit and administer requests for payment. Some indicated that they use the DFAA Guidelines to model provincial disaster financial assistance programs.

However, some interviewees felt there was still room for improvement citing that it is difficult to navigate between the old and the new guidelines; that guidelines could be providing greater clarity; that the old guidelines offered more flexibility; that issues remain around the current ceiling for large business owners (e.g. feedlot owners); and that, in some cases, the new DFAA Guidelines and the provincial guidelines contain divergent definitions in some areas (e.g. small business and farms).

Mitigation provisions have been positively received although some raised the issue that the provisions only apply to damaged infrastructure therefore new infrastructure cannot be built and that the various administrative costs associated with these types of projects (e.g. engineering studies) are mostly incurred by the provinces. It was noted that there would be low uptake of the mitigation provision since reimbursements are higher for recovery expenses (the federal share is on average about 50%, and can go up to 90%) and more immediate, whereas mitigation benefits are long term and limited to cost-sharing for up to 15%.

In relation to noted gaps, Program representatives indicated that the Annual DFAA Managers Meetings provide a mechanism to air issues, share views and put forward proposals to address outstanding issues and challenges. Most provincial interviewees indicated that the DFAA Program is well managed and that a good relationship exists with DFAA Program staff. Comments included views that Program representatives are very collaborative, professional, and well organized. They also indicated that they are very open, and able to engage and work through issues. It was noted that the clarity and consistency of roles between PS Regional Directors and Headquarters could be improved. Half of interviewees indicated that there were communication issues such as the need for a more direct relationship with DFAA Program staff in Ottawa, rather than through PS regional offices or Audit Services Canada. They also commented on the need for regional offices to be more consistent in terms of response and knowledge level.

Overall Impact on the Provincial/Territorial Financial Burden

Provincial interviewees were asked whether the DFAA Program has had an impact on the provincial financial burden related to major natural disasters. All interviewees indicated that the support was "good"^{Note 22}, especially for large scale events and for smaller provinces. They mentioned that the DFAA Program plays an important role socially and politically and that the reimbursements are fair.

DFAA Program management indicated that the concept behind the funding formula is sound because it matches the Program intent; cost-sharing is on a progressive scale and contributions are proportionate to the disaster. However it was noted that, for larger provinces, the disaster has to be very large-scale before they can qualify for assistance citing a possible inequity because the threshold is too high.

A review of quantitative information supports the view that financial support has been provided to provinces and territories over the past five years. On average, the federal share represents about half of the cost of the disaster. As summarized in Table 4, actual payments to provinces/territories represent about $300 million over the past five years.

It is noted that Table 4 does not include outstanding committed payment amounts (payables at year end^{Note 23}) of approximately $119 million for years 2008-2009 to 2010-2011. This separation has been made so that Table 4 illustrates financial support actually received by provinces to ease financial burden as opposed to committed amounts they may eventually receive. The figures in Annex C include actual and committed amounts for the purposes of program costing.

4.2.2 Support to Canada for Response and Recovery

In order to assess the impact of the DFAA Program on response and recovery in Canadian communities, the evaluation gathered perceptions on the contribution that the DFAA Program has made in communities affected by disasters realizing that most of the impact is attributable to the actions of provinces/territories. The evaluators also reviewed audit reports in order to understand the types of ultimate recipients that have benefited from the DFAA Program reimbursements.

In terms of community response and recovery, about half of interviewees expressed the DFAA Program contribution as being "good". Comments included sentiments that, where the DFAA Program was involved, it was good for the community; that the DFAA Program helped individuals; and that some provinces could not accomplish recovery without federal involvement. More specifically, some interviewees stated that the impact has been significant on start-up businesses that had recently made initial investments; that the DFAA Program allows individuals to get "back on their feet" and lessens the severity on individuals; and that it allows the province to be more generous. A review of audit reports also demonstrates that Canadian communities are benefiting from DFAA reimbursements. Aside from provincial governments, the types of ultimate recipients noted in the audit reports were municipal governments, small businesses (including farms), not-for-profit organizations and individuals.

The other half of interviewees expressed that the DFAA Program contribution to community response and recovery was "low" and that this outcome was more attributable to the province than to the DFAA Program. Some stated that in more populous provinces, the vast majority of events do not attain the DFAA Program funding threshold; thus, over time, the province pays out much more in reimbursements to those affected by disasters.

4.3 Performance—Efficiency and Economy

4.3.1 Program Efficiency – Minimizing the Use of Resources

The efficiency of the DFAA Program was assessed by determining how the average Program administration ratio^{Note 24} compares to other PS programs. The evaluators also examined examples of measures taken to improve Program administration and efficiency and gathered perceptions related to Program administration.

A review of financial information combined with Program estimates approximates the annual administrative cost of the DFAA Program to have varied between $2.8 million and $4.2 million over the last five years. In terms of an administrative ratio, the DFAA Program compares very favourably to other PS transfer payment programs with an average Program administration to payment ratio of 4.8%. Details of the calculations are contained in Annex C.

The DFAA Program administration has implemented several measures to improve the Program. At the request of PS, Audit Services Canada developed a presentation to explain the provincial versus the federal audit role. Six presentations have been delivered to date. Another measure taken to improve Program administration and communication is the annual DFAA Managers Meetings, which began in 2007. The objectives of the meetings are to share best practices for providing financial assistance following natural disasters and to discuss administrative issues associated with the DFAA Program. Provincial representatives agree that these meetings are required and important. A review of meeting minutes indicates that the meetings advance specific issues and that actions have been taken to improve communication and achieve a more efficient process.

All provincial interviewees cited issues with the audit process. Some indicated that there is too much granularity in the audit process which extends the timeframe for audits as small amounts are contested. Some expressed concern of the loss of Audit Services Canada as a service provider because of the loss of "corporate memory" related to the audit of the DFAA Program.

Noted challenges to improving Program administration included:

• Providing accurate business planning and forecasting due to the number of unknown events each year;
• Meeting corporate reporting commitments because they are increasingly time- consuming for DFAA resources and no new resources have been added; and
• Potential loss of corporate memory with the dissolution of Audit Services Canada and potential retirement of the PS DFAA Program Director, who has been in place for 25 years.

As a final note, in a May 2011 PS internal audit^{Note 25}, the DFAA was identified as one of four sampling groups. The DFAA Program has implemented some of the action items related to the internal audit and continues to advance on others.

4.3.2 Program Economy – Delivery Alternatives

Program economy was assessed in the context of the dissolution of Audit Services Canada and interviewee comments regarding the timeliness of federal audits. Alternatives to the audit approach that might deliver the same outcome of providing financial support to provinces within a shorter timeframe (at a lower cost) are presented. The assessment of economy also includes a wider discussion, mainly based on document review, that situates the DFAA Program within other planned prevention/mitigation initiatives and their potential economic impact.  

When asked if DFAA payments were made in a timely fashion, most interviewees disagreed. There was dissatisfaction with the audit process in that it was onerous in terms of the threshold for materiality and amount of work to substantiate requests for payment; and it was too resource-intensive both at the federal and provincial levels. In some cases, provinces incur interest costs while the audits proceed slowly, and some events have taken years to be resolved. It is noted that these interviewees referred to the audit process used under the former DFAA Guidelines. Under the revised guidelines, the provinces will have to conduct their audit prior to submitting for federal audit. No request for payment has yet been submitted using the new guidelines.

Negative perceptions of provinces were challenged by the fact that the federal audit cannot begin until a request for payment has been received from a province. Thus, for example, if a province has a disaster in year one; recovery efforts are conducted up to year three; and in year six, a request for payment is made. The audit cannot begin until year six and in year seven, the federal auditors conduct the audit. Thus, in reality, it took less than a year from the time of the request for payment to make the payment. It was also noted that provinces sometime wish to dispute the audit findings to a low level of materiality and this can lengthen the process. A review of six Audit Services Canada audit reports and related Program information revealed that the average time to conduct an audit was 14 months^{Note 26}. In the absence of service standards, it cannot be determined if the audit process has, in quantitative terms, been "timely".

Some interviewees made the distinction between the former guidelines and the new audit requirements. They indicated that this should improve the process but that it is too early to determine. The first event to be administered under the new guidelines is forthcoming from an event in Manitoba so this may be the "test case" to understand how the new process works.

Having noted the above points, there was agreement among all interviewees that the audit process should change and that the current context provides the opportunity to update audit methodology to be more risk-based. This would mean that provincial auditors would have the responsibility for due diligence; and there would be assurances that the expenses are linked to provincial accounts. Under a risk-based process, the DFAA Program would verify if the provinces have the proper systems and controls in place. Thus, the federal auditors could reduce oversight where provinces have strong controls or processes in place, and spend less time with provinces that are strong. In addition, federal auditors could spend time educating provinces if the same errors appear in a number of audits; this approach would be more proactive and less time-consuming. As a final note, in January 2011, the Canadian Institute of Chartered Accountants adopted the International Financial Reporting Standards as the Canadian Audit Standards. The new standards should allow for streamlined audits of transfer payment programs since the standards are better aligned with information needs regarding public sector institutions.

The evaluation also examined the DFAA Program in the larger policy context of the planned programming related to mitigation/prevention (one-time disaster mitigation program and National Disaster Mitigation Strategy) in terms of economic impacts. In this regard, some interviewees, while recognizing the high cost of mitigation measures, expressed that it would be beneficial to do more completely preventative work (on works that were not impacted by a disaster). They stated that it was estimated that there was between $3 and $10 savings for every $1 invested in mitigation. They also cited non-quantifiable aspects such as how people's lives and whole communities are affected. Document review supported these sentiments stating that "Mitigation actions provide significant return on investment. Benefit-cost ratios for flood prevention measures in Australia, the United States and the United Kingdom are 3:1, 4:1 and 5:1, respectively. In Canada, $63.2 million invested in the Manitoba Red River Floodway in 1960 has saved an estimated $8 billion in potential damage and recovery costs. By emphasizing mitigation, Canada's built environment (e.g. public utilities, transportation systems, telecommunications, housing, hospitals and schools) can be designed to withstand the impacts of extreme natural forces."^{Note 27} Thus, it appears that mitigative measures implemented by the DFAA Program were insightful in this regard.

5. CONCLUSIONS

5.1 Relevance

There is a continuing need for the DFAA Program

The activities of PS in delivering the Program continue to provide a mechanism by which the Government of Canada helps Canadians in their "time of need", as it has done since 1970. Nationally and internationally, there is an upward trend in the number of major natural catastrophes, and the severity and costs of the events are increasing. In 2011, there continues to be disasters that overburden provincial/territorial resources.

While the DFAA Program is aligned with federal government and departmental priorities, there is emergent policy direction for consideration

Emergent policy direction includes the concept of prevention/mitigation which is being implemented in two emerging PS programs^{Note 28} and has been integrated into the existing Guidelines for the Disaster Financial Assistance Arrangements. In light of these emergent programs and policy direction, it will be important to position the DFAA Program in such a way as to balance the federal support offered by the combination of these programs; balance between short-term (response/recovery) and long-term (prevention/ mitigation) objectives will need to be considered.

It is the federal government's role to share in response and recovery efforts. According to interviewees, there exists the possibility for duplicative payments from other federal organizations

Legislative and policy authorities demonstrate that it is within the role of the federal government, under certain circumstances, to assist provinces/territories in response and recovery efforts. There is no duplication with regard to provincial disaster financial assistance programs or other PS programs. However, according to interviewees, there exists the possibility for duplicative payments from other federal organizations within the sectors that the DFAA Program serves.

5.2 Performance—Effectiveness

Through the efforts of PS staff that administer the Program and through financial reimbursements, provinces/territories have been supported in their response and recovery efforts for large-scale natural disasters.

Program guidelines have been modified based on engagement of provinces/territories and these changes have been generally well received. The new guidelines bring new challenges that will require monitoring, examination and engagement for resolution in order to support provinces/territories in their response and recovery efforts. Overall, the DFAA Program has supported provinces/territories in easing the financial burden of major natural disasters as the federal government has paid out about $300 million over the past five years or about half of the total provincial/territorial cost of eligible disasters.

The DFAA Program supports Canada in its response and recovery from events affecting the national interest

In terms of supporting Canadian communities, because the Program is accessed more often in less populous provinces, the Program has more of a direct impact in these provinces. Interviewees from these provinces highlighted the importance of the federal program in supporting provincial efforts benefiting individuals and business in the communities. More populous provinces noted a smaller impact on their communities due to the DFAA Program and attributed community benefits to provincial support.

5.3 Performance—Efficiency and Economy

The DFAA Program is administered efficiently but there are gaps in strategic management

In terms of efficiency, the average annual Program administration ratio (4.8%) compares very favourably to other PS transfer payment programs. Actions have been taken to improve program administration during the past five years. From a strategic management perspective, the program is at risk because it has no succession plan and there is the potential for loss of corporate memory, particularly at this juncture with the dissolution of Audit Services Canada and a Director who has been involved with the Program for 25 years.

The audit process requires improvement

The dissolution of Audit Services Canada provides an opportunity to update audit methodologies to a risk-based approach. A sample of six audits revealed an average delivery time of 14 months; however, because there are no service standards in place, it is not possible to state, in quantitative or absolute terms, whether the audit process has been timely.

Mitigation enhancements are poised provide economic benefits

There are documented economic benefits to prevention/mitigation programs and the DFAA Program has integrated mitigation enhancements into its programming; however, as noted above, going forward the DFAA Program must consider its role in the larger policy context and in relation to emerging programs.

6. Recommendations

We recommend that the Assistant Deputy Minister, Emergency Management and Regional Operations Branch implement the following recommendations.

1. Going forward, ensure proper positioning of the DFAA Program with respect to related emergent prevention/mitigation programs. Proper positioning will help ensure that the support offered by the DFAA Program does not create deterrents for provinces/territories in accessing these other programs, and should promote balance between short-term (response/recovery) and long-term (prevention/mitigation) economic objectives.
2. Ensure that the Program's management:
   1. Include other federal departments and agencies that operate in sectors similar to the DFAA Program (e.g. agriculture, infrastructure, and Aboriginal affairs) in future consultations to explore possible duplication issues; and
   2. Develop and implement a staff succession plan to protect program and corporate memory.
3. Review the federal audit process with a view to a more risk-based, streamlined approach to enhance the program delivery.

7. Management Response and Action Plan

Emergency Management and Regional Operations Branch will:

1. Ensure that future Emergency Management and Regional Operations policy and program activities include and document how the DFAA Program affects, and may be affected by, emergent prevention/mitigation programs.
2.  
   1. Include other federal departments/agencies such as Agriculture and Agri-food Canada, Aboriginal Affairs and Northern Development Canada, and Infrastructure Canada in program consultations, which are planned to occur prior to 2016, as part of DFAA Program renewal in order to ensure that there is no duplication among federal program criteria.  Program management will consult with other departments and agencies as required during the application of the DFAA to specific natural disasters to ensure that there is no program overlap. 
   2. Develop and implement a staff succession plan in 2012-2013 to preserve program expertise and corporate memory. 
3. Review the federal audit process to evolve to a more risk-based approach during the federal audit transition process.   The initiative will begin in 2011-2012, in consultation with the Internal Audit Directorate, and should be completed and implemented in 2012-2013. 

Annex A: List of Documents Reviewed

1. Aboriginal Affairs and Northern Development Canada, Evaluation of the Emergency Management Assistance Program, 2010.
2. Agriculture and Agri-Food Canada, A Guide to Agri-Recovery.
3. Audit Services Canada, Final Audit Reports.
4. Audit Services Canada, Presentation to Provincial Auditors.
5. Bevere, Lucia, Grollimund, Dr. Balz, and Rogers, Brian, Natural Catastrophes and Man-made Disasters in 2010: A Year of Devastating and Costly Events (Zurich: Swiss Reinsurance Company Limited, 2011).
6. Department of Justice Canada, Emergency Management Act (S.C. 2007, c. 15).
7. Infrastructure Canada Building Canada Fund http://www.infrastructure.gc.ca/prog/bcf-fcc-eng.html
8. Media Release June 6, 2011, Prime Minister's Website.
9. National Round Table on the Environment and the Economy, 2011, Climate Prosperity, Paying the Price: The Economic Impacts of Climate change for Canada.
10. Public Safety Canada, Terms and Conditions for the Payment of Contributions under the Disaster Financial Assistance Arrangements.
11. Public Safety Canada, Guidelines for the Disaster Financial Assistance Arrangements (2007).
12. Public Safety Canada, Revised Guidelines for the Disaster Financial Assistance Arrangements (effective January 1, 2008).
13. Public Safety Canada, Canada's National Disaster Mitigation Strategy (2008).
14. Public Safety Canada, Report on Plans and Priorities 2011-2012.
15. Public Safety Canada, Performance Measurement Framework 2011-2012.
16. Public Safety Canada, Agendas and Records of Decision, Disaster Financial Assistance Arrangements Managers Meetings
17. Speech from the Throne, June 2011

Annex B: Scan of Other Federal Programs

Annex C: Cost of the DFAA Program

TITLE-MISSING PDF Version (## KB)

* Regional Directors' time calculated at 20% for five provinces per year based on average involvement of provinces over the past five years.

** Audit costs for a given year are not necessarily related to final payments made in that same year because the audit timeframe can extend over more than one fiscal year.

Date modified:

2022-07-26