Evaluation of the Search and Rescue New Initiatives Fund

Evaluation of the Search and Rescue New Initiatives Fund PDF Version (1,160 KB)

Table of contents

• Executive Summary
• 1. Background
• 2. Evaluation Purpose and Methodology
  • 2.1 Lines of Evidence
  • 2.2 Limitations
• 3. Findings
  • 3.1 Relevance
  • 3.2 Design and Delivery
  • 3.3 Performance
  • 3.4 Efficiency
• 4. Conclusions
• 5. Recommendations
• 6. Management Action Plan
• Annex A – Logic Model
• Annex B – Annual Priorities Identified in the SAR NIF Call for Proposals

Executive Summary

This report presents the results of the evaluation of the Search and Rescue New Initiatives Fund (SAR NIF)

What we examined

The purpose of the evaluation was to examine the relevance, design and implementation, and performance (achievement of outcomes, efficiency and economy) of the SAR NIF. The evaluation covered the period from fiscal year 2015-16 to 2019-20. The evaluation used multiple lines of evidence, both qualitative and quantitative, to ensure triangulation of findings.

What we found

1. The SAR NIF funded 49 projects over the evaluation period and was found to provide necessary support for SAR services across Canada, particularly in the area of ground SAR. However, while the SAR NIF supports the priorities and response and prevention objectives associated with national SAR, there is no formally defined framework coordinating national SAR policy. As such, it is difficult to assess the SAR NIF’s strategic role in supporting Public Safety’s coordination of national SAR.
2. Governance structures are in place although the SAR NIF Terms and Conditions were last updated in 2010, prior to the 2015 transition to Public Safety. Concerns were raised with respect to eligibility and application requirements as well as administration of the application process and outreach efforts to promote awareness of the call for proposals. Some respondents suggested that the Merit Board could be more representative of the SAR community and Canadian society to ensure a diversity of perspectives, similar to the ‘whole-of-community’ approach recently adopted by the National SAR Advisory Council.
3. There is inconsistency in the collection and reporting of SAR related data across the country and there is no centralized or standardized accounting of search and rescue activities across Canada. Public Safety’s management of performance data and GBA+ related data was limited and project results were not found to be shared within the SAR community. The unavailability of data limits a full picture of the existing capabilities and needs of search and rescue in Canada.
4. Notwithstanding, interview respondents overwhelmingly affirmed the SARNIF’s positive contribution to Canada’s SAR community.

Recommendations

The Assistant Deputy Minister of the Emergency Management and Programs Branch should:

1. Develop and communicate the priorities of the National Search and Rescue Secretariat (NSRS) and overall policy direction for SAR support within Public Safety Canada, and ensure that the priorities and objectives of the SAR NIF are aligned.
2. Explore ways to improve collection and sharing of project data to support national SAR policy and enhance knowledge management at the national level.
3. Improve the SAR NIF’s application and approval processes to ensure sufficient outreach, support and accessibility.

Management Action Plan

Program Management accepts all recommendations and will implement an action plan.

1. Background

Canada has one of the world’s largest areas, covering 18 million square kilometers of land and water, more than 240,000 kilometers of coastline, three oceans, three million lakes (including the Great Lakes), and the St. Lawrence River system.

The challenges associated with search and rescue for such an enormous area are compounded by the varied and often austere terrain, extreme weather conditions and low population density that characterize many parts of the country, making Canada a difficult environment in which to conduct search and rescue operations.

Canadian search and rescue (SAR) services are a shared responsibility amongst federal and provincial/territorial governments in collaboration with municipalities, volunteers, the private sector and other partners (Table 1). This wide-ranging list of authorities and partners, combined with Canada’s expansive geography, diverse topography and contrasting climates, make elements such as cooperation, collaboration and capacity essential to the successful provision of SAR services.

The Search and Rescue New Initiatives Fund (SAR NIF) was established as a contribution program in 1988 by the Department of National Defence. It supports the objectives of the National SAR Program (NSP) (Annex A). Both the NSP and the National SAR Secretariat (NSS) were created in 1986 in response to recommendations from the Royal Commission on the Ocean Ranger Disaster.

The NSS is responsible for the development and coordination of overall SAR policy in consultation with federal, provincial and territorial SAR partners, as well as supporting and promoting the activities of the NSP as a means of fostering interoperability and coordination within the SAR community and its trained volunteers.

The NSP is a Canada-wide horizontal program that aspires to integrate organizations and resources involved in the provision of search and rescue (SAR) services to Canadians. The Program’s vision is grounded by dual objectives – response and prevention – and has the following vision: “A Canada where the critical importance of Search and Rescue is reflected in a multi-jurisdictional approach to promoting individual, collective and organizational behaviour that minimizes the risk of injury or loss of life while maintaining timely and effective response services.”

The NSS, NSP and the SAR NIF were located at the Department of National Defence until 2015 when they were transferred to Public Safety Canada.

While the NSP continues to be associated with national SAR coordination, Public Safety is currently exploring options to develop a modernized and formalized framework for SAR in Canada.

Responsibility for the SAR NIF resides within Public Safety’s Emergency Management and Preparedness Branch. 

To be eligible for the SAR NIF, a project proposal must demonstrate that it contributes to one of the core objectives of the national SAR program (i.e., improved response capabilities or incident prevention). In addition, a proposal must address one or more of the priorities identified in the SAR NIF call for proposals. These annual funding priorities (see Annex B) are established by Public Safety in consultation with the Interdepartmental Committee on Search and Rescue, the provincial/territorial Ground Search and Rescue Council of Canada, and with input from the three national SAR volunteer associations.

Under the SAR NIF Terms and Conditions, any organization, association, agency or person in Canada can apply for funds by submitting proposals to one of the following federal managing departments and agencies: Canadian Coast Guard, Fisheries and Oceans Canada; Canadian Forces, Department of National Defence; Meteorological Service of Canada, Environment Canada; Parks Canada Agency; Royal Canadian Mounted Police; Transport Canada; other federal departments with SAR responsibilities, as designated by Public Safety; the National Search and Rescue Secretariat; or Provincial or Territorial governments.

Project proposals are reviewed by a Merit Board that is managed by Public Safety and comprised of representatives from federal government departments and agencies with SAR responsibilities, relevant national volunteer organizations and provincial and territorial SAR counterparts.

Public Safety compiles the scores submitted by each member of the Merit Board and based on the results, develops a list of recommended projects which is then forwarded to the appropriate authority for final approval. Public Safety then determines, based on available funds, which projects will be funded. The annual SAR NIF budget is $7.6M. With multi-year projects ongoing, the availability of funds for new projects varies from year to year.

2. Evaluation Purpose and Methodology

The purpose of the evaluation was to examine the relevance, design and implementation, and performance (achievement of outcomes, efficiency and economy) of the SAR NIF. The evaluation covered the period from fiscal year 2015-16 to 2019-20. The evaluation used multiple lines of evidence, both qualitative and quantitative, to ensure triangulation of findings.

2.1 Lines of Evidence

Data collection for this evaluation included the following lines of evidence:

Interviews and survey:

Twenty-five interviews were conducted with program staff, SAR NIF partners, stakeholders and funding recipients. A survey among nineteen funding recipients was utilized to further gain perspective on the program.

Literature and program document review:

Literature (e.g. academic research, media, governmental and independent reports) and documents (e.g. project proposals, recipient activity reports, final activity reports) were reviewed.  

Performance and financial data:

Performance data from the program was reviewed. Program financial data was analysed.

2.2 Limitations

There was a lack of data pertaining to SAR activities undertaken across Canada, creating challenges when assessing relevance and continued need.

Repeated efforts to secure interviews with officials from some federal departments and provincial/territorial governments were unsuccessful and their perspectives were unable to be considered in the report.

3. Findings

3.1 Relevance

Given its mandate to improve search and rescue response capabilities and incident prevention, the SAR NIF aligns with governmental priorities and departmental objectives to strengthen national emergency preparedness to help prevent, mitigate, prepare for, respond to, and recover from all-hazards events. This supports Public Safety's ultimate outcome of building a safe and resilient Canada.

The positive contribution made by SAR NIF to the Canadian SAR community was overwhelmingly affirmed by both interviewees and survey respondents. It was noted that without the funding provided by SAR NIF, there would be a reduction in the response and prevention capabilities of the search and rescue community. It was felt that ground SAR organizations would be particularly impacted as the SAR NIF is viewed as an essential funding source for equipment and training. Over the evaluation period, 65% of SAR NIF-approved projects (approx. $15.5 million) supported ground SAR activities (Table 2).

Assessment of continuing need was hindered by a lack of consistency in the collection and reporting of SAR-related data across the country which limits a full picture of the existing capabilities and needs of SAR in Canada. The 2013 Quadrennial Search and Rescue Review prepared by the NSS (DND) found that information on the rates and nature of SAR incidents varied considerably across jurisdictions and that there was “no centralized or standardized accounting of SAR across Canada.” These concerns were echoed in the 2013 Spring Report of the Office of the Auditor General and the 2015 Final Report of the Evaluation of the SAR NIF, conducted by DND which cited limitations in the SAR NIF’s data collection and knowledge management processes with respect to funded projects. There have been no noticeable improvements identified over the evaluation period.

3.2 Design and Delivery

3.2.1 Governance

Sufficient governance structures were found to be in place for the SAR NIF and provided through two main instruments: the SAR NIF Terms and Conditions (last updated in 2010) and the SAR NIF Merit Board which reviews and scores project proposals. Terms of Reference for the Merit Board were updated in 2018.

The Merit Board’s mandate is to evaluate, score and prioritize the various SAR project proposals submitted by SAR organizations in response to the annual call for proposals. Its membership includes one representative from each of the partnering federal departments and agencies with SAR responsibilities, a Provincial/Territorial representative from the Ground Search and Rescue Council, and one member representing the three national volunteer associations.

Merit Board members from managing Federal departments and agencies are expected to provide clarification and details on proposals that are supported by their own department and the provincial and territorial representative and the SAR national association representative are expected to provide some insight into the needs and issues of the regions.

Respondents suggested that Merit Board membership could be more representative to ensure a diversity of perspective and SAR expertise from various jurisdictions and Canadian society as a whole. 

While SAR NIF governance is in place, it is unclear as to what larger policy framework the SAR NIF feeds into or is informed by. Documents and interviews show that although the SAR community possesses a certain degree of understanding of the NSP, it has never been formally defined. Public Safety has acknowledged this gap and is actively engaged in developing a framework for national SAR. However because of the diversity of jurisdictions and organizations with SAR responsibilities, addressing this need is viewed as a significant challenge and a national, strategic-level policy document has not been created or approved.

The main Federal-Provincial-Territorial body that would address SAR is the Senior Officials Responsible for Emergency Management (SOREM). Given SOREM’s main area of concern is crisis management and preparing for and responding to disasters, there is limited opportunity to discuss a national SAR Framework. 

Given these higher-level considerations, it is difficult to assess the SAR NIF’s strategic role in supporting Public Safety’s coordination of national SAR. Interviews suggest a lack of consensus concerning the SAR NIF’s mandate. This included discussions on whether the SAR NIF should support national or pan-regional initiatives or smaller local/regional capacity-building projects.

One key improvement in national SAR oversight has been the 2017 establishment of the National Search and Rescue Advisory Council (NSARAC). It is endorsed by both federal and provincial/territorial governments to provide strategic advice on SAR policy and program matters. Membership includes federal departments, provinces and territories, Indigenous organizations, the tri-services (police, fire and paramedics) and the three national volunteer associations. It is unclear as to how the NSARAC impacts the SAR NIF, as it is not included in the current governance model.

3.2.2 Synergies

The SAR NIF funded a total of seven projects with the three national volunteer organizations supporting SAR response in Canada: two projects with the Civil Air Search and Rescue Association (CASARA), one project with the Canadian Coast Guard Auxiliary, and four projects with the Search and Rescue Volunteer Association of Canada (SARVAC). Outside of project specific SAR NIF funding, the Coast Guard Auxiliary and CASARA receive funding from the Department of Fisheries and Oceans and the Department of National Defence, respectively. SARVAC has a broad contribution agreement with Public Safety that was not assessed as part of this evaluation.

There were no comparable federal grant or contribution programs found to be funding SAR response or incident prevention projects with ground or aeronautical SAR organizations. Transport Canada manages the Boating Safety Contribution Program which supports projects that promote public awareness and knowledge about boating safety and regulatory compliance. Over the evaluation period, the SAR NIF funded eight projects related to boating safety. The evaluation was unable to identify synergies or links between these two programs.

3.2.3 Gender-based analysis plus

While evidence of GBA+ considerations in SAR NIF design and implementation are limited, Northern SAR and Indigenous SAR were identified as priority areas in the 2017-18 and 2018-19 call for proposals. Four proposals corresponding to these priorities were funded. In total, the SAR NIF funded six projects that specifically impacted Indigenous populations over the evaluation period. The SAR NIF also funded a project designed to enhance national SAR awareness with respect to Autism Spectrum Disorders.

A recent scan of Canada’s SAR Risk Environment identified the following GBA+ considerations: a lack of knowledge of SAR capabilities or capacities in many Indigenous communities; increases in missing persons living with dementia and mental illness; and increased participation of new Canadians in recreational activities.

Currently SAR NIF projects do not collect or track GBA+ related data. Given identified risk factors and the lack of related data, the SAR NIF’s ability to assess needs and target funding to underrepresented or at-risk populations is limited. The NSS is currently exploring opportunities to enhance the availability of GBA+ data with other federal partners.

Approved projects supporting Indigenous populations were found to significantly favour incident prevention activities over response-based initiatives with five of six projects supporting the former. Interviews and document review suggest that encouragement of community-level project proposals addressing response capabilities is hindered by limitations in data, outreach and the capacity for communities to commit resources to the SAR NIF application process.

Another significant potential barrier to Indigenous communities and other underrepresented populations concerns access to current national SAR governance structures. Indigenous Services Canada is not a member of the SAR NIF Merit Board and as such, it is not a recognized Managing Department or Agency that can provide support and sponsorship for proposals.

Public Safety has made efforts to improve Indigenous participation and outreach in SAR governance. The establishment of the National SAR Advisory Council was highlighted as one example. As part of its ‘whole-of-community’ approach, several Indigenous organizations are members of the Council.

3.3 Performance

3.3.1 Prevention and response capabilities

Over the course of the evaluation, 49 projects were funded by SAR NIF, of these 18 (37%) were under the prevention objective, including two projects that supported research and development. 13 projects were national in scope with 11 identified as making innovative contributions within the SAR community including the development of new technologies and enhanced outreach to underrepresented populations.

Interviewees highlighted the success of AdventureSmart, a national prevention education initiative which combines online and on-site awareness with targeted grass-roots outreach to try and reduce the number and severity of SAR incidents. Its website, adventuresmart.ca, is now managed by SARVAC with content developed by Public Safety. It is delivered in partnership with not-for-profit organizations with the assistance of federal, provincial/territorial and municipal governments.

Over the evaluation period, the SAR NIF funded two initiatives that contributed programming to AdventureSmart and another project that explored potential collaborative opportunities with Indigenous communities. The synergy between the SAR NIF and AdventureSmart was regarded as strengthening incident prevention capabilities.

Of the 31 projects under the objective of response, the majority (84%) supported ground SAR, with a strong focus on funding for equipment or training for provincial organizations. There were a limited number of response-focused projects that were cross-jurisdictional.

Many respondents were able to articulate the impact their funded project had on improving ground SAR response capabilities within their jurisdiction. Examples of the results of funded projects included ground SAR training in remote areas, purchase of radio communications systems, and incident management logistic support and training. While these projects improved response capabilities within local or provincial SAR teams, there is limited data on the impact of national SAR response capabilities.

3.3.2 Collaboration and communication

The SAR NIF facilitated partnership building within the SAR community. The application process requires applicants to engage a federal or provincial organization as a partner at the outset. This was designed to promote collaboration between larger federal agencies and smaller organizations. There is no performance data related to the impact or continuation of these partnerships.

One of the stated objectives of the SAR NIF is the facilitation and sharing of best practices in response and incident prevention. Public Safety has not taken an active role in the sharing of best practices or innovative projects stemming from funded projects.

Over the evaluation period, there have been challenges in coordinating SAR-related data across jurisdictions. The SAR NIF supported 13 projects related to standardizing and reporting data and information management. Public Safety is currently exploring options to enhance knowledge management at a national level.

Many respondents felt that sharing the results of SAR NIF-funded projects would be beneficial and could improve the overall coordination amongst SAR organizations across Canada. It was also noted that summaries of approved projects, published perhaps on a web portal, would be helpful for organizations looking for innovative approaches or opportunities for collaboration.

The annual SARScene conference, organized by the NSS until 2017, was viewed as a beneficial venue for facilitating collaboration and communication within the SAR community. Documents and interviews highlighted the showcase of SAR NIF projects and Public Safety-hosted workshops clarifying SAR NIF application and reporting processes as positive elements featured at the conference. In 2019, SAR NIF funds were made available to SARVAC to relaunch a national SAR conference.

3.4 Efficiency

3.4.1 Economy and administration

Review of financial data found that the SAR NIF lapsed funds in each of the fiscal years under review (Table 3). This included an overall 22% variance between allocated and actual expenditures for contribution funding, and a 27% variance in the operating expenditures, including salary.

Reduced staffing and frequent changes in personnel resulted in limited support to funding applicants and recipients and inconsistent advice provided by the program. Interviews and survey respondents indicated that the application process and level of effort required was discouraging to potential applicants.

Over the evaluation period, SAR NIF calls for proposals received an average of 20 to 30 applications. However, this figure dropped to 17 in the final year of the evaluation and 13 in response to the 2020-21 call for proposals. While the latter is outside the scope of this evaluation, it is important to note as it may indicate an emerging trend.

Challenges were identified with the application guidelines and process for the SAR NIF. Each province, territory, or volunteer SAR organization, is limited to one application per call for proposals. This could potentially limit the submission of innovative and/or collaborative projects.

The short time frame for applications was also seen as a barrier for smaller organizations and those interested in developing horizontal submissions. These organizations rely on volunteer resources to prepare applications, and some viewed the process as having a low success-to-effort ratio. Adding to the issues around the limited time for application, there is limited outreach from Public Safety to inform potential applicants of the availability of funds.

The list of Managing Departments and Agencies, who can sponsor applications, was viewed as too narrow. As previously discussed, while the need for improved outreach to Indigenous communities is acknowledged and Indigenous SAR has been included in SAR NIF priorities, ISC is not a recognized Managing Department.

Interview and survey respondents were invited to make suggestions for streamlining or improving the SAR NIF. Key areas of consensus focused on streamlining the application process by establishing predictable timelines and improving outreach to ensure widespread awareness of the call for proposals and annual priorities to permit sufficient preparation time. It was also noted that a simplified process would encourage more proposals and potential innovators from within the SAR community. 

In terms of administration, the management of performance data was found to be limited. While the collection and reporting of performance data is required from funding recipients, and these reports are provided to Public Safety, project results were not found to be shared within the SAR community and no evidence was found that they were used to support policy making or SAR priority setting. These concerns were raised previously in the 2013 Office of the Auditor General Spring Report and the 2014 DND Evaluation of the SAR NIF.  

4. Conclusions

The SAR NIF funded 49 projects over the evaluation period and was found to provide necessary support for SAR services across Canada, particularly in the area of ground SAR. However, while the SAR NIF supports the priorities and response and prevention objectives associated with national SAR, there is no formally defined framework coordinating national SAR policy. As such, it is difficult to assess the SAR NIF’s strategic role in supporting Public Safety’s coordination of national SAR.

Governance structures are in place although the SAR NIF Terms and Conditions were last updated in 2010, prior to the 2015 transition to Public Safety. Concerns were raised with respect to eligibility and application requirements as well as administration of the application process and outreach efforts to promote awareness of the call for proposals. Some respondents suggested that the Merit Board could be more representative of the SAR community and Canadian society to ensure a diversity of perspectives, similar to the ‘whole-of-community’ approach recently adopted by the National SAR Advisory Council.

There is inconsistency in the collection and reporting of SAR related data across the country and there is no centralized or standardized accounting of search and rescue activities across Canada. Public Safety’s management of performance data and GBA+ related data was limited and project results were not found to be shared within the SAR community. The unavailability of data limits a full picture of the existing capabilities and needs of search and rescue in Canada.

Notwithstanding, interview respondents overwhelmingly affirmed the SARNIF’s positive contribution to Canada’s SAR community.

5. Recommendations

The Assistant Deputy Minister of the Emergency Management and Programs Branch should:

1. Develop and communicate the priorities of the National Search and Rescue Secretariat (NSRS) and overall policy direction for SAR support within Public Safety Canada, and ensure that the priorities and objectives of the SAR NIF are aligned.
2. Explore ways to improve collection and sharing of project data to support national SAR policy and enhance knowledge management at the national level.
3. Improve the SAR NIF’s application and approval processes to ensure sufficient outreach, support and accessibility.

6. Management Action Plan

Annex A – Logic Model

Annex B – Annual Priorities Identified in the SAR NIF Call for Proposals

Annual Priorities by Fiscal Year

2015-16

• Priority 1: Interoperability and Coordination
• Priority 2: Information Management and Data Standardization
• Priority 3: Strengthening prevention
• Priority 4: Supporting volunteers

2016-17

• Priority 1: Interoperability and Coordination
• Priority 2: Information Management and Data Standardization
• Priority 3: Strengthening prevention

2017-18

• Priority 1: Standardize reporting and improve data management to inform decision making.
• Priority 2: Strengthen National SAR Program
• Priority 3: Emphasize and effectively coordinate SAR prevention efforts across NSP partners
• Priority 4: SAR in Northern and Remote Areas
• Priority 5: Coordinated Communications

2018-19

• Priority 1: Strengthen the National SAR Program
• Priority 2: Standardize Reporting and Improve Data Management
• Priority 3: Emphasize and effectively coordinate SAR prevention efforts
• Priority 4: Indigenous SAR

2019-20

• Priority 1: Strengthen the national coordination of SAR
• Priority 2: Emphasize and effectively coordinate SAR Prevention efforts

Date modified:

2022-07-22