Evaluation of the Memorial Grant Program for First Responders 2021

PDF Version (904 KB)

Table of contents

• Executive Summary
• What we examined
• What we found
• Recommendations
• Management Action Plan
• 1. Background
• 2. Evaluation Purpose and Methodology
  • 2.1 Lines of Evidence
  • 2.2 Limitations
• 3. Findings
  • 3.1 Program Design
  • 3.2 Early Outcomes
  • 3.3 Efficiency and Economy
• 4. Conclusions
• 5. Recommendations
• 6. Management Action Plan

Executive Summary

This report presents the results of the evaluation of the Memorial Grant Program for First Responders.

What we examined

The purpose of the evaluation was to assess the implementation of the program, including its design, early outcomes, and efficiency and economy. The evaluation covered the three-year period from fiscal year 2018-19 to 2020-21, the timeline of which does not include the expansion of the program to include correctional, parole and probation officers. Multiple lines of evidence were collected and analyzed to ensure triangulation of findings.

What we found

1. There is a strong rationale for the program and its specific design. The program addresses gaps in existing mechanisms that provide financial recognition to families of fallen first responders.
2. While GBA+ was considered in program design, some barriers continue to exist due to limited awareness amongst some underrepresented groups.
3. The MGP instituted numerous mechanisms to support program delivery, including privacy and security protocol and MOAs with PTs, the effectiveness of which varied.
4. There is a clear and appropriate division of roles and responsibilities between PS and RCGT.
5. The program’s eligibility criteria were generally appropriate and incorporated sufficient flexibility to support the program’s inclusive mandate. Some concerns were expressed that program communications could include additional details on incident eligibility relating to occupational illness.
6. The application process is clearly described and includes reasonable flexibility, but limited awareness created barriers to access amongst some stakeholder groups.
7. Internal communications processes and the nature and extent of communications with applicants and recipients is sufficient, but engagement with external stakeholder groups could be improved.
8. Despite some initial delays, key program activities have been implemented and the program is on track to achieve its expected results. Best practices and facilitators of program performance include sufficient resources and early outreach. Limited broader awareness of the program constrained the achievement of outcomes.
9. While the MGP utilized significantly less of its budget than planned in 2018-19 and 2019-20, its funding was fully expended in 2020-21. The program was economically delivered but there are areas in which efficiency could be improved. Although a robust performance measurement strategy was developed, it was not formally implemented.

Recommendations

The Assistant Deputy Minister, Emergency Management and Programs Branch, should:

1. Review applicant and incident eligibility criteria to ensure clarity of terms and conditions, as well as alignment with program intent and the needs of family members of first responders.
2. Enhance engagement with and outreach to organizations and associations that represent first responders and PT counterparts to increase awareness and uptake of the program. A focus on organizations in rural, remote and Indigenous jurisdictions and volunteer first responders should be considered in addition to providing information on an ongoing basis and in multiple formats.
3. Review and monitor program resources to ensure they are sufficient to support delivery, particularly in the context of expanded eligibility criteria and increasing program uptake.
4. Review the nature and extent of performance data collected in light of current program needs and identify opportunities for further data collection opportunities to better support program reporting and management.

Management Action Plan

Program Management accepts all recommendations and will implement an action plan.

Background

In recognition of the critical role of first responders in protecting Canadians, the Memorial Grant Program for First Responders (MGP) provides a one-time lump sum payment to the families of first responders who have died as a result of their duties. The payment is free from federal tax and has a maximum value of $300,000. Grant eligibility is dependent upon meeting both eligibility conditions noted below.

The program’s five-year budget is $117.4M for fiscal years 2018-19 to 2022-23, which includes funding for 72 annual grant payments of $300,000, as well as program operating costs.

Evaluation Purpose and Methodology

The purpose of the evaluation was to assess the implementation of the program, including its design, early outcomes, and efficiency and economy. The evaluation covered the three-year period from fiscal year 2018-19 to 2020-21, the timeline of which does not include the expansion of the program to include correctional, parole and probation officers. Multiple lines of evidence were collected and analyzed to ensure triangulation of findings.

Lines of Evidence

Key Informant Interviews: Thirty-six interviews were conducted with Public Safety (PS) personnel, Raymond Chabot Grant Thornton Consulting Inc. (RCGT) representatives, representatives of provinces and territories (PTs) that did and did not sign memoranda of agreement (MOA), and representatives of organizations and associations that represent first responders (including paramedics, firefighters, police officers, and correctional, parole, and probation officers).

Document, Literature, and Performance Data Review: Program policies and procedures, business processes, communication documents, planning documents, and meeting summaries were reviewed, as well as performance data like website analytics, service standards data, and application statistics. The evaluation also included a scan of similar initiatives and a review of research and literature pertaining to Canadian line of duty deaths.

Limitations

The program cycle was not yet complete at the time of the evaluation. Therefore, the evaluation does not include full results on program performance. Additionally, it was not possible to reach representatives in three jurisdictions for interviews, therefore the perspectives and experiences of these jurisdictions may not be fully captured.

Findings

Program Design

Validity of Design Premises and Assumptions

The MGP was developed to recognize the service and sacrifice of first responders and their family members. It was designed to fill previous gaps and has provided a level of equity across the country.

As it is a national program, it provides a consistent approach for recognition regardless of the class of first responder or their jurisdiction. As well, the MGP provides the same financial amount regardless of the deceased first responder’s circumstances (e.g., salary, years of service, etc.) and the grant mechanism allows beneficiaries to use the funds without restriction. Other existing mechanisms are often taxable and can be used to offset other benefits a family may receive (e.g., Old Age Security, the Canada Child Benefit, etc.). Memorial grants have tax-free status under the Income Tax Act and the program instituted the MOA process with PTs to try to mitigate the risk of grants being reduced by regional policies or legislation or being used to offset other benefits a family may receive.

The broad nature of the eligible beneficiaries addresses gaps with other initiatives which typically provide benefits only to surviving spouses and dependent children (i.e., under the age of 19 years). Eligible beneficiaries of the MGP include the surviving spouse, children, siblings, parents, and the deceased’s estate.

The MGP incident eligibility criteria include a broad range of occupational illnesses. It also includes death assuccess result of psychological impairment as an eligible criterion, which acknowledges the significant impact of mental illness on first responders.

“The overwhelming majority of [other initiatives] in Canada…don’t handle suicide and mental health. From a policy perspective…acknowledging suicide is unique. It’s recognizing service and sacrifice but taking it to that extra level of recognizing that the job can take its toll on mental health.”

Other elements that distinguish the MGP include the following:

Grant Model - Program staff reported that the type of payment program reflected stakeholder feedback and administrative needs to provide a direct, one-time payment to eligible beneficiaries.

Use of Third-Party Contractor - PS contracted a third-party service provider (RCGT) to assess grant applications as the department lacked the highly specialized expertise needed to manage the type of information required of the applications. An additional benefit of using an external contractor is its arms-length status.

Application Process - Availability of multiple options for submission of grant applications supports the program’s goals around inclusivity and accessibility, as outlined in program documents. The program also implemented a pre-screening process to eliminate ineligible individuals from applying to the program, which internal interviewees described as an expectation management strategy.

Grant Value - The value of the grant payment was selected to reflect recognition provided to family members of fallen first responders in other jurisdictions, and to be similar to the amount provided to family members of Canadian military personnel.

Gender-Based Analysis Plus (GBA+)

GBA+ factors have been included in the design of the MGP. For example, a brochure was developed to increase awareness of the program and was translated into seven languages, including five Indigenous languages. The program also supports applicants with a range of needs by offering multiple application formats, including by phone, email, and mail as well as a teletypewriter to support applicants that are deaf, deafened, or hard of hearing.

In the early stages of the program’s development and implementation, PS conducted an engagement and outreach campaign to increase program awareness, which included travel to and engagement with individuals and organizations from rural, remote, and Indigenous communities.

No significant barriers inherent to the program’s design were identified that would limit access amongst underrepresented groups. However, several interviewees expressed concern that first responders in Indigenous and rural, remote communities may have limited awareness of the program which could limit their access to grants.

Suitability of Processes to Support Program Delivery

Privacy & Security - The MGP undertook rigorous privacy impact and security assessment activities prior to program implementation to ensure processes were in place to safeguard applicants’ personal information. Privacy considerations are highlighted to prospective applicants on the program website. These mechanisms are significant as the program stores sensitive medical information and internal interviewees reported that they were very satisfied with the privacy structure instituted.

MOA - In order to ensure grant beneficiaries are not negatively financially impacted by receiving a grant payment, the program implemented an MOA process with PTs. This is a formal agreement whereby signed jurisdictions agree to undertake efforts to prevent grant payments from being reduced or offset by other benefits or made taxable.

However, the MOA process was perceived to be somewhat ineffective. While a number of PS staff reported that the MOA was the only policy tool available to the program, several indicated that the non-binding nature of the agreement minimizes its utility and suggested that the administrative burden of signing an MOA prevented some jurisdictions from participating. The intent was that all PTs would sign an MOA prior to administering grants but there were four unsigned MOAs as of March 31, 2021. The lack of a signed MOA does not prevent a resident of those PTs from applying for the grant.

Roles and Responsibilities

As the third-party contractor, RCGT has overall responsibility for fielding applicant inquiries, managing applications (including receiving completed applications), collecting and reviewing information regarding application eligibility, and providing written comments regarding the assessment of applications to PS. Following RCGT’s provision of application assessments, PS has final decision-making authority over the approval of applications.

The division of roles and responsibilities between RCGT and PS supports a streamlined application process whereby RCGT is the key entry point for prospective applicants. Applicants must only submit documents to and communicate with one entity to complete the application process.

PS and RCGT staff generally agreed that their respective roles and responsibilities were appropriate, understood, and being fulfilled. Several interviewees expressed that the collaboration and partnership between the two parties was working effectively to complete application processing and approvals, and some reported that the smooth program operations was an indicator of this effective working relationship. A few interviewees reported that RCGT and PS had established regular communications (e.g., weekly meetings) which had further improved the working relationship in recent months.

Eligibility and Flexibility

The program’s applicant eligibility criteria reflect commonly referenced classes of first responders as well as evidence regarding high and/or increasing rates of line-of-duty deaths amongst Canadian police officers, paramedics, firefighters, correctional, parole, and probation officers. Most interviewees indicated that the applicant eligibility criteria were appropriate and several commented that there were no gaps. A few interviewees described the flexible terms and conditions as inclusive of a broad range of applicants (e.g., individuals in small communities that fulfill the role of first responders and volunteer first responders). However, a few interviewees expressed concerns regarding the expansiveness of the criteria (e.g., inclusion of the estate as an eligible beneficiary) and questioned whether it dilutes the intent of the program. Additionally, several representatives of first responder organizations expressed disappointment that the criteria excluded some sworn peace officers and deaths prior to April 1, 2018.

The incident eligibility criteria align to the most common causes of death amongst the eligible classes of first responders. Most interviewees indicated that the incident eligibility criteria were appropriate, and many expressed significant satisfaction with the inclusion of suicide in the incident eligibility criteria. While many interviewees were pleased with the program’s recognition of mental illness in first responders, several indicated that the burden of proof for suicide may be difficult for some applicants to obtain. Several interviewees also reported that the terms and conditions could include more specific details and clearer communications, for example regarding the years of service required to qualify for the benefits, the types of cancers and other illnesses that are eligible, and whether a Workers’ Compensation decision is required prior to applying for the MGP.

“If [PS] would just be more specific [about eligible occupational illnesses] that would help provide more information. Just be upfront from the beginning rather than having families getting excited about benefits but then don’t qualify.”

Suitability of Application Process

The application process is broadly described on the MGP website, including how to initiate an application and a high-level description of documents required. Additional details are available in the terms and conditions outlined on the PS website and in the MGP awareness brochure. Documentary evidence requirements appear reasonable and appropriate for a broad range of applicants. They are also comparable to the requirements of other similar initiatives. The program also implemented a pre-screening process to ensure that prospective applicants are eligible prior to completing the application process.

Flexibility - The program provides some flexibility in the documentary evidence required. The flexibility in the requirements for suicide-related deaths is particularly significant given the complexities surrounding post-traumatic stress injuries (PTSI).

Service Standards - The program’s service standards are to communicate a funding decision within 30 weeks of an application submission and issue a payment within 30 business days of receiving all required documentation. The program met both service standards 100% of the time in 2018-19 and 2019-20.

Awareness - Many external interviewees indicated that limited awareness of the program prevents some families from utilizing it, particularly family members of volunteer first responders and first responders in smaller jurisdictions (e.g., small provinces and territories and rural, remote communities). Some external interviewees suggested that increasing the availability of information and communications to first responder organizations and families could help to address access barriers related to limited awareness.

“There hasn’t been a whole lot of information released to us on the program or how it works. We know we are included but the process and how it works hasn’t been communicated to us. We have one member that fits into the category, but we are unaware of how [the] family would make the application.”

Sufficiency of Program Communications

Throughout the application assessment process, RCGT and PS maintain ongoing communications to fulfill their established roles as outlined in program process mapping documents. Internal interviewees were very satisfied with the level and nature of communications between PS and RCGT, noting that the two parties have established frequent and regular communications that have improved their relationship over time.

Communications efforts were made during program development to engage external stakeholder groups, but more targeted outreach may be needed to specific groups (e.g., paramedics and first responders in Quebec). Several external interviewees indicated that they had had little to no interaction with the MGP and that PS could undertake greater efforts to establish and maintain connections and communications with key stakeholder groups that have limited resources to seek out services and supports (e.g., volunteer first responders).

Interviewee suggestions for communication strategies and approaches included yearly or twice-yearly emails/notifications to external stakeholders regarding the existence of the program and using non-digital communications methods, such as posters that can be placed in common areas of first responder facilities (e.g., bulletin boards), cards and brochures that can be included in orientation packages, and information pamphlets that can be passed along to family members.

RCGT maintains continuous communications with applicants throughout the application process via telephone or mail, depending on the preference of the applicant. Communications are supported by templates and scripts that are tailored to various circumstances (e.g., out of country beneficiaries or beneficiaries in Quebec). Several interviewees were satisfied with the level and nature of communications to beneficiaries and unsuccessful applicants, identifying a high level of customer service provided by RCGT, as well as generally quick turnaround times in communications and application processing. Some interviewees indicated that there could be more upfront communications regarding eligibility criteria, including more specific information on eligible occupational illnesses.

“PS has done a good job of advertising on their website, but if they could reach out to stakeholders on an evergreen basis to remind people how to access the program that might be helpful.”

Early Outcomes

Implementation of Planned Activities

As of April 2019, the program was fully operational in terms of its three key pillars: service provider (i.e., RCGT) management, policy development, and program administration. There were some notable deviations from the planned implementation, including delayed onboarding of RCGT and limited uptake of MOAs across PTs.

One of the key early barriers to program implementation was the difficulty securing a suitable third-party service provider, which was addressed through targeted outreach to pre-qualified bidders. Implementation of MOAs was limited by the perceived redundancy of the policy tool due to its non-binding nature. Additionally, some PTs indicated that significant administrative effort was required of the MOA, which prevented some from being signed.

While the program had planned to administer 72 grants per year, it paid just 25 in 2018-19 and 44 in 2019-20. However, all 72 grants were administered in 2020-21. It is important to note that there is no time restriction on the payment of the grants, so deaths on or after April 1, 2018 remain eligible.

Progress Towards Expected Results

Program Awareness - Interviewees were generally familiar with the MGP, but several indicated that there was limited broader awareness of the program. Specifically, they reported that awareness varies across regions and first responder groups and indicated that PS could undertake efforts to increase awareness, particularly for paramedics and first responders with limited resources to seek out services and benefits (e.g., first responders in smaller jurisdictions, volunteer first responders, and Indigenous first responders). Suggested methods for increasing awareness included, but were not limited to, ongoing outreach to stakeholder groups to remind and/or inform them of the program, leveraging online methods (e.g., webinars), and distributing print advertisements and communications (e.g., magazine advertisements, brochures and posters for organizations and associations, etc.).

Program Management - The program was generally well-managed and becoming better established, as evidenced through the maintenance of relatively low administrative costs (7%), achievement of service standards in 2018-19 and 2019-20, and positive stakeholder perceptions of the program.

Grant Administration - Stakeholders did not perceive there to be any significant barriers for eligible beneficiaries to receive a grant in recognition of a service-related death. A majority of grants were paid to family members of fallen firefighters, followed by police officers and paramedics. On December 21, 2020, PS announced the expansion of the MGP to include the beneficiaries of correctional, parole and probation officers. As of March 2021, no grant payments had been issued for correctional, parole and probation officers as they were not yet eligible to apply until April 1, 2021. As of March 2021, most applications were from Ontario, British Columbia, Alberta, and Quebec.

Lessons Learned

Few unintended consequences (positive or negative) were associated with the MGP. However, some interviewees highlighted the emotional toll on individuals and families that are not eligible for the program, as well as the potential negative impact of the grant on family dynamics particularly when there are familial disagreements over the priority order of eligible beneficiaries.

A best practice that facilitated program implementation and delivery was the strong working relationship and partnership between RCGT and PS. Additionally, having sufficient resources for early engagement and outreach activities helped to establish some initial awareness of the program, and the robust privacy structure in place effectively safeguarded applicant information and streamlined application processing.

Factors that constrained the program’s achievement of outcomes included delays in contracting RCGT, the inability to process some grant applications in a timely manner as a result of COVID-19 public health restrictions, and limited program awareness which constrained overall uptake of the program in 2018-19 and 2019-20. Some interviewees also identified human resource issues that constrained the program’s performance, including limited full time equivalents (FTEs) to operate the program as planned and the transfer of the MGP to another branch within PS which resulted in the transfer of unfilled FTEs and a loss of corporate knowledge.

Efficiency and Economy

Use of Planned Program Resources

The MGP used just over one third (35%) of its available budget in 2018-19, which increased to 62% in 2019-20. The significant variance in use of funds was primarily attributable to the delayed program implementation due to third-party contracting delays, limited uptake of the program in 2018-19 and 2019-20, and a lapse of 10 grant payments in 2019-20 due to the impacts of COVID-19 public health restrictions. However, the 10 lapsed grant payments were paid in 2020-21. Staff turnover also contributed to lower-than-planned operational expenditures.

The program expended the full budget for contribution funds (i.e., memorial grants) for the 2020-21 fiscal year and lapsed a small percentage relating to operating expenditures. The program does not anticipate lapsing funds in future years. In fact, with the addition of correctional, parole and probation officers, there is a possibility of over 72 applications in a year, which may exceed the current budget allocation.

Efficiency of Resource Use

Despite lapses in program spending in 2018-19 and 2019-20, the program demonstrated efficiency through the annual increase in the number of applications received and paid which indicates an improvement in mechanisms and processes for managing grant applications.

Applications Processed, 2018-19 to 2020-21

Image Description

This graph presents the number of applications received and the number of applications paid per year from 2018-19 to 2020-21.

In 2018-19 there were 46 applications received and 25 applications were paid.
In 2019-20 there were 86 applications received and 44 applications were paid.
In 2020-21 there were 106 applications received and 72 applications were paid.

The program maintained reasonable administrative costs (7%) which were comparable to other federally delivered grants and contributions programs. Additionally, the program instituted a pre-screening process to assess prospective applicants for initial eligibility and prevent ineligible individuals from completing the application process; this mechanism also contributed to the program’s overall efficiency. As of March 31, 2021, no completed applications have been rejected. Additional efficiencies may be gained by considering the operational impacts of program expansion and subsequent need for additional resources to support ongoing delivery (i.e., reviewing the number of FTEs allocated to the program to ensure applications can be managed in a timely manner). It is important to note that each application could have multiple beneficiaries, resulting in the processing of multiple payments.

Performance Measurement

A comprehensive performance measurement strategy was developed in 2017 to assess the performance of the MGP on an ongoing basis. The strategy included 14 indicators to measure seven outputs and outcomes and outlined the frequency of data collection, targets and timelines for when targets will be achieved, the organization, unit and position responsible for data collection, and the data management systems used for collecting and storing data. The intent was that the strategy would be an evergreen document that would be reviewed each year.

There was limited evidence identified to suggest that the performance measurement strategy was implemented as intended. While some performance data was readily available (e.g., quarterly reports on application statistics), data on several measures outlined in the strategy did not seem to be captured (e.g., number of requests for information received, percentage of stakeholders who indicate program awareness). Some barriers related to performance measurement identified by interviewees included the absence of client satisfaction data and limited staff resources to adequately manage information generated by the RCGT case management system. The transfer of the MGP to another branch within PS may have also contributed to loss of corporate knowledge with respect to the performance measurement strategy.

“The ability to track and report on [case management] data is very strong. Every data element is captured – the challenge is the lack of resources.”

Conclusions

There is a strong rationale for the MGP. The program’s design was based on available evidence and aligned to federal priorities, which helped to address certain gaps by providing a tax-free grant that is consistent across regions and classes of first responders and includes death as a result of psychological impairment as an eligible incident criterion. While the program’s design is inclusive in nature and incorporated a GBA+ lens, limited awareness amongst underrepresented groups (including Indigenous first responders) was identified as a barrier to accessing the program.

The program design was generally appropriate, as it clearly delineated roles and responsibilities of PS and RCGT in terms of program delivery and incorporated several mechanisms to support the program’s overall objectives, such as robust privacy and security protocol and MOAs with PTs to prevent regional taxation or reductions of memorial grants paid. While applicant and incident eligibility criteria were inclusive and flexible, some concerns were identified that the broadness of terms and conditions could dilute the program’s intent and that they lack some clarity in terms of occupational illnesses covered.

In the first three years of the program, 141 grants have been paid. While the program is generally demonstrating progress towards achievement of expected outcomes as of 2020-21, there are ongoing concerns around program awareness, for example amongst volunteer first responders and first responders in small and rural, remote communities.

Some efficiencies were lost as a result of low program uptake and human resource constraints, but the program demonstrated economy through low administrative costs. A comprehensive performance measurement strategy was developed for the MGP, but was not strictly adhered to due, in part, to limited human resources.

Recommendations

The ADM, Emergency Management and Programs Branch, should:

1. Review applicant and incident eligibility criteria to ensure clarity of terms and conditions, as well as alignment with program intent.
2. Enhance engagement with and outreach to organizations and associations that represent first responders and PT counterparts to increase awareness and uptake of the program. A focus on organizations and associations in rural, remote and Indigenous jurisdictions and volunteer first responders should be considered in addition to providing information on an ongoing basis and in multiple formats.
3. Review and monitor program resources to ensure they are sufficient to support delivery, particularly in the context of expanded eligibility criteria and increasing program uptake.
4. Review the nature and extent of performance data collected in light of current program needs and identify opportunities for further data collection opportunities to better support program reporting and management.

Management Action Plan

Date modified:

2021-08-04