Evaluation of the Initiatives to Address Post-Traumatic Stress Injuries (PTSI) Evaluation Report

Evaluation of the Initiatives to Address Post-Traumatic Stress Injuries (PTSI) Evaluation Report PDF Version (1,298 KB)

Table of contents

• Executive Summary
• 1. Background
• 2. Engagement Purpose and Methodology
• 3. Findings
  • 3.1 Relevance
    • 3.1.1 Continued Need
  • 3.2 Effectiveness
    • 3.2.1 Implementation
    • 3.2.2 The CIPSRT Hub
    • 3.2.3 ICBT Pilot
  • 3.3 Gender-Based Analysis Plus (GBA Plus)
  • 3.4 Efficiency
    • 3.4.1 Initiative resources
    • 3.4.2 Challenges
• 4. Conclusions
• 5. Recommendations
• 6. Management Action Plan
• Annex A – Other Available Programming

Executive Summary

Public safety officers play a critical role in keeping Canadians safe, which often exposes them to traumatic incidents that can lead to post-traumatic stress injuries (PTSI). Recognizing the need to support the mental health of public safety officers, the Government of Canada announced “Supporting Canada’s Public Safety Personnel: An Action Plan on Post-Traumatic Stress Injuries” in April 2019. Under this Action Plan, Public Safety Canada funded the Canadian Institute for Public Safety Research and Treatment (CIPSRT) at the University of Regina to develop and pilot an Internet-based Cognitive Behavioural Therapy (ICBT) pilot for public safety officers, as well as establish a Knowledge Exchange Hub for research related to PTSI among public safety officers.

What we examined

The evaluation covered the Initiatives’ activities under the Contribution Agreement with CIPSRT from January 2019, when the agreement was signed, to April 2022.

What we found

1. PTSI is a serious issue among the public safety community, and there is a continued need for the type of programming and research provided by the Initiatives. Research related to PTSI and public safety officers – its prevalence, its impacts, what treatments are efficacious – is in its early stages with numerous gaps in knowledge that remain to be filled. CIPSRT address a need by providing a national research hub that has public safety officers’ mental health as a focus.
2. The ICBT pilot has a unique set of attributes, including being designed for public safety officers, using clinicians that specialize in treating trauma in the public safety setting, being confidential, and being free to access.
3. The CIPSRT Hub has a growing reputation as a hub for public safety officer mental health research and has made strides in facilitating knowledge translation and mobilization, although knowledge products could be made more accessible to the public safety community in terms of content and delivery.
4. The ICBT pilot’s early results indicate that it is effective in assisting public safety officers who are experiencing the effects of trauma, and there is  desire to see it expand to other jurisdictions.
5. More could be done to maximize the potential of each Initiative by increasing awareness of them and broadening their reach within the public safety and research communities. The nature of the Initiatives mean that there are many opportunities to incorporate GBA Plus. The current performance measurement processes could be strengthened to allow for a more fulsome assessment of progress.

Recommendations

The Assistant Deputy Minister, Emergency Management and Programs Branch, should, as part of renewed/continued funding:

1. Explore methods to support the sustainability and expansion of the ICBT pilot by collaborating and coordinating with provinces, territories, and relevant organizations.
2. Ensure resources provided to CIPSRT are sufficient for the communication and dissemination of knowledge products and consider working with federal partners (e.g., Statistics Canada) to support the development of a national baseline of information.
3. Revisit and revise the current approach to performance measurement, including the development of a logic model, a performance measurement framework, and an approach that better incorporates GBA Plus.

1. Background

Public safety officers play a critical role in keeping Canadians safe, which often exposes them to traumatic incidents that can lead to post-traumatic stress injuries (PTSI). These injuries include clinically diagnosed post-traumatic stress disorder (PTSD) and other mental health issues, such as anxiety, depression, and substance abuse. It is estimated that approximately 20% to 45% of public safety officers have PTSD or some other mental health issue.

Recognizing the need to support the mental health of public safety officers, the Government of Canada announced a joint initiative in April 2019 between the Public Safety portfolio (including Public Safety Canada [PS]) and the Health portfolio, “Supporting Canada’s Public Safety Personnel: An Action Plan on Post-Traumatic Stress Injuries” (the Action Plan). The Action Plan is intended to provide national leadership and coordination in efforts to “support research, prevention, early intervention, stigma-reduction, care and treatment for all types of public safety personnel, all across the country” while recognizing that public safety officers work in many jurisdictions that each have their own responsibility for their employees’ mental health.

Public safety officers: A term that broadly encompasses front-line personnel who ensure the safety and security of Canadians across all jurisdictions, such as: the tri-services (i.e., paramedics, firefighters, and police, including the RCMP), correctional and border services officers, operational and intelligence personnel, emergency managers, Indigenous emergency managers, search and rescue volunteers, and dispatch officers. Notably, the research community employs the analogous term “public safety personnel.”

The two initiatives covered by this evaluation were provided funding from 2018-19 to 2022-23 through a contribution agreement with the University of Regina for certain activities of the Canadian Institute for Public Safety Research and Treatment (CIPSRT) (hereinafter, collectively referred to as the Initiatives). The two initiatives are as follows:

CIPSRT Knowledge Exchange Hub (CIPSRT Hub): $5 million was allocated over five years for establishing the CIPSRT Hub for research related to PTSI among public safety officers. A key activity of the CIPSRT Hub is to establish and maintain the Academic, Research, and Clinical Network (ARC Network), which is intended to encourage and enable coordination of research among academics, researchers, and clinicians, and to engage in knowledge translation and mobilization activities with the public safety officer community. The Public Safety Steering Committee (PSSC) provides advice and guidance on CIPSRT’s strategic direction, and helps to determine CIPSRT’s research agenda. PSSC membership is drawn from public safety officer executives, organizations, and associations. CIPSRT serves a secretariat role to the PSSC.

Internet-based Cognitive Behavioural Therapy (ICBT) pilot: $10 million was allocated over five years for CIPSRT to develop and pilot an ICBT program to increase the accessibility of quality care and treatment, particularly to reach public safety officers in more rural or remote areas. The pilot is hosted by PSPNET and offers two eight-week long therapist-guided ICBT courses specifically designed for public safety officers: a Wellbeing Course that assists public safety officers with various mental health concerns, such as depression or anxiety; and a PTSD Course that assists public safety officers who have symptoms of PTSD. Both courses have lesson materials presented online on a weekly basis as well as support from a therapist trained in ICBT treatment for post-traumatic stress issues. Therapists review client progress and respond to client questions or comments via a secure email system, typically once or twice a week. For clients who need additional support, therapists may contact the client by telephone or have more frequent contact. Support from the therapist is typically eight weeks in duration but can be extended to 16 weeks, depending on client needs.

2. Engagement Purpose and Methodology

The purpose of the evaluation was to examine the relevance of the PTSI Initiatives, the effectiveness of their implementation, the achievement of their outcomes, and the efficiency of their delivery. The evaluation covered the Initiatives’ activities under the Contribution Agreement with CIPSRT from January 2019, when the agreement was signed, to April 2022, and was conducted in accordance with the Treasury Board Policy on Results and the Directive on Results.

Interviews: Twenty-three interviews were conducted with 27 individuals, including CIPSRT personnel, representatives of PS, members of the PSSC, and members of the Academic, Research and Clinical Advisory Committee.

Literature and program document review: Literature (e.g., academic research, and governmental and independent reports) and corporate documents (e.g., terms of reference, action plan, contribution agreement) were reviewed.

Performance and financial data: Performance data from interim and annual performance reports was reviewed. Program financial data was also analyzed.

Limitations: The Initiatives do not have a logic model or performance measurement framework, and some of the outcomes listed in performance reporting documents would be more appropriately called ‘activities.’ As a result, available performance data has a greater focus on outputs rather than outcomes. Performance reporting also uses different measures and different time periods so trends over time cannot easily be established. In addition, few key informants were able to address all evaluation issues; in particular, most key informants could not address questions related to the ICBT pilot project. Due to privacy/confidentiality issues, ICBT pilot participants were not consulted in the conduct of this evaluation.

Note: Outside the scope of the evaluation is the joint initiative between the Canadian Institutes of Health Research and CIPSRT for a research consortia to gather data and conduct research on PTSI among public safety officers.

3. Findings

3.1 Relevance

3.1.1 Continued Need

By the very nature of their work, public safety officers are repeatedly exposed to crime, violence, accidents, natural disasters, disturbing materials, and other emergency situations that increase their risk of developing PTSI. This has broad, far-reaching, and interrelated impacts on public safety officers. Some of the more common impacts of PTSI include depression and anxiety disorders, anger issues, emotional distress, substance abuse disorders, self-harm, sleep disorders, a sense of hopelessness, increased aggression and conflict-seeking behaviours, becoming increasingly withdrawn, weight fluctuations, and suicidal behaviour.

PTSI also affects the families of public safety officers. PTSI can lead to withdrawal from one’s family, loss of income, familial and spousal conflict (including separation, divorce, and custody battles), stress for children and spouses, intergenerational trauma, and intimate partner violence.

In addition, PTSI has various impacts on public safety services, including staffing shortages (e.g., absenteeism, recruitment and retention challenges), poor job performance, impaired decision-making, increased stress and responsibilities placed on colleagues, and decreased workplace morale. Accordingly, PTSI can cost public safety organizations considerable amounts of money in the form of overtime costs, the recruitment and training of new staff, the cost of the treatment itself, and insurance premiums.

Recent Studies show:

• 85% of public safety officers have had exposure to at least one traumatic incident (11 times higher than other segments of the Canadian population)
• 45% of public safety officers have one or more mental health disorders (compared to 10% of the general population)
• 23% of public safety officers screened positive for PTSD
• 40% of RCMP employees cited “psychological health” for using Employee Assistance services

The Initiatives offered by CIPSRT are part of a wider federal approach to addressing PTSI and respond to the three pillars of the Action Plan by supporting research activities through knowledge mobilization and facilitation (Pillar 1); by working on prevention, early intervention, and stigma-reduction (Pillar 2); and by piloting care and treatment for all types of public safety personnel that could be offered across the country (Pillar 3). 

In terms of its research activities, the creation and continued growth of the ARC Network is one tangible way that CIPSRT is seeking to address research gaps. By identifying researchers who are working in this area (or are interested in working in this area), gathering them in a network, and connecting them with PSSC members, there is greater coordination and information sharing, which should lead to more relevant research products and reduce duplications and inefficiencies in the research being conducted. No other organization in Canada provides that role.

“We need CIPSRT. It is the only national organization that looks at PTSI and public safety personnel.”- Interviewee

In addition, the evaluation found little evidence to suggest duplication of Initiative programming related to the prevention, care, and treatment of PTSI among public safety officers. The lack of duplication was attributed to a number of factors, including the large scope of the issue and the high level of unmet need (thus indicating that there is room for several stakeholders to be involved). 

The overall landscape of mental health services for public safety officers is a patchwork of services that lack coordination and may leave gaps in delivery, content, and reach. For federally employed public safety officers, there are a number of mental health services and programs offered by their federal departments that are specific to their sector (e.g., RCMP, Canada Border Services Agency [CBSA], Correctional Services Canada [CSC]). Other levels of government that employ public safety officers (provinces, territories, municipalities) may also offer mental health programs or provide access to services through their employee assistance programs or through workers’ compensation programs. In addition, not-for-profit organizations and private businesses offer mental health services.

However, few of the programs are specifically tailored to public safety officers, and even fewer still are evidence-based, extend beyond one public safety sector, provide clinical support by clinicians who have experience with public safety officers, and are offered free at point of use – all of which the ICBT pilot offers.

Annex A provides a non-exhaustive list of programs similar to the ICBT pilot that were identified through an online scan.

CIPSRT works collaboratively with both the Canadian Institute for Military and Veteran Health Research (CIMVHR) and the Atlas Institute for Veterans and Families (the Atlas Institute), which are organizations with similar mandates to address PTSI, but only for military and veterans. These organizations engage in similar activities in terms of knowledge translation and building networks that unite researchers and frontline members. CIPSRT has signed memoranda of understanding with each organization to reduce possible duplications (retired RCMP officers are also considered veterans), develop and share resources, and learn from one another. CIPSRT has also engaged in collaborative activities with the Canadian Armed Forces and the Mental Health Commission of Canada in relation to the Road to Mental Recovery (R2MR), although the pandemic appears to have impacted the timely delivery of these activities and the relationship among the parties is reported to be unclear. 

CIPSRT is less involved with provincial, territorial, or municipal initiatives and there may be opportunities for greater collaboration and coordination with those levels of government.

There are many research needs and gaps related to PTSI among public safety officers. Some of the more pressing ones include a need for: longitudinal and baseline studies; understanding of risk factors for developing PTSI; the effectiveness of current interventions and programs; the effectiveness of prevention and mitigation strategies; sector-specific research, especially for sectors that have traditionally not been the focus of research; the impacts of PTSI on families; and the intersections between PTSI and Gender-based Analysis Plus (GBA Plus).

The extent to which the CIPSRT Hub has identified and addressed research needs and gaps has increased over the course of the Initiative. The CIPSRT Hub identifies gaps and accessibility issues in a number of ways, including through the ARC Network, the National Research Consortium on PTSI, as well as through various knowledge sharing fora, including webinars, town halls, and resources on their website.

The ICBT pilot addresses many of the pervasive, interconnected, and mutually reinforcing barriers to the accessibility of mental health programming for public safety officers. This is limited to the regions of the country where it is available (currently in five provinces).

Accessibility barrier

• Stigma and fear of consequences if seeking assistance
• Lack of system capacity
• In-person, business hours
• Service gaps in rural and remote areas
• Lack of awareness of PTSI risks and services available
• Cost

ICBT mitigation

• Confidential, does not require employer referral
• Adds to existing capacity
• Virtual, flexible schedule
• Virtual, internet connection
• Promotion of the ICBT pilot improves awareness
• Free at point of use

3.2 Effectiveness

3.2.1 Implementation

The CIPSRT Hub took time to develop but now has key structures in place. The first two years of funding focussed on building the CIPSRT operations team and developing the roles of the PSSC, the ARC Network, and the ARCNAC. ARCNAC is an elected body which represents the ARC Network, informs its development, and seeks to facilitate its sustainability.

While progress was impacted by the COVID-19 pandemic, the CIPSRT Hub demonstrated its flexibility by shifting its activities to support frontline public safety officers with the COVID Readiness Response Project. This project quickly brought together stakeholders and produced knowledge products that would assist the public safety officer community with the COVID-19 pandemic’s potential impact on officers’ mental health. This was accomplished while continuing to work on building capacity as well as increasing its knowledge sharing.

Since its inception, the ARC Network has grown to 140 members with ongoing efforts to increase membership.

The ICBT pilot developed its model and content by adapting proven models as well as tailoring content based on the experiences and needs of public safety officers. A best practice is the evergreen nature of the ICBT pilot as it has a continuous feedback loop where participants’ experiences inform improvements.

The ICBT pilot has exceeded expectations by expanding from two to five provinces. It launched in the intended two initial provinces (Saskatchewan and Québec) on schedule and has since expanded to three other provinces (Nova Scotia, New Brunswick, and Prince Edward Island). The expansion was able to occur because PSPNET had the available capacity and federal funding was leveraged with both provincial and private funding; this, coupled with the fact that provinces were able to benefit from economies of scale created by joining concurrently. The expansion meant that the ICBT pilot has had a greater geographic reach than was originally planned.

The pilot is also expanding in terms of the services offered. It began with two courses (PTSD and Wellbeing) and is currently studying the suitability of a self-guided version of  the Wellbeing Course.

As of February 2022, over 560 public safety officers had received services under the ICBT pilot in the five provinces.

The primary challenge identified for both the CIPSRT Hub and the ICBT pilot is the need to increase awareness and reach. For the CIPSRT Hub, this takes the form of needing to broaden and diversify its reach among both the public safety officer and academic/research community. For the ICBT pilot, this includes the need to increase uptake, which is considered slower than anticipated in the participating jurisdictions (the target in the contribution agreement was 3,000 public safety officers).

Various activities to support broadening the Initiatives’ reach (including through social media) are underway. CIPSRT is working on expanding the ARC Network, and is also developing a media campaign strategy to increase awareness of CIPSRT and PSPNET. Other suggestions for expanding the uptake of the ICBT pilot include greater use of the PSSC members’ networks and associations connected with frontline staff to promote the program.

Another challenge for the ICBT pilot is hiring clinicians, which will be important to support efforts at expanding its reach. This challenge stems from a system-wide lack of clinicians with experience working with public safety officers, as well as the inability to offer a competitive pay scale.

3.2.2 The CIPSRT Hub

According to stakeholders, CIPSRT has established what was variously described as an “excellent,” “strong,” and “very positive” reputation as a national hub for public safety officer mental health information. This reputation exists within both the Canadian public safety and mental health communities. In addition, CIPSRT’s positive reputation is growing both nationally, as provinces and territories learn of its work, as well as internationally, in particular with interest from Australia and the United States.

The key factors that have facilitated the CIPSRT Hub’s growing reputation include:

• the strong communication between CIPSRT and the PSSC;
• the expertise and reputation of the CIPSRT team, including its scientific director; and
• the demonstrated commitment by CIPSRT to improve conditions for public safety officers with PTSI.

As evidence of its reputation, CIPSRT is receiving a growing number of requests to assist the public safety officer community in making informed, evidence-based decisions related to available mental health programs. In addition, CIPSRT is engaged in international projects and has built relationships with other major research centres and universities, including partnerships with 62 universities and organizations representing 12 essential frontline sectors.

In terms of facilitating knowledge translation and mobilization, progress includes:

• Offering webinars and virtual town halls to share research with both the academic and public safety officer communities. While the number of webinars dropped between 2020-21 and 2021-22 (from 30 to 15, although it still exceeded its target number), the number of participants (attendees and online views) averaged 100 participants per session in 2021-22, a 45% increase from 2020-2021. The webinars offered have had over 8,500 views (live and video). Webinar attendance is not reported on by sector so the reach within the public safety officer community is not fully known.  
• Developing products that are accessible to the public safety officer community by offering shorter versions of the webinars for frontline workers, lay publications, and research summaries and briefs.
• Making connections between the public safety officer community and clinicians/researchers, such as through the development of the Research Engagement Form, the public safety officer network of mental health champions, and the Families Matter Research Group. The Research Engagement Form collects information on public safety officer-related research activities in Canada. CIPSRT then shares this information with its stakeholders, thus helping public safety officers and leadership decide which research projects to participate in.

Stakeholder response:

• 91% of webinar participants found them to be useful
• 55% of respondents considered the research summaries useful

CIPSRT was praised for taking the time to understand the public safety officer community before offering solutions. However, more work could be done to improve knowledge products by making them more relevant to the public safety officer community through more accessible/less academic language and by showing how research findings are transferrable into actionable items, tools, or resources. 

It was suggested that there be more meetings between the PSSC and ARCNAC, and the ARC Network could improve communications and create more understanding of the needs of public safety officers as well as of the value and utility of the research being conducted. A national conference may enable this communication and promote the work that has been done. Methods to increase accessibility of current products included making the CIPSRT website more easily searchable, increasing the number of one-page  summaries of the research, and ensuring that content is available in both official languages.

There have been improvements in communications and knowledge translation. However, as some stakeholders noted, this may require a large communications team to (e.g., to develop a podcast). Creating effective communications products is also complicated by the need to have different messaging to make the content meaningful/relevant to different public safety sectors. Currently, CIPSRT has three full-time communications personnel, which is not considered sufficient, particularly when compared to the much larger communications team at similar organizations (e.g., the Atlas Institute).

Another area of improvement is the development of a national baseline of information and the identification of effective interventions. Efforts are underway to fill this research gap, though results from such studies will take a while to be produced. In the interim, the CIPSRT Hub has been instrumental in gathering studies to determine consensus-based best practices using systematic or synthesis reviews of the literature.

3.2.3 ICBT Pilot

There is an existing evidence base to support the use of cognitive-based therapy in treating PTSI. While the literature on ICBT is more recent and is typically not focussed on the public safety officer community, it generally indicates that ICBT is effective and helps to address issues of accessibility to mental health treatment for those living in rural areas. In addition, given current knowledge, the studies underway on ICBT more generally, and on the impact of the ICBT pilot, will have substantial value in building an evidence base for ICBT in addressing PTSI among public safety officers.

The ICBT pilot is showing positive results and is a success based on the results of its post-treatment questionnaire. 

• 97% - show significant improvement compared to pre-treatment symptom scores
• 91% - reported increased confidence in managing their symptoms
• 98% - found PSPNET worth their time
• 97% - would refer a friend to PSPNET

Many stakeholders stressed the need for the ICBT pilot to be offered in more provinces and territories. In addition, the geographically restricted offering could be impacting uptake as national organizations cannot promote programs that only some of their members can access. Another gap is the need to involve families, who are key supports for public safety officers, and provide programming targeted to family members who may experience secondary trauma.

“I was told by a colleague that the course was ‘life changing.’”- Interviewee

3.3 Gender-Based Analysis Plus (GBA Plus)

GBA Plus considerations were included in the design and implementation of the PTSI Initiatives from the outset as PS’ contribution agreement with CIPSRT includes performance indicators related to GBA Plus. Moreover, CIPSRT has equity, diversity, and inclusion (EDI) requirements in place for its internal hiring processes and the ARC Network sub-committee selection process. In addition, given that public safety officers are predominately male, training spots in webinars are set aside for female public safety officers to ensure that they have an opportunity to participate.

The nature of the ICBT pilot incorporates GBA Plus by making the programming available to people in rural settings where mental health services, particularly those specifically targeted to public safety officers, are scarce. PSPNET enables access to the ICBT pilot for those without high-speed, continuous Internet access by making it easy for users to use the program over many sittings. The program also enables less represented groups, such as women, BIPOC (Black, Indigenous, people of colour), and 2SLGBTQI+ (Two-Spirit, Lesbian, Gay, Bisexual, Transgender, Queer, and Intersex), to have completely confidential access to programming.

 Of the 560 public safety officers who had enrolled in the pilot, the majority lived in communities with fewer than 90,000 residents (61%), or were female (53%). Sixteen percent were members of an ethnic minority.

Areas of improvement were identified for each Initiative in order to better incorporate and document GBA Plus.

The CIPSRT Hub should:

• Review performance metrics related to GBA Plus in order to better track progress (e.g., track GBA Plus-related content in research, demographics of ARC Network members).
• Encourage increased focus on addressing gaps in research related to the intersectionality of PTSI and various identity factors.
• Ensure that knowledge translation and messaging is consistently made available in both official languages.

The ICBT pilot should:

• Ensure that it takes into account different cultural perspectives. Outreach has been conducted with families from diverse cultures to build a greater understanding of the needs of these families and clients, but key informants from all stakeholder groups thought that more could be done.

3.4 Efficiency

3.4.1 Initiative resources

The effective collaboration of CIPSRT with the PSSC, ARCNAC, ARC Network, PSPNET, and PS was cited as a key factor helping the efficient use of resources. Some of these collaborations have taken time to develop and some remain a work in progress, such as better integrating the PSSC into decisions regarding research priorities. Other elements that supported efficiency including hosting town halls and webinars virtually, and having dedicated administrative support for Initiative-related meetings and activities.

The CIPSRT Hub is credited with having encouraged efficiency in research by promoting better alignment of research with the needs of frontline public safety officers, supporting coordination of research teams with shared relevant interests/expertise in the research priority areas (e.g., through the Research Engagement Form), and encouraging collaborative (rather than competitive) approaches to research. 

For the ICBT pilot, the groundwork to develop the content and infrastructure resulted in efficient launches in the initial two jurisdictions which meant there was capacity to support the expansion to three Maritime provinces. The approach to expansion – leveraging the federal funding used for developing the infrastructure and content of the pilot with provincial and private funding to fund service delivery –  serves as a model for potential expansion of the pilot to other jurisdictions.

In addition, offering the courses online is more cost-effective than an in-person approach, particularly for public safety officers in rural areas, and allows public safety officers to access services in a more timely manner.

3.4.2 Challenges

Mandate or scope creep for CIPSRT and the Hub has occurred in a few ways. For example, CIPSRT receives requests to confirm or validate different PTSI programming being considered by different organizations. They have also been asked by organizations such as those representing nurses to consider expansion to other public safety sectors.

Bilingual capacity has been an issue for CIPSRT as there have been difficulties ensuring all materials are translated into both official languages and offering in-person or virtual live presentations in French.

CIPSRT requires further human resources supports to work efficiently, in particular administrative staff (or access to University of Regina’s administrative resources), and a larger communications team to support the expansion to other public safety sectors, as well as the requests for more accessible materials (plain language, different formats).

There remains the need to create greater awareness of the CIPSRT Hub as well as the ICBT pilot. Greater uptake and usage will help ensure the maximum benefit of the Initiatives are achieved.

Performance measurement: The Initiatives do not have a logic model or performance measurement framework. As a result, CIPSRT reporting documents focus on outputs, which are framed as outcomes. The Initiatives would benefit from developing these tools, which would assist with identifying activities, outputs, outcomes, and indicators of success, but also with streamlining reporting and strategic planning.

While regular and detailed reporting is provided to PS, the content and styles are not consistent which makes it difficult to show progress over time. 

Cost-effectiveness: Currently, there is a lack of information to support a determination of all associated costs and benefits in order to understand the true cost effectiveness of the ICBT pilot. This is a challenging task, as obtaining such data to determine ICBT cost effectiveness per user requires time to acquire and analyze information from public safety personnel related to the number of days off work due to mental health issues and health care use. Much of this information is self-reported, is not yet available, and is not linked to other data sources in order to maintain patient confidentiality, which makes analysis a challenge. While such information was not necessarily to be gathered by the pilot, it was raised by stakeholders as an efficiency consideration of interest.

4. Conclusions

PTSI is a serious issue among the public safety community, and there is a continued need for the type of programming and research provided by the Initiatives. Research related to PTSI and public safety officers – its prevalence, its impacts, what treatments are efficacious – is in its early stages with numerous gaps in knowledge that remain to be filled. CIPSRT addresses a need by providing a national research hub that has public safety officers’ mental health as its focus. In addition, while there are numerous programs available to public safety officers that address mental health issues, the ICBT pilot has a unique set of attributes, including being designed for public safety officers, using clinicians that specialize in treating trauma in the public safety setting, being confidential, and being free to access.

The Initiatives have largely been implemented as planned, despite some early challenges. More could be done to maximize the potential of each Initiative by increasing awareness of them and broadening their reach within the public safety and research communities.

Progress has been made toward several of the Initiatives’ outcomes. More specifically, the CIPSRT Hub has a growing reputation as a hub for public safety officer mental health research and has made strides in facilitating knowledge translation and mobilization, although knowledge products could be made more accessible to the public safety community in terms of content and delivery. The ICBT pilot’s early results indicate that it is effective in assisting public safety officers who are experiencing the effects of trauma, and there is a desire to see it expand to other jurisdictions. Finally, the nature of the Initiatives mean that there are many opportunities to incorporate GBA Plus, and the evaluation found that this could be done to a greater extent and be better documented.

The Initiatives’ resources have largely been used effectively and efficiently, though there are opportunities for improvement. The current performance measurement processes could be strengthened to allow for a more fulsome assessment of progress.

5. Recommendations

The Assistant Deputy Minister, Emergency Management and Programs Branch, should, as part of renewed/continued funding:

1. Explore methods to support the sustainability and expansion of the ICBT pilot by collaborating and coordinating with provinces, territories, and relevant organizations.
2. Ensure resources provided to CIPSRT are sufficient for the communication and dissemination of knowledge products and consider working with federal partners (e.g., Statistics Canada) to support the development of a national baseline of information.
3. Revisit and revise the current approach to performance measurement, including the development of a logic model, a performance measurement framework, and an approach that better incorporates GBA Plus.

6. Management Action Plan

Annex A – Other Available Programming

Date modified:

2023-06-23