Cyber-Physical Capstone Exercise After-Action Report

Delivered October 24-26, 2023

Table of contents

Executive Summary

Cyber-Physical Exercise Pilot Initiative: How Cy-Phy Helped Test Critical Infrastructure Participants' Security Posture

The current threat landscape in Canada has seen a rise in insider threats and ransomware attacks, putting Critical Infrastructure (CI) owners and operators acutely at risk. The Cyber-Physical (Cy-Phy) Exercise Pilot Initiative was conceived by Public Safety Canada (PS) to help fill a gap in exercises, with the objective of increasing Canada's CI resiliency to cyber incidents with physical consequences, and to strengthen collaboration between CI sectors and organizations navigating in both the cyber and physical domains.

In the current digital age, CI is increasingly interconnected and interdependent within and across domestic and international jurisdictions. In an ecosystem of interconnected networks and assets, disruptions to CI can result in significant harm to Canadians, Canadian businesses and CI organizations.

The Cy-Phy Exercise Pilot Initiative was conceptualized to examine the interconnectedness between the cyber and physical realities and increase knowledge on organizational security measures through a series of virtual tabletop exercises, seminars and workshops, and culminated in a national multi-sector capstone exercise (“Capstone") delivered virtually over three days in October 2023. These activities were designed to encourage the exchange of risk-based information, identify best practices and lessons learned, help bolster cross-sector interdependency literacy, and test incident response capabilities to enhance CI resiliency.

The Capstone took place in a simulated environment in which participants could actively test their security protocols and procedures in response to an organized and intentional cyber-attack against individual and combined Canadian infrastructure.

The broader collective objectives of Cy-Phy Exercise Pilot Initiative were to educate and raise awareness, develop partnerships, enable decision-making, and share information among and within participating organizations.

Approximately 150 private and public sector organizations participated in the Capstone exercise with over 650 individual players from coast to coast.

Participant feedback broadly denotes that the Capstone was considered an overall success in helping them identify areas of improvement with the goal to increase their resiliency as Canadian CI when faced with cyber-attacks with physical consequences.

Playing organizations were able to execute their responses to various scenarios, and thus identified strengths and weaknesses in their existing procedures and protocols. Participants have reported that they are using the exercise as a model for organizational continuous improvement.

This Cy-Phy Capstone Exercise After-Action Report (AAR) presents a walkthrough of the event and a summary of lessons learned captured during the Capstone.

Introduction

The Cy-Phy Exercise Pilot Initiative aimed to further the understanding of risks and capabilities in responding to cyber events with physical consequences and increase collaboration between those responsible for cyber and physical infrastructure in response to realistic threats.

While organizational objectives could vary, four overarching strategic objectives were identified at the onset to drive the building block activities and the Capstone.

Education and Awareness – Ongoing user education for technology security within organizations is reinforced and builds a culture of security awareness through proactive measures.

Partnerships – Organizations develop partnerships for their cyber security operators and service providers. These strategic partnerships help organizations reduce cyber risks, embrace innovation and disruptive technologies, and improve operational resilience.

Information Sharing – By exchanging cyber threat information within a sharing community, organizations leverage the collective knowledge, experience, and capabilities to gain better understanding of threats and best practices.

Decision-Making – Training, capacity and capability building are vital to understanding the complexities of cyber risks and making proactive decisions for cyber security. Management experience alone does not compensate for uncertainties of events.

PS co-designed the Capstone with participating organizations to provide participants with credible scenarios that would put in practice, in real time, their capabilities and help them prepare, identify, prevent, mitigate, and respond to threats, tailored to their organizational objectives. PS' Communications Directorate and the Royal Canadian Mounted Police’s (RCMP) National Cybercrime Coordination Centre (NC3) also supported the design and delivery of the Capstone as chief controllers for the communications and law enforcement cells, respectively.

Throughout the planning process, exercise designers from playing organizations were required to invest several hours per month towards collectively building the Capstone event. They collaborated to produce the global narrative and scenario paths, which were later scaled at the organizational level.

Participating organizations represented a wide range of sectors and jurisdictions, which offered tremendous opportunity for intersecting scenarios and cross-sectoral play. However, over the Capstone planning period, the focus of the exercise shifted to a more cyber-centric event with lesser physical impacts and interdependencies, affecting the participant mix, the adversary details, and consequently, the timely delivery of the master scenario events list.

The Capstone was a realistic simulation of a major national cyber event involving criminal activities threatening the safety of Canadians such as impeding access to banking, food, health services or transportation. The Capstone displayed cyber and physical impacts arising from scenarios involving ransomware, malicious software, data loss, and affects against operational systems. At the discretion of players, some compromised organizations were featured in the simulated news to prompt their players and bolster the overall verisimilitude of the narrative. For example, there were regional and national reports of hospital equipment malfunctioning leading to medical services being postponed, as well as airport baggage systems and air traffic being completely halted.

Playing organizations scripted their injects according to their organizational objectives, and sample scenario paths could be combined or modified. With upstream and downstream impacts and threats on productivity or customer service; it was essential to involve subject matter experts and explore communications, legal, and senior decision-making protocols.

The Capstone exercise took place in real-time in a virtual environment and was deliberately timed to coincide with the cyber security awareness month recognized by many NATO allies. Participants from across the country logged onto the platform to play, monitor or observe the activities taking place over the three days.

This Cy-Phy Capstone Exercise After-Action Report (AAR) seeks to highlight findings specific to the Capstone exercise. The AAR further serves to identify strengths to build upon, as well as identify potential areas of improvement in managing such an initiative.

Playing organizations are encouraged to consider updating any relevant plans, policies and procedures identified for improvement through their participation in the Capstone.

Data Collection Methodology

Data used to produce this report was sourced from players, evaluators and controllers. It encompasses information from diverse channels including live polls and chat rooms within the platform, participant feedback and testimonials provided by email during or after the Capstone, the post-exercise questionnaire, and comments provided during the players' national hotwash session held on November 2nd, 2023.

Chronology

From the concept and design, to roll out and evaluation of the Capstone, the various elements and activities of Cy-Phy Exercise Pilot Initiative took close to two years to complete.

The planning and engagement activities were initiated in the Fall of 2021 and continued until the Capstone in October 2023. During this period, PS delivered standard exercise planning meetings, such as:

In parallel, PS organized a series of supporting seminars and Virtual Tabletop Exercises (VTTXs) designed to both educate participants on security concepts such as ransomware and insider risks, and serve as opportunities to onboard players. In addition, a VTTX  for federal partners took place to explore an escalating cyber event affecting Canadian CI that resulted in the activation of the Federal Cyber Incident Response Plan (FCIRP) to its highest level (Level 4), which triggered the Government of Canada Cyber Security Emergency Management Plan (GC CSEMP) and the Federal Emergency Response Plan (FERP) in parallel.

Timeline Snapshot

September and October 2021
Cy-Phy Launch Seminars
May 2022
Concept and Objectives Meeting
October 2022
Virtual Tabletop Exercise on Ransomware
November 2022
Initial Planning Meeting
February 2023
Virtual Tabletop Exercise on Insider Risk
February 2023
Now You Know Webinar on Cy-Phy
March 2023
Distribution of Participants Kits (Overview of Exercise Scenario Design: A User Guide, the Cy-Phy 23 Storyboard, the Cy-Phy 23 Workbook)
March 2023
Inject writing clinics
May 2023
Virtual Tabletop Exercise on the Federal Cyber Incident Response Plan (FCIRP)
June 2023
Main Planning Meeting
July/August 2023
Development of Organizational Scenarios
September 2023
Final Planning Meeting
October 2023
Virtual Platform Training Sessions
October 2023
Communications Meeting
October 2023
Cy-Phy 23 Capstone Exercise
November 2023
National Hotwash Meeting
Spring 2024
After-Action Report

Participation Breakdown

The Capstone brought together participants from multiple sectors and areas of discipline from federal departments, provincial, territorial, and municipal governments, law enforcement, emergency management agencies and associations as well as private owners and operators of CI assets.

Approximately 150 organizations were involved during the Capstone, with over 650 active individuals who logged into the platform throughout the three days. Some CI partners were simulated by PS to ensure a realistic representation of a federal response to a significant national cyber event.

Participants could approach the Capstone in various ways. Such as:

Player / Victim Organizations (32 total) - Playing organizations appointed one or more lead exercise designer(s) who set their organizational objectives and crafted their exercise play with the support and guidance of PS through a series of working group meetings, inject writing clinics, bilateral meetings, and exercise planning activities. Each playing organization accessed the virtual environment from their chosen venue. Players responded to injects (simulated news, phone calls, emails) prompting them to react and interact as they would during the course of an event, practicing their organizational response actions and mitigation measures.

Monitor and Respond Organizations (32 total) - Monitor and respond organizations participated in planning meetings and could choose to have some scripted injects to prompt player action. With their primary function remaining to support the realism of the scenarios, they also benefited from exercising their communication protocols or incident response procedures.

Observing Organizations (86 total) - Observers were invited to take part in planning meetings and accessed the virtual platform throughout the Capstone

Once in the virtual environment, all participants could access an email inbox, social media, the newsroom, the tech corner, the live polls and the dark web.

None of the participants could see what other playing organizations were simulating, unless they had scripted injects to explore interdependencies. Which a few participants opted to do.

Exercise Delivery

The Capstone simulation was executed in a controlled environment using Calian's ResponseReady™, a virtual platform, which provided players with an immersive exercise environment throughout the three days of play. The virtual platform allowed for multiple information sources such as social media, email, news outlets, to be presented to participants during the exercise. The tool also allowed PS to replicate technical blogs and a dark web experience.

Participants received training on how to use the platform prior to the Capstone delivery.

Virtual play was administered from Ottawa as the main hub and exercise control venue (EXCON) on eastern standard time, with organizational play occurring at participant's chosen exercise control venues across Canada.

Through a range of injects disseminated by PS via the virtual platform, all participants including observers, were introduced to the scene-setting narrative.

The narrative involved retaliation from a fictitious group of patriotic “Westinians” by way of cyber attacks in response to Canada supporting sanctions against the fictitious country of “Westinia” for its treatment of its inhabitants. Cyber based intrusions across critical infrastructure by way of a zero-day vulnerability through a popular operating system led to simulated ransomware activity, malicious software, loss of data and in some cases compromised operational systems.

Participating organizations chose how many days they wished to play and planned injects to prompt their players to react and interact accordingly.

Day One (Oct 24) – Mainly for situational awareness through key events such as news, broadcasts, social media activity and vendors notifying that a vulnerability had been detected in a widely used operating system (a facsimile of Microsoft). A few security articles were also available at this time.

Day Two (Oct 25) –Mainly consisted of incident response activities. Events ranged from ransomware notifications, loss of service and affected systems, prompted investigations, incident response plans and continuity plans to be tested.

Day Three (Oct 26) – Mainly focused on the recovery of data, restoration of services and validation of business continuity plans, putting an end to the event state and mitigation activities.

For organizations interested in testing hands-on responses, a package containing technical information and artifacts was made available. This package included executable files (benign), malware details including indicators of compromise (file names, hash values, file locations, registry entry), images of the ransom message, instructions on how to use the technical package and a removal tool to delete all entries after exercise conduct.

Throughout the Capstone, EXCON conducted coordination calls for controllers and evaluators at two different times in the day to better serve participants from various time zones and disseminated a “start state” document via the platform. Once activities were paused for the day, EXCON convened controllers and evaluators for a daily debrief. Controllers and evaluators were encouraged to conduct daily internal organizational hot washes to identify any issues, gaps or best practices learned from the day.

The Chief Controllers for the law enforcement and public communications pillars, as well as observers visiting from the Australian Cyber Security Centre (ACSC), joined the planning team in EXCON.

Virtual Platform: Calian ResponseReady™

Information Flow

During the course of the Capstone, injects and scene-setting information were delivered to players using a variety of methods and formats such as through social media, email, news outlets, as well as from niche sources such as technical blogs and the dark web. Other methods such as phone calls and direct email were used on case by case basis, where applicable.

Methods were chosen in order to best mimic how players would receive information during a real-life situation to accurately test responses and prompt player action.

The Inbox was the email messaging capability within the platform and was used to simulate standard email business communication. Phone calls from the simulation cell were used to deliver certain injects to players. This included simulation of calls from media outlets, as well as simulation of information reporting in to coordination centers.

Platform Building Blocks

The following section offers a description of the platform options that were used during the Capstone:

Canadian Virtual News Network (CVNN) - CVNN news mimicked legitimate media coverage and served as a trusted source of information for ongoing situations of interest, both domestically and internationally. CVNN published content in the form of video news broadcasts, national news articles, and regional news articles. Certain articles were used as player injects to prompt player action in response. In order to increase realism, certain players received emails or phone calls from CVNN reporters asking them to provide statements which were incorporated in subsequent articles.

News Room – The News Room served as a repository for direct source publications from industry partners. This included advisories and information from the operating system vendor, advisories and alerts from government sources such as the Canadian Centre for Cyber Security, RCMP, and United State’s Federal Bureau of Investigation (FBI), industry and sector specific newsletters, and other industry releases.

Social Media - The primary form of social media available on the platform was “Chatter”, a facsimile of X, formerly known as Twitter. Chatter was key in reflecting public sentiment and breaking stories that may not have been reflected in traditional media yet. Adversary groups were also found on social media preforming recruitment and leaving clues to their identities and affiliations.

Tech Corner – The Tech Corner served as a repository for technical tools and resources allowing for technical players to more accurately follow usual procedures when investigating a cyber incident.. Tools and resources were updated throughout the exercise in order to display new findings and up to date information.

Dark Web - A Dark Web portal was created in order to allow players to simulate threat hunting activities. It contained pages for each adversary group with hints about each group's mission and identity as well as various hidden pieces of information allowing players to piece together adversary backstory.

Technical Blog - The “Screen Saver” blog was a niche industry publication delivering in-depth coverage of the technical aspects of the ongoing situation, but not meant to be consumed by the general audience. Blog posts were key in the delivery of unconfirmed details and speculation that had no place in traditional media.

The full potential of Calian's ResponseReady™ tool was maximized and the resulting experience was well-received by participants. During the Capstone, a daily average of 563 active users logged into the environment. Other data high notes include the distribution of 2,933 social media posts and 4,098 emails as well as 848 scenario injects. The CVNN broadcasted 24 national news posts, 16 regional new posts, and 8 news broadcasts – along with the 8 technical blog posts targeting the more technical players. These activities allowed to recreate a verisimilitude scenario which vastly enhanced the simulation and drove up participant zeal.

Key Findings

What We Heard from Capstone Participants

In line with the Capstone's core objectives, the following notable observations were drawn from data collection points and overall exercise participation.

Education and Awareness

Objective: Raise awareness and understanding of CI and the convergence of cyber security, physical security and emergency management amongst CI Stakeholders.

Through exercise play, participants gained insights into the potential cascading effects of cyber incidents on physical infrastructure, mostly within their organizations. Participants recognized the inherent complexity of addressing cyber and physical threats concurrently within their organization, underscoring the necessity of comprehensive strategies and cross-functional collaboration.

The intersection of cyber and physical realms as the central theme for the Capstone was in and of itself a challenging environment to navigate. Each maintains distinct policies, procedures, and communication channels, necessitating careful consideration within organizational incident response strategies. Participants recognized the difficulties inherent in operating within silos, emphasizing the importance of furthering education and awareness of roles and responsibilities of internal and external partners, a principle underscored in the Partnerships objective.

The exercise highlighted organizational need for ongoing training and education initiatives to equip their personnel with the knowledge and skills necessary to address cyber-physical security challenges effectively. This finding underscored the importance of investing in continuous improvement activities tailored to unique requirements of critical infrastructure organizations.

The exercise prompted organizations to review and revise existing policies and procedures to address the convergence of cyber and physical security threats. The improvement of business continuity planning was of importance for many organizations, with several identifying areas of concern related to continuity of services.

Partnerships

Objective: Strengthen public and private sector collaboration among CI stakeholders with responsibilities for cyber security and emergency management.

Despite the establishment of working groups aimed at fostering collaboration, there was minimal interaction between players during the Capstone. This lack of engagement hindered the effectiveness of cross-sector and cross-jurisdictional coordination efforts, highlighting the need for enhanced communication channels and engagement strategies.

Many participants predominantly focused on exploring their organizations' cyber security posture, overlooking the importance of addressing physical impacts and understanding interdependencies within the critical infrastructure community. A more holistic approach that integrates both cyber and physical security considerations would be a key factor for enhancing collaboration and resilience. Establishing cyber and physical security tag teams to address incidents collaboratively was noted as a possible next step.

Effective communication methods emerged as a crucial factor for facilitating collaboration and decision-making processes. Ensuring consistency, trustworthiness, and responsiveness in communication channels as well as integrating set protocols  and thresholds for communication with partners was recognized as vital for enabling timely responses to emerging threats and incidents.

Interaction with law enforcement and regulatory bodies highlighted the importance of coordination and cooperation in responding to cyber security incidents. Participants recognized the value of establishing collaborative relationships with law enforcement agencies and other relevant stakeholders to facilitate timely and effective incident response efforts.

Decision-Making

Objective: Examine decision-making processes and coordination between cyber and emergency responders in the public and private sector.

The exercise highlighted the importance of clearly defining roles and establishing effective communication channels between cyber security, communications, legal, or other subject matter experts and emergency responders within organizations. Participants identified the need for designated points of contact and established protocols for sharing information and coordinating responses from an organizational point of view.

Coordination between cyber security, or other information technology teams and emergency management players presented challenges. Participants encountered difficulties in aligning priorities, exchanging information, and coordinating actions, emphasizing the need for enhanced decision-making frameworks.

Participants identified legal and regulatory considerations as critical factors impacting decision-making processes and coordination efforts. Navigating data protection laws, and jurisdictional boundaries posed challenges for players, highlighting the importance of integrating legal and regulatory experts into decision-making circles.

Effective incident management necessitated cross-functional collaboration among various departments and teams within organizations. Participants recognized the value of involving representatives from IT, cyber security, legal, communications, and other relevant departments in decision-making processes to ensure comprehensive and coordinated responses to incidents.

Participants observed the importance of identifying and assessing the severity and potential impact of incidents to prioritize response efforts and allocate resources accordingly. Conducting thorough risk assessments and prioritizing incident response actions emerged as critical factors in effective incident management.

Information Sharing

Objective: Identify information requirements and information sharing processes among CI partners with responsibilities for cyber security and emergency management.

Throughout the Capstone exercise, participants frequently underscored the occurrence of delays in information sharing, particularly concerning sensitive information. These delays were noted to occur not only among organizations within the same sector or jurisdiction but also among teams within organizations. Timely sharing of information emerged as a critical factor in effective incident response. Participants stressed the importance of defining timelines for information sharing and ensuring that relevant stakeholders receive timely updates to facilitate informed decision-making and response coordination.

Players expressed a pressing need to expedite the sharing process of sensitive information among key partners within the CI community during incidents. Furthermore, participants emphasized the value of testing alternative communication methods among stakeholders and sought more comprehensive information regarding potential impacts, interdependencies, and the overall scope of threats to CI operations. They expressed a desire for case studies illustrating incidents stemming from vulnerable cyber security postures, neglected contingency plans, or underutilized partnerships. Additionally, participants articulated a need for a deeper exploration of cross-sector and cross-jurisdictional incidents.

The exercise highlighted the necessity for improved interoperability between emergency management, communications, and cyber security incident response plans within organizations. Participants stressed the criticality of rehearsing and mapping out the timing of information sharing, including external communications, and identifying key partners and support services, along with their respective roles and responsibilities.

Throughout the exercise, participants identified the need for interoperability of information systems and platforms as vital for information sharing among partners. This includes compatible technology solutions that enable data exchange and collaboration across organizational boundaries.

Internally, collaboration and the formalization of procedures for exchanging information were identified as top priorities. Participants emphasized the requirement to test alternate communication modes before events and the necessity of overcoming organizational silos. They highlighted the importance of adopting a holistic approach from a cyber response standpoint and expressed a desire to ensure the inclusion of all relevant stakeholders in future exercises.

Final Thoughts

The Capstone achieved its overall goal of helping participants identify opportunities to improve their resiliency and those of the broader Canadian critical infrastructure community when faced with cyber-based attacks with some physical impacts.

Overall, findings identified the presence of federal leadership as crucial in coordinating the CI community for continuous and increased knowledge of cyber and physical security requirements, and the need for deeper exploration of cross-sector interdependencies. This leadership role was said to be pivotal in ensuring continued collaboration and alignment of efforts across the different sectors and jurisdictions, thereby enhancing the overall resilience of critical infrastructure against increasing and emerging threats.

Moreover, participants shared that it is important to emphasize the interplay between cyber and physical security, as protocols and strategies often vary among organizations and jurisdictions. Understanding this nexus is essential for developing cohesive and comprehensive security measures to combat complex threats. Furthermore, a noticeable gap was identified across organizations in terms of preparedness, training, and adherence to protocols.

Participating organizations recognized that an exercise of this scale requires early commitment and continuous and meaningful engagement with organizational subject matter experts (information technology, legal, communications, etc.) to develop quality scenarios and injects that fittingly tests plans, protocols and roles and responsibilities. Participants also shared that they would value the opportunity to further explore interdependencies as they recognize that cross-sector, cross-jurisdiction and cascading disruptions are  vulnerabilities for many CI organizations.

Participants voiced their desire for Lead Federal Departments (LFDs)Footnote 1 to increase their participation in exercises, as it serves as an opportunity for collaboration and innovation to improve CI security. Participants see value in having federal partners responsible for cyber security or emergency management maintain a leadership role and foster meaningful information sharing.

From a planning and exercise delivery perspective, the complexity, the scale and the scope of the Capstone was immense. Narrowing the scope of the exercise would have helped focus the collective efforts required to test interdependencies and the capacity of planners, partners and players to be in a better position to comprehend and explore the real impacts of a cyber incident with physical consequences to critical infrastructure from a cross-sector and cross-jurisdiction standpoint. Early endorsements and commitments from key federal departments and agencies are essential for the successful delivery of an exercise of this magnitude. Many participants had competing priorities and some faced some level of exercise fatigue. It is critical to focus capacity on building a robust master scenario that explores CI as an interconnected ecosystem, rather than divesting efforts into tailoring the exercise to players on an individual level.

In conclusion, proactive leadership, a holistic approach to security, and collaborative efforts have been recognized as key elements for safeguarding critical infrastructure against evolving threats. By prioritizing information sharing, fostering collective action, investing in the improvement of the security posture across CI, the CI community should be able to enhance its resilience and mitigate risks to national security while improving public safety.

Date modified: