Internal Audit of Policy, Priority-setting and Planning (May 2012)
Table of contents
- Executive Summary
- 1. Introduction
- 2. Findings, Recommendations and Management Responses
- Appendix A: Audit Criteria
- Appendix B: Summary Risk Analysis
- Appendix C: Generic Policy Development Process Map
- Appendix D: References
Executive Summary
Background
The mandate of Public Safety Canada (PS) is horizontal in its nature: in building a safe and resilient Canada, it exercises national leadership and contributes to Canada's resiliency through the development and implementation of innovative policies and programs and the effective engagement of domestic and international partners. To this end, as portfolio lead, it must set, communicate and help deliver on directions (whether in the form of policy, priorities or plans) that reflect the Government of Canada's priorities, but also respond to emerging risks, threats and priorities of Canadians. The establishment and communication of clear, appropriately informed and cohesive direction is critical to solidify the role of the department as portfolio lead and to ensure that all actions and related resources (both within the department and across the portfolio) are aligned and appropriately used. Simply stated, it is critical for departmental and portfolio effectiveness and efficiency and is vital to secure the credibility and reputation of the department, both with its portfolio agencies, as well with as the Canadian public. In keeping with these imperatives and in response to the inherent complexity and risk associated with them, an internal audit of Policy, Priority-setting and Planning was planned and conducted.
Audit Objective
The audit objective was to provide reasonable assurance that the mechanisms in place to set and communicate clear and cohesive strategic and operationalFootnote 1 direction for the department and the portfolio are adequate and effective. Lines of enquiry included:
- Governance arrangements, including oversight, decision-making and communication mechanisms, as well as accountability mechanisms;
- Direction-setting processes and the appropriateness and comprehensiveness of the information that feeds them; and,
- The use and usefulness of the outputs of direction-setting, including the mechanisms that enable horizontal and vertical alignment of directions, as well as the appropriate resourcing of them.
Summary of Results
The audit noted many strengths and positive developments. It is understood that many of these initiatives will take time to fully implement and realize value; continued and sustained effort to evolve not only practice but also culture will yield benefit over time. Of note, recent improvements aimed at moving the departments towards a more collaborative model of direction-setting are notable and are attributable to positive 'tone at the top' set by the Deputy and Associate Deputy Ministers. While positive, additional, complementary measures are needed for such a horizontal, complex organization.
In examining the formal processes, the audit concluded that direction-setting mechanisms are defined and consistently applied, although in some cases, more robust and regularized informational inputs and challenge are needed for both policy development and priority-setting. We understand the value that comes from organic idea generation and policy development and are pleased that an enabling environment is being built to engender "bottom-up" policy development. The audit noted opportunities for improvement and evolution in relation to the use of research and risk information as well as the conduct of consultation and engagement of key stakeholders.
On the matter of engagement, the audit concluded that internal engagement across Branches is not optimal, although it is improving; external engagement is strong on individual files, and is more institutionalized and valued than internal engagement. This may be attributable to the natural affinity that exists between PS Branches and their portfolio counterparts. The audit notes that these bilateral relationships represent strengths and should be complemented with stronger internal, branch-to-branch engagement, where appropriate.
As well, while we note that strong bilateral mechanisms are important to support direction-setting on a file-by-file basis, we have also concluded that more systematic mechanisms are needed to engage portfolio and other relevant partners on a holistic basis. We understand that there is a stated desire to eventually involve portfolio agencies in the Assistant Deputy Minister (ADM) Policy Committee, although this has not yet occurred due to the early stages of the committee's maturity. We feel that the eventual involvement of portfolio agencies in this forum is appropriate and strongly encourage this evolution.
Effective portfolio coordination requires more than the establishment of communication channels for horizontal engagement. It also requires a common starting point on which the relationship and interactions can be built. The audit found that the overarching policy agenda of the department is neither systematically nor holistically developed nor managed. This appears to be due to the limited attention that has been placed on this function to date and is compounded by the lack of definition of overarching policy outcomes and expected results at the departmental and portfolio level. As a result, no single PS "story line" or strategic policy framework exists to provide a common foundation from which individual departmental and portfolio-wide policy orientations can be aligned. It must be stated that this is not atypical in light of the organization's maturity. Indeed, important developments including, but not limited to the Policy Committees and the Medium Term Policy Planning (MTP) process are laying the foundations which are necessary for the department to address these issues. At this juncture, considerable opportunity exists for the department to take an active leadership role in setting and developing a long-term, integrated policy agenda for itself and for the portfolio at large. In so doing, the department can solidify its role as portfolio lead and principal policy advisor to the Minister.
The audit also examined the degree to which directions, once established, are set on a good course towards implementation through the identification of outcomes and allocation of resources. The audit noted that required resources for specific directions are identified during the Memorandum to Cabinet / Treasury Board submission phases; however, no systematic tracking of resource usage or outcomes achieved is yet done. Positive steps to cost and track resources associated with priorities are being taken as part of the 2012-2103 business planning cycle. Similarly, while outcomes are often identified, they are not systematically tracked over time.
Lastly, the audit examined the degree to which human resources and knowledge management practices enable the development and sustaining of a strong cadre of policy makers. The audit concluded that, while it may not be an issue of immediate urgency, more opportunity exists to think strategically about what knowledge is needed in support of departmental and portfolio policy. It is important to note that appropriate Human Resource (HR) tools are in place and well defined, but could be leveraged more strategically for the organization's benefit. Related to this, improvements to the management of research would complement this strategic focus in relation to knowledge.
Risk Analysis
At the outset of this audit, a wide range of inherent risks were identified and laid the foundation for the audit focus and approach. Audit results indicate that many of these risks are under control; however, we remain concerned that the department is exposed to a range of inter-related and mutually compounding risks. First, from a process perspective, audit results indicate that the direction-setting processes remain characterized by insufficient internal substantive discussion and challenge function in relation to emerging issues, policy options and priorities. This risk is exacerbated by the risk that the organization's culture and operating practices do not yet fully allow for the optimal sharing of information and engaging of necessary stakeholders. Together, this "process" and "culture" risk lead to insufficient informational inputs for decision-making and oversight which ultimately results in a significant risk of misalignment. Specifically, we are concerned that departmental policy, priorities and related resources may not reflect the most critical imperatives. Related to this, there is a high risk that interactions and liaison with portfolio agencies may be misaligned or uncoordinated. Taken together, the impacts of these risks include not only sub-optimal policy development, inefficient use of scarce resources and implementation challenges around key directions, but fundamentally, may result in a missed opportunity for the department to play its important leadership role in relation to the portfolio and other key stakeholders.
Audit Option
In my opinion, the mechanisms in place to support departmental and portfolio-wide direction are generally adequate and effective, although some important areas require management attention to address some residual risk exposure.
Statement of Assurance
In my professional judgment as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided and contained in this report. The opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed upon with management. The opinion is applicable only to the entity examined.
Summary of Recommendations
The following recommendations are made to the Assistant Deputy Minister, Strategic Policy Branch:
- Continue to pursue open and collaborative policy development mechanisms, to build on early progress that has been made in relation to the engagement of policy thinkers at all levels of the organization. The goal will be to create and sustain an open, creative and enabling environment for idea generation, which can then be harvested by the more formal practices referenced in the following recommendations.
- Formalize the departmental and portfolio governance arrangements and processes in relation to the planning, management and review of the overarching Public Safety policy agenda. To this end,
- Elaborate and specify the role of Strategic Policy Branch (SPB) in relation to the management of the departmental and portfolio policy agenda, equipping it, as necessary, with the resources needed to effectively play this role. This role should include, but may not be limited to the setting of the integrated, strategic policy priorities for the department and the portfolio through processes such as the MTP and through active engagement with ADMs and portfolio policy leads. A formal portfolio coordination function may need to be established for this purpose.
In elaborating this role, care should be taken to ensure SPB plays a discrete role relative to ADMs and portfolio policy leads, to allow the latter to continue to develop and implement policy in their own realms. Rather, the role of SPB should be focused on setting and reviewing overarching directions, within which policy-makers work. - Consolidate these integrated policy orientations into a Strategic Policy Framework (SPF) that sets out the medium to long term policy directions for the department and the portfolio. It is expected that this policy framework will provide the basis for portfolio engagement on matters of direction and, as noted below, will provide the high-level (but not prescriptive) frame according to which lower-level directions pursued within branches can be developed, reviewed and endorsed.
The framework should include, but may not be limited to:- Long-term policy vision and outcomes being pursued;
- Policy context, strategic drivers and resultant policy imperatives;
- Governance, roles, responsibilities and accountabilities for direction-setting;
- Touch points and areas of complementarity in relation to substantive policy topics; and,
- Policy priorities to be pursued over the medium to long-term.
- Update the terms of reference for key committees that have an influence from a direction-setting perspective. This review should examine, clarify and where necessary, streamline the manner in which various committees inter-relate, in support of information sharing and engagement related to direction-setting.
- Elaborate and specify the role of Strategic Policy Branch (SPB) in relation to the management of the departmental and portfolio policy agenda, equipping it, as necessary, with the resources needed to effectively play this role. This role should include, but may not be limited to the setting of the integrated, strategic policy priorities for the department and the portfolio through processes such as the MTP and through active engagement with ADMs and portfolio policy leads. A formal portfolio coordination function may need to be established for this purpose.
- The SPF should be reviewed for its currency on an annual basis by a senior committee with composition from across the department and, eventually, the portfolio. The ADM Policy Committee would be an appropriate mechanism for this.
- Ensure that new departmental policy directions or priorities, early in their lifecycle, are efficiently reviewed against the SPF, for assurance that their expected results are in line with the broader policy directions and orientations. It is strongly recommended that this review take place at a departmental forum with horizontal composition, to enable open dialogue. The use of the Director General (DG) policy committee for this 'signals checking' would be appropriate.
- Continue to enhance the MTP process such that it incorporates and uses key inputs such as risk, threat and performance information as well as research and portfolio agency input.
- Aligned with the improvements to the MTP process, develop an inventory of available research and research plans, leveraging the material that exists across PS branches and the portfolio agencies. The goal of this will be to ensure that policy planners have efficient access to available research. The consolidation of plans will have the added benefit of compelling a review of potential synergies between research projects and potential complementarity and streamlining of efforts. Gaps in knowledge will also be more easily identifiable.
- In light of the importance of risk and performance information as key drivers of direction-setting, as part of the evolution of Integrated Risk Management (IRM), develop department-wide guidance on "risk- and results-enabled decision-making" to clearly outline the expectations and approaches for integrating risk, threat and performance information into the following key direction setting processes:
- Policy planning
- Policy development
- Priority-setting
- Program design
- Early in the Integrated Human Resource Business Plan (IHRBP) process, as annual departmental policy priorities are being identified, the ADM Policy Committee should review the inventory of proposed policy priorities, to ensure their alignment with the SPF. The goal of this review will be to better integrate and align strategic policy with the annual priority setting process. Once reviewed and vetted by ADM Policy Committee, these priorities should be formally endorsed and approved by Departmental Management Committee.
- Mid-year and year-end departmental results against priorities (both policy and management priorities) should be formally discussed at Departmental Management Committee, accompanied by Branch specific results for information.
Management Response
The Audit on Policy, Priority-Setting and Planning acknowledges the complexity and interconnectedness of policy issues that fall under the mandate of the Minister of Public Safety. Understanding these issues requires an integrated approach to policy development. While the Strategic Policy Branch (SPB) plays a central role in this process, the involvement of departmental and Portfolio partners at all levels remains a critical success factor. In order to develop a cohesive storyline for Public Safety Canada, SPB will reinforce horizontal collaboration across the Department and Portfolio to better inform policy decisions in support of Ministerial and Government priorities. The actions flowing from the Management Response and Action Plan are expected to improve the efficiency and effectiveness of decision making by implementing an appropriate governance structure to streamline processes and better align policy priorities and resources, both financial and human.
SPB supports the audit's findings and recognizes the importance of better integrating policy development across the Department and the Portfolio. Delivering on all elements of the action plan, will be contingent on securing additional resources for the development and monitoring of the Strategic Policy Framework and to support a new approach to research. Where appropriate, SPB will aim to leverage existing mechanisms and processes to meet the audit's objectives.
Audit Team Members
Carmen Abela - Windreach Consulting Services Inc.
Yolande Andrews
Deborah Ann Duhn
Melissa Greenland
Cathy Kwan
Kyle Abonasara
Acknowledgements
Internal Audit would like to thank the following individuals for providing advice and assistance during the audit:
Marie-Josée Bernier, Canada School of Public Service
René Bouchard, Department of Canadian Heritage
Jules Leblanc, Canadian Security and Intelligence Service
Dr. Ivan Fellegi, Emeritus Statistician, Statistics Canada
Sandra Lafortune, Transport Canada
Debbie Winker, Agriculture and Agri-food Canada
Introduction
1.1 Background
On December 12, 2003, the Prime Minister announced the creation of the Department of Public Safety & Emergency Preparedness. The goal was to better integrate government efforts to secure the safety of Canadians, as well as protect against and respond to national crises, natural disasters and emergencies. The creation of the department and the portfolio was intended to provide a more coordinated, integrated approach to public safety. While the individual components of the portfolio are expected to deliver on their distinct mandates, according to their own enabling legislation, a common reporting relationship to a single minister is intended to provide a means by which the government can promote the effective and efficient coordination of efforts.
The Department of Public Safety & Emergency Preparedness Act mandates the Minister of Public Safety with a unique role in leading the portfolio which includes authority to establish strategic priorities and to initiate, recommend, coordinate, implement or promote policies for those entities relating to public safety and emergency preparednessFootnote 2. Similarly, the Deputy Minister of Public Safety Canada (PS) is considered the principal policy advisor to the Minister of Public Safety. As part of this role, he is mandated to examine, coordinate, provide advice on and support the efforts of portfolio agencies in support of the overarching mandate of the Minister, portfolio and department.
The development of cohesive and aligned directions is critical in the context of today's environment. The increasing horizontality, breadth of scope and complexity of files, combined with scarce resources, require mechanisms to enable the development of well-informed and appropriately integrated directions (whether policy, priorities or plans). It is important to note that full integration on all matters is not necessarily required; however, to achieve the objectives set out with the creation of the department and the portfolio, appropriate and efficient mechanisms are needed to enable a comprehensive approach to strategic and operational direction-setting. Such mechanisms enable the achievement of mandates with the best possible use of resources while exploiting inherent synergies within and between organizations.
While coherence and coordination is vital, so also is flexibility. The business of direction-setting can be "messy": sometimes purely reactive, at times organic and entrepreneurial and often done on short timeframes. Thus, direction-setting mechanisms must simultaneously provide structure and coherence, but still be adaptable and efficient enough to navigate the inherent realities facing departments such as PS.
The complexity of the direction-setting process inherently exposes PS to a range of risks, ranging from the potential for mis-alignment of directions internally, or with their portfolio leads to the risk of inappropriate governance or processes in support of direction-setting. In light of these risks and the importance of direction setting to PS's mandate, an internal audit of Policy, Priority-setting and Planning was conducted.
1.2 Audit Objective
The audit objective was to provide reasonable assurance that the departmental mechanisms in place to set and communicate clear and cohesive strategic and operational direction for itself and the portfolio are adequate and effective.
1.3 Scope and Approach
The scope of the audit included all PS branches and focused on the policy, priority-setting and planning processes that collectively enable direction-setting and communication and which are illustrated below:
Figure 1: Overview of Direction-setting Processes
Image Description
A chart entitled Departmental and Portfolio Direction-Setting Processes displays a set of four boxes linked by a series of arrows from top to bottom. This top-down approach illustrates how the Government of Canada directs Departmental and Portfolio Policy and their Priorities and Departmental Plans. These boxes are labelled, from top to bottom, as follows:
- Government of Canada Direction (PM, Cabinet, National Security Advisor, Minister, etc.)
- Departmental and Portfolio Policy (What direction is being taken and why?)
- Departmental and Portfolio Priorities (What needs to be done to follow the direction?)
- Departmental Plans (How will the Department implement the direction?)
The output (results, risks and emerging issues) from the Departmental Plans are then looped back to the Departmental and Portfolio Priorities and the Departmental and Portfolio Policy and the cycle continues.
Three key lines of enquiry were used to target the audit. The key questions posed in relation to each are provided below:
- Governance: The audit examined the adequacy and effectiveness of the governance structures and processes that enable the identification of portfolio and departmental directions and priorities;
- Processes and Informational Inputs: The audit examined the degree to which the processes are designed to incorporate appropriate horizontal and vertical linkages from within and beyond the department. It also assessed the degree to which the processes were appropriately enabled by sound and comprehensive information for direction-setting; and,
- Use and Usefulness of Outputs: The audit examined the relevance and usefulness of the planning documents in support of effective resource allocation and active monitoring.
The audit generally covered the 2011-12 periods, but in light of the continued evolution of a number of processes, some of the testing was done on an "under development" basis, examining the processes as they are evolving to provide management with timely advice and insight.
It is important to note that the audit focused solely on the processes and mechanisms owned by PS. While mechanisms to engage the portfolio and other stakeholders were examined, those that were owned by parties outside the department were outside the scope of the audit.
WindReach Consulting Services Inc. was retained to lead the audit and to direct a team comprised of internal departmental auditors. The audit team used interviews and documentation review to test the adequacy and effectiveness of controls. Limited consultations with other similar departments and agencies were used during the planning phase to gain an appreciation of various direction-setting mechanisms. As well, five strategic initiatives (directions) were sampled. The goal of this was to determine the degree to which the departmental mechanisms enabled effective and efficient setting of direction. Initiatives were 'traced back' from completion to their origins to understand and assess the degree to which the departmental mechanisms being audited were used and effective in support of the directions.
To ensure that the audit was relevant and covered all types of direction-setting circumstances, the sample of initiatives examined was selected to provide adequate coverage of the following types of direction-setting environments:
Type | Description | Sampled Initiative |
---|---|---|
Planned | The 'planned' model of direction-setting is characterized by known requirements, a reasonable line of sight in terms of critical path and often is accompanied by longer time frames for direction-setting. Transfer payment program design or renewal activities would constitute a 'planned' approach to direction setting. | Youth Gang Prevention Fund Program Renewal Cyber Security Strategy North American Plan for Animal and Pandemic Influenza |
Reactive | Directions set under the 'reactive' model tend to be unforeseen and directed from outside parties (including but not limited to the Minister) and /or external events. These are often developed under tight timeframes and often have limited to no precedent. Directions set in response to emergencies, crises or political imperatives often fall into this category. | Air India Action Plan |
Entrepreneurial | Directions set under the 'entrepreneurial' model tend to be developed organically and often serendipitously. They are often not planned, but are rather evolutionary in their identification and formulation. | Perimeter Security |
The audit criteria against which this audit was conducted were developed based on a variety of sources, including:
- Research and best practice in the areas of policy development and public administration;
- Guidance from central agencies and other key organizations on integrated planning;
- The Treasury Board Secretariat (TBS) Policy on Management, Resources and Results Structures; and,
- The Government of Canada's core management control model.
Appendix A provides a summary of the audit criteria against which the audit was conducted.
Appendix D provides the key information sources upon which the criteria were based.
1.4 Risk Analysis
A number of risks were identified during the planning phase of the audit as being inherently relevant. These have been presented in Appendix B. In keeping with a risk-based audit approach, the audit was targeted such that it focused on the mechanisms in place to manage these risks. Audit evidence suggests that a number of these risks are under control or that appropriate mechanisms are being built that will, over time, help to address the exposure. However, some outstanding exposure remains.
From a process perspective, audit results indicate that the direction-setting processes remain characterized by insufficient internal substantive discussion and challenge function in relation to emerging issues, policy options and priorities. This risk is exacerbated by the risk that the organization's culture and operating practices do not allow for the optimal sharing of information and engaging of necessary stakeholders. It is important to note, however, that recent efforts to create an open policy dialogue and increased collaboration are taking hold and are expected to have positive effects.
However, we remain concerned that together, the above-noted "process" and "culture" risks may lead to insufficient informational inputs for decision-making and oversight which ultimately results in a significant risk of misalignment. Specifically, we are concerned that departmental policy, priorities and related resources may not reflect the most critical imperatives.
Related to this, there is a high risk that interactions and liaison with portfolio agencies may be misaligned or uncoordinated. Taken together, the impacts of these risks include not only sub-optimal policy development, inefficient use of scarce resources and implementation challenges around key directions, but fundamentally, may result in a missed opportunity for the department to play its important leadership role in relation to the portfolio and other key stakeholders.
This is graphically depicted in the following illustration.
Figure 2: Risk Analysis
Image Description
A figure explaining risk analysis includes a rectangle with four text boxes inside it. The first text box, in the shape of a block arrow pointing towards the right, is in the upper left hand corner of the rectangle and has the words "Process risk" written inside it. The second text box, in the shape of a block arrow pointing towards the right, is located just below the first text box in the lower left hand corner of the rectangle and has the words "Culture risk" written inside it. These two text boxes are connected by a double-headed arrow that goes from the lower left hand side of the first text box to the upper left hand side of the second text box. Next to the double headed arrow are the words "Compounding effect leading to" followed by an ellipsis. To the right of the first and second text box is a curly bracket that starts in the middle of the first text box and ends in the middle of the second text box and points to the right and towards the third text box. This third text box, in the shape of a block arrow pointing towards the right, has the words "Information for decision-making risk" written inside it. To the right of the text box are the words "Results in" followed by an ellipsis and written at a 90° angle. There is a curly bracket that spans these words and points to the right and towards the fourth text box which has the words "Alignment risk" written inside it. There is a loop-back arrow that points upwards from the middle of the top of the fourth text box, flattens out and points back down, with the arrow-head ending in the middle of the top of the third text box. Above the flattened-out portion of this arrow are the words "Compounding effect".
In a separate rectangular box located below the first rectangle and connected by two downward arrows, the following ultimate impacts are listed:
- Sub-optimal policy design
- Implementation challenges around key directions
- Inefficient or suboptimal use of limited resources
- Missed opportunity to lead
- Diminished credibility
1.5 Audit Opinion
In my opinion, the mechanisms in place to support departmental and portfolio direction are generally adequate and effective, although some important areas require management attention to address some residual risk exposure.
1.6 Statement of Assurance
In my professional judgment as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided and contained in this report. The opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed upon with management. The opinion is applicable only to the entity examined.
Findings, Recommendations and Management Responses
2.1 Governance
The audit sought to determine whether the department has appropriate governance mechanisms for the effective development and oversight of policy, priorities and plans at the portfolio and departmentalFootnote 3 levels. To this end, the audit examined PS's oversight, decision-making and communication mechanisms as well as the means by which accountability for direction-setting is established. Mechanisms that enable an open and collaborative policy dialogue were examined, as were processes that allow for the scrutiny, challenge, endorsement and monitoring of directions. The following are the key findings.
2.1.1 Mechanisms for Engagement and Collaboration
The audit found that numerous committees and fora exist for information-sharing purposes both within and beyond the department. Recently reconstituted Director General (DG) and Assistant Deputy Minister (ADM) level Policy committees are positive steps forward and have been formed specifically to foster open discussion on matters of common interest. For this reason, in an effort to create a collegial environment, presentation of new directions at either policy committee has not been made mandatory.
The audit also found that the engagement of portfolio partners and other stakeholders is done well and is supported by effective structures at various levels of the hierarchy. This engagement tends to be on an issue-by-issue basis, rather than on a comprehensive or holistic basis (i.e., in relation to cross-cutting portfolio priorities or issues). These bilateral mechanisms are important to support direction-setting and addressing of issues on a file-by-file basis; however, the audit has concluded that more systematic mechanisms are needed to engage portfolio and other relevant partners on a collaborative and holistic basis. We understand that there is a stated desire to eventually involve portfolio agencies in the ADM Policy Committee, although this has not yet occurred due to the early stages of the committee's maturity. We feel that the eventual involvement of portfolio agencies in this forum is appropriate and strongly encourage this evolution.
The audit also concluded that external collaboration and information sharing appears to be more systematic and institutionalized than it is internally. The emphasis on external collaboration is indicative of the natural synergies and relationships that exist between PS Branches and their portfolio counterparts and is positive. However, in contrast, the audit found that the value of collaborating and information sharing internally, across Branches, is not always appreciated. We are concerned about the potential negative impact this may have on the comprehensiveness and quality of inputs to direction-setting, a finding which relates to issues raised in section 2.2 of this report. It is important to note, however, that recent efforts to institutionalize bi-weekly ADM meetings are positive steps towards creating a culture of collaboration and engagement within the department. Other efforts such as the Open Policy forum are viewed positively in this regard and are beginning to create an environment in which "bottom-up" policy ideas can be generated.
In examining the governance and information sharing bodies that are relevant to direction-setting, we noted that a number of important committees exist. Among them are the ADM and DG policy committees noted above, as well as the Planning and Reporting Network, the Portfolio Research Committee and others. By design, these mechanisms are useful to support information sharing among participants; however, the linkages between them are not clear. Thus, is it difficult to discern the manner in which information generated from one feeds others and how well and how efficiently they collectively support direction-setting.
2.1.2 Roles, Responsibilities and Accountabilities
The audit examined the mechanisms in place to establish clear roles, responsibilities and accountabilities in relation to direction-setting. This focus was taken as clear accountability is vital to support not only the efficiency of the direction-setting processes, but also the alignment of the directions themselves.
Many strength were noted. First, the audit concluded that the Deputy and Associate Deputy Ministers model the appropriate, constructive behaviour, setting a strong "tone at the top" and laying the foundation for effective accountability. These behaviours have had a positive impact by explicitly and implicitly creating an expectation and a standard for constructive engagement.
More formally, the audit also examined the mechanisms that exist to hold senior executives to account for their policy mandates. We found that the roles of Branches are defined, although at a relatively high level. We also concluded that appropriate mechanisms exist to hold ADMs individually accountable for their roles in policy- making. Branch level accountability mechanisms, such as the mid-year and year-end reviews, complement individual mechanisms, although it is not always clear how they are used to hold Branch heads accountable.
Finally, the audit examined the role of Strategic Policy Branch (SPB), relative to line Branches - all of whom are involved in the setting of direction. The audit concluded that SPB plays a unique and recognized role in some key areas, including risk management and integrated business planning. Opportunity exists however, to further clarify and elaborate their role in relation to policy development. At present, their mandate is to play a "leadership role in policy development and integration across the Department and the Public Safety Portfolio" and to "provide strategic policy advice on the development and implementation of the Government's and Department's broad public safety agenda". The audit concluded that in practice, the scope of this role is not clear, leading to concerns of overlap, duplication and possible misalignment of activities.
We understand that the function of strategic policy is complex and is often characterized by "grey zones" and natural (often valuable) tension between players. Risk of overlap and 'competition' between policy centres is, to some degree, inevitable in light of these factors and of the horizontality of files being managed. It is even more so when, like with PS, strong policy functions exist in Branches, resulting in challenges for SPB in differentiating itself from other Branches in this regard. While understandable, there is nonetheless an increased risk of inefficient or misaligned activities and therefore, directions.
Further, while it is acknowledged that SPB may have a role to play on specific policy files, the audit identified an opportunity for the Branch to play a unique role that is currently not being addressed and that, if it was, would increase complementarity, not competition, between Branches. This role relates to the setting and management of the strategic policy agenda for the department and portfolio. This issue is further elaborated below.
2.1.3 Management of the Policy Agenda and Portfolio Coordination
As outlined above and as further elaborated upon in Section 2.2.3, the audit concluded that the governance and processes in place to support direction-setting for individual policy files is generally adequate and effective. The audit also examined the processes and governance that support holistic policy direction, in keeping with the mandate of the department as a portfolio lead. The term holistic refers to the overarching direction for the department and the portfolio.
Audit results indicate that there are limited mechanisms in existence to holistically or substantively examine, scrutinize and make decisions on the departmental or portfolio policy directions. Specifically,
- While the department has established various policy priorities through different means, it does not have in place an overarching policy framework or roadmap that sets out the medium to long term policy imperatives for itself and for the portfolio as a whole. Such a framework is critical to allow for the establishment of key policy priorities for the department and to provide policy makers within the department and across the portfolio with a common starting point and broad, expected policy outcomes with which they should be aligned;
- As noted earlier in the report, within the department, there are limited formal mechanisms for systematic, horizontal "signals checks" in relation to policies under development. ADM and DG level policy committees exist but, as noted above, are not mandated to play a challenge role, in part because of management's specific objective of creating a collegial environment. Scrutiny, review and endorsement of directions are generally done through the vertical accountability channels of the lead branch, in accordance with the delegated authority of the ADM. The limited horizontal challenge role that is played is strained when directions are being set under tight timeframes. It is important to note that the vertical channels are appropriate and in line with delegated authorities. As well, we fully support senior management's objective of creating opportunities for openness and collegiality. However, the absence of a horizontally-focused forum to systematically review, discuss and endorse individual directions against departmental and portfolio objectives increases the risk of mis-alignment.
- Similarly, opportunity exists to considerably strengthen mechanisms that enable holistic portfolio coordination in relation to overarching departmental or portfolio directions. It is important to note, however, that roles, responsibilities and accountabilities for portfolio coordination are decentralized and work well on a bilateral, issue-specific basis. And while the Cabinet agenda and mandate priorities are closely tracked, no single party has responsibility for the systematic setting of overarching policy priorities and for reviewing progress against commitments made. Indeed, over time, it may undermine the very objectives of collegiality that senior management is so rightly trying to institutionalize.
It must be stated that the above-noted conditions are not atypical in light of the organization's maturity. Indeed, important developments such as the use of Policy Committees and the Medium Term Policy Planning (MTP) process (see Section 2.2.2) are laying the foundations which are necessary for the department to address these issues.
At this juncture, considerable opportunity exists for the department to take an active leadership role in setting and developing its long term, integrated policy agenda. In so doing, the department can solidify its role as portfolio lead and principal policy advisor to the Minister. Findings and recommendations related to the MTP process are provided in Section 2.2.2 and will help to address this issue. In addition, a stronger and more clearly defined role for SPB in the area of policy planning, management and monitoring is also needed. Resources should be appropriately aligned with SPB's role.
Recommendations
The following recommendations are made to the Assistant Deputy Minister, Strategic Policy Branch:
- Continue to pursue open and collaborative policy development mechanisms, to build on early progress that has been made in relation to the engagement of policy thinkers at all levels of the organization. The goal will be to create and sustain an open, creative and enabling environment for idea generation, which can then be harvested by the more formal practices referenced in the following recommendations.
- Formalize the departmental and portfolio governance arrangements and processes in relation to the planning, management and review of the overarching Public Safety policy agenda. To this end,
- Elaborate and specify the role of Strategic Policy Branch (SPB) in relation to the management of the departmental and portfolio policy agenda, equipping it, as necessary, with the resources needed to effectively play this role. This role should include, but may not be limited to the setting of the integrated, strategic policy priorities for the department and the portfolio through processes such as the MTP and through active engagement with ADMs and portfolio policy leads. A formal portfolio coordination function may need to be established for this purpose.
In elaborating this role, care should be taken to ensure SPB plays a discrete role relative to ADMs and portfolio policy leads, to allow the latter to continue to develop and implement policy in their own realms. Rather, the role of SPB should be focused on setting and reviewing overarching directions, within which policy-makers work.
- Consolidate these integrated policy orientations into a Strategic Policy Framework (SPF) that sets out the medium to long term policy directions for the department and the portfolio. It is expected that this policy framework will provide the basis for portfolio engagement on matters of direction and, as noted below, will provide the high-level (but not prescriptive) frame according to which lower-level directions pursued within branches can be developed, reviewed and endorsed.
The framework should include, but may not be limited to:
- Long-term policy vision and outcomes being pursued;
- Policy context, strategic drivers and resultant policy imperatives;
- Governance, roles, responsibilities and accountabilities for direction-setting;
- Touch points and areas of complementarity in relation to substantive policy topics; and,
- Policy priorities to be pursued over the medium to long-term.
Once departmental and portfolio policy orientations are established, mechanisms should be put in place to review and track progress against them and in so doing, determine the degree to which individual directions are contributing to and enabling the achievement of overarching policy outcomes.
- Update the terms of reference for key committees that have an influence from a direction-setting perspective. This review should examine, clarify and where necessary, streamline the manner in which various committees inter-relate, in support of information sharing and engagement related to direction-setting.
- Elaborate and specify the role of Strategic Policy Branch (SPB) in relation to the management of the departmental and portfolio policy agenda, equipping it, as necessary, with the resources needed to effectively play this role. This role should include, but may not be limited to the setting of the integrated, strategic policy priorities for the department and the portfolio through processes such as the MTP and through active engagement with ADMs and portfolio policy leads. A formal portfolio coordination function may need to be established for this purpose.
- The SPF should be reviewed for its currency on an annual basis by a senior committee with composition from across the department and, eventually, the portfolio. The ADM Policy Committee would be an appropriate mechanism for this.
- Ensure that new departmental policy directions or priorities, early in their lifecycle, are efficiently reviewed against the SPF, for assurance that their expected results are in line with the broader policy directions and orientations. It is strongly recommended that this review take place at a departmental forum with horizontal composition, to enable open dialogue. The use of the Director General (DG) policy committee for this 'signals checking' would be appropriate.
Management Response
Action Planned | Planned Completion Date |
---|---|
SPB will continue to provide leadership to initiatives such as Open-Source Policy discussions and the Community Resilience tiger team to promote new approaches to collaboration and innovation in policy development. | Ongoing |
SPB will strengthen its leadership role for horizontal processes such as the medium-term policy planning (MTP) process and environmental scanning across the Portfolio, to help identify policy gaps (see recommendation 5). | November 2013 |
SPB will maximize opportunities to share information across the Department and Portfolio on research and policy development. | November 2013 |
In consultation with Departmental and Portfolio colleagues, SPB will review the existing governance arrangements and make recommendations on a process to better integrate policy development across the Department and the Portfolio. | March 2013 |
SPB does not envision a formal Portfolio coordination function. SPB's ability to lead on the development and monitoring of the Strategic Policy Framework and, in support, a new approach to research will be contingent on securing additional resources, as detailed below and in recommendations 2b, 2c, 3, 4 and 6. |
November 2013 |
SPB's Portfolio management role will encompass the following:
|
November 2013 |
SPB will develop a Strategic Policy Framework (SPF) with overarching policy orientations through engagement with departmental and Portfolio partners. The SPF will be informed by horizontal process such as the MTP process, environmental scanning and by Portfolio-wide research as key drivers for: Existing policy directions based on:
Forward-looking, medium to longer-term elements based on:
|
November 2013 |
When the SPF is developed, departmental policy centres will establish individual policy directions aligned with the overarching policy outcomes of the SPF. In setting the policy agenda in consultation with the Department and the Portfolio, SPB will review the alignment of the individual policy directions to the SPF. |
Starting November 2013 |
After conducting a review of existing governance structures supporting direction setting (see recommendation #2), SPB will develop options to support increased information sharing and engagement and make recommendations on adjustments to the terms of reference of appropriate committees, where necessary. | Starting November 2013 |
When the SPF is developed, departmental policy centres will establish individual policy directions aligned with the overarching policy outcomes of the SPF. In setting the policy agenda in consultation with the Department and the Portfolio, SPB will review the alignment of the individual policy directions to the SPF and assess the currency of the SPF. |
Starting November 2013 |
When the SPF is developed, departmental policy centres will establish individual policy directions aligned with the overarching policy outcomes of the SPF. In setting the policy agenda in consultation with the Department and the Portfolio, SPB will lead a review of the alignment of the individual policy directions to the SPF. As part of this process, SPB will lead a DG-level 'signals checking" of the SPF in advance of the review by a senior management Committee, rather than in response to the timing of all new initiatives. |
Starting November 2013 |
2.2 Processes and Informational Inputs
The second key line of enquiry of this audit related to the adequacy and effectiveness of the department's direction-setting processes and the appropriateness of the informational inputs that enable them. To this end, the audit examined whether PS employs structured, systematic and integrated processes for the development of policy, priorities and plans, and whether these processes are appropriately informed by comprehensive and relevant information. The audit also examined the human resources processes that help to plan for, create and sustain the knowledge and expertise needed to support effective direction setting. Formal processes as well as the informal organizational culture and environment within which the processes operate, were examined.
2.2.1 Culture of Collaboration
As noted at the outset of this report, the environment, issues and directions pursued by PS are extremely horizontal and growing more so over time. Inter-dependencies and synergies abound and, if exploited appropriately, enable effective and efficient direction-setting. A key enabler of collaboration is the organization's culture and "control environment"; as such, the audit assessed its strength.
The audit found that a culture of collaboration is not yet fully institutionalized across the department and indeed, that the value of horizontal collaboration is not always acknowledged. While the cause of this is difficult to definitively ascertain, audit evidence suggests that contributing factors include:
- Accountability mechanisms that do not incent collaboration and consultation;
- A culture of secrecy that works against disclosure and open dialogue on some policy issues; and,
- Formal direction-setting processes that do not compel internal players to come together early in the policy development activities.
The audit has concluded that relationships are generally relied upon, although important efforts are being made to begin to institutionalize horizontal collaboration at various levels of the organization.
2.2.2 New and Evolving Approaches to Policy Planning
In 2011-12, the department initiated, on a pilot basis, an MTP exercise, the goal of which was to examine policy gaps, challenges and opportunities for PS over the coming three to 5 year horizon. Generally speaking, medium term policy planning is an exercise that consolidates evidence, knowledge and policy expertise from key stakeholders to identify and plan for policy gaps over a longer period of time. The pilot process at PS relied on internal consultations and literature review to gain insight from analysts across the department into the emerging policy priorities.
We are pleased and highly supportive of this planning approach and see it as an important mechanism by which more rigors can be brought to the management of an integrated PS policy agenda. Further, we see it as an important tool of culture change, as the process compels horizontal consultation and engagement which is necessary to break down branch-specific silos. As the department deploys this on a regularized basis, the audit has identified a few areas that, if addressed, will strengthen the process and increase the value of its outputs. These are discussed below.
First, interview evidence indicates that the expected results of the MTP process have not yet been precisely or formally confirmed. While there is a general expectation that it will generate information in support of direction-setting, the specific linkages to the planning and formulation of directions - whether the overall policy agenda, specific policy initiatives or departmental priorities - are not yet clear.
As well, improvements to the nature and quality of the process's inputs would strengthen the process considerably. Of note are the following:
- Better, more systematic linkages to research are needed. While the pilot process relied on some in-house diagnostics that were developed based on a literature review, opportunity exists to leverage existing research products in a more fulsome fashion. As well, opportunities exist to use the outputs of the MTP process to help focus longer-term research priorities. At present, we understand that no such linkages are envisioned.
- Limited environmental scanning was done. Where existing scans existed (e.g., RCMP), they were leveraged. We understand that resource constraints were one of the main reasons for this. Despite this reality, we are concerned that the lack of a fulsome examination of external and emerging factors may lead the department to draw conclusions on policy drivers and imperatives that may not be fully or appropriately informed.
- Similarly, more opportunity exists to integrate risk and threat information into the process. As designed, the pilot process did not use existing risk or threat information that resides within PS or across the portfolio. Best practice for risk management suggests that plans and directions should be set, in part, based on a consideration of risk, with the goal being that resources are allocated to areas where there is a need or a value in terms of risk mitigation. The setting of medium term policy priorities is no exception. Thus, as the MTP process evolves, risk information from various sources should be leveraged.
- Portfolio partners and other key stakeholders were not engaged in the pilot MTP process. As noted earlier, much opportunity exists to more formally engage these players in the discussion and setting of an overarching direction for the portfolio. We also note that the MTP process has the potential to lay a strong foundation for this overarching direction. Given this, the involvement of external parties in the MTP process going forward is of paramount importance. Not only does this provide a mechanism for engagement - an outcome that is valuable in itself - it also ensures that portfolio perspectives are considered early, as part of planning efforts.
It is important to acknowledge the relative newness of the MTP process. Indeed, while we understand that the MTP process is still under development, anecdotal evidence suggests that it provides a good forum to help break down internal silos and effect culture change. Enhancing the process with stronger inputs as suggested above, will further realize benefits.
2.2.3 Direction-Setting Mechanisms: Policy
As noted earlier in this section, the audit examined the appropriateness, robustness and consistency with which individual policy directions were developed. Unlike the priority setting process (see section 2.2.4), the department does not (nor would we expect them to) have a formal policy development map, against which the audit could take place. Accordingly, a generic map of the policy development process was developed based on standard practice and validated by management. This map is appended to the report (Appendix B).
Overall, the audit found evidence that while all aspects are not formally documented, policy makers adhered to the same principles and practices related to policy formulation. Some aspects of the process (e.g., Memorandum to Cabinet (MC) development activities) are more formally documented and implemented than others while other aspects of the process are more informally implemented (e.g., idea generation, development of policy options, etc.).
Inputs to the policy development process varied across initiative and across Branch, yet in general, they were balanced with the nature of the policy file and the environment in which it was developed. For instance, directions developed under the 'entrepreneurial' model relied more on informal consultation, with fewer inputs being documented, which is to be expected given the organic nature of the early policy formulation stage. Some opportunity exists, however, to strengthen the challenge function around some of the inputs used. This reflects the concern raised earlier in the report related to scrutiny and 'signals checking'. While we do not foresee the creation of a heavy, bureaucratic process, more rigors in the review and challenge processes for individual directions at specific times in their lifecycle would nonetheless be valuable to ensure their alignment with broader orientations.
As well, regardless of the nature of the policy development process, some key inputs were consistently not used to inform policy directions, among the most notable of which are information on risk, threats and past performance. Research inputs are used, but we are concerned that opportunities exist for better integration of research, yet are being missed. In all cases, policy development is characterized by consultation; however, as noted earlier, consultation with external parties was more institutionalized that it was with internal players. The causes of this have been noted above and appear to be reinforced by the organizational culture that in some cases does not fully appreciate the value of early, internal dialogue on specific policy files.
In addition to these cultural issues, the policy development process itself does not compel key players to come together early in the formulation of directions. While Cabinet Affairs makes attempts to line up the right players internally, it cannot compel horizontal collaboration and engagement. Moreover, their involvement begins at the MC formulation stage, by which stage much of the policy thinking has already taken place. Relationships are relied upon to initiate dialogue and while the DG and ADM policy committees are well positioned to stimulate dialogue, they have not been designed as mandatory "gates" for policy initiatives. Some other departments address this by creating "cabinet coordination teams" designed to establish important linkages early on in the policy development process; however, no such teams are in use at PS. As a result, while the audit found evidence of internal review and sign off within a Branch in accordance with the delegated authority of the ADMs, it concluded that there are no formal or systematic mechanisms for horizontal signals checking as policies are being developed. Thus, there is limited ability to review, scrutinize and endorse individual directions against holistic or horizontal considerations at the departmental or portfolio levels.
2.2.4 Direction-Setting Mechanisms: Priority-Setting and Planning
The audit concluded that the department's processes for the setting of annual priorities is robust, adheres to expected practice and continues to evolve in positive directions. Specifically, the department has a defined and accessible departmental planning and reporting cycle which provides a high-level overview of how priorities are set annually for the department. Appropriate guidance and tools are prepared and distributed by SPB for the development of Branch plans.
At a high-level, the process as designed aims to consider and respond to a wide range of inputs including: emerging issues, risks, Management Accountability Framework (MAF) results, and current and past performance, evaluation and audit outcomes, and strategic review commitments. Interview evidence indicated that inputs are indeed provided to management during the priority-setting process and are aggregated, by Branch, in the planning material (e.g., 'placemats'); however, there is no evidence that these inputs directly drive the setting of priorities. In particular, like the setting of policy directions noted above, the links between priorities and the risks they are intended to help manage are not explicit or clear.
Once Branch plans are developed, prior to their finalization, "Integration Sessions" are held which allow for horizontal discussion and challenge to take place. Until this year, these sessions brought together individual Branches with key stakeholders from Internal Services. In 2011-12, the process was expanded somewhat to bring together all Branches to challenge each other - an evolution that we view as a positive contribution to transparency and horizontality. In addition, in 2011-12, ADMs engaged collectively around a discussion of departmental priorities, prior to the formal integration sessions.
While internal horizontal discussion is being strengthened, more opportunity exists to engage portfolio agencies on the setting of priorities. This could be done by consulting them early in the process of priority identification, or to review and discuss the final priorities, once identified.
Finally, we noted that the linkages between priorities and the policy process are not as explicit as they could be. While the Integrated Human Resources Business Plan (IHRBP) identifies policy priorities (in addition to management priorities), the policy process identified above and the IRHBP process are not formally linked, either through governance or process. There is an acknowledgement that the policy agenda needs to more formally drive annual priority setting. At present, individual relationships between key parties help to bridge the gap; moreover, it is expected that the MTP process, once fully deployed will also provide a formal connection. However, at the present time, a gap exists which, if left unaddressed may have a negative impact on the alignment and cohesiveness of directions.
2.2.5 Human and Knowledge Management
As noted above, the audit sought to examine whether the department has adequate tools and processes to ensure it has the necessary knowledge and expertise now and into the future to support its current and emerging policy priorities. The audit found that human resources tools such as Human Resource (HR) Plans, Individual Learning Plans, etc. are in place and well defined, although the sole "beneficiary" of these tools tends to be the individual employee. While this focus is appropriate, the potential benefit of these tools to the organization as a whole must also be borne in mind. In particular, when married with a strong sense of strategic direction and priorities, HR planning and learning tools can be harnessed as a powerful mechanism by which the organization can manage its corporate knowledge strategically and position itself more effectively. While this issue may not be pressing in its urgency, the audit has concluded that opportunity exists to leverage them as strategic tools by which the organization can develop and sustain capacity.
Organizational knowledge extends beyond the talents and competencies of individuals and extends to the consolidated informational assets that the department and portfolio creates through its research activities. While we did not conduct a detailed assessment of the research function, the audit did note that some opportunity exists to manage the departmental and portfolio research activities in a more strategic fashion to maximize the benefit from the scarce resources that are allocated. For instance, research is planned for and conducted in silos, making it difficult to identify potential synergies and economies of scale that could be exploited. This issue relates to the broader issues raised earlier in this report in relation to policy planning and should be addressed by management in a concerted fashion. Simply stated, in light of increasing horizontality of files and resource constraints, more opportunity exists to think strategically about what research and knowledge is needed in support of departmental and portfolio policy.
2.2.6 Communication of Directions
Finally, the audit examined the processes that exist to communicate directions, once established. As noted under Governance, numerous mechanisms exist to engage, consult and communicate in relation to directions. Further, once directions are set, external communications are well controlled. However, as noted above under Policy Agenda, no mechanisms yet exist to develop and communicate the overarching goals, priorities and policy agenda of the department in a holistic fashion.
Recommendations
The following recommendations are made to the Assistant Deputy Minister, Strategic Policy Branch:
- Continue to enhance the MTP process such that it incorporates and uses key inputs such as risk, threat and performance information as well as research and portfolio agency input.
- Aligned with enhancements to the MTP process, develop an inventory of available research and research plans, leveraging the material that exists across PS branches and the portfolio. The goal of this will be to ensure that policy planners have efficient access to available research. The consolidation of plans will have the added benefit of compelling a review of potential synergies between research projects and potential complementarity and streamlining of efforts. Gaps in knowledge will also be more easily identifiable.
- In light of the importance of risk and performance information as key drivers of direction-setting, as part of the evolution of Integrated Risk Management (IRM), develop department-wide guidance on "risk- and results-enabled decision-making" to clearly outline the expectations and approaches for integrating risk, threat and performance information into the following key direction-setting processes:
- Policy planning
- Policy development
- Priority-setting
- Program design
- Early in the IHRBP process, as annual departmental policy priorities are being identified, the ADM Policy Committee should review the inventory of proposed policy priorities, to ensure their alignment with the SPF. The goal of this review will be to better integrate and align strategic policy with the annual priority setting process. Once reviewed and vetted by ADM Policy Committee, these priorities should be formally endorsed and approved by Departmental Management Committee.
- Mid-year and year-end departmental results against priorities (both policy and management priorities) should be formally discussed at Departmental Management Committee, accompanied by Branch specific results for information.
Management Response
Action Planned | Planned Completion Date |
---|---|
When the SPF is developed, departmental policy centers will establish individual policy directions aligned with the overarching policy outcomes of the SPF. In setting the policy agenda in consultation with the Department and the Portfolio, SPB will lead a review of the alignment of the individual policy directions to the SPF. As part of this process, SPB will lead a DG-level 'signals checking" of the SPF in advance of the review by a senior management Committee, rather than in response to the timing of all new initiatives. | Starting November 2013 |
MTP is one of several inputs into the yearly departmental priority-setting exercise and other decision-making processes. SPB will continue to develop a more robust MTP process by incorporating key information, including input from the Corporate Risk Profile. | November 2013 |
In coordination with the departmental and Portfolio research community, SPB will provide leadership in setting priorities for data collection, knowledge housing and dissemination, and engagement with outside researchers and networks, to support and shape policy development. Horizontal processes such as the MTP process and environmental scanning across the Portfolio, will help identify policy gaps and thereby guide priority setting for research. SPB will focus on developing, maintaining, and making available the evidence base needed to support priority policy issues. As such, options being considered include how existing research committees can better interrelate and ensure that research is shared across the Portfolio. The options will also delineate the parameters of aninventory of available research and research plans. | March 2013 |
2.3 Use and Usefulness of Outputs
The final line of inquiry for this audit was the examination of the mechanisms in place to ensure policy, priorities and plans are clear and useful to managers. While the objective was not to assess the degree to which the directions were effectively implemented, the audit did examine the mechanisms by which the directions, once determined, were put on a solid course towards implementation through the application of some key management practices. Specifically, we assessed whether directions, once set at the higher levels, effectively and clearly drove lower level plans, activities and resourcing decisions. It is important to note that the audit did not duplicate earlier audit work that was done in relation to planning and resource allocationFootnote 4. Indeed, this audit relied on findings from this closely-related project in the area of implementation planning and resource allocation. However, the audit did examine the degree to which lower level priorities were driven by and vertically aligned with higher order directions. An examination of the robustness of work planning was also done to determine if methods exist to operationalize higher level directions into tactical plans.
2.3.1 Alignment of Directions
First, the audit sought to determine whether directions, once set, were vertically and horizontally aligned with one another and were therefore cohesive. Directions that are aligned with one another are more likely to be effectively and efficiently communicated and implemented. Thus, the audit assessed whether:
- Policy directions for the department and portfolio were reflected concretely in corporate plans and further trickled down to lower level priorities and plans (i.e., were vertically aligned and cohesive); and,
- Whether priorities set within a single branch were coordinated and aligned with those in other branches (i.e., were horizontally aligned and cohesive). On this point, the audit also examined the internal mechanisms that allow for aligned departmental and portfolio priorities.
As noted in the earlier sections of the report, we remain concerned that there are insufficient mechanisms to enable horizontal alignment of directions. Findings and recommendations related to these issues are outlined elsewhere in this report.
While processes for horizontal alignment are not sufficiently mature, the audit did note that overall, directions within the department are generally effectively vertically aligned with one another. Specifically, we were able to trace the departmental priorities described in the IHRBP down to the lower level branch business plans and, within them, Directorate or Divisional work plans. Further, "milestones" or key outputs were also in evidence. Similarly, the audit could trace the five sampled initiatives to broader departmental priorities.
2.3.2 Work Planning, Resourcing and Monitoring
The audit also sought to determine if departmentalFootnote 5 work planning and resource allocation processes were adequately and effectively used to operationalize lower level directions and further, whether they laid the foundation for effective monitoring over time. Although we concluded that higher level directions could be traced down to lower level plans, we did note that lower level work planning at the Directorate or Divisional level is not done consistently, in part because there is no corporate standard for this practice.
From a resourcing perspective, the audit found that required resources for specific initiatives are determined during the MC and Treasury Board submission process (if applicable). However, the allocation of financial and non-financial resources and the tracking of their utilization by initiative are not easily done due to lack of consistent or robust work planning tools.
Similarly, general outcomes associated with a particular direction are established in the same fashion, yet neither financial nor non-financial results associated with a particular policy direction are systematically tracked. It must be noted that if the policy direction is articulated as a departmental priority (i.e., it is in the IHRBP), some tracking of non-financial results is done through the mid-year and year-end review processes. However, as noted elsewhere in this report, while mid-year and year end performance is assessed, it is not clear how the results are, in practice, used. As well, apart from the mid-year and year-end review process, at the present time, these priorities are not costed and as such, monitoring of financial results against specific priorities is not possible. We understand that a commitment has been made to identify and track costs associated with all departmental priorities and strongly encourage this development.
Recommendations
No recommendations are made for this line of enquiry.
Appendix A: Audit Criteria
Lines of Inquiry & Audit Criteria
Line of Inquiry 1: Governance
The department has appropriate governance mechanisms for the effective development and oversight of policy, priorities and plans at the portfolio, departmental and operational levels.
- Oversight, Decision-making & Communication Mechanisms:
Appropriate and effective departmental and inter-departmental oversight, decision-making and communication exist to allow for the discussion, setting and monitoring of directions. For in-year course corrections, monitoring mechanisms exist to obtain information that may signal the need to re-evaluate / adjust directions. - Overarching Policy Direction & Roadmap:
A clear and communicated departmental and portfolio direction, philosophy or framework exists and provides the starting point for the setting of directions within the department and across the portfolio. - Roles, Responsibilities & Accountabilities:
Roles, responsibilities and accountabilities related to the setting of directions are well articulated, aligned, communicated and understood by all parties in the department. - Accountability Mechanisms:
Processes exist within the department to hold management accountable for their role in the setting of directions. Managers are not only held accountable for these functions, but also the manner in which they are applied. (i.e., appropriate processes and inputs are used to support high quality outputs). - Challenge Function:
A formal, authoritative, constructive and substantive challenge function exists in relation to the setting of directions, whether policy, priorities or plans. - Approval & Endorsement of Directions:
Directions are formally endorsed and approved by senior management, prior to their communication. Approval processes allow for the consideration of horizontal / holistic examination of priorities (i.e., there is some mechanism to allow for the detection of Branch priorities that are overlapping, duplicative or working at cross-purposes). - Monitoring of Progress:
Processes exist to allow management to monitor and measure progress against their directions, in support of governance and oversight.
Line of Enquiry 3: Process & Informational Inputs
The department employs structured, systematic and integrated processes for the development of policy, priorities and plans, which are appropriately informed by comprehensive and relevant information.
- Processes for Information Gathering:
The department has in place methods, processes and structures that allow for the cohesive, comprehensive and coordinated identification and consideration of:- Emerging issues
- Risks (strategic, operational and financial)
- Current and past performance (operational and financial)
- Needs and opportunities.
- Policy Development Process:
The department has a formal and rigorous approach to policy development for short, medium and long-terms. The information inputs referred to in criterion 8 are formally and systematically considered and used in the setting of policy direction. - Priority-setting Process:
The department has a well-defined, effective and aligned priority-setting and planning process that establishes strategic and management priorities and expected results in line with emerging issues, risks and opportunities as well as the department's mandate, PAA, portfolio and government priorities. The information inputs referred to in criterion 8 are formally and systematically considered and used in the setting of priorities and plans. - Reporting & Information Sharing Processes:
Appropriate and formal reporting and information sharing mechanisms exist to ensure that directions (policy, priorities and plans) are effectively shared with key stakeholders (internally and externally). This includes the setting of new direction and the course corrections on existing directions. - Human Resources Planning & Capacity Building
The department has mechanisms (including but not limited to HR planning) in place to ensure sufficient, dedicated and capable resources to support research, policy analysis and priority setting. HR plans are developed in the broader context of the department's policy imperatives to allow for optimal alignment of human resources to policy imperatives.
Line of Enquiry 3: Use & Usefulness Of Outputs
Public Safety policy plans and priorities are clear and useful to managers across the portfolio.
- Vertical & Horizontal Alignment:
Directions, once set, are vertically and horizontally aligned with one another and are therefore cohesive. Specifically:- To ensure that policy directions for the department and portfolio are reflected in corporate plans (i.e., RPP, IHRBP) and further trickle down to lower level priorities and plans
- To ensure that priorities set within a single branch are coordinated and aligned with those in other branches
- To ensure the alignment of departmental and portfolio priorities
- Work Planning:
The outputs of the policy, planning and priority setting processes at both the department and branch levels effectively drive lower-level work planning and budgeting. Management considers them useful and constructive as aids to managerial decision-making.
Appendix B: Summary Risk Analysis
Risk Areas and Potential Risk Event | |
---|---|
Process Risks | There is a risk that there will be insufficient substantive discussion or challenge function exercised in relation to emerging issues, policy options and priorities |
There is a risk that direction-setting processes will not allow for comprehensive inputs | |
Setting of direction on a bi-lateral rather than horizontal basis (whether at the department or portfolio level) may lead to lack of transparency and challenges in communicating and executing directions. | |
Culture Risk | Organizational culture and operating practices do not allow for the optimal sharing of information and engaging of necessary stakeholders |
Information for Decision Making Risks | There is a risk that those responsible for the setting or oversight of direction-setting will not have access to sufficient information in support of their roles. |
There is a risk that the outputs of the direction-setting process will not be relevant, useful or meaningful in supporting management. |
|
Alignment Risks | There is a risk that there will be unclear or misaligned roles, responsibilities, decision-making processes or authorities. |
There is a risk that there will be insufficient coordination of players involved in policy, priority setting or planning, leading to directions that are at cross purposes to one another, contradictory or which do not fully address the underlying issue. |
|
There is a risk that interactions and liaison with portfolio agencies will be misaligned or uncoordinated, leading to overlap, duplication or missed opportunity to gather and consider comprehensive information. |
|
There is a risk that individual directions will be set without the benefit of substantive or horizontal analysis and may therefore work at cross-purposes, be contradictory or may not fully address the underlying issue(s) to be addressed. |
|
There is a risk that directions and related resources will not reflect the most critical imperatives, which may lead to sub-optimal program design, ineffectiveness or inefficiency. |
|
There is a risk that policy imperatives will not be appropriately reflected in corporate priorities or plans. |
|
Priorities will not be sufficiently resourced, resulting in implementation challenges |
|
Human Resources Risk | There is a risk that there will be insufficient policy development capacity or functionality |
Communications Risk | Directions will not be effectively communicated, leading to misaligned or divergent actions within the department and/or portfolio |
Appendix C: Generic Policy Development Process Map
Appendix D: References
- Bakvis, Herman and Luc Juillet. The Horizontal Challenge: Line Departments, Central Agencies and Leadership. 2004.
- Bourgault, Jacques and René Lapierre. Horizontality and Public Management. December 2000.
- Canada School of Public Service. Integrated Planning: Best Practice Learning Tool. 2009.
- Environment Canada. Guide to Building Dialogues on Horizontality: Discussion Paper. November 2000.
- Expert Panel on Integrated Planning. Better Choices for Better Results: Report of the Expert Panel on Integrated Business and Human Resources Planning in the Federal Public Service. 2008.
- Fellegi, Ivan. Strengthening our Policy Capacity. Deputy Minister Task Force Report. 1996
- Fitzpatrick, Tom. Horizontal Management: Trends in Governance and Accountability. 2000.
- Lahey, James. Moving from the Heroic to the Everyday. Lessons Learned from Leading Horizontal Projects. Canadian Centre for Management Development Roundtable on the Management of Horizontal Initiatives. 2001.
- Office of the Comptroller General of Canada. Core Management Control Model.
- Peters, Guy. Managing Horizontal Government. The Politics of Coordination. January 1998.
- Peters, Guy. The Policy Capacity of Government. Canadian Centre for Management Development. 1996.
- Privy Council Office of Canada. Accountable Government. A Guide for Ministers and Ministers of State. 2011.
- Steinberg, Samuel; Elizabeth Moore and Nathalie Burlone. The Management of Horizontal Issues: an Annotated Bibliography. A Working Paper for the CCMD's Action Research Roundtable on the Management of Horizontal Initiatives. Spring 2000.
- Treasury Board of Canada Secretariat. Management Accountability Framework Assessment Criteria. Round 8 (2010-11)
- Treasury Board of Canada Secretariat. Key Leadership Competencies. 2006
- 1
The mechanisms for operational direction setting were limited in scope to the department of Public Safety. As the department does not have this mandate in relation to Portfolio Agencies, this was not examined from a portfolio perspective.
- 2
Public Safety and Emergency Preparedness Act. 2004. Sections 5 & 6.1(a)
- 3
Departmentally, the audit also examined direction-setting mechanisms at the operational level (e.g., branch plans and priority setting mechanisms). This was not done for the portfolio level, as operational direction setting for the portfolio agencies are the responsibility of agency Deputy Heads.
- 4
See 2012 Audit of Financial Planning, Forecasting and Monitoring.
- 5
Note that this line of inquiry was not examined through the lens of the portfolio, as accountability for operational activities, including work planning, rests with the deputy heads of portfolio agencies.
- Date modified: