ARCHIVE - Evaluation of Two Components of the Effective Corrections Initiative: Public Education/Citizen Engagement PSECPC's Community Corrections -
Final Report

Adobe Acrobat version (PDF 116KB)

Table of Contents

• I. Introduction
  • 1.1. The Effective Corrections and Citizen Engagement Initiative
  • 1.2. Purpose of the Evaluation
  • 1.3. Evaluation Methodology
• II. Summary of Findings
  • 2.1. Enhancing the Community Corrections Infrastructure
    •  Was this a relevant response?
    •  Is it the most cost-effective response
  • 2.2. Public Education/Citizen Engagement
  • 2.3. Overall Findings
    •  Is the Initiative Relevant?
    •  Were the elements of due diligence (including eligibility) applied by the ECI?
    •  What can be done to deliver ECI in a more cost-effective manner?
    •  What are alternatives to ECI in attempting to meet the stated objectives?
• III. Analysis and Interpretation of Findings
• IV. Conclusions and Suggestions
  •  PSEPC Staff
  •  Project Contacts

I. Introduction

Cathexis Consulting Inc. was engaged to conduct an evaluation in order to assess the extent to which the objectives of the Results-based Management and Accountability Framework (RMAF) have been met in relation to two components of the Effective Corrections and Citizen Engagement Initiative (ECCEI).

1.1 The effective corrections and citizen engagement initiative

The Effective Corrections and Citizen Engagement Initiative is funded at $45M over five years from 2001 – 2005 and is intended to achieve the following goals:

• Address the over-representation of Aboriginal people in prison
• Support a better criminal justice policy
• Improve public confidence in the criminal justice system

Although the initiative includes an Aboriginal component designed specifically to address the over-representation of Aboriginal people in prison, this interim evaluation address the following two components:

• Enhancing the Community Corrections Infrastructure with the primary goal of expanding research and development aimed at evidence-based policy and program development in community corrections. It is intended to support innovative pilot projects with a focus on restorative justice. This evaluation includes the following projects related to this component:
  • Lanark County Community Justice Program – Perth
  • The Collaborative Justice Project – Ottawa
  • Toward an Integrated Model of Justice - Ottawa
  • Victim Companion & Contracting Safe Justice – Winnipeg
  • Restorative Justice Options to Parole Suspension – Victoria
  • Restorative Parole/Community Reintegration – Winnipeg
• Citizen Engagement/Public Education Strategy is intended to increase opportunities for Canadians to exchange views on criminal justice issues and to strengthen current learning opportunities and increase the number and extent of media used. It is hoped that the activities in this component will lead to more informed criminal justice policy and improved public confidence in the criminal justice system. The following projects are reviewed in this evaluation:
  • Solicitor General Canada Speakers Series
  • Advancing Restorative Justice Conference – Hull
  • What is Working in Restorative Justice Community Forum – Moncton
  • John Howard Society Community Forum – Calgary
  • John Howard Community Forum – Charlottetown
  • John Howard Society Community Forum – Dartmouth
  • Corrections and Conditional Release Statistical Overview
  • National Associations Active in Criminal Justice Policy Forum
  • Semaganis Gatherings
  • CCRA 10th Anniversary Publications
  • “What Works” in Public Safety – Montreal
  • Offender Art Auctions – Vancouver, Ottawa

1.2 Purpose of the evaluation

This interim evaluation is intended to address evaluation questions from the initiative RMAF in the following two key areas:

Relevance: Does ECI make sense?

1. Was ECI an appropriate response to the needs identified?
2. Have the needs changed from those ECI was originally intended to meet?
3. Should ECI continue?
4. Are the objectives of ECI consistent with current government, Portfolio and Departmental priorities and objectives?

Cost-Effectiveness: Given alternatives, is ECI the most cost-effective way to achieve the objectives?:

1. Is ECI the most cost-effective way to achieve the stated objectives?
2. Were the elements of due diligence (including eligibility) applied by the ECI?
3. What can be done to deliver ECI in a more cost-effective manner?
4. What are alternatives to ECI in attempting to meet the stated objectives?

At this point the emphasis is determining whether the activities and outputs of the initiative occurred according to plan. Because this initiative is aimed at major social change, it is too early to determine whether those activities and outputs have actually resulted in improved criminal justice policy or increased public confidence in the criminal justice system.

1.3 Evaluation methodology

Because of time constraints, this evaluation was conducted primarily by using secondary sources of information supplemented by qualitative data obtained through interviews. It should be viewed as a preliminary report and will point to areas that may be fruitful to explore further as the initiative progresses. The following documents were reviewed:

• Material relating to each of the projects
• The Effective Corrections and Citizen Engagement RMAF
• Fear of Crime and Attitudes to Criminal Justice in Canada: A Review of Recent Trends, November, 2001
• Corrections and Conditional Release Statistical Overview, December, 2003
• Effective Corrections Status Report, February, 2002

The following people were interviewed:

• Mary Campbell – Department of Public Safety and Emergency Preparedness
• Robert Cormier – Department of Public Safety and Emergency Preparedness
• Kimberly Feavor – Department of Public Safety and Emergency Preparedness
• Christine Clouteir – Correctional Services of Canada
• Jim Murphy – Correctional Services Canada
• Elaine St.-Amour – Correctional Services Canada
• Kimberly Mann – Church Council on Justice and Corrections (Ottawa)
• John Vandoremalen - National Parole Board
• Andrew McWhinnie – Victoria Restorative Justice Options
• Patricia Rainer – Lanark County Community Justice Program
• Elizabeth White – Canadian Criminal Justice Association

Interview guides are attached in Appendix A. A draft report was produced, based on the information available from the above sources and reviewed by key people in the Department of Public Safety and Emergency Preparedness.

ll. Summary of findings

While many of the individual projects are quite interesting, this evaluation focuses on the initiative as a whole, so this section provides summary data, referring to individual projects in the context of their contributions to the overall initiative.

2.1 Enhancing the community corrections infrastructure

The Community Corrections component is intended to provide opportunities for ‘action research’ by applying some of the theory of restorative justice in real life situations. Most of the projects included an education component as well as a conferencing component. Summaries of significant data for each of the projects are attached in Appendix B.

All of the projects were established to provide evidence that a restorative justice approach is an appropriate response to some situations. To date there have been six projects funded. The Victim Companion & Contracting Safe Justice project was just recently funded in January, 2004 so would not be expected to produce results. Table 2 outlines the activities and outcomes achieved by the remaining projects.

Table 2: Achievements of Programs

*Agreements reached/offender taking responsibility
**This number does not represent the total number of people involved

The table is intended to give an indication of the cumulative impact the projects, not as a comparison between projects. Each of the projects is quite different and produced different reports. For example, with Lanark County, the report only covers a six month period and their community forums involved a number of participants. With the Collaborative Justice Project, the data is from two reports, covering a much more extensive period. Many of their meetings were one-on-one.

Was this a relevant response?

In a word, yes. This component was intended to demonstrate not only the value of restorative justice, but to explore under what conditions it works best. The environment in which the projects operated supported projects being critical of themselves. Two projects indicated that they were not as successful as they had hoped and emphasis was placed on learning from the experience. Winnipeg’s Restorative Parole/Community Reintegration had problems getting the involvement of victims. Two key learnings were: 1) it is much more difficult to involve victims later in the process and 2) it is essential to involve a victim support organization as the primary initiator. A modified program has been developed with the primary emphasis on victim support. It has just received funding. The Victoria Restorative Justice Options to Parole Suspension Project was disappointed at the lack of referrals from the parole office, but felt that there were substantial learnings: 1) that it is essential to have the local parole office committed to the project; and 2) that it would make more sense to work with a single team rather than trying to engage all parole officers. These projects both indicated that despite the lack of involvement of the victims, those offenders who were involved gained a greater understanding of the impact of their behaviour on others and seemed more willing to move towards taking responsibility for their own behaviour.

The other projects provide some excellent examples of learnings from success. Perhaps the most persuasive evidence of success is reflected in the contacts between victims and offenders. The following are two examples:

1. A young man was arrested about 2 years after committing armed robbery at a convenience store. He was holding the gun. The victim asked him why he was doing this to himself. The question struck home. He moved back home and returned. Through our process he pled guilty right away. He met with the victim to apologize and thank victim for turning his life around. They came to an agreement that was sent to the court. The victim indicated that he did not feel jail was appropriate because the offender was turning his life around. The judge gave him a stringent conditional sentence - saying this was the first time he had given such a sentence for an armed robbery.
2. A young man had broken into a man’s house and found a large sum of money. He had no criminal record, this event seemed to be aberrant behaviour. Through our process, the offender and victim met. The victim came in and was somewhat fatherly, talking about the impact on him and his family. The victim gave the offender some fatherly advice about choosing friends carefully and asking for help when he needed it. The young man apologized. Towards the end the victim reached out to shake the young man’s hand (which was what the young man wanted). The victim asked that the young man be given his phone number so the offender could call him if he ever needed help. The victim indicated that he hoped some day the young man could come to his house, being welcomed through the front door as it should be. The young fellow had paid restitution. The victim handed back the certified check – as a symbol of faith. There was incredible impact on this young man – it virtually left him speechless. The judge gave him a conditional discharge. This young man was determined to prove to the victim that he would take advantage of this chance.

While not all outcomes can be this positive, where there is such tremendous moving forward, project staff who were interviewed indicated that conferences such as the above frequently result in healing on the part of the victim and the offender gaining greater empathy.

The Collaborative Justice Project in Ottawa conducted a participant satisfaction survey which found:

• All but one offender felt they were held adequately accountable for their crimes
• Only one offender felt he had not made sufficient reparations to the victims
• 87% of the offenders indicated they would choose a restorative justice approach in the future
• 11% of victims found their opinion was not adequately considered
• 15% of victims found that the offenders efforts to make reparations were insufficient
• 7% would select the traditional justice system over restorative justice in the future
• 10% of victims indicated they were treated unfairly in the meeting
• 90% of victims indicated the restorative justice approach to a fair approach.

While there were more victims not happy with the process than there were offenders, there is still a very high percentage of both victims and offenders who were positive about the experience and found it to be fair.

Is it the most cost-effective response?

The information provided did not focus on the cost of the approaches taken by the demonstration projects compared to the cost had the situation been addressed in a different way. The project-level reporting does not give sufficient information to determine the cost compared to successful outcome or the cost compared to alternative outcomes. Table 3 provides a summary of the amount of funding expended and the amount gained from other resources.

Table 3: Funding of Enhancing Community Corrections Projects

There is not evidence one way or another to determine whether this is the most costeffective way to achieve the stated objectives. This is an area that will require additional attention in the final evaluation.

2.2 Public Education/Citizen engagement

Figure 1 presents Prochaska’s theory of change model for individuals. This initiative attempts to translate the theory into practice for the entire population of Canada.

Figure 1: Stages of Change

It is almost impossible to determine whether a public education or citizen engagement process has had the desired impact within the first few years of an initiative. This particular initiative is looking for change with policy development and public confidence. It also very difficult to determine just how far reaching efforts have been. Despite these limitations, it is possible to point to some specific achievements:

• A National Conference in Hull (2002) brought together approximately 350 people from across Canada, including all provinces and territories. The conference summary provides information, opinions and ideas for a ranges of topics related to restorative justice.
• The Corrections and Conditional Release Statistical Overview (2003) provides some excellent baseline data regarding trends in the justice system. This is an excellent resource, if it is done at regular intervals, for tracking changes in the system response even though those changes will be attributable to factors other than this initiative.
• The Fear of Crime and Attitudes to Criminal Justice in Canada provides some baseline data for assessing whether attitudes towards restorative justice are changing.
• The reports from the New Brunswick and Nova Scotia community forums provide information as well as having given the opportunity for the participants to gain a better understanding of restorative justice.
• The Semaganis Gatherings brought together Aboriginal people with government officials to learn from each other and gain an increased understanding of restorative justice

Table 4 provides examples of funded projects with the funding provided. It is clear from this list that a variety of activities have occurred. It is not possible to determine, from the information provided, the extent to which participants found that the sessions directly contributed to different ways of looking at the justice system. Feedback from the evaluation forms (e.g. What Works Conference) and informal reports on the Speakers Series indicated the participants found the events and information useful.

Table 4: Examples of Citizen Engagement/Public Education Funding

2.3 Overall findings

Is the Initiative Relevant?

The overwhelming response is yes, it is relevant. Table 5 outlines the response of key informants regarding the relevancy of the ECCEI initiative. Consistently, respondents indicated that there would be little opportunity to introduce restorative justice without this targeted funding.

Table 5: Relevancy: Key Informant Responses

Two people who were not sure if ECI was an appropriate response indicated that various components had varying effectiveness; in other words the respondents felt the initiative is an appropriate response to the need, but did not know whether all funded projects were the best response. Others pointed strongly to the opportunities for learning even with projects that did not achieve their specific goals.

Were the elements of due diligence (including eligibility) applied by the ECI?

Proposals were solicited from organizations with knowledge and experience of restorative justice. Because this is a relatively small number of agencies that it is fairly well networked across Canada, those involved on the committee that made the selections felt they were able to get fairly high quality proposals. One person did indicate that the process could be improved by having clearer parameters for proposals. The criteria for selection seems to have been understood by committee members:

• The project related to restorative justice
• The organization had a track record in the area
• The project was consistent with the goals of the initiative

This evaluation process did not include an in-depth exploration of the process with a review of the records. Because the records provided to us do not include financial, progress or final reports for all of the projects, we assume that we do not have full documentation. Therefore, we cannot comment as to whether due diligence was followed in all cases.

Based on the information available it is evident that the funding went to projects that were consistent with the goals of the initiative, that funding went to established organizations and that those organizations have experience with restorative justice.

What can be done to deliver ECI in a more cost-effective manner?

One of the challenges has been that this initiative is peripheral to the core work of the department, and staff have been delivering it by contributing their own time. Hence it is highly cost-effective from one perspective. But one cannot help but wonder if in the long run such added demand might lead to burn out and increased turnover, which can be costly. For the next evaluation, it will be important to establish the extent to which restorative justice creates cost savings within the system. If such savings can be established, it should support putting additional resources into administering the initiative.

What are alternatives to ECI in attempting to meet the stated objectives?

No one posed alternatives. Respondents indicated the need for expansion rather than alternatives.

lll. Analysis and interpretation of findings

Table 6 provides an overview of the conclusions in relation to each of the questions.

Table 6: Conclusions In Relation to Evaluation Questions

The initiative has also provided the opportunity for some key learnings such as the following:

• The restorative justice system works best when the needs and wishes of the victim are a priority
• The victim does not always need to be involved for the offender to gain empathy and take responsibility
• The judicial system presents some barriers to alternative approaches at two levels:
  • Attitudinal barriers where there is not buy-in from corrections or judicial staff
  • Systemic barriers where there is more than one offender involved, and the cases against each may be different

This particular initiative appears to have taken some difficult but appropriate steps for supporting innovation. Almost all respondents indicated that as much could be learned from some of the projects that did not achieve their goals as from projects that did. This initiative is being implemented in a ‘learning culture’.

lV. Conclusions and suggestions

The Effective Corrections and Citizen Engagement Initiative appears to be successful. The key changes suggested for the initiative are:

• Staffing that is specifically responsible for this initiative that is additional to current resources
• Replicate successful projects in other locations
• Continue to try innovative projects

This interim evaluation provides only a preliminary picture of that success. In order for the next evaluation to be able to come to more definitive conclusions, it will be important to establish a more systematic mechanism for collecting and interpreting information.

The following are some considerations for further evaluation of the initiative:

• Develop a cluster evaluation framework for each of the two components (and other components, if appropriate)
• Develop project level evaluation frameworks that are consistent with RMAF and can feed into the cluster evaluation, using a standardize reporting format. The cluster evaluations can feed into a roll-up evaluation of the overall initiative.

Some areas to explore with the Community Corrections pilots:

• The cost of restorative justice compared to current approach, looking at the cumulative savings of all the projects taking into account the cost of a trial as well as incarceration
• The benefits to the victims in terms of emotional and healing benefits as well as financial
• Changes in attitudes in the victims
• To extent to which there is evidence that restorative justice reduces re-offending

Some areas to explore with Citizen Engagement/Public Education:

• The extent to which Canada is covered
• A longitudinal study looking at changes in attitude/ if possible determine if there are differences in communities were key educational activities have occurred compared to where they have not.

Appendix A: Interview Guides

PSEPC Staff

1. To what degree do you believe that Effective Corrections and Citizen Engagement initiative has been effective? (Probe for examples of alternatives to prison for Aboriginal people, improved criminal justice policy, improved public confidence in the criminal justice system)
2. Of the projects that have been funded, which have been most successful in achieving the goals of the initiative? (Probe the reasons)
3. What lessons have been learned from both successful and less successful projects?
4. To what extent are the objectives of the initiative consistent with current government policy and priorities? What needs to be changed?
5. Describe the process for selection of the projects. (Probe: the rfp process, who selected, what criteria were used, how was the criteria applied)
6. Is the initiative still needed? Should the initiative continue? Why or why not?

Project Contacts

1. Describe the request for proposal process and the proposal development process.
2. In what ways are the goals of your project consistent with the goals of the Effective Corrections and Citizen Engagement initiative?
3. To what extent did you achieve the goals you set? (Probe: evidence that goals were achieved)
4. What unexpected results, if any, occurred?
5. What lessons have you learned from this experience? (Probe: things to be done differently, things to be repeated or replicated)
6. Is the initiative still needed? Should the initiative continue? Why or why not?
7. If it continues, what changes, if any, should be made?

Appendix B: Data Summary Matrix

Data Summary Matrix – Enhancing Community Corrections Infrastructure

Management action plan (map) Evaluation of two components of the Effective Corrections Initiative

Date modified:

2015-12-15