Translating 'What Works' into Sustainable Everyday Practice: Program Design, Implementation and Evaluation 2009-05

Translating 'What Works' into Sustainable Everyday Practice: Program Design, Implementation and Evaluation 2009-05 PDF Version (122 KB)

Abstract

A major challenge for community correctional agencies is translating "what works" knowledge into effective everyday practice. Service delivery adherence to the principles of Risk, Need and Responsivity (RNR) is threatened by a variety of design, implementation, and evaluation issues found in the "real world" of community supervision. The Strategic Training Initiative in Community Supervision (STICS) was designed as a service delivery model and implementation training package, which attempted to translate these principles into the routine practice of probation officers. In this report, we describe the challenges and issues that should be addressed when trying to bring research to the real world of community supervision faced by probation officers and we evaluate our efforts to overcome these challenges through STICS. Firmly rooted in RNR principles, the STICS supervision model emphasizes officers' interventions that facilitate prosocial attitudinal/cognitive change in moderate to high-risk offenders. In order to maintain service integrity and officers' skill maintenance, STICS not only provided probation officers with initial three-day training in the model and basic cognitive-behavioural interventions but the model also provided ongoing clinical supervision. Random assignment and direct observation of probation officer behaviour during interactions with their clients are key components of the evaluation of this initiative. Preliminary results suggest that STICS had a significant impact in improving probation officers' use of effective core correctional practices.

Authors' Note

The views expressed are those of the authors and do not necessarily reflect those of Public Safety Canada. Correspondence concerning this report should be addressed to: Guy Bourgon 340 Laurier Avenue West, Ottawa, Ontario, Canada, K1A 0P8, or by email at Guy.Bourgon@ps-sp.gc.ca.

Acknowledgements

We would like to thank Leticia Gutierrez and Kyle Simpson for their tireless efforts in coding the audiotapes and entering this data, and Jobina Li for providing assistance to the probation officers with data collection.

December 2009

When Lipton, Martinson, and Wilks (1975) published their review of offender treatment studies and launched the "nothing works" movement, there were two major consequences. First, many scholars and policy-makers abandoned offender rehabilitation as a goal of corrections and turned to deterrence and punishment as the new goal. This new purpose was embraced particularly in the United States where the penal harm movement was firmly implanted (Clear, 1994). A second important outcome of "nothing works" was that proponents of the rehabilitation ideal not only continued to conduct research on offender treatment but did so with renewed vigour (e.g., Cullen & Gilbert, 1982; Gendreau & Ross, 1979; Palmer, 1975). The evidence supporting offender rehabilitation continued to accrue to the point that there has been a return to "what works".

With the return to "what works", correctional agencies have made significant investments into bringing this empirical knowledge into practice; unfortunately, the results have frequently been disappointing (e.g., Barnoski, 2004; Goggin & Gendreau, 2006; Wilson & Davis, 2006). With discrepancies between expectations based on small scale well controlled empirical studies and what was found in large scale implementations of "what works", there has been a growing interest in the importance of program design, the integrity of implementation, and the evaluation of these effectiveness mediators (Andrews, 2006, 2008; Rhine, Mawhorr & Parks, 2006; Taxman & Marlowe, 2006). Designing effective programs/ services for offenders, implementing them, and evaluating them in a manner that provides insights into the development, delivery, and evaluation, is a considerable challenge for clinicians, program managers, administrators and researchers alike (Welsh, 2006).

This paper begins with a brief overview of the "what works" literature within the context of community supervision. Next, some of the critical issues and challenges that are commonly faced by efforts to bring "what works" practices to the supervision of offenders in the community are identified. Many of these issues were considered in the design of the Strategic Training Initiative in Community Supervision (STICS) - a comprehensive model, with an implementation strategy, to transfer "what works" knowledge into the real world of everyday community supervision. Therefore, we conclude with a description of how these issues were addressed through STICS and how we evaluated our efforts to determine success.

The Emergence of the Risk-Need-Responsivity (RNR) Model

An important advance in summarizing research evidence is the application of meta-analytic techniques. One of the first important meta-analytic reviews of the offender rehabilitation literature was Mark Lipsey's (1989) analysis of 400 interventions with juvenile delinquents. He found that treatment was associated with an average 10 percent reduction in recidivism. In addition, Lipsey made a significant contribution to the field by listing some of the characteristics of the more effective programs. His list, however, consisted mostly of methodological factors (e.g., sample size, length of follow-up). Subsequent meta-analyses confirmed that offender treatment more often than not led to reductions in recidivism whereas "get tough" sanctions showed little impact on recidivism (Lösel, 1995; Redondo, Garrido, & Sanchez-Mecca, 1999).

In 1990, Andrews, Bonta, and Hoge described a set of principles for effective intervention. At the time, the following four principles were presented: (1) Risk (direct services to higher risk offenders), (2) Need (target criminogenic needs in treatment), (3) Responsivity (use cognitive-behavioural treatment methods and tailor the intervention to the offender's learning style, motivation, abilities, and strengths), and (4) Override (deviate from the principles for specified reasons; now called the principle of professional discretion). Since the original formulation, the Risk-Need-Responsivity (RNR) model and its principles have been greatly expanded. Today there are 21 principles articulated (Bonta & Andrews, 2007), ranging from broad, overarching themes (e.g., use a general personality and cognitive social learning theory) to organizational factors (e.g., clinical supervision of staff in accordance with RNR). Although the RNR model is far more comprehensive now than in 1990, the Risk, Need and Responsivity principles remain at its core.

The validity of the RNR principles was tested in a meta-analysis conducted by Andrews, Zinger, Hoge, Bonta, Gendreau, and Cullen in 1990. Eighty studies of adult and juvenile treatment interventions were reviewed and the results showed that the effectiveness of treatment varied in accordance with the RNR principles. Studies that followed the principles of Risk, Need, and Responsivity had a mean effect size (phi) of .32, whereas studies that failed to adhere to the three principles showed a mean effect size of -.07. In the latter case, these inappropriate interventions actually resulted in increased recidivism (sanctions were also associated with increased recidivism effects; phi = -.08). The robustness of the RNR model has continued to grow through extended meta-analyses of the offender treatment literature (e.g., Andrews & Bonta, 2006) and independent tests of the principles (Hanley, 2006; Lovins, Lowenkamp, Latessa, & Smith, 2007; Lowenkamp, Latessa, & Holsinger, 2006; Lowenkamp, Latessa, & Smith, 2006; Marlowe, Festinger, Lee, Dugosh, & Benasutti, 2006; Palmer, McGuire, Hatcher, Hounsome, Bilby, & Hollin, 2008). Undoubtedly, the evidence in support of RNR accounts for it being the predominant model in the rehabilitation of offenders (Taxman & Marlowe, 2006).

From Demonstration Projects to the Real World

Translating empirical knowledge into system-wide everyday practice has proven difficult. Andrews and Bonta (2006) examined 47 treatment demonstration projects and 209 "real world" evaluations. Real world projects were defined as interventions with samples greater than 100 and where external researchers conducted the evaluations. These 256 studies were then rated on their adherence to the three major principles of Risk, Need, and Responsivity. The results are shown in Table 1 below. It is clear that the effect size diminishes when we move from a demonstration project to a real world application. This is congruent with Lipsey's (1989) earlier findings that the best results are found when the sample size is small and those designing and delivering the treatment conduct the evaluation. It is also apparent from Table 1 that the RNR principles remain important; as adherence to the RNR principles increases, so does the mean effect size. Other studies have confirmed this pattern of results (Andrews, 2006; Lipsey & Cullen, 2007; Lowenkamp, Latessa, & Smith, 2006).

(from Andrews & Bonta, 2006)

The findings shown in Table 1 suggest that in the "real world", treatment has less of an effect than in demonstration projects. The "real world" treatments with their large samples make it difficult to adhere to the RNR principles. As such, integrity in the delivery of services is critical. Furthermore, ensuring that the intervention is delivered to the higher risk cases, targets criminogenic needs, and uses cognitive-behavioural techniques are major challenges for correctional systems. When we consider the expanded principles in the RNR model (e.g., use structured assessments of Risk, Need, and Responsivity; managers select, train, and supervise staff according to RNR) the challenges are compounded.

A recent example of the difficulty of translating knowledge to practice is illustrated by the experiences in United Kingdom where they undertook perhaps the largest social experiment in corrections ever conducted. Guided by the "what works" knowledge and RNR model, the National Offender Management Service rolled out over 1,350 individual intervention projects. Expecting to find the same effects on recidivism reported in the various meta-analyses, over 400 million pounds were invested in the delivery of treatment services (Homel, Nutley, Webb, & Tilly, 2005). Although early evaluations showed reductions in recidivism, the results from the national roll-out were not on the same scale as reported in the literature (Raynor, 2004; 2008). What went wrong? Although there are many explanations for the somewhat disappointing results (see Goggin & Gendreau, 2006), a primary reason is the failure of the services to adhere to the RNR principles. Integrity of adherence to the RNR principles is critical not only for formal treatment programs but also for effective community supervision.

Community Supervision

Most offenders in Canada and the United States are under a sentence of probation. For example, there are approximately 100,000 individuals each month who are supervised on probation in Canada (Public Safety Canada, 2008). Probation is not only less expensive than imprisonment, but it is also thought to be more effective in reducing recidivism. Community supervision presents an opportunity for probation officers to use empirical knowledge about "what works" to facilitate prosocial change in their clients and thereby reduce re-offending. However, studies that actually examine the interactions between probation officers and their clients cast some doubt on the efficacy of probation officers in promoting change in their clients. In a meta-analysis of 15 studies yielding 26 effect size estimates, Bonta, Rugge, Scott, Bourgon and Yessine (2008) found that community supervision was associated with a reduction of only two percentage points in recidivism. Furthermore, in the same report, an analysis of audiotaped interviews between probation officers and their clients revealed that probation officers adhered to relatively few RNR practices (e.g., spent too much time on low risk cases, did not target criminogenic needs sufficiently). Despite these findings, there is a considerable body of evidence that offers a range of suggestions as to what probation officers can do to facilitate change.

Andrews and colleagues (Andrews, 1979; Andrews & Bonta, 2006; Andrews & Carvell, 1997; Andrews & Kiessling, 1980; Dowden & Andrews, 2004) have contributed significantly to our understanding of the therapist/officer behaviours that result in reduced recidivism among offenders. These researchers described what they call "Core Correctional Practices," which are practices derived from the RNR model and are demonstrably linked to reduced recidivism. These include certain relationship skills, prosocial modeling, the effective use of reinforcement and disapproval, and problem-solving.

Armed with this knowledge, the obvious next step is to train probation officers in some, if not all, of these core correctional practices and ensure that they utilize these skills during supervision of their clients. Surprisingly, there is almost a complete absence of evaluations of training probation officers in any of the skills described. The sole exception is the work conducted by Chris Trotter (1996, 2006), who trained 12 officers in a "prosocial approach" that emphasized prosocial modeling and reinforcement, problem-solving, and empathy (as a relationship skill). For the 93 clients of the trained officers, the four-year recidivism rate was 54%. However, for the 273 clients of the 18 untrained officers, the rate was 64% (a non-random evaluation design was used). In spite of these encouraging results, this study was a small demonstration project. For those attempting to translate RNR principles into everyday practice, the study does not provide concrete guidance on how to address the myriad of issues that threaten the integrity and fidelity of translating empirical knowledge into sustainable everyday practice.

The Strategic Training Initiative in Community Supervision (STICS)

The goals of the Strategic Training Initiative in Community Supervision (STICS) were to design a model of community supervision that was consistent with the RNR model, put together a means to implement the model into everyday practice, and create an evaluation strategy that would inform the design and implementation of effective "what works" community supervision. How to achieve these goals ultimately became the challenge. The three major challenges STICS attempted to address were to: (a) translate the RNR model into specific, concrete actions that would be useful for probation officers (i.e., the model of community supervision), (b) develop an implementation strategy that included officer training and on-going clinical supervision/support (i.e., the implementation strategy), and (c) evaluate the model and implementation efforts on the behaviour of both the officers and the offenders they supervise (i.e., the evaluation design).

The Challenges of Translating RNR Research into Practice

In translating the RNR principles into everyday community supervision, there are a number of issues that need to be addressed to maintain the integrity of services. Table 2 summarizes the key issues, as well as our response to address them. These issues pertain to the three critical components of program design, implementation, and evaluation. These are familiar challenges to anyone who is interested in translating the research on "what works" and the RNR model into practice. Whether it is a treatment program for offenders or community supervision, the issues are the same. The program or service must be guided by the evidence and be attentive to the general principles of Risk-Need-Responsivity and its underlying theory of criminal behaviour. Moreover, for probation specifically, there is the issue of how probation services should be structured and delivered. Implementation concerns hinge on key pre-existing or prerequisite organizational practices, staff training, and skill maintenance to aid the delivery of services as intended. Evaluation issues include the research design/methodology and identifying critical data to collect.

Notes: GPCSL: General Personality and Cognitive Social Learning
PO: probation Officer

I. Program Design

The first step in bringing "what works" from the research world to the practical world is the design of the program or service. The importance of this part of the process (i.e., translating research to practice) should not be underestimated (Welsh, 2006). The actual program and its components, targets, and intervention strategies are often what meta-analytic studies have used to assess treatment programs' adherence to RNR principles (Andrews & Bonta, 2006; Hanson, Bourgon, Helmus, & Hodgson, 2009). In this section, we identify key considerations for those responsible for developing programs and services and explain how STICS addressed them.

General Theory of Criminal Behaviour: The RNR model and consequently, the STICS model, are based on a General Personality and Cognitive Social Learning (GPCSL) theoretical perspective (Andrews & Bonta, 2006; Bonta & Andrews, 2007). Briefly, this perspective makes three important points: a) criminal behaviour is a learned behaviour that follows the laws of classical, operant, and vicarious learning; b) learning occurs via the interactions of an individual with his/her environment; and c) some risk/need factors are more important than others, with one of the most important risk factors being procriminal cognitions and attitudes.

In developing the STICS model and training program, one practical concern was how to convince probation officers that their clients' antisocial behaviour is a product of learning and that behaviour is primarily under the control of the individual's cognitions and attitudes. If probation officers accepted the GPCSL view rather than a medical model (e.g., offenders are sick) or a sociological perspective (e.g., poverty causes crime), then they would more readily accept the idea that offenders can learn prosocial behaviour through the same processes that govern the learning of criminal behaviour. The importance of probation officers "buying into" a theoretical view has been grossly underestimated in many studies. The psychotherapy literature has long recognized the importance of an "explanation" for the problems of the patient and how these problems can be overcome (Wampold, 2007). Like the patient of the psychotherapist, probation officers also need an explanation as to why they should change their behaviours and that of their clients. Therefore, the importance of GPCSL was explicitly addressed in the STICS training with specific reference to and presentations of the research in support of GPCSL.

Risk Principle: The evidence in support of the Risk Principle indicates that direct services should focus on higher risk offenders with 'dosage' increasing as risk increases. This, of course, requires an assessment of risk. Following the Risk Principle, STICS was designed for probation officers who supervise higher risk clients. We made two decisions during the initial design that we believed would enhance adherence to the Risk Principle. First, we reviewed existing policies that helped us identify which probation officers were eligible for STICS (i.e., those that supervised higher risk clients). It is common to have policies, at least in Canada, that dictate what probation officer level is responsible for supervising clients of different levels of risk. For example, two of our sites (British Columbia and Saskatchewan) have policies where probation officers supervise higher risk clients and assistant probation officers supervise lower risk clients. However, in our third site (Prince Edward Island), there is a variation on this policy as one group of officers supervise medium and high-risk cases and the other group supervises low and medium-risk clients. In this last case, since we wanted to capture all the medium-risk clients possible, we agreed to allow both groups of POs to participate in STICS. Secondly, the decision was made to over-sample high-risk clients. This was done to maintain a focus on high-risk clients and in recognition of probable higher attrition levels for this group. Consequently, STICS required officers to recruit only medium and high-risk clients with an over-sample of the high-risk (i.e., they were asked to recruit two medium-risk clients and four-high risk clients). It was hoped that these program design decisions would aid adherence to the Risk Principle.

Need Principle: The Need Principle indicates that services should target criminogenic needs. As with risk, adherence to need requires an assessment of the client's criminogenic needs. In addition to the identification of a client's criminogenic needs, the Need Principle also requires officers to focus intervention efforts on these specific needs (e.g., relapse prevention for substance abuse need). During the initial design of STICS, we debated on which criminogenic needs would be the focus of STICS and how this might be practically translated to community supervision (e.g., including relapse prevention for substance abuse and anger management for aggressive behaviours). However, it became apparent to us that the efforts of probation officers during face-to-face supervision sessions needed to be flexible and yet at the same time, be able to address the majority of criminogenic needs. Procriminal attitude is one of the criminogenic needs that ultimately applies to all other criminogenic needs (e.g., the attitude "working is for suckers" taps into two criminogenic needs: procriminal attitudes and employment). As a result, we decided to focus STICS towards helping probation officers target procriminal attitudes and cognitions.

To facilitate adherence to the Need Principle with an emphasis on procriminal attitudes, two specific strategies were undertaken. First, we developed what we called the STICS Action Plan. This tool assisted officers in understanding how procriminal attitudes and cognitions are intertwined with all other criminogenic needs and assisted them in developing RNR-based supervision plans using the client's risk-need assessment. Secondly, we included specific interventions in the STICS training. Officer skills and learning components required staff to identify expressions of procriminal attitudes and facilitate the client's learning of how to replace procriminal attitudes with prosocial ones. By incorporating these two strategies, we expected probation officers to generalize STICS interventions and skills, to be able to apply them to the dysfunctional attitudes and cognitions that underlie a variety of criminogenic needs (e.g., "I only drink on weekends" for substance abuse needs, "I make more money in an hour stealing than you make in a week" for employment needs).

Responsivity Principle: The evidence in support of the Responsivity Principle indicates that services must be tailored to the client's learning style, motivation, abilities, and strengths in order to establish and promote an effective learning environment. Adherence to the Responsivity Principle is arguably the most challenging because there are a number of techniques, skills, and intervention strategies that can promote or diminish an effective learning environment for offenders. In practice, the STICS supervision model requires officers to provide the client with information to learn (e.g., key concepts and skills) in an understandable manner within the context of the officer-client relationship. The following four factors were considered critical to the Responsivity Principle for the STICS model of community supervision: the officer-client relationship; cognitive-behavioural techniques; concepts and skills relevant for clients under community supervision; and structuring of individual sessions and the supervision period.

a) The Officer-Client Relationship: The importance of establishing a good relationship with the client has been demonstrated with correctional clientele (Dowden & Andrews, 2004), substance abusers (Caroll, Ball, Nich, Martino, Frankforter, Farentinos, et al., 2006), and general psychiatric outpatients and attendees at counselling clinics (Ackerman & Hilsenroth, 2003). Some scholars argue that establishing a good rapport or therapeutic working alliance with the client is essential for effective intervention (Ahn & Wampold, 2001; Barlow, 2004; Frank & Frank, 1991; Wampold, 2007).

Adherence to this aspect of the Responsivity Principle is promoted directly through the implementation procedures of STICS. Most probation officers, at least in Canada, are hired based on their ability to establish rapport and many have also been trained in motivational interviewing. In addition, congruent with GPCSL is the view that relationship building is a skill that is learned. The skills include expressions of warmth, demonstrating flexibility, and engaging in various forms of active listening and constructive feedback (e.g., showing understanding, reflecting to the client what was heard, etc.). To facilitate establishing an effective working relationship, STICS ensures that officers learn these relationship building skills and use them in supervision. By incorporating specific session processes (e.g., role clarification and collaborative goal setting), STICS supervision fosters good working alliances between the officer and the client.

b) Cognitive-Behavioural Techniques: Employing cognitive-behavioural techniques with the offender population in general and with clients during one-on-one supervision sessions is considered critical (Andrews & Bonta, 2006). In order to promote adherence to this component of the Responsivity Principle, the STICS model ensured that all aspects of supervision (i.e., concepts, interventions, skills) were based on a sensible cognitive-behavioural theory to account for client problems. Clear, concrete, and simple concepts, techniques, and skills derived from the model to facilitate prosocial change were used. For example, a simple tool (i.e., the Behaviour Sequence) was used to illustrate the cognitive-behavioural model so that it could demonstrate to both officer (the change agent) and client (the consumer) that there is a concrete link between thoughts and behaviour. Derived from this tool, specific techniques and skills are used to identify cognitions and attitudes that promote procriminal behaviours. In addition, individuals can be taught concrete skills to change cognitions (i.e., cognitive restructuring) and behaviour with multiple opportunities for practice and generalization. This cognitive-behavioural model and its functional client-friendly tool allows for an examination of behaviour as a function of antecedent stimuli, cognitions, and consequences with the emphasis on how internal cognitive cues (i.e., attitudes and thoughts) are the root causes of behaviour. Furthermore, we incorporated specific ways of applying this tool to demonstrate a concrete method of identifying attitudes and cognitions that promote criminal behaviours, as well as a specific technique to teach cognitive restructuring skills.

c) Relevance to Client: It is one thing to discuss and describe a complex model of human behaviour and quite another to translate this model into one that is easy to understand and personally relevant to offenders. STICS made every effort to ensure that key concepts, interventions, and skills involved in facilitating change were "client friendly". This meant that concepts, tools, and skills were kept as simple and concrete as possible, and the language employed was free of 'psychobabble jargon'. "Antecedent stimuli" were called "outside cues", "procriminal thoughts" were called "tapes", and "prosocial thoughts" were called "counters". In addition, with the interventions focused specifically on attitudes and cognitions, STICS was designed to ensure that the key concepts and skills were applicable to each and every criminogenic need. In this fashion, officers could tailor their change efforts to each individual client.

d) Structuring Supervision: Finally, the STICS model provided a specific concrete structure for individual sessions and a broad overview of how supervision should progress from intake to completion. Such structure fosters adherence to the Responsivity Principle by facilitating the creation and maintenance of an effective learning environment. Policies in most probation departments are relatively silent on what a probation officer should do when s/he meets a client for supervision. The only exception is to ensure that the client is complying with the conditions of probation. This lack of structure is reasoned to be one factor contributing to supervision sessions focusing more on compliance rather than efforts to promote change (Bonta et al., 2008).

STICS includes a structure for both the individual session and the overall probation period. For each individual session, the structure consisted of four components. The first component is a brief "check-in" lasting no more than five to ten minutes. The check-in involved spending time enhancing the working relationship with the client, checking for any new developments in the client's situation that may require immediate attention, and making sure that the probation conditions were being addressed. The second component is a "review" of the last session, including the homework assigned. The review was designed to facilitate learning via discussions and/or rehearsal of previous material and provide linkages from one supervision session to the next. The third component is to actually conduct an "intervention" (about 20 minutes). This could be teaching the Behaviour Sequence tool or doing a structured problem-solving exercise. Finally, the STICS session structure ended with "homework". Homework that is agreed upon by the client and reinforces the learning of new concepts, skills, and/or prosocial cognitions is assigned and confirmed at the end of the session.

In addition to this session structure, there was also the recognition that community supervision changes with time. As such, there was the expectation that both the focus and content of supervision sessions change over the course of supervision. In STICS, the supervision period was divided into eight distinct stages and their associated goals. The stages range from conducting an intake assessment (beginning of supervision), establishing a relationship and collaborative goals (early in supervision), to using cognitive-behavioural techniques to change procriminal thoughts (later in the course of supervision). These elements of structure, both in the individual sessions as well as the overall probation period, were deemed to assist probation officers in achieving their goals with their clients.

II. Implementation of STICS

It is one thing to have a RNR-based service but it is quite another thing to implement it effectively in the real world. There are a number of implementation factors that are believed to influence the quality or integrity of the services that are actually delivered behind 'closed doors" (Andrews, 2006). For example, in recognition of these organizational or implementation factors, the Correctional Program Assessment Inventory (CPAI) was developed to assess many of these factors and their relationship to effective correctional interventions (Lowenkamp, Latessa, & Smith, 2006). In this section, we describe and discuss what we consider to be three major challenges that are faced with those attempting to bring "what works" to everyday practice and how STICS attempted to address them.

Jurisdictional Prerequisites: In order to facilitate a quality implementation of STICS community supervision, we considered what were the prerequisites or pre-existing conditions necessary prior to implementing an RNR-based service. As mentioned in the Program Design section, a jurisdiction must already utilize a validated risk-needs assessment instrument. Use of such instruments was considered necessary for community supervision to have any chance of adhering to the principles of risk and need. In addition, policies regarding levels of supervision and services should be congruent with the principle of Risk (i.e., higher risk cases receive higher levels of service).

Another prerequisite was managerial support for STICS. One aspect included top-down verbal commitments to support probation officers' participation in all aspect of STICS (i.e., the initial training as well as on-going implementation and maintenance). This meant that managers had to provide both the time and resources resulting from the additional demands placed on officers who were learning and applying a new way of conducting community supervision. A second aspect of support was that all managers of the frontline officers who were to be involved in STICS were required to attend the initial three-day STICS training. In practical terms, training of officers and managers occurred at the same time but a separate trainer led the managers through the exercises and role plays during the training. We reasoned that attendance of managers not only showed support to their staff but would also yield dividends when staff returned to the field to practice what was taught. The managers would be more cognizant of the demands placed upon the officers by STICS and would work with staff to organize their workload in order to facilitate participation.

Initial 3-Day Training: The next major hurdle was bringing the model to probation officers and teaching them the new skills and knowledge. In developing the 3-day STICS Training, we considered the training as a starting point to initiate change in the probation officers' behaviour when working with their clients. Just like the principles that have been shown to facilitate change in their clients, our training program also followed the Responsivity Principle to foster change in the officer's behaviour.

Responding to the learning style of the officers, we recognized that probation officers need an explanation as to why they should change their behaviours and that of their clients. One practical concern was how to convince probation officers that the antisocial behaviour of their clients is a product of learning and that behaviour is primarily under the control of the individual's cognitions and attitudes. The STICS training included exercises to demonstrate the power of cognitive restructuring, prosocial modeling, reinforcement, and punishment. In particular, probation officers were trained to teach their clients how cognitions control their behaviour, how rewards and punishments influence future behaviour, and what the clients themselves can do to bring about change. Exercises were practical and included ones that addressed both client examples as well as ones that focused on the officer's thoughts and behaviours regarding community supervision work.

Good working relationships are considered critical to facilitate change with clients. When providing training for professionals where the goal is to change their behaviour, the relationship between the "student" and the "trainer" is likely just as important. The trainers modeled the skills being taught throughout the training. For example, the training included various components and exercises to ensure that the presentations, exercises, and discussions were collaborative, reciprocal, and experiential. Not only did the trainers describe real life experiences with clients to illustrate concepts, skills, and interventions, exercises permitted officers to bring and discuss their own experiences and examples to the training. The trainers were not just "academics" with no real life clinical experience, rather the trainers were considered ones with substantial experience and knowledge collaboratively helping them do a more efficient and effective job.
One of the foundations of the STICS model is a coherent and comprehensive cognitive-behavioural model of criminal behaviour. STICS training relied heavily on such a model and cognitive-behavioural techniques to facilitate the learning of STICS material. Just like the client whose behaviour is determined by his/her cognitions and attitudes, so too is the officer's behaviour when working with his/her client. Attention was paid to demonstrate that the cognitive-behavioural model was applicable to their behaviour as well as that of their clients. Exercises permitted the officers to not only identify and facilitate change in the client's attitudes and cognitions, but also to examine their own attitudes and cognitions in relation to their work as probation officers and their behaviour behind closed doors during supervision.

When designing STICS, efforts were made to keep the model, language, and skills relevant to the client. It was also important that the training itself demonstrated relevance to the officer. The training modules explicitly demonstrated empirically, theoretically, and practically how STICS was relevant to the daily work of community supervision (e.g., in general and in particular, to the officer's interactions with the client and the client's criminal behaviour). Much of this was achieved through the consistent use of the STICS model, language and skills with one component building on the previous one. Interactive exercises were developed that were practical and relevant to everyday probation work and that acknowledged and attempted to address realistic challenges presented in community supervision (e.g., high-risk unmotivated clients and communities with minimal resources).

Program or service structure was also an important component of the STICS supervision model and as a result, it was a vital part of the training. We developed a formal training manual that structured the three days. The training consisted of 10 modules (see Table 3 for an overview of the 3-day training) covering all aspects of STICS supervision including providing information for the officers to understand the STICS model, and the skills and tools necessary to utilize and implement it.

Skill Maintenance: Repetition is the hallmark of skill maintenance. However, the limited practice during a three-day training program is not sufficient to maintain new behaviours over a period of weeks, let alone months (e.g., Miller & Mount, 2001). One noteworthy feature of STICS was that it went beyond the three days of training in RNR-based interventions and included ongoing clinical supervision. After the training, probation officers met monthly in small groups to discuss their use of STICS concepts and skills. At these meetings, specific exercises were assigned, completed, and discussed among the probation officers. The meetings also included a teleconference with the trainers/evaluators during which officers were provided with clinical supervision. Clinical supervision included various types of exercises consisting of reviewing small samples of audiotaped sessions between officers and clients. In addition to the monthly meetings, we also encouraged officers to submit audiotapes to the trainers for individual clinical feedback. This feedback was provided orally and/or in written format. Finally, skill maintenance and development was fostered with officers attending a one-day STICS refresher workshop facilitated by one of the trainers approximately one year after the initial three-day training.

Table 3. Initial three-day training and skill maintenance components of STICS

	Method	Goal		Program Issue
Initial Three-Day STICS Training
Day 1	Module 1: Overview and Rationale for STICS		Lay the theoretical groundwork and evidence for many of the specific skills taught in later modules	GPCSL Theory
	Module 2: Differential Supervision & the Risk Principle			Risk Principle
	Module 3: Criminogenic Needs			Need Principle
	Module 4: Procriminal Attitudes			Need Principle
	Module 5: Responsivity Principle: Building Rapport in a Therapeutic Relationship	Show importance of rapport and teach relationship-building skills		Responsivity Principle
Day 2	Module 6: The Cognitive-Behavioural Model	Teach concrete concepts/skills, relevant to various criminogenic needs, applicable to range of clients		Responsivity Principle
	Module 7: Cognitive-Behavioural Interventions: Cognitive Restructuring			Responsivity Principle
	Module 8: Prosocial Modeling and Reinforcement	Teach modeling techniques, effective use of reinforcement/punishment		Responsivity Principle
Day 3	Module 9: Other Specific Cognitive-Behavioural Interventions	Teach core skills of problem-solving and self-management		Responsivity Principle
	Module 10: Strategic Supervision	Provide structure for each session and overall probation period		Responsivity Principle
Skill Maintenance
Monthly Meetings		Support, skill development & integrity		Skill Maintenance
Formal Clinical Feedback on offender-client sessions		Support, skill development & integrity		Skill Maintenance
Refresher Course (approximately one year post-training)		Support, skill development & integrity		Skill Maintenance

III. Evaluation Issues
After issues of implementation are addressed, evaluation issues need to be considered. The quality of research or evaluation efforts in the field of correctional treatment have been criticized ever since the famous Martinson's "nothing works" review and is often believed to be a major problem in knowledge accumulation (Farrington & Welsh, 2005; Lipsey & Cullen, 2007). In this section, guided by the work of the Collaborative Outcome Data Committee on study quality of sex offender treatment outcome research (Collaborative Outcome Data Committee, 2007a; 2007b), we present two key evaluation issues and our responses to them regarding STICS. The two main evaluation issues that we faced pertained to the implementation of the best research design and the measurement of the impact of STICS training on both the behaviour of the probation officers and their clients.

Research Design: There are various types of research designs but the randomized experiment is considered to be the "gold standard". Although random assignment designs are difficult to employ in "real world" corrections (Farrington, 2006), the evaluation of STICS included the random assignment of probation officers to either the training or control conditions. Recruitment of research participants was accomplished as described below.

Senior management from three provinces (British Columbia, Saskatchewan, and Prince Edward Island) who had agreed to all the necessary prerequisites for participating in STICS sent an email to staff informing them of the opportunity to participate in a three-day training on learning "what works" techniques. It was also explained that this was a research project with certain data collection requirements and that staff would be randomly assigned to either an experimental or a control group. Given that maintaining participation in a research project is challenging, particularly for a lengthy and demanding one such as STICS, we tried to maximize participation through efforts directed to the probation officers and their managers. First, we recruited only volunteering probation officers in order to capitalize on motivation. For the evaluation, we did not see volunteers as problematic. Although volunteers may not be representative of all probation officers, our intention was to evaluate the effectiveness of STICS among an amenable staff. Second, we asked the probation officers to submit an audiotaped session with one of their clients prior to training. Only two of the eighty probation officers (one from the experimental group and one from the control group) failed to submit a sample audiotape prior to group assignment (due to problems with the distribution of the audio recorders). This request was used to assure us that the probation officers were motivated and capable of fulfilling one of the basic requirements of the study (i.e., submitting multiple audio taped sessions, discussed later) and the audiotape also provided a pre-test baseline measure of their interactions with clients. Finally, through random assignment of probation officers to groups, motivation was held constant. The probation officers were assigned to the experimental (i.e., STICS training) or control (i.e., no training) group using a 60:40 ratio. We over-sampled the experimental cases in order to have sufficient power for planned analyses specific to the trained officers.

For staff assigned to the experimental group participated in an initial 3-day training and were expected to attend monthly meetings. It was in these monthly meeting where skills were developed and maintained and on-going commitment to the project was encouraged. For those officers who were assigned to the control condition, we anticipated that motivation to participate in the research could diminish significantly. Therefore, similar to those assigned to the experimental group, officers assigned to the control group were brought together for a half-day seminar. In this seminar, the probation officers were given an overview of the "what works" literature, the research requirements, and the importance of random assignment. By providing an overview of the offender rehabilitation literature, we raised the possibility that probation officers may be encouraged to engage in some core correctional practices if they were not doing so already. In addition, the research team held bi-monthly teleconferences with the control group to answer their questions about the research and to reiterate their importance in the evaluation. Finally, the control group was promised the three-day training at the end of the evaluation in the event of favourable research results.

In total, 80 probation officers volunteered for the study with 51 officers assigned to the experimental "trained" group and 29 officers assigned to the control group. Table 4 presents an overview of the characteristics of the officers. No significant differences were found between those assigned to the two groups. Overall, the probation officers were well-educated with all of them having a university degree and most with specializations in the social sciences (e.g., psychology, criminology, and social work). On average they had approximately ten years of experience in probation.

Attrition is a problem in almost all experiments in criminology and this study was no different. Despite recruiting volunteers and having other structures in place to minimize attrition, we had 28 probation officers (18 experimental, 10 control) who, after training, did not recruit any clients for the study. This represents an attrition rate of 35%. The drop-outs fell into two main groups. The first group consisted of 18 probation officers (11 experimental, 7 control) who did not participate because they felt that they could not meet the demands of the extra work required by the project. The second group of 10 officers (7 experimental and 3 control) did not recruit clients because they were transferred to new jobs or withdrew from the project for personal reasons (e.g., maternity leave) soon after training. Comparison of the characteristics (see Table 4) of probation officers who continued with the project and those who dropped out found no statistically significant differences. Finally, it should be noted that seven officers submitted post-training data but ended their project participation early due to a new job or maternity leave. Nevertheless, data from these seven officers were included in all analyses following the intent-to-treat principle.

Assessment of the Client:

As in all "what works" research projects of this type, gathering offender data included recruiting clients and collecting client information. For client recruitment, officers were asked to recruit six clients with new probation orders who had recently come onto the officer's caseload. In accordance with the Risk Principle, officers were asked to recruit four high-risk and two medium-risk clients (offenders risk was determined by the existing risk-needs assessment used by the jurisdiction). Not only was risk-need assessment a prerequisite for RNR adherence, it was also a factor in determining the effectiveness of correctional interventions (Collaborative Outcome Data Committee, 2007a; 2007b). In addition to the risk-need assessments used in a particular jurisdiction, cross-jurisdiction risk-need information was collected. For example, probation officers across all three sites provided data that permitted scoring the 10 items of the Criminal History subsection of the Level of Service Inventory – Revised (LSI-R; Andrews & Bonta, 1995). We also asked officers to rate the severity of problems on seven criminogenic need areas (i.e., personality, attitudes, peers, family/marital, employment/education, substance abuse, leisure/recreation). These needs were assessed again after three and six months of supervision.

Finally, we gathered various demographics and criminal history information on the clients. This data was assessed at multiple times using two sources. Client self-report instruments were used to measure attitudes and problem-solving skills at intake and again after six months of supervision. Officer ratings and reports were used to measure the client's compliance and their criminal behavior after three and six months of supervision. The evaluation also included a plan to collect one year recidivism outcomes using official records from provincial and national sources.

In accordance with our client selection criteria, almost all of the 143 probationers recruited for STICS were assessed as medium (40%; n = 57) or high-risk (55% n = 79), determined by the respective jurisdiction's risk-need assessment. Despite the risk criteria for client recruitment, a small percentage of low-risk clients were recruited (5%; n = 7) owing to one site's client-assignment procedures (e.g., there were a couple of participating probation officers in Prince Edward Island who supervised only low and medium-risk clients). No significant differences were noted between experimental (STICS) and control group client risk level, the Criminal History subsection of the LSI-R, or demographic variables (see Table 5). Given the over-representation of medium and high-risk offenders, it was not unexpected to find that the large majority of the clients had prior convictions and sentences of incarceration.

Assessment of Probation Officer Behaviour: Clearly underlying STICS is the assumption that the behaviour of probation officers during supervision sessions influences the behaviour of the clients. In that respect, a primary concern of this study was evaluating the effects of the implementation components on the behaviour of probation officers. Even though officers were randomly assigned to the two groups, we wanted to ensure that the baseline "effectiveness" of officers was comparable. To assess this, we plan to retrospectively gather recidivism data on six offenders that the officer supervised approximately one year prior to the start of the project. Data was also gathered on the officers' participation in the various aspects of the maintenance components (e.g., initial training, monthly meetings, formal clinical feedback, and attendance at the refresher course).

The behaviour of probation officers during client sessions was assessed through the use of audiotape recordings. Probation officers were asked to audiotape three separate sessions with each client recruited for the project: one soon after the intake assessment, after 3 months of supervision, and again after approximately 6 months of supervision. Audiotapes have been widely used for training purposes (Aveline, 1997; Gordon & Arbuthnot, 1988), evaluating the fidelity of treatment interventions (Ball, Martino, Nich, Frankforter, Van Horn, Crits-Christoph et al., 2007; Barber, Krackauer, Calvo, Badgio & Faude, 1997; Gondolf, 2008) and monitoring the supervision provided by probation officers to their clients (Bogue, Pampel & Vanderbilt, 2007). Audiotapes have the advantage of being relatively unobtrusive and non-threatening compared to videotapes or observers sitting in on sessions.

In addition to the 78 pre-training audiotapes (50 experimental and 28 control), we received a total of 299 audiotapes of which 295 were valid for coding (four audiotape files cut abruptly shortly after the interview started). This included 220 audiotapes submitted by 33 STICS trained officers and 75 tapes submitted by 19 Control group officers. The majority of these sessions were at intake (n = 140) with fewer being recorded at three months (n = 93) and six months (n = 62). Unlike the assessment prior to training where there was a single audiotape sample for each probation officer, officers were requested to submit multiple post-training audiotapes. On average, STICS officers submitted significantly (t (50) = 2.43; p = .019) more audiotapes (M = 6.76; SD = 4.35) than did Control group officers (M = 4.00; SD = 3.09). In order to reduce potential bias in the data introduced by officers with more tapes, aggregate mean scores across tapes for each individual officer were calculated and then between-group differences were examined.

The audiotapes were assessed by trained raters, in teams of two, using a detailed coding guide (available upon request from the authors). The coding focused on behaviours that adhered to the Risk-Need-Responsivity principles. The coding was conducted in two steps. First, raters coded each audiotape in five minute segments, examining the presence or absence of specific topics of discussion (i.e., the various criminogenic needs that were identified for that client, non-criminogenic needs, conditions of probation, crisis). Next, the audiotape was listened to in its entirety without interruption, and the raters coded the presence and quality of the specific skills and interventions used by the officers (e.g., active listening, prosocial modeling, and cognitive restructuring). A 7-point Likert scale was used to assess quality. For example, if procriminal attitudes were targeted in a session then it was rated from "1" (confrontational identification, not getting client buy-in or understanding, etc.) to "7" (identification of procriminal attitudes in a non-confrontational manner, discussion about effects of procriminal attitudes, confirmation of client understanding, etc.).

Although there was a large amount of data coded from the audiotapes, we will highlight the results of some key variables. Two 'frequency' variables were simply counts of the number of 5-minute segments during which: (a) officers discussed criminogenic needs as identified by the intake risk/need assessment, and (b) officers targeted procriminal attitudes. In addition, five key intervention/skill quality constructs based on adherence with the RNR principles were calculated from ratings on individual audiotape-coded items which were grouped a priori into the broader constructs. These constructs included: (a) the level of structure of the session, (b) skills to building a collaborative working relationship, (c) cognitive techniques (e.g., focus on procriminal attitudes, cognitive restructuring), and (d) behavioural techniques (e.g., reinforcement, modeling, rehearsal). These four constructs were also combined into an overall 'Effective Correctional Skills' score.

Results on these variables reflect the officers' application of the RNR-based STICS model of community supervision with their clients. As can be seen in Table 6, the trained probation officers spent significantly more of their sessions focusing on criminogenic needs (p < .01) and procriminal attitudes (p < .01) than the control officers. In addition, they demonstrated significantly (p < .01) higher quality of RNR-based skills and interventions than the Control group officers with the exception of behavioural techniques, where the difference between the groups was not statistically significant (p = .06). The average session length was comparable between the two groups with the STICS group averaging 26:45 minutes a session and 24:36 minutes for the Control group (p > .05).

Significant between group post-training differences: * p< .05; ** p< .01.
1 Each construct was measured using a different set of items and had different theoretical range of scores. Although all had a theoretical minimum score of 0, the theoretical maximum score varied for each construct. The maximum theoretical score was 56 for Structure, 35 for Relationship building, 28 for Cognitive techniques, 49 for Behavioural techniques, and 168 for Effective Correctional Skills. The observed range for each is shown in brackets.

Summary
There is a significant body of research demonstrating that offender rehabilitation can reduce recidivism. For those individuals seeking to design, implement, and evaluate evidence-based offender treatment services, the principles of Risk-Need-Responsivity provide guidance. However, articulation of these principles fails to provide concrete guidance or solutions to the practical challenges of translating these principles into the "real world" of everyday correctional work. This paper attempted to assist those "real world" efforts by identifying some of the key challenges and issues in the areas of program design, implementation, and evaluation of transferring "what works" to community corrections and by illustrating how they were addressed at each of these three stages. With RNR principles at the nexus of the strategies to address design, implementation, and evaluation challenges, we believe that efforts to bring "what works" into everyday corrections can significantly advance our knowledge and practice of effective corrections.

The first step to bringing "what works" to the "real world" is designing the service, intervention, or program. Developers must ensure that the "nuts and bolts" of the service/intervention adhere to the RNR principles. Although Andrews (2006) has provided a "to do" list to comply with the principles, it lacks specific guidance on the "what" and "how exactly" the service/interventions should be done behind closed doors. For example, Andrews' list includes employing cognitive-behavioural and social-learning interpersonal-influence strategies, but what exactly should a probation officer do and when should they do it during a supervision session with an offender? With explicit identification of the design challenges for community supervision, the STICS model was developed. As a model of community supervision, STICS included a simple, concrete cognitive-behavioural approach with specific interventions that targeted procriminal attitudes. Moreover, the model had a clear structure for the entire supervision period as well as a structure for the individual officer-client sessions. And, at the same time, STICS had flexibility to permit addressing a wide spectrum of criminogenic needs with a variety of clients.

The second step of bring "what works" to the "real world" is implementation. Numerous researchers (Andrews, 2006; Goggin & Gendreau, 2006; Lowenkamp Latessa, & Smith, 2006) have recognized the importance of implementation and the challenges of having a RNR-based service or intervention delivered as intended. It is relatively straight forward to identify some prerequisite organizational conditions (e.g., risk-need assessment is utilized, policies that support the RNR principles, management support and commitment for implementation). However, there are clear challenges regarding how to encourage front-line staff to consistently engage in RNR-based interventions when the actual client work is being done. In community supervision, it is critical to facilitate and maintain behaviour change in the officers themselves. Of course, the question is how best to accomplish this change? It is believed that traditional training approaches in corrections have not sufficiently addressed the challenges we have outlined. These challenges need be faced and overcome when implementing "what works" into community supervision.

The authors of STICS took considerable effort developing the initial training program and its on-going maintenance components. Practically, this meant that implementation had to recognize the significant demand that was placed on probation officers who must make the transition from a surveillance/enforcer role to a teaching/therapist role. Many probation agencies emphasize the former role for probation officers and treatment is usually referred out to structured group programs. In a departure from the norm, we asked probation officers to structure their supervision session by spending the majority of the session actively teaching a new prosocial skill or attitude. Doing this kind of teaching in one-on-one supervision is likely intimidating to almost all officers who are accustomed to the enforcement/monitoring role. Practically speaking, the initial three-day training and the on-going maintenance components of STICS included specific components to address this personal uneasiness as well as any potential skill deficit in this teaching role. The training explicitly addressed why the officers should change their own behaviours and what behaviours (i.e., skills and interventions) would be more effective.

Just as adherence to the Responsivity Principle (e.g., using cognitive-behavioural strategies; key language, concepts, and core skills moulded to the learning style of the client) is pivotal to promote offender change, all aspects of the implementation attempted to adhere to the same Responsivity Principle. Consequently, the training and on-going maintenance for officers utilized cognitive-behavioural techniques to 'teach' and 'change' officer behaviour. Efforts were made to help officers understand the STICS model, learn the concepts, skills, and techniques, practice them, and learn how to 'teach' and facilitate 'change' in their clients. As the early results of STICS suggest, it is important for correctional agencies considering implementing RNR-based services to recognize the level of support (e.g., three days of training, refresher course, monthly meetings, and individual feedback) that was provided to change officer behaviour.

The final piece to translating "what works" into practice is evaluation. We value every effort a researcher makes to enhance a research project's potential contribution to the field. Of course, there are a number of methodological challenges to evaluating recidivism reduction efforts. For example, the research design, what factors are measured, how they are assessed, as well as the overall level of "contamination" (e.g., attrition, breakdown in randomization) impacts a study's internal and external validity. For STICS, we recognized that the study's fundamental assumption was that change in offender behaviour was, at least partially, dependant on the probation officer's behaviour during supervision sessions. The research methodology paid particular attention to the officers as study participants. This was apparent through the random assignment of officers and the collection of data on the officer's behaviour during supervision sessions. The results showed that STICS-trained officers, compared to controls, demonstrated significantly more and qualitatively better effective correctional practices during their interactions with clients.

There were two main limitations of this project. The first limitation is that the officers were volunteers, and therefore perhaps more likely open to this type of model and training. This raises the question, to use Goldkamp's (2008) words, were we "missing the target"? Volunteering probation officers are certainly a subset of the officer population and the generalizability to the population is limited. The impact of STICS on less inclined officers would require further research. A second concern was the level of officer attrition, a potential contaminate of the study's overall validity. In spite of our efforts to maintain motivation and enhance participation, 35% of the original volunteer officers did not provide any post-assignment data to the project. It is important to note that the majority of the officers who did not submit post-assignment data cited reasons concerning additional workload. This post-assignment attrition threatens the generalization of our findings. We are attempting to address this potential threat by undertaking a retrospective file review of pre-project cases to examine whether there were pre-existing differences in the probation officers' effectiveness to reduce reoffending. Future projects attempting to bring "what works" into the real world of everyday corrections would be wise to put in place organizational (e.g., additional management support and resources) and methodological (e.g., incentives for compliance) strategies to reduce officer attrition.

In conclusion, this paper attempts to add more concrete and practical guidance to bringing "what works" into the "real world" of community corrections. The description of our "how to" for STICS development, implementation, and evaluation illustrates one way in which a comprehensive RNR package to offender supervision can address the issues and challenges that can potentially erode a service's impact on facilitating offender change. The overall purpose of the STICS evaluation component was to demonstrate that the key ingredients of the RNR model can be successfully taught to probation officers and applied to their clients. Although the results on officer behaviour during supervision sessions are encouraging, the next step is to evaluate the impact of these changes in officer behaviour on client attitudes, behaviours, and ultimately recidivism. Overall, it is hoped that our experiences developing STICS will provide insight and further guidance into how to effectively transfer empirical knowledge into the real world of community corrections.

References

Ackerman, S. J., & Hilsenroth, M. J. (2003). A review of therapist characteristics and techniques positively impacting the therapeutic alliance. Clinical Psychology Review, 23, 1-33.

Ahn, H., & Wampold, B. E. (2001). Where oh where are the specific ingredients? A meta-analysis of the component studies in counseling and psychotherapy. Journal of Counseling Psychology, 48, 251-257.

Andrews, D. A. (1979). The dimensions of correctional counselling and supervision process in probation and parole. Toronto: Ontario Ministry of Correctional Services.

Andrews, D. A. (2006). Enhancing adherence to risk-need-responsivity: Making quality a matter of policy. Criminology and Public Policy, 5, 595-602.

Andrews, D. A. (2008). Extensions of the Risk-Need-Responsivity model of assessment and correctional treatment. In G. Bourgon, R.K. Hanson, J.D. Pozzulo, K.E. Morton Bourgon, & C.L. Tanasichuk (Eds.), The Proceedings of the 2007 North American Correctional & Criminal justice Psychology Conference (User Report 2008-02: pp. 7-11). Ottawa, Public Safety Canada.

Andrews, D. A., & Bonta, J. (1995) The Level of Service Inventory - Revised. Toronto: Multi-Health Systems.

Andrews, D. A., & Bonta, J. (2006). The psychology of criminal conduct (4th ed.). Newark, NJ: LexisNexis.

Andrews, D. A., Bonta, J., & Hoge, R. D. (1990). Classification for effective rehabilitation: Rediscovering psychology. Criminal Justice and Behavior, 17, 19-52.

Andrews, D. A., & Carvell, C. (1997). Core correctional treatment-core correctional supervision and counselling: Theory, research, Assessment and practice. Ottawa: Department of Psychology, Carleton University.

Andrews, D. A., & Kiessling, J. J. (1980). Program structure and effective correctional practices: A summary of the CaVIC research. In R.R. Ross & P. Gendreau (Eds.), Effective correctional treatment (pp. 439-463). Toronto: Butterworth.

Andrews, D. A., Zinger, I., Hoge, R. D., Bonta, J., Gendreau, P. & Cullen, F. T. (1990). Does correctional treatment work? A psychologically informed meta-analysis. Criminology, 28, 419-429.

Aveline, M. (1997). The use of audiotapes in supervision of psychotherapy. In G. Shipton (Ed.), Supervision of psychotherapy and counseling: Making a place to think (pp. 80-92). Buckingham, England: Open University Press.

Ball, S. A., Martino, S., Nich, C., Frankforter, T. L., Van Horn, D., Crits-Christoph, P., et al. (2007). Site matters: Multisite randomized trial of motivational enhancement therapy in community drug abuse clinics. Journal of Consulting and Clinical Psychology, 75, 556-567.

Barber, J. P., Krakauer, I., Calvo, N., Badgio, P. C., & Faude, J. (1997). Measuring adherence and competence of dynamic therapist in the treatment of cocaine dependence. Journal of Psychotherapy Practice & Research, 6, 12-24.

Barlow, D. H. (2004). Psychological treatments. American Psychologist, 59, 869-878.

Barnoski, R. (2004). Outcome evaluation of Washington State's Research-based programs for juvenile offenders. Olympia, Washington: Washington State Institute for Public Policy.

Bogue, B., Pampel, F., & Vanderbilt, R. (2007). Evaluation of alternative sanction programs, Connecticut probation and related services. Report to the Connecticut Judicial Branch, Court Support Services Division, Wethersfield, CT.

Bonta, J., & Andrews, D.A. (2007). Risk-need-responsivity model of offender assessment and rehabilitation (User Report 2007-06). Ottawa: Public Safety Canada.

Bonta, J., Rugge, T., Scott, T., Bourgon, G., & Yessine, A. (2008). Exploring the black box of community supervision. Journal of Offender Rehabilitation, 47, 248-270.

Caroll, K. M., Ball, S. A., Nich, C., Martino, S., Frankforter, T. L., Farentinos, C., et al. (2006). Motivational Interviewing to improve treatment engagement and outcome in individuals seeking treatment for substance abuse: A multi-site effectiveness study. Drug and Alcohol Dependence, 81, 301-312.

Clear, T. (1994). Harm in American penology: Offenders, victims and their communities. Albany, NY: State university of New York Press.

Collaborative Outcome Data Committee. (2007a). Sexual offender treatment outcome research: CODC Guidelines for evaluation Part 1: Introduction and overview (Corrections Research User Report No. 2007-02). Ottawa, Ontario: Public Safety Canada.

Collaborative Outcome Data Committee. (2007b). The Collaborative Outcome Data Committee's guidelines for the evaluation of sexual offender treatment outcome research Part 2: CODC guidelines (Corrections Research User Report No. 2007-03). Ottawa, Ontario: Public Safety Canada.

Cullen, F. T., & Gilbert, K. E. (1982). Reaffirming rehabilitation. Cincinnati, OH: Anderson.

Dowden, C., & Andrews, D. A. (2004). The importance of staff practices in delivering effective correctional treatment: A meta-analysis of core correctional practices. International Journal of Offender Therapy and Comparative Criminology, 48, 203-214.

Farrington, David P. (2006). Methodological quality and the evaluation of anti-crime programs. Journal of Experimental Criminology, 2, 329-337.

Farrington, David P., & Welsh, B.C. (2005). Randomized experiments in criminology: What have we learned in the past two decades? Journal of Experimental Criminology, 1, 9-38.

Frank, J. D., & Frank, J. B. (1991). Persuasion and healing: A comparative study of psychotherapy (3rd ed.). Baltimore, MD: John Hopkins University Press.

Gendreau, P., & Ross, R. R. (1979). Effective correctional treatment: Bibliotherapy for cynics. Crime & Delinquency, 25, 463-489.

Goggin, C., & Gendreau, P. (2006). The implementation and maintenance of quality services in offender rehabilitation programs. In C. R. Hollin & E. J. Palmer (Eds.), Offending behaviour programmes: Development, application, and controversies. Chichester (pp. 209-246). Chichester, England: John Wiley & Sons.

Goldkamp, J. S. (2008). Missing the target and missing the point: 'successful' random assignment but misleading results. Journal of Experimental Criminology, 4, 83-115.

Gondolf, E. W. (2008). Program completion in specialized batterer counseling for African-American men. Journal of Interpersonal Violence, 23, 94-116.

Gordon, D. A., & Arbuthnot, J. (1988). The use of paraprofessionals to deliver home-based family therapy to juvenile delinquents. Criminal Justice and Behavior, 15, 364-378.

Hanley, D. (2006). Appropriate services: Examining the case classification principle. Journal of Offender Rehabilitation, 42, 1-22.

Hanson, R.K., Bourgon, G., Helmus, L., & Hodgson, S. (2009). The principles of effective correctional treatment also apply to sexual offenders: a meta-analysis. Criminal Justice and Behavior, 36, 865-891.

Homel, P., Nutley, S., Webb, B., & Tilly, N. (2005). Investing to deliver: reviewing the implementation of the UK crime reduction programme. Home Office Research Study 281. London, England: Home Office Research.

Lipsey, M. W. (1989, November). The efficacy of intervention for Juvenile delinquency: results from 400 studies. (Paper presented at the 41st annual meeting of the American Society of Criminology, Reno, Nevada).

Lipsey, M. W., & Cullen, F. T. (2007). The effectiveness of correctional rehabilitation: A review of systematic reviews. Annual Review of Law and Social Science, 3, 297-320.

Lipton, D., Martinson, R., & Wilks, J. (1975). The effectiveness of correctional treatment: A survey of treatment evaluation studies. New York: Praeger.

Lösel, F. (1995). The efficacy of correctional treatment: A review and synthesis of meta-evaluations. In J. McGuire (Ed.), What works: Reducing reoffending (pp.79-111). Chichester, England: John Wiley & Sons.

Lovins, L. B., Lowenkamp, C. T., Latessa, E. J., & Smith, P. (2007). Application of the risk principle to female offenders. Journal of Contemporary Criminal Justice, 23, 383-398.

Lowenkamp, C. T., Latessa, E. J., & Holsinger, A. M. (2006). The risk principle in action: What have we learned from 13,676 offenders and 97 correctional programs. Crime & Delinquency, 52, 77-93.

Lowenkamp, C. T., Latessa, E. J., & Smith, P. (2006). Does correctional program quality really matter? The impact of adhering to the principles of effective intervention. Criminology & Public Policy, 5, 575-594.

Marlowe, D. B., Festinger, D. S., Lee, P. A., Dugosh, K. L., & Benasutti, K. M. (2006). Matching judicial supervision in clients' risk status in drug court. Crime & Delinquency, 52, 52-76.

Miller, W. R., & Mount, K. A. (2001). A small study of training in motivational interviewing: Does one workshop change clinician and client behavior? Behavioral and Cognitive Psychotherapy, 29, 457-471.

Palmer, T. (1975). Martinson revisited. Journal of Research in Crime and Delinquency, 12, 133-152.

Palmer, E. J., McGuire, J., Hatcher, R. M., Hounsome, J. C., Bilby, C. A. L. & Hollin, C. R. (2008). The importance of appropriate allocation to offending programs. International Journal of Offender Therapy and Comparative Criminology, 52, 206-221.

Public Safety Canada (2008). Corrections and Conditional Release Statistical Overview 2008. Ottawa, Public Safety Canada.

Raynor, P. (2004). The probation service "pathfinders": finding the path and losing the way? Criminal Justice, 4, 309-325.

Raynor, P. (2008). Community penalties and Home Office research: On the way back to "nothing works"? Criminology & Criminal Justice, 8, 73-87.

Redondo, S., Garrido, V., & Sanchez-Mecca, J. (1999). The influence of treatment programs on the recidivism of juvenile and adult offenders: A European meta-analytic review. Psychology, Crime, and Law, 5, 251-278.

Rhine, E.E., Mawhorr, T.L., & Parks, E.C. (2006). Implementation: The bane of effective correctional programs. Criminology and Public Policy, 5, 347-358.

Taxman, F. S., & Marlowe, D. (2006). Risk, needs, responsivity: In action or inaction? Crime & Delinquency, 52, 3-6.

Trotter, C. (1996). The impact of different supervision practices in community corrections: Cause for optimism. Australian and New Zealand Journal of Criminology, 29, 1-18.

Trotter, C. (2006). Working with involuntary clients: A guide to practice (2nd ed.). Thousand Oaks, CA: Sage.

Wampold, B. E. (2007). Psychotherapy: The humanistic (and effective) treatment. American Psychologist, 62, 857-873.

Welsh, W.N. (2006). The need for a comprehensive approach to program planning, development, and evaluation. Criminology and Public Policy, 5, 603-614.

Wilson, J.A. & Davis, R.C. (2006). Good intentions meet hard realities: an evaluation of the Project Greenlight reentry program. Criminology & Public Policy, 5, 303-338.

Date modified:

2022-08-08