Youth Advocate Program

Youth Advocate Program PDF Version (249 KB)

ISSN: 978-1-100-20387-4

Table of contents

Building the Evidence - Evaluation Summaries 2012-ES-25

1. IntroductionFootnote 1

Halifax Regional Police (HRP) data showed an increase in youth crime from 2005 to 2006. HRP had been tracking gang-related activity and identified seven gangs active within the Halifax Regional Municipality (HRM), including the former cities of Halifax and Dartmouth.

At this time, it was estimated that there were 80 youth directly involved in gangs with 20 more youth identified as affiliated with gang members or gang activities. Four HRM Public Housing projects had active gangs or evidence of gang-related violence. Public housing data for the four areas identified over 400 youth between the ages of 9 to 14, primarily from single parent families on income assistance.

In response to the increased level of gang-related activity, HRM applied to the National Crime Prevention Centre (NCPC) Youth Gang Prevention Fund (YGPF) and subsequently received 1.9 million in funding to implement the Youth Advocate Program (YAP) from 2008-2011. This included the cost of conducting an impact evaluation. The YAP was piloted in six communities in the HRM as a response to gang activity, four of which were the above-noted housing areas. Based on police information and community stakeholder consultation six sites were selected.

The YAP focused on preventing youth in the target age group of 9 to 14 years, which are typically in Grades 4 to 9, from engaging in gang-related activities, anti-social and criminal behaviours.

YAP proposed a community-based intervention to best meet the needs of the targeted population. In keeping with this emphasis, the Wraparound method was chosen as the theoretical base of the YAP. Wraparound is meant to provide a comprehensive suite of support to the participants. Elements of effective wraparound include: putting people and their individual needs first; a focus on the family as a whole system; participation of people in their case planning; safety; respect for culture; continuity over time; a non-blaming approach; a single integrative plan for a community of supports and the service provider; and an emphasis on doing what is efficient and effective (VanDenBerg & VanDenBerg, 2005Footnote 2).

2. Program Description

The goal of the YAP was to address the risk factors that make youth vulnerable to gang influence and involvement, such as a lack of school attachment and positive role models, low self-esteem, engagement in anti-social behaviour, poor family relationships and bullying.

The program focused on three key factors:

The project involved six Youth Advocate Workers (YAW), one per neighbourhood, who worked directly with a maximum of five participants. Referrals could come from anyone affiliated with the youth. A YAW received the referral, opened a case file and contacted the parent/guardian(s) of the young person to schedule an appointment to discuss the program. An assessment was then scheduled. Based on the outcomes a youth would be accepted into the program or referred to a more suitable alternative. At each step the parent/guardian(s) were involved.

The YAW developed the participant action plan in consultation with the young person and their family, the school and other stakeholders in the community. The action plan included five hours weekly of components tailored to the youths' needs: recreation activities, therapy, academic tutoring, group counselling and discussion, youth support teams, one-on-one time, referrals for parents and direct advocacy to service providers for the youth and his/her family. The action plan was a flexible document, amended as the circumstance and needs of the participant required. Finally, it included performance measures for determining exit.

Life skills training and development topics, to direct youth towards healthy and pro-social behaviours, were also included in the action plan. These were delivered as experiential learning exercises by the YAWs during 'teachable moments', rather than in workshops settings. They were opportunity-based occasions, either during one-on-one time or around the time of a significant event/incident. Life skills modules were also reviewed with parents individually or in groups.

Finally, the YAP intervention involved joint activities for the youth and family. This involved collaboration with school representatives to improve school performance including attendance, behaviour, homework, and participation in extracurricular activities. This intervention identified and assisted the youth to participate in community activities, for example, by supporting parent access to transportation.

Program Participants

Youth were accepted into the program based on several variables including their scores on the Youth Advocate Program Screening Tool (YAPST), the reasons for the referral and the risky behaviours described by the youth themselves. Participants displayed characteristics that would make them vulnerable to gang membership and/or gang activities. These included difficulty forming age-appropriate relationships, greater issues with anger management and impulsiveness, and higher levels of delinquent behaviour and substance use. Finally, participants demonstrated a greater acceptance of gang-related behaviour and violence.

Among the youth accepted into the program, across the six high-risk neighbourhoods, 84% were male and 16% were female, 61% were between 12-15 years old and 39% between 9-11 years. The majority lived with their parents or guardians, of which 68% lived with a single parent. The youth accepted into the program showed a higher degree of engagement with child/family or correctional services, which would indicate a higher level of risk. Information from external sources, such as the police, school or probation staff, was also considered prior to admittance.

As of November 2010, 73 youth were referred and assessed, and 57 subsequently accepted into the program. Of the 57, 15 had graduated, 15 left for other reasons (e.g. incarceration, moving away) and 27 remained active participants. On average, graduates were in the program for 16 months with approximately five hours weekly of intervention.

3. Evaluation of the Program

The evaluation objectives were to establish the viability of the YAP as an intervention to prevent youth gang membership, in terms of process, cost and outcomes. The evaluation utilized a quasi-experimental matched comparison group design, and included both qualitative and quantitative data collection methods.

The evaluation involved triangulated methods such as surveys, interviews and file review, so that multiple viewpoints were incorporated in the study to avoid the bias of any one stakeholder.

Finally, a process of dialogical reciprocity was built into the design to mitigate issues of construct validity the evaluators also checked in with participants, parents, and professionals to see if the research team's interpretation of the data made sense to the evaluation's stakeholders.

The researchers used several instruments and data collection methods:

The evaluation involved a repeated measures design. Youth who were referred to the YAP were initially assessed using the YAPST survey. After being admitted to the YAP, youth were assessed every six months while participating in the program. Upon graduation or departure from the program, youth were given an exit assessment, followed by a final assessment six months after exit.

The YAPST was the primary quantitative data-gathering tool for the evaluation. It has two assessments, one for youth aged 9 to 11 years and another for youth aged 12 to 14 years. The YAPST contains validated scales of factors central to the prevention of youth gang involvement.

The three key domains are:

The YAPST – PMK was administered to participant parent/guardian(s). It investigates the same three key areas to gain their perspective. Focus groups and interviews were conducted on 20% (n = 14) of the active and inactive files and the case records of the YAWs were reviewed. The analysis followed the constant comparative method as the continued response patterns were identified.

Reported statistics were largely descriptive in nature. Three statistical tests were used to detect changes over time. Dependent sample t-tests were used for within-group comparisons. Independent Sample t-tests were used to test for statistical differences over time, relative to the community comparison group. Finally, one-way repeated measures ANOVA tests were used to determine changes over time.

4. Evaluation Findings

Process Findings

The Wraparound model

The YAP engaged family and community stakeholders in the delivery of services outlined in the youths' action plan. This included services and supports to the participant's families. The philosophy of the model was adhered to, including, putting the young person and his/her needs first, focusing on the family as a whole, respecting culture, safety, and non-blaming.

The one area that did not strictly follow the Wraparound model was Youth Service Team meetings. Service providers were linked together, though in a more virtual manner. Face-to-face meetings were rare and the YAW was the link between each element. The outcome was similar in terms of service coordination, though with significantly more work required on the part of the YAW.

Target population

The YAP program successfully reached its at-risk target population. Youth who were accepted into the YAP showed difficulty forming age-appropriate relationships with peers, had a higher likelihood of manifesting anger and impulsiveness through problematic conduct, and engaged in significantly higher levels of delinquent behaviour and substance use. They showed significantly more positive attitudes towards displays of aggression, and more knowledge of the existence of gangs than youth who were not accepted into the program.

Family engagement

The experiences of the families were positive. Parents noted they felt supported by the YAWs in their parenting and within the greater community. Parents learned more about the services that existed in the community for their children and became better able to advocate for their children and themselves, especially when they had previously faced barriers to services. An unexpected component of the program was the very high degree to which some parents came to rely on the YAW for securing and maintaining support.

Personnel and training

The program had very little staff turnover, though concern arose about staff burn-out and stress. Over the lifetime of the program there were over 36 staff training events. Formal qualifications among staff varied. Not all had university degrees but all were required to have a very high standard of community awareness and direct intervention skills. The reports from the participants and families on the skills of YAW were extremely positive. The only area in which YAWs themselves specifically noted concern was in their ability to provide interventions for girls. They felt the regular programs did not always address some of the specific issues faced by girls at risk of gang involvement.

Partnerships

YAP program minutes noted that various organizations within the community attended meetings both to share information with the YAWs and to seek out partnerships and professional relationships with the YAP. YAP staff were frequently out in the community participating in events and raising awareness of the program. The evaluation noted that the YAP formed partnerships with 25 different non-profit, private and government organizations to support the program and participants.

Outcome Findings

The scores for participants improved over most variables measured by the YAPST. Although in many cases not statistically significant, the results showed positive trends among key variables. The analysis showed an increase in all pro-social/life skills competencies. This increase is equated with an increase in resilience, which is considered to be a buffer against delinquency and related behaviours, and gang involvement.

According to the evaluation results, the YAP appears to have reduced isolation and negative rushes among youth at risk of gang activity and it increased their pro-social and life skills competencies. The results did demonstrate that youth scores sometimes became worse before they improved. This could be attributed to the changing relationships with the YAWs or the age of the participants.

Awareness

Awareness was a key area of improvement with parents. Interviews with parents demonstrated that prior to the youth/parent involvement in the program they had often been unaware of the various programs and services in their own neighbourhoods. Increasing this awareness was a crucial first step in supporting parents to access service for their children.

Attitudes

The main two concerns of the YAP, that youth were showing positive attitudes towards guns, violence and gangs and were normalizing aggression, showed positive change at the time of graduation. Participant's positive beliefs about aggression, decreased slightly at exit relative to their beliefs at entry to the YAP. Participants' positive attitudes towards guns and violence greatly diminished at exit compared to their baseline, as shown by a reduction in mean score of nearly 40% from baseline. Attitudes towards gangs also became more negative, with a mean score approaching the scale minimum of zero. These outcomes were further validated qualitatively through interviews with participants, their families and police.

Though not statistically significant, the attachment to school (as measured by attendance) scores for youth who graduated from YAP increased (with the mean score of 4.08 (SD = 0.50) at exit nearing the maximum possible score on the scale of 5.00). Attachment to teachers also increased slightly and peer problems decreased slightly. While not statistically significant, these results represent a combined decrease in isolation and increase in positive attachment. The comparison group demonstrated decreasing teacher attachment, increased family detachment and peer difficulties, while the YAP participants improved their scores in these areas.

Skills

Youth in gangs showed lower scores on pro-social and life skills competencies (except for self-esteem on their baseline and exit assessments) compared to the comparison group. Importantly, the exit scores were generally higher than the baseline assessment. As a group, YAP participants show a longitudinal increase in their pro-social skills and self-esteem scores between baseline and exit. None of the findings were statistically significant; however, exit scores did demonstrate improvement relative to the scores at the baseline assessment.

Behaviour

When analyzed longitudinally, youth exiting the program showed a decrease in conduct problems, victimization, impulsivity and delinquency. The change in the victimization scores is significant (p = 0.22), with a large effect size (53%, n2 = 0.53). As a group, their normative beliefs about aggression, attitudes towards gangs and impulsivity decreased at time two (T2) though increased again at exit. With respect to negative rushes, improvement was noted on the conduct problem scale, the initial mean score of 5.20 in the borderline/abnormal range improved to 4.72 which falls into the normal range for the YAP target group.

Cost Analysis Findings

The evaluation showed that the YAP was cost-effective. More specifically, the cost per participant was significantly less than the cost of having a child in care or in custody. In the third year of the program, the cost per participant was $27,758 per year or $2,313 per month.

Costs associated with the YAP are approximately one-fifth of those associated with incarcerating a youth offender of approximately $120,000 per youth per year or $10,000 per youth per month (Canadian Psychological Association, 2008).

Given that high-risk youth tend to use multiple services concurrently, service use costs and ultimately societal costs, are more likely to be compounded and therefore higher than the incarceration costs presented here. This further supports the cost-effectiveness of the YAP program.

Evaluation Limitations

This evaluation employed research methods that, according to NCPC Evaluation StandardsFootnote 4, would be deemed as good, with a strong ability to attribute program activities to outcomes. However, some limitations and challenges should be considered when interpreting the results.

The evaluation was initially designed to include a comparison group of wait-listed youth; however a wait list was never established. To compensate, the evaluator used a number of different cohorts as comparison groups for YAP youth. This included a non-random community comparison group, a sample of youth from junior high schools in the same target communities as YAP youth, and youth matched by gender, age and scores on measures of delinquency who are participants in other Resilience Research Centre studiesFootnote 5.

Exit and post-exit data for a small number of youth was not completed because some had exited the program early, and evaluators were unable to contact the youth. Most of these youth had moved away and/or their contact information had changed.

There was inconsistent reporting noted in the file review data in the early stages of project implementation. Evaluators worked with the YAWs and the situation improved considerably by year three.

Other important considerations include:

Statistical validity

The YAP had small sample sizes resulting in very few outcomes that were statistically significant. However, the majority of the results did indicate improvement on key variables in participants' scores over time. Confidence in these results is increased as they are supported by family and service provider concurrence.

Instrumentation validity

YAP staff administered the questionnaires to the youth while they were in the program and at exit. This may have introduced bias into the surveys whereby youth may have been inclined to provide more positive responses given their relationship with the YAW.

Internal validity

The limitations noted and other typical threats to validity such as maturation, history and testing should be considered when interpreting the results. However, the reader can place a reasonably high degree of confidence in the findings based on the rigor of the evaluation design implemented, the quality of the data and the statistical analysis applied.

5. Lessons Learned and Recommendations

Program Delivery

Youth advocate workers

The YAWs para-professional status and community awareness were continuously cited as positives. Their availability and accessibility increased the profile of the YAP in the community thereby increasing referral sources. The para-professional designation facilitated participation of youth and their families, 'street smarts' being a better credential for building trust than professional designation.

Clinical support

A gap identified in the design of the program was the need for stronger clinical support behind the YAW. Although, parents and youth generally cited overall satisfaction with the way in which the program was administered, it was recognized that more support for the staff working with youth and families with complex needs should be a part of the structure of the YAP. Given the intense nature of the work, it was recommended that the YAP secure a clinical consultant to support the YAWs with their complex caseloads. Additional training on the specific needs of female participants was suggested.

Youth Service Teams (YSTs)

Another unexpected challenge was the amount of resistance the YAWs experienced from service providers. The reasons were varied, including systemic barriers to coordination. Wraparound relies heavily on buy-in from not only the youth and their family, but also community-based, professional and government service providers.

As a result, the YSTs for each participant were never firmly established. To meet YSTs objectives, YAP staff consulted with each YSTs member individually to collect their opinions and expertise when making decisions on how to move forward. This ensured consistency across each action plan so participant/parent/guardian needs were communicated to all formal and informal supports.

While this sequential approach is an adaptation of the Wraparound model, keeping the young person and his or her family at the center of all decision-making, it dramatically increases the workload of the YAWs.

Selection area

The evaluators recommended that future iterations of the YAP consider a school-based selection area instead of neighbourhoods. This would provide a larger participant group while still having a static number of risk factors, e.g. per association and school attendance. As well, schools are virtual boundaries that are more meaningful to youth, especially of this younger age group, than municipal neighbourhoods.

Finally, it was strongly recommended that the program be expanded geographically and additional YAWs hired. In support of this it was suggested that the program build stronger ties with the advisory committee and establish community committees.

Evaluation

Control group

The initial evaluation plan had intended to use the program wait list as the comparison group. In the absence of a wait list, a community sample was drawn from the six neighborhoods and from the youth not accepted into the program. The evaluators suggested that future evaluation projects prepare a contingency plan in the absence of a wait list.

Assessment completion

A primary challenge was having the assessments completed on time, at the six-month intervals Late assessments and/or lack of timely follow-up with youth who withdrew from the program made exit assessments more difficult.

As a result, meaningful comparisons were not possible using post-exit assessments since only three youth completed them. Maintaining contact with the youth participants is a common problem as they tend to be a transient population who move frequently and change schools.

Data Tracking

Improved data tracking was recommended, particularly in the area of survey administration, client case management files and updating participant contact information.

It was difficult to assess the amount of time participants spent in the program, as they would sometimes quit, then return in the same week. In these circumstances, the action plans could be disrupted and/or services delayed, inflating the duration of participation due to difficulty in tracking actual dosage.

The evaluation recommended the administration of a fidelity checklist to be implemented, both for confirming adherence to the core principles of the program and to assist with tracking activities. It could also help with future model replication.

6. Conclusion

Overall, the evaluation results show that the YAP appears to be an effective community-based response to youth who are at risk of joining gangs or who are gang-involved.

The outcomes demonstrated reduced levels of isolation, stresses and negative rushes among the target group as well as increased resiliency and pro-social skills.

Finally, the community partners, police and families engaged in the project strongly affirmed that the YAP gave them more resources and tools for addressing youth gangs and youth violence in their neighbourhoods.

For more information or to receive a copy of the final evaluation report, please contact the National Crime Prevention Centre by e-mail at ps.prevention-prevention.sp@canada.ca.

If you wish to register for the NCPC mailing list to receive information from the Centre, please visit the subscription page at: http://www.publicsafety.gc.ca/cnt/bt/mlng-lst-eng.aspx

Footnotes

  1. 1 This synthesis note is based on the NCPC's research and evaluation team review and analysis of the final evaluation report prepared by Resilience Research Centre (2011) YAP Evaluation Final Report (File# 6735-H1).
  2. 2 As cited in the evaluation: VanDenBerg, J. & VandDenBerg, V. (2005). Integrated systems of care, team based support models, and the recent research on the Wraparound process. Presentation at the National Wraparound Conference, Toronto, ON.
  3. 3 Examples of negative rushes include socially undesirable behaviours like drug use, risky sexual activity and vandalism.
  4. 4 National Crime Prevention Centre. (2007) Evaluation Standards. Ottawa, Ontario.
  5. 5 Resiliency Research Centre, Dalhousie University, Nova Scotia.
Date modified: